In an earlier paper, we presented methods for estimating achievement gaps when test scores are coarsened into a small number of ordered categories, preventing fine-grained distinctions between individual scores. We demonstrated that gaps can nonetheless be estimated with minimal bias across a broad range of simulated and real coarsened data…

Ho and Reardon (2012) present methods for estimating achievement gaps when test scores are coarsened into a small number of ordered categories, preventing fine-grained distinctions between individual scores. They demonstrate that gaps can nonetheless be estimated with minimal bias across a broad range of simulated and real coarsened data…

This manual provides guidance and documentation for users of the longitudinal kindergarten-fourth grade (K-4) data file of the Early Childhood Longitudinal Study, Kindergarten Class of 2010-11 (ECLS-K:2011). It mainly provides information specific to the fourth-grade round of data collection. The first chapter provides an overview of the…

Literature shows that most researchers are unaware of some of the characteristics of reliability. This paper clarifies some misconceptions by describing the procedures, benefits, and limitations of reliability generalization while using it to illustrate the nature of score reliability. Reliability generalization (RG) is a meta-analytic method…

This guide is designed to help researchers make sure that their research reports include enough information about study measures so that readers can assess the quality of the study's methods and results. The guide also provides examples of write-ups about measures and suggests resources for learning more about these topics. The guide assumes…

This manual provides guidance and documentation for users of the longitudinal kindergarten-fourth grade (K-4) public-use data file of the Early Childhood Longitudinal Study, Kindergarten Class of 2010-11 (ECLS-K:2011), which includes the first release of the public version of the third-grade data. This manual mainly provides information specific…

After presenting a general linear model as a framework for discussion, this paper reviews five methodology errors that occur in educational research: (1) the use of stepwise methods; (2) the failure to consider in result interpretation the context specificity of analytic weights (e.g., regression beta weights, factor pattern coefficients,…

A major problem in the research concerning distributional and other properties of reliability coefficients has been the non-existence or inaccessibility of adequate test data for use in empirical verification of hypothetical conclusions. The purpose of this paper is to develop a technique for the simulation of test item scores through the use of…

To meet the requirements of the No Child Left Behind Act, school districts and states must compile summary reports of the levels of student achievement in reading and mathematics. The levels are to be described in broad categories: "basic and below," "proficient," or "advanced." Educational units are given considerable latitude in defining the…

Supports arguments against general use of change scores and recommends the Lord/McNemar estimates of true change. Provides a numerical example illustrating the reliability problem and the problem of the prediction of true change from various linear composites of initial and final measures. (Author)

An examination of the weaknesses of the common statistical approaches suggests that change scores are unnecessary and often even inappropriate for statistical studies. Alternative statistical procedures are suggested. (Author/VW)

Because assessment instruments of the future may well be composed of a combination of types of questions, a way to combine those scores effectively is discussed. Two new graphic tools are presented that show that it may not be practical to equalize the reliability of different components. (SLD)

A model and computation procedure based on classical test score theory are presented for determination of a correlation coefficient corrected for attenuation due to unreliability. Delta and Monte Carlo method applications are discussed. A power analysis revealed no serious loss in efficiency resulting from correction for attentuation. (TJH)

A measurement and psychometric examination of cognitive tasks was carried out with 195 undergraduates, investigating stability of latencies and number correct scores over a 2-week period and relations with standardized test and ability measure scores. Results are discussed in terms of the need for evaluating cognitive task measurement properties.…

This study validated computerized versions of selected tests from the Kit of Factor-Referenced Cognitive Tests of 1976, compared performance of higher education students on computerized and paper-and-pencil versions of the tests, and explored whether gender, academic status, or age interacted with the tests. (MBR)