The application of diagnostic classification models (DCMs) in the field of educational measurement is getting more attention in recent years. To make a valid inference from the model, it is important to ensure that the model fits the data. The purpose of the present study was to investigate the performance of the limited information…

Cognitive diagnosis models (CDMs) are an increasingly popular method to assess mastery or nonmastery of a set of fine-grained abilities in educational or psychological assessments. Several inference techniques are available to quantify the uncertainty of model parameter estimates, to compare different versions of CDMs, or to check model…

Traditional estimators of item response theory scale scores ignore uncertainty carried over from the item calibration process, which can lead to incorrect estimates of the standard errors of measurement (SEMs). Here, the authors review a variety of approaches that have been applied to this problem and compare them on the basis of their statistical…

Background: When studying student performance across different countries or cultures, an important aspect for comparisons is that of score comparability. In other words, it is imperative that the latent variable (i.e., construct of interest) is understood and measured equivalently across all participating groups or countries, if our inferences…

Test equating is a statistical procedure for adjusting for test form differences in difficulty in a standardized assessment. Equating results are supposed to hold for a specified target population (Kolen & Brennan, 2004; von Davier, Holland, & Thayer, 2004) and to be (relatively) independent of the subpopulations from the target population (see…

There is widespread interest in using various statistical inference tools as a part of the evaluations for individual teachers and schools. Evaluation systems typically involve classifying hundreds or even thousands of teachers or schools according to their estimated performance. Many current evaluations are largely based on individual estimates…

There are relatively few examples of quantitative approaches to quality control in educational assessment and accountability contexts. Among the several techniques that are used in other fields, Shewart charts have been found in a few instances to be applicable in educational settings. This paper describes Shewart charts and gives examples of how…

Statistical modeling of school effectiveness data was originally motivated by the dissatisfaction with the analysis of (school-leaving) examination results that took no account of the background of the students or regarded each school as an isolated unit of analysis. The application of multilevel analysis was generally regarded as a breakthrough,…

An assumption underlying multiple linear regression is that the standard error of measurement is equal for all observations. The literature has not addressed the procedures to be used when this assumption is violated. It was clear that data analysis to be performed on districts in California would severely violate this assumption, since district…

The authors present results of a generalizability study of running record assessment. They conducted 2 decision studies to ascertain the number of raters and passages necessary to obtain a reliable estimate of a student's reading ability on the basis of a running record assessment. Ten teachers completed running record assessments of 10…

The role that measurement error plays in the regression effect is discussed with particular reference to the RMC models for evaluating the effects of Elementary Secondary Education Act Title I programs. The norm referenced evaluation model assumes a theory of growth where the relative ranking of students remains the same from pretest to posttest…