The difficulties in comparing profile variability (a measure of test scatter) are briefly discussed and the limitations of current techniques pointed out. Test scatter is defined as individual variation in test scores between or within various psychological and educational tests. Currently, no statistical technique for the comparison of profile…

Cut-off scores from two approaches for setting standards are examined. Standards determined from judgments about groups and from inspection of test content are compared. Results indicate that there was neither consistency nor pattern to cut-off scores set from the two procedures. (Author/RD)

Stochastic models are developed in this article to examine the rate of test misgrading in educational and psychological measurement. The estimation of inadvertent grading errors can serve as a basis for quality control in measurement. Limitations of traditional Poisson models have been reviewed to highlight the need to introduce new models using…

Simulated tailored tests were used to investigate the relationships between characteristics of the item pool and the computer program, and the reliability and bias of the resulting ability estimates. The computer program was varied to provide for various step sizes (differences in difficulty between successive steps) and different acceptance…

The ultimate goal of physics education research (PER) is to develop a theoretical framework to understand and improve the learning process. In this journey of discovery, assessment serves as our headlamp and alpenstock. It sometimes detects signals in student mental structures, and sometimes presents the difference between expert understanding and…

In mastery testing a linear relationship between an optimal passing score and test length is presented with a new optimization criterion. The usual indifference zone approach, a binomial error model, decision errors, and corrections for guessing are discussed. Related results in sequential testing and the latent class approach are included. (CM)

A method for analyzing test-item responses is proposed to examine differential item functioning (DIF) in multiple-choice items within the latent class framework. Different models for detection of DIF are formulated, defining the subgroup as a latent variable. An efficient estimation method is described and illustrated. (SLD)

This paper introduces a probabilistic model that is capable of diagnosing and classifying cognitive errors in a general problem-solving domain. Item response theory is used to deal with the variability of response errors. Responses from a 38 item fraction addition test given to 595 junior high school students are used to illustrate the model.…

Tailored testing procedures for achievement testing were applied in a situation that failed to meet some of the specifications generally considered to be necessary for tailored testing. Discrepancies from the appropriate conditions included the use of small samples for calibrating items, and the use of an item pool that was not designed to be…

Bayesian estimation procedures are summarized and numerically illustrated by means of simulation methods. Procedures of data generation for simulation purposes are also delineated and computationally demonstrated. The logistic model basic to the Bayesian estimation procedures is shown to be explicit with respect to the probability distribution…

A brief survey of the mathematics involved in the theory of educational testing. (MM)

The insightful discussions by Raudenbush, Rubin, Stuart and Zanutto (RSZ) and Reckase identify important challenges for interpreting the output of VAM and for its use with test-based accountability. As these authors note, VAM are statistical models for the correlations among scores from students who share common teachers or schools during the…

This paper develops theory and methods for the application of the Bayesian Model II method to the estimation of binomial proportions and demonstrates its application to educational data. (Author/RK)

Three item selection methods for criterion-referenced tests are examined: the classical theory of item difficulty and item-test correlation; the latent trait theory of item characteristic curves; and a decision-theoretic approach for optimal item selection. Item contribution to the standardized expected utility of mastery testing is discussed. (CM)

In three lectures, Benjamin D. Wright of the University of Chicago introduces the Rasch model and its basic concepts. The first lecture, March 30, 1994 discusses the model created by Georg Rasch, a Danish mathematician, which Dr. Wright initially saw as merely a way to make raw scores into measures. Eventually, the model developed into a…