The purpose of this simulation study was to assess the accuracy of a classical test theory (CTT)-based procedure for estimating the alternate-forms reliability of scores on a multistage test (MST) having 3 stages. We generated item difficulty and discrimination parameters for 10 parallel, nonoverlapping forms of the complete 3-stage test and…

Quantitative information about test score reliability can be presented in terms of the distribution of equated scores on an alternate form of the test for test takers with a given score on the form taken. In this paper, we describe a procedure for estimating that distribution, for any specified score on the test form taken, by estimating the joint…

Article commented on a study by Harris, who presented formulas for the variance of errors of estimation (of a true score from an observed score) and the variance of errors of prediction (of an observed score from an observed score on a parallel test). (Author/RK)

Leo Nedelsky developed a method for determining absolute grading standards for multiple choice tests. His method required a group of judges to examine each test question and eliminate those responses which the lowest D- student should be able to reject as incorrect. The correct answer probabilities remaining were used in computing an expected test…

Author replies to article TM 500 559. (MB)

A method is presented for estimating the accuracy and consistency of classifications based on test scores. The reliability of the score is used to estimate effective test length in terms of discrete items. The true-score distribution is estimated by fitting a four-parameter beta model. (SLD)

The procedure of estimating true scores by means of a transformation of the obtained score based on the reliability coefficient is compared with the use of the obtained score without transformation. Using the mean squared error as a criterion, the transformed score is a better estimate for most examinees but poorer for those whose true scores lie…

A distinction is made between reliability of measurement and reliability of classification; the "criterion-referenced reliability coefficient" describes the former. Application of this coefficient to the probability distribution of possible scores for a single student yields a meaningful way to describe the reliability of a single score. (Author)

The traditional reliability coefficient and standard error of measurement are not adequate measures of reliability for tests used to make pass/fail decisions. Answering the important reliability questions requires estimation of the joint distribution of true and observed scores. Lord's "Method 20" estimates this distribution without the…

Much previously published material for estimating the reliability of classification has been based on the assumption that a test consists of a known number of equally weighted items. The test score is the number of those items answered correctly. These methods cannot be used with classifications based on weighted composite scores, especially if…

This article is a reply to a previous paper (see TM 500 488) interpreting Livingston's original article (see TM 500 487). (CK)

The assumptions of the classical test-theory model are used to develop a theory of reliability for criterion-referenced measures which parallels that for norm-referenced measures. It is shown that the Spearman-Brown formula holds for criterion-referenced measures and that the criterion-referenced reliability coefficient can be used to correct…

Procedures are described for studying the reliability of decisions based on specific passing scores with tests made up of discrete items and designed to measure continuous rather than categorical traits. These procedures are based on the estimation of the joint distribution of true scores and observed scores. (CTM)

A reliability coefficient for criterion-referenced tests is developed from the assumptions of classical test theory. The coefficient is based on deviations of scores from the criterion score, rather than from the mean. (Author/CK)

This issue of Measurement in Education is presented in the form of a dialogue between Dr. Robert L. Ebel, Distinguished Professor of Educational Measurement at Michigan State University, and Dr. Samual A. Livingston, Program Research Scientist at the Educational Testing Service. Alternative views on some aspects of the use of tests in assessing…