Assessments scored with diagnostic models are increasingly popular because they provide fine-grained information about student achievement. Because of differences in how diagnostic assessments are scored and how results are used, the information teachers must know to interpret and use results may differ from concepts traditionally included in…

The process of implementing intensive reading interventions using data-based decision-making (DBDM) becomes increasingly challenging as students move into the secondary grades and reading tasks correspondingly become more complex. This article provides teachers with guidelines to support effective implementation of DBDM for students with or at…

In mastery testing, the raw agreement index and the kappa index may be estimated via one test administration when the test scores follow beta-binomial distributions. This paper reports formulae, tables, and a computer program which facilitate the computation of the standard errors of the estimates. (Author/CTM)

A coherent system of decision making is described that may be incorporated into an instructional sequence to provide a supplement to the experience-based judgment of the classroom teacher. The elements of this decision process incorporate prior information such as a teacher's past experience, experimental results such as a test score, and…

The use of setting standards for criterion-referenced tests is defended in a response to two papers by Gene Glass and Nancy Burton. (JKS)

Decision rules for assigning students to treatments based upon aptitudes or criterion scores are discussed. Popular procedures are criticized and a Bayesian approach is recommended. The effect of unreliability of aptitude or criterion scores is also discussed. (JKS)

The problem of establishing appropriate passing scores is one of evaluation rather than estimation and not amenable to exact solution. It must therefore be approached by (1) identifying criteria for judging the acceptability of the passing score, (2) collecting the data appropriate to assessing each relevant criterion, and (3) judging how well the…

The utility of setting standards for educational decisions, even though those standards may be somewhat arbitrary, is defended in this response to Glass's article (TM 504 031). (JKS)

The use of cut-off scores with criterion referenced tests is defended in this response to two papers by Gene Glass and Nancy Burton. Suggestions for setting cut-off scores are made. (JKS)

The traditional reliability coefficient and standard error of measurement are not adequate measures of reliability for tests used to make pass/fail decisions. Answering the important reliability questions requires estimation of the joint distribution of true and observed scores. Lord's "Method 20" estimates this distribution without the…

An approach to simultaneous optimization of assignments of subjects to treatments followed by an end-of-mastery test is presented using the framework of Bayesian decision theory. Focus is on demonstrating how rules for the simultaneous optimization of sequences of decisions can be found. The main advantages of the simultaneous approach, compared…

Three methods for setting performance standards for criterion referenced tests--theories, expert judgments, and practical necessity--are reviewed. The author concludes that the use of performance standards is not a promising vehicle for making societal decisions. (Author/JKS)

Procedures based on latent trait models and the Rasch model are described for computation and asymptotic statistical inference for two decision consistency indexes often used in mastery or competence testing. Simulations were conducted for typical mastery testing situations to illustrate the procedures. (SLD)

Procedures are described for studying the reliability of decisions based on specific passing scores with tests made up of discrete items and designed to measure continuous rather than categorical traits. These procedures are based on the estimation of the joint distribution of true scores and observed scores. (CTM)

Political and educational consequences of standard setting for criterion-referenced tests are discussed. Suggestions for setting standards are given. (JKS)