Many mastery decisions are made based on assessment devices that are known to be less than perfectly reliable. A significant portion of examinees who have scores near the cutting score risk misclassification, even for instruments with relatively high retest reliability. Suggestions for minimizing this risk are offered. (Author/MT)

Two indices for assessing the efficiency of decisions in mastery testing are proposed. The indices are generalizations of the raw agreement index and the kappa index. Empirical examples of these indices are given. (Author/JKS)

Techniques for classifying individuals into mastery or non-mastery status based on criterion-referenced test performance are investigated. Extensions of mastery classification techniques developed by Macready and Dayton are presented. The techniques are based on latent class and quasi-independence models. (Author/JKS)

The dependence of reliability indices for mastery tests on mean and cutoff scores was examined in the case of three decision-theoretic indices. Dependence of kappa on mean and cutoff scores was opposite to that of the proportion of correct decisions, which was linearly related to average threshold loss. (Author/BW)

The classical estimate of a binomial probability function is to estimate its mean in the usual manner and to substitute the results in the appropriate expression. Two alternative estimation procedures are described and examined. Emphasis is given to the single administration estimate of the mastery test reliability. (Author/CTM)

A procedure is described for determining the minimal length of a mastery test given certain constraints. The procedure assumes that the testor is indifferent to misclassifying some testees who score within a specified range about the passing score. An example and table are provided. (JKS)

Suggests a process for setting a cutoff score on knowledge tests for promotion, certification, and licensing: a modified Angoff method, in which a competency estimate is determined by subject matter experts, is combined with analysis of potential impact on any groups protected by the Civil Rights Act. (SK)

A model for the setting of mastery cut scores is presented. The model, based on the beta-binomial test distribution, allows for hand calculation of cut scores. The model provides a simple way to explore the consequences of selecting a particular cut score. (Author/JKS)

Descriptions are presented of two related probabilistic models that can be used for making classification decisions with respect to mastery of specific concepts or skills. Determination of cutoff scores, minimal number of items necessary, and the assessment of the quality of tests provided by the model are discussed. (Author/JKS)

In mastery testing, the raw agreement index and the kappa index may be estimated via one test administration when the test scores follow beta-binomial distributions. This paper reports formulae, tables, and a computer program which facilitate the computation of the standard errors of the estimates. (Author/CTM)

A coherent system of decision making is described that may be incorporated into an instructional sequence to provide a supplement to the experience-based judgment of the classroom teacher. The elements of this decision process incorporate prior information such as a teacher's past experience, experimental results such as a test score, and…

Four recent indices emphasizing the interrelationships of score distribution shape, modality, mean, and variance were investigated to determine the reliability of mastery tests. Attention was focused on the values of the indices when the cutoff score was near to or far from the modes of distribution. Five types of score distributions were…

This study examined the efficacy of teacher judgment in the process of setting mastery scores (cut scores) for fourth-grade mathematics in local school districts in Nebraska in terms of agreement between teacher classification of students and classification by the cut score obtained by this classification. The study also examined cut scores in…

A procedure for estimating the rates of false positive and false negative classification in a mastery testing situation is described. Formulas and tables are described for the computations of the standard errors. (Author/JKS)

Wilcox (1977) examines two methods of estimating the probability of a false-positive on false-negative decision with a mastery test. Both procedures make assumptions about the form of the true score distribution which might not give good results in all situations. In this paper, upper and lower bounds on the two possible error types are described…