In the case of dichotomous decisions, the total set of all assumptions/specifications for which the decision would have been the same is the robustness region. Inspection of this (data-dependent) region is a form of sensitivity analysis which may lead to improved decision making. (Author/BW)

Huynh (1976) suggested a method of approximating the reliability coefficient of a mastery test. The present study examines the accuracy of Huynh's approximation and also describes a computationally simpler approximation which appears to be generally more accurate than the former. (Author/RL)

A number of different definitions and indices of reliability for mastery tests have recently been proposed in an attempt to cope with possible lack of score variability that attenuates traditional coefficients. One promising index that has been suggested is the proportion of students in a group that are consistently assigned to the same mastery…

A new procedure is proposed for developing, evaluating, and implementing routing procedures for use with individualized instruction programs. An item response theory model for describing the relationship between performance on sequentially arranged units of instruction, and procedures for using the model to evaluate sequential relationships and…

Using a linear loss function, a procedure is described for computing a cutting score that minimizes the risk for a given decision rule. The procedure is demonstrated with a criterion-referenced achievement test of elementary statistics administered to 167 students. (Author/CTM)

The Libby-Novick class of three-parameter generalized beta distributions is shown to provide a rich class of prior distributions for the binomial model that removes some restrictions of the standard beta class. A numerical example indicates the desirability of using these wider classes of densities for binomial models. (Author/BW)

Single administration (beta-binomial) estimates for the raw agreement index p and the corrected-for-chance kappa index in mastery testing are compared with those based on two test administrations in terms of estimation bias and sampling variability. Bias is about 2.5 percent for p and 10 percent for kappa. (Author/RL)

Macready and Dayton introduced two probabilistic models for mastery assessment based on an idealistic all-or-none conception of mastery. Alternatively, an application of latent trait theory to mastery testing is proposed (a three parameter logistic model) as a more plausible model for test theory. (Author/CTM)

False-positive and false-negative dicisions are the fundamental errors committed with a mastery test; yet the estimation of the likelihood of committing these errors has not been investigated. Accordingly, two methods of estimating the likelihood of committing these errors are described and then investigated using Monte Carlo techniques.…

Three statistical problems related to criterion-referenced testing are investigated: estimation of the likelihood of a false-positive or false-negative decision with a mastery test, estimation of true scores in the Compound Binomial Error Model, and comparison of the examinees to a control. Two methods for estimating the likelihood of…

A large number of seemingly diverse coefficients have been proposed as indices of dependability, or reliability, for domain-referenced and/or mastery tests. In this paper, it is shown that most of these indices are special cases of two generalized indices of agreement: one that is corrected for chance, and one that is not. The special cases of…

A Markov model for predicting performance on criterion-referenced tests is presented,. The model is expressed mathematically as a function of transition matrix, a current state vector, and a future state vector. The matrix is defined in terms of conditional probabilities, i.e., the probability of making a transition to a specific future…

The Mastery-Learning test model is extended. Methods for estimating prior probabilities are described. The use of an adjustment matrix to transform a probability of mastery measure and empirical methods for estimating adjustment matrix parameters are derived. Adjustment matrices are interpreted as indicators of instructional effectiveness and as…

The use of Bayesian decision theory to solve problems in test-based decision making is discussed. Four basic decision problems are distinguished: (1) selection; (2) mastery; (3) placement; and (4) classification, the situation where each treatment has its own criterion. Each type of decision can be identified as a specific configuration of one or…

Four different procedures were used for estimating the proportion of persons who would be classified consistently as either passing both of two parallel tests or failing both. These four methods were applied at each of four different mastery level scores for each of three different length tests. Data were based on 50 replications of each procedure…