This study is an attempt to answer the following research question: can the reliability of a criterion-referenced test be accurately determined according to a multiple classification of the student's performance? Specifically, the study pursues the beta-multinomial model, which postulates the probability distribution of an examinee's degree of…

A classical problem in mastery testing is the choice of passing score and test length so that the mastery decisions are optimal. This problem has been addressed several times from a variety of viewpoints. In this paper, the usual indifference zone approach is adopted, with a new criterion for optimizing the passing score. Specifically,…

Ways in which the Statistical Package for the Social Sciences (SPSS) can be used to perform some Rasch analyses are described in detail. It is shown how SPSS and a set of item calibrations can be used to estimate person abilities, standard errors of measurement, test characteristic curve, test information curve, classification consistency on a…

A computer simulation of Bayes' Theorem was conducted in order to determine the probability that an examinee was a master conditional upon his test score. The inputs were: number of mastery states assumed, test length, prior expectation of masters in the examinee population, and conditional probability of a master getting a randomly selected test…

Summarizing work which is part of an Army research program on Methodological Issues in the Construction of Criterion Referenced Tests, the focus of this paper is on a Bayesian model, which gives the probability of correctly classifying an examiner as a master or as a nonmaster while taking into consideration the test length and the mastery cut-off…

The use of Bayesian decision theory to solve problems in test-based decision making is discussed. Four basic decision problems are distinguished: (1) selection; (2) mastery; (3) placement; and (4) classification, the situation where each treatment has its own criterion. Each type of decision can be identified as a specific configuration of one or…

The problems associated with misclassifying students when pass-fail decisions are based on test scores are discussed. One protection against misclassification is to set a confidence interval around the cutting score. Those whose scores fall above the interval are passed; those whose scores fall below the interval are failed; and those whose scores…

The sequential probability ratio test (SPRT), developed by Abraham Wald, is one statistical model available for making mastery decisions during computer-based criterion referenced tests. The predictive validity of the SPRT was empirically investigated with two different and relatively large item pools with heterogeneous item parameters. Graduate…