To address the computational constraints of parameter estimation in the polytomous Cognitive Diagnosis Model (pCDM) in large-scale high data volume situations, this study proposes two two-stage polytomous attribute estimation methods: P_max and P_linear. The effects of the two-stage methods were studied via a Monte Carlo simulation study, and the…

In this paper, we report on developments in the Mastery Learning (ML) curriculum and assessment model that has been successfully implemented in a metropolitan university for teaching first-year mathematics. Initial responses to ML were positive; however, we ask whether the nature of the ML tests encourages a focus on shallow learning of…

Computerized testing has created new challenges for the production and administration of test forms. This paper describes a multi-stage, testlet-based framework for test design, assembly, and administration called computer-adaptive sequential testing (CAST). CAST is a structured testing approach that is amenable to both adaptive and mastery…

This study is an attempt to answer the following research question: can the reliability of a criterion-referenced test be accurately determined according to a multiple classification of the student's performance? Specifically, the study pursues the beta-multinomial model, which postulates the probability distribution of an examinee's degree of…

Current methods of obtaining reliability coefficients for mastery tests are laborious from a practitioner's perspective. Some methods require two test administrations; while others require access to computer facilities and/or advanced measurement and statistical procedures. This report provides tables from which practitioners can read such…

A number of different definitions and indices of reliability for mastery tests have recently been proposed in an attempt to cope with possible lack of score variability that attenuates traditional coefficients. One promising index that has been suggested is the proportion of students in a group that are consistently assigned to the same mastery…

Four recent indices emphasizing the interrelationships of score distribution shape, modality, mean, and variance were investigated to determine the reliability of mastery tests. Attention was focused on the values of the indices when the cutoff score was near to or far from the modes of distribution. Five types of score distributions were…

Two approaches to criterion-referenced measurement are described and contrasted--domain-referenced testing and mastery testing. This paper is organized according to ten issues or stages in test construction: (1) content domain specification; (2) item construction; (3) item domain specification; (4) item analysis; (5) item selection; (6) parallel…

This study examined the efficacy of teacher judgment in the process of setting mastery scores (cut scores) for fourth-grade mathematics in local school districts in Nebraska in terms of agreement between teacher classification of students and classification by the cut score obtained by this classification. The study also examined cut scores in…

A comparison was made between scores and grades of college students taking a development and learning course, using either a modified mastery grading system (MMGS) or a modified norm-referenced grading system (MNRGS). Under the MMGS, students could take each unit exam up to three times to meet minimum or higher criteria levels. Under the MNRGS,…

Comparisons were made among various methods of estimating the reliability of pass-fail decisions based on mastery tests. The reliability indices that are considered are p, the proportion of agreements between two estimates, and kappa, the proportion of agreements corrected for chance. Estimates of these two indices were made on the basis of…

A general model along with four illustrations is presented for the consideration of budgetary constraints in the setting of passing scores in instructional programs involving remedial action for poor test performers. Budgetary constraints normally put an upper limit on any choice of passing score. Given relevant information, this limit may be…

The purposes of this study were to: (1) extend the sequential probability ratio testing (SPRT) procedure to polytomous item response theory (IRT) models in computerized classification testing (CCT); (2) compare polytomous items with dichotomous items using the SPRT procedure for their accuracy and efficiency; (3) study a direct approach in…

Two types of classification error are possible in competency tests: erroneous classification of an individual as a "master" of the subject (Type II error), and erroneous classification of a master as a "nonmaster" of the subject (Type I). If steps are taken to minimize Type II errors, an artificially high number of true masters…

A solution is offered to problems associated with the inequality in the manipulability of probabilities of classification errors of masters versus nonmasters, based on competency test results. Eschewing the typical arbitrary establishment of observed-score standards below 100 percent, the solution incorporates a self-correction of wrong answers.…