Two types of classification error are possible in competency tests: erroneous classification of an individual as a "master" of the subject (Type II error), and erroneous classification of a master as a "nonmaster" of the subject (Type I). If steps are taken to minimize Type II errors, an artificially high number of true masters…

The 10 criticisms of criterion-referenced tests addressed in this paper are: the domains tested; pedagogical influence; difficulty of items; cumbersome reports; reliability; arbitrary criteria; local objectives; labeling; predictive validity; and repeated testing. (SJL)

Three methods for setting performance standards for criterion referenced tests--theories, expert judgments, and practical necessity--are reviewed. The author concludes that the use of performance standards is not a promising vehicle for making societal decisions. (Author/JKS)

A detailed analysis of standard setting and criteria for test scores and educational decisions is presented. The author contends that present procedures are in need of re-examination. (JKS)

Mastery tests are analyzed in terms of the number of skills to be mastered and the number of items per skill, in order that correct decisions of mastery or nonmastery will be made to a desired degree of probability. It is assumed that a random sample of skills will be selected for measurement, that each skill will be measured by the same number of…

Computer adaptive testing systems are feasible for certification and licensure testing. This is in part due to the availability of extensive yet inexpensive computers. Modern item response theory, combined with computerized adaptive testing, yields a powerful new method of testing which provides greater accuracy and efficiency and less boredom for…

The feasibility of criterion referenced testing is held to be dependent on the tenability of two postulates: (1) that bias can be controlled in a principled manner from one test to the next; and (2) that one mental process measured by such tests may lawfully interact with another. Without the first postulate, criterion scores could not be…

The concept of performance levels in criterion-referenced assessment is explored by applying the idea to different types of tests commonly used in schools, mastery tests (including diagnostic tests) and achievement tests. In mastery tests, a threshold performance standard must be established for each criterion. Attainment of this threshold…

A basic technical framework is provided for the design and use of mastery tests. The Mastery Testing Project (MTP) prepared this framework using advanced mathematics supplemented with computer simulation based on real test data collected by the South Carolina Statewide Testing Program. The MTP focused on basic technical issues encountered in using…

A procedure is demonstrated for constructing tables showing, for each score on a commercial reading achievement test, the percentage of real-world materials that the testee is likely to comprehend with at least a criterion level of proficiency, the percentages of students in a local or national sample who can competently comprehend a given…

Three areas of concern for those considering competency-based graduation requirements are discussed. First, what is involved in implementing a competency-based graduation requirement? The first tasks are the specification of performance standards and the performance skill areas. Then, appropriate certification procedures can be selected, and their…