This research examined 2 elements of multiple-choice test construction, balancing the key and optimal number of options. In Experiment 1 the 3 conditions included a balanced key, overrepresentation of a and b responses, and overrepresentation of c and d responses. The results showed that error-patterns were independent of the key, reflecting…

A series of studies on conceptual and design problems in competency-based measurements are explained. The concept of validity within the context of criterion-referenced measurement is reviewed. The authors believe validation should be viewed as a process rather than an end product. It is the process of marshalling evidence to support…

Achievement tests which are specifically linked to an instructional program and have been developed in relation to an objectives base and/or to an item generation rule are considered, as well as student response data. Three types of studies are outlined and the kind of procedures thought useful illustrated. As various methods for examining…

Multiple matrix sampling of items and examinees was applied to an 18-item rank order instrument administered to a randomly assigned group and compared to the ordering and ranking of all items by control subjects. High correlations between ranks suggest the methodology may viably reduce respondent effort on long rank ordering tasks. (Author/CM)

Both criterion-referenced testing and behavioral assessment share the basic assumption that test behavior is a sample rather than a sign. In addition, both types of assessment focus on response capabilities and performance in specified content domains. Although content validity has been traditionally recognized as essential to criterion-referenced…

This study examined the effects of certain item selection methods on the classification accuracy and classification consistency of criterion-referenced instruments. Three item response data sets, representing varying situations of instructional effectiveness, were simulated. Five methods of item selection were then applied to each data set for the…

Formulas are presented for decision reliability and for classification validity for mastery/nonmastery decisions based on criterion referenced tests. Two item parameters are used: the probability of a master answering an item correctly, and the probability of a nonmaster answering an item incorrectly. The theory explores the relationships of…

Author discusses and defines criterion tests in the context of classroom needs that have created much of the interest in the theory at this time. The primary source of interest is related to the growing implementation of individualized curricula. (Author/CB)

As an alternative to a classical test theory basis for criterion-referenced test construction, it is proposed that a strict item-sampling model be used. The computer's role in such a model is outlined. The assumptions of the model are carefully defined and its properties reviewed. The relationship between mastery criteria and such sampling plans…

Technical considerations associated with item selection and reliability assessment are considered in relation to criterion-referenced tests constructed to provide group information. The purpose is to emphasize test building and the evaluation of test scores in program evaluation studies. It is stressed that an evaluator employ a performance or…

An approach was developed based on item-response models defined at the level of salient subject groups rather than at the level of individuals, designed for use with multiple-matrix sampling designs. In each of three National Assessment of Educational Progress (NAEP) mathematics subtopics, Reiser's group-effects latent trait model was fitted to…

This study attempted to determine the effectiveness of cloze procedures as norm-referenced instruments by comparing the differential responses of four groups of college students of English as a second language on two identical cloze passages. The responses were scored using both exact-answer and acceptable-word methods. The results indicate that…

Mastery tests are analyzed in terms of the number of skills to be mastered and the number of items per skill, in order that correct decisions of mastery or nonmastery will be made to a desired degree of probability. It is assumed that a random sample of skills will be selected for measurement, that each skill will be measured by the same number of…

This manual outlines the rationale for using the Criterion Referenced Test (CRT) approach and suggests specific guidelines for test developers to use in constructing test items. Methods for assessing the adequacy of a CRT are also provided. (Author/RC)

The existence of criterion-referenced (CR) measurement is questioned in this paper. Despite beliefs that differences exist between two alternative forms of measurement, CR and Norm Referenced (NR), an analysis of philosophical and psychological descriptions of measurement, as well as a growing number of empirical studies, reveal that the common…