This study compared a domain referenced approach with a traditional psychometric approach in the construction of a test. Results of the December, 1975 Quarterly Profile Exam (QPE) administered to 400 examinees at a university were the source of data. The 400 item QPE is a five alternative multiple choice test of information a "safe"…

Some basic statistical concepts relevant to the teacher--mean scores, standard deviation, normal and skewed distributions, z scores, item analysis, standard error of measurement, reliability--and their use by the teacher are explained. (RM)

The reliability of a criterion referenced test was defined as a measure of the degree to which the test discriminates between an individual's level of performance and a predetermined criterion level. The variances of observed and true scores were defined as the squared deviation of the score from the criterion. Based on these definitions and the…

The assumptions of the classical test-theory model are used to develop a theory of reliability for criterion-referenced measures which parallels that for norm-referenced measures. It is shown that the Spearman-Brown formula holds for criterion-referenced measures and that the criterion-referenced reliability coefficient can be used to correct…

This project began with the delineation of the roles and functions of respiratory therapy personnel by the American Association for Respiratory Therapy. In Phase II, The Psychological Corporation used this delineation to develop six proficiency examinations, three at each of two levels. One exam at each level was designated for the purpose of the…

This issue of Measurement in Education is presented in the form of a dialogue between Dr. Robert L. Ebel, Distinguished Professor of Educational Measurement at Michigan State University, and Dr. Samual A. Livingston, Program Research Scientist at the Educational Testing Service. Alternative views on some aspects of the use of tests in assessing…

A large number of seemingly diverse coefficients have been proposed as indices of dependability, or reliability, for domain-referenced and/or mastery tests. In this paper, it is shown that most of these indices are special cases of two generalized indices of agreement: one that is corrected for chance, and one that is not. The special cases of…

A summary is provided of the rationale for questioning the applicability of classical reliability measures to criterion referenced tests; an extension of the classical theory of true and error scores to incorporate a theory of dichotomous decisions; a presentation of the mean split-half coefficient of agreement, a single-administration test index…

Scores of high school students responding to the same biology cognitive preference test using both normative and ipsative procedures were compared. All subtests, whether ipsative or normative, had high alpha Cronbach coefficients. A number of similarities were found in the mean scores obtained by the two procedures, as well as moderate positive…

A diagnostic test in English as a second language should be a series of miniature tests on specific problems. Subscores in each area should be considered rather than a total score. The results should be used to probe mastery in an area rather than provide the means for comparing one student against another. The statistical reliability of the…

Using the basic principles of generalizability theory, a psychometric model for domain-referenced interpretations is proposed, discussed, and illustrated. This approach, assuming an analysis of variance or linear model, is applicable to numerous data collection designs, including the traditional persons-crossed-with-items design, which is treated…

In the late 1970s, the Department of Defense requested that the reference population for the Armed Services Vocational Aptitude Battery (ASVAB) be changed and updated to reflect the current youth population. Analyses of new data collected in 1980 indicated that speeded subtest scores of the new sample were atypically low and that the sample might…

This paper examines the development and evaluation of criterion-referenced measures, and elaborates on the distinction between them and norm-referenced measures. The concept of sensitivity is introduced as an appropriate method for evaluating such measures, and a sensitivity index is proposed. The traditional model for the response of a subject to…

The relationship between the Instructional Process, Instructional Objectives, and Assessment Tasks, identified at the School City of Gary, Indiana, necessitate an effective testing program. Four characteristics perceived crucial to a sound program were: (1) The program should be continuous, (2) The testing program should be comprehensive, (3)…

A central problem for the user of domain-referenced tests in instruction is deciding who has passed and who has failed. Two procedures were presented and discussed. The first, employing classical test theory, was found to be more useful for larger domains and where the passing standard is 70 percent or less. The sampling procedure suggested by…