Since mid-1971 the Instructional Objectives Exchange has been engaged in a major effort to develop and disseminate criterion-referenced tests in the fields of reading, mathematics, language arts, and social studies. This paper isolates the chief technical decision-alternatives faced in this project, such as: (1) the optimal number of tests to…

General guidelines exist for reporting and interpreting test scores, but there are short comings in the available technology, especially when applied to criterion-referenced tests. Concerns that have been expressed in the educational measurement literature address the uses of test scores, the manner of reporting scores, limited testing knowledge…

Because criterion referenced measurement is a relatively new approach to testing, systematic guidelines for constructing criterion referenced tests (CRTs) have not yet been developed. A number of questions and issues face the test developer who wishes to provide a test that will be helpful in a particular decision-making situation. There are three…

Consideration is given to the degree of congruence between objectives and test items in the evaluation of objective-based testing. An analytic technique that measures congruence is described, and a bibliography is provided. (CK)

Criticisms of norm referenced standardized tests are contrasted with the assumption and claims of proponents of Rasch model testing and item banking. The Rasch test theory model is based on the assumption of equal interval curriculum referenced scales for both students' abilities and test item difficulties. The Portland school district has…

The application of conventional psychometric procedures to instructional product development is outlined. Selected non-psychometric requirements of developing and delivering effective instruction are described and the consequent implications for psychometric procedures set forth. Present instructional development technology is circumscribed in…

Problems in constructing criterion referenced tests related to inadequate technical criteria, and guidelines for their solution are discussed. The need to select a well-defined domain of behaviors is emphasized as the key to test construction and to understanding students' test performance. Considerations must also be given to the most effective…

Although dealing broadly with issues involving program evaluation, this paper is especially concerned with large-scale evaluations of federally funded education programs that are national in scope and, to some extent at least, compensatory in nature. The paper concentrates on a few basic problem areas that seem inherent in the concept of…

Six essays on technical measurement problems in criterion referenced tests and four essays by psychometricians proposing solutions are presented: (1) "Criterion-Referenced Measurement" and Other Such Terms, by Marvin C. Alkin which is an overview of the first six papers; (2) Selecting Objectives and Generating Test Items for Objectives-Based…

A discussion of performance contracting, defined as an agreement between a group offering instruction and a school needing the services, is presented. Four major hazards to direct measurement of specific learning are considered: poor statement of objectives; selection of the wrong tests; misinterpretation of test scores; and depersonalization of…

Criterion-referenced and norm-referenced tests are compared, and norm-referenced tests are found to have inadequacies as a basis for evaluation or instructional improvement. Disadvantages of norm-referenced tests, which measure a student's performance relative to the performance of other students, are: (1) mismatches between local curriculum and…

The logical and psychological bases for setting cutting scores for criterion-referenced tests are examined; they are found to be intrinsically arbitrary and are often examples of misdirected precision and axiomatization. The term, criterion referenced, originally referred to a technique for making test scores meaningful by controlling the test…

A parallel Spanish edition was developed of released objectives and objective-referenced items used in the National Assessment of Educational Progress (NAEP) in the field of Career and Occupational Development (COD). The Spanish edition was designed to assess the identical skills, attitudes, concepts, and knowledge of Spanish-dominant students…

This article analyzes potential problems in using standardized achievement tests in early childhood education settings. The first problem discussed is whether the outcomes typically measured by standardized achievement tests are the outcomes about which early childhood educators are most concerned. Second, it is argued that a concern for test…

The purpose of these studies was to develop and demonstrate a model for identifying criterion levels of performance that can be rationally defended as being the best level of performance for a particular instructional task. The specific objective was to identify the score on a cloze test that represents the most desirable level of performance on…