This article examines what types of language tests are suitable in evaluating instructional programs in testing for educational accountability. Norm-referenced measurements are found unsuitable for this purpose; new, criterion-referenced tests must be developed. (CHK)

Two ESEA Title I evaluation models developed by the Resource Management Corporation (RMC), were field tested simultaneously with 560 Title I reading students, grades 2-8. Measuring instruments for models 1 and 2 were, respectively, the California Achievement Test (reading vocabulary section), a norm-referenced test; and the Tarmac Reading…

Technical considerations associated with item selection and reliability assessment are considered in relation to criterion-referenced tests constructed to provide group information. The purpose is to emphasize test building and the evaluation of test scores in program evaluation studies. It is stressed that an evaluator employ a performance or…

The question of how standardized tests can be better developed to improve educational program evaluation is probed in this paper. After the first section's brief introduction, section 2 explores the thesis that tests developed in terms of selection and inference may not serve current social functions of educational testing. To clarify this thesis,…

A procedure was developed to combine subsets of curriculum-relevant items from a norm-referenced achievement test (NRT) with items comprising a locally-developed supplementary test in order to produce objective-referenced test (ORT) data for program evaluation. The study was designed to determine whether the ORTs were valid instruments for program…

The validity of the Comprehensive Tests of Basic Skills (CTBS) and problems associated with inappropriate norms were investigated in eleven senior high schools in Phoenix, Arizona. CTBS validity was assessed in three ways: (1) correlation between test scores and number of courses taken in that subject; (2) correlation between CTBS scores and…

Threats to the validity of the Title I Evaluation and Reporting System are covered in two parts: an annotated bibliography of reports concerned with technical issues, and a discussion of threats to validity--from the reporting system in general and from each model in particular. Threats common to all three evaluation models are reported:…

The effectiveness of an Education Consolidation Improvement Act Chapter 1 program was studied in a large urban school district. A common problem in evaluating remedial programs is that pretest/posttest achievement gains often indicate the effects of the entire curriculum, rather than those specific to the remedial class. In this district, over…

A norm-referenced achievement test, in combination with supplementary items, can be used to produce norm-referenced data as well as objective-referenced data. The experiences of the Palm Beach County (Florida) school district in developing and using such a test are described. (EGS)

Traditionally, the use of control groups, regression discontinuity/projection, or norm referenced models has been advocated for educational program evaluation. The small numbers of and the heterogeneous characteristics of severely and profoundly handicapped make these evaluation methods inappropriate for this population subgroup. Furthermore, no…

A classical model for achievement test development was influenced by: the educational tradition of selecting a sample of questions appropriate to a subject and counting the proportion of acceptable responses; the emphasis on obtaining a large and homogeneous sample of questions to insure reliability; and the tradition in psychological testing of…

This paper highlights some special characteristics of criterion-referenced tests while comparing them with norm-referenced tests. Psychometric considerations involved in constructing a criterion-referenced test, including item analysis, reliability, and validity, are discussed, along with application of criterion-referenced testing to individual…

Focusing on the use of the Metropolitan Achievement Test (MAT) with Project Follow Through evaluations, this report represents an extended discussion of the appropriateness of standardized tests for evaluation and policy research. First, the development of the MAT and its roots in the Army Alpha intelligence test are described in detail. The next…

The objectives of this booklet are to provide suggestions, procedures, and rules of thumb to aid the evaluator in using achievement tests in program evaluation. It also discusses some of the theories underlying the procedures for developing and selecting achievement tests and interpreting their results. Each of the six chapters offers advice on…

In this discussion of the use of achievement tests in evaluating Title I programs, matching content between test and curriculum is the main theme. The point is made that unless a test measures what is taught, it cannot be sensitive to whatever gains the instruction produces. Thus, if different instructional treatments have different objectives, it…