As a result of the federal No Child Left Behind Act (NCLB) of 2002, the field of education has seen a heavy emphasis on the use of "scientifically based research" for designing and testing the effectiveness of new and existing educational programs. According to NCLB, when addressing basic cause and effect questions scientifically based…

When evaluating the effectiveness of an educational program, multiple matrix sampling is particularly effective and efficient when the goal of the evaluation is estimating group (as opposed to individual) performance. The technique is described in some detail, with its advantages and disadvantages, and examples of its application are given.…

Describes two types of criterion-referenced testing that districts can use to measure achievement outcomes of their instructional programs: a random sampling assessment plan and an item-response theory assessment plan. (Author/JM)

Described is a method to assist developers of mathematics programs to make formative evaluations. The method involves periodic achievement testing via item sampling. Details of item construction, periodic testing, and profile construction are given in a discussion of how this method was applied to modify a sixth-grade mathematics program. (JP)

Monte Carlo research with TAILOR, a program using implied orders as a basis for tailored testing, is reported. TAILOR typically required about half the available items to estimate, for each simulated examinee, the responses on the remainder. (Author/CTM)

The Michigan Oral Language program was installed in the Santa Paula School District for a 2-month tryout and student performance before and after the program was assessed through multiple matrix sampling. Student performance over seven content areas was determined for both the English and Spanish language components. In all seven content areas,…

The governance options matrix is provided to offer a way for state and university policymakers to examine the functioning environments of specific university-owned public teaching hospitals. With it, they can consider the benefits and problems involved with different options for governance. The issues related to the environmental factors affecting…

Argues that even for purposes of applied decision making, reliance upon criterion validity -- the degree to which measures correlate with specific criteria -- or content coverage is not enough, that the meaning of the measure must also be analyzed in order to evaluate responsibly the possible consequences of the proposed use, it is stated.…

Demands for more complete information on educational programs have emanated from national, state and local sources. Their focus is on the processes that are occurring in individual classrooms. The information that is collected to provide insight into educational programs is customarily summative in nature, answering, for example, questions…

One solution to the problem of obtaining test scores in order to assess multiple outcomes of educational programs is a basically simple technique called matrix sampling. Matrix sampling is the estimation of total test score statistics by administering random subsets of test items to randomly selected students. This chapter lays out the specific…

This handbook for training program evaluators provides background information and practice activities in the following areas: (1) measurement: purposes, ideals, possibilities; (2) defining measurement domains; (3) person and item sampling; (4) test and item selection--with a selected list of standardized tests with pertinent information on each,…

A matrix model for the collection, management, and utilization of data was developed at Eastern Iowa Community College District, (Davenport, Iowa) for evaluating institutional effectiveness. The model was examined in relation to various assessment instruments and the evaluation of functional areas and programs. Surveys of six different target…

Technical considerations associated with item selection and reliability assessment are considered in relation to criterion-referenced tests constructed to provide group information. The purpose is to emphasize test building and the evaluation of test scores in program evaluation studies. It is stressed that an evaluator employ a performance or…

Representing a set of required readings for those who would keep abreast of developments in the field of educational evaluation, this book presents nine chapters, each written by a recognized scholar on the topic being considered. Several chapters are also available as separate booklets, thereby increasing the flexibility with which the materials…

A new Joint Comprehensive Evaluation System for the assessment of 15 different federal programs has been developed by the U.S. Office of Education. In this system these diverse program services will be thought of as resources available to meet the needs of critical target groups. Using this approach, a set of nine crucial questions that need to be…