The administrative and political ramifications of a form of multiple matrix sampling where all students in the population are tested on a subtest is discussed in the context of a statewide testing program. The conclusion is drawn that large-scale testing programs will have to adopt a testing framework which samples items from the total item domain…

When performing large scale evaluations (e.g., on a state-wide or national level) it may not be possible to administer all items in the item universe to all respondents in the subject population. One method which has been proposed to sample both items and respondents is multiple matrix sampling (MMS) in which a sample of the items is administered…

Several techniques exist to automatically put together a test meeting a number of specifications. In an item bank, the items are stored with their characteristics. A test is constructed by selecting a set of items that fulfills the specifications set by the test assembler. Test assembly problems are often formulated in terms of a model consisting…

When evaluating the effectiveness of an educational program, multiple matrix sampling is particularly effective and efficient when the goal of the evaluation is estimating group (as opposed to individual) performance. The technique is described in some detail, with its advantages and disadvantages, and examples of its application are given.…

Pennsylvania's Educational Quality Assessment Program provides each participating school with a building level report in which state percentiles are a prominent part. Multiple matrix sampling was being considered as a technique to reduce testing time. However, there was great concern that the error associated with estimating the school mean might…

This document presents more useful statewide testing as a rationale for a revision to the California Assessment Program (CAP). A brief description of the revision procedures, and an illustrative outline of the skills and concepts covered by CAP are included. Approximately 270 districts and counties participated in the test development process to…

The National Education Association's Task Force on Testing has stated its opinion that standardized tests are overused. The task force suggests that the application of sampling techniques and a variety of alternatives to current testing practices would accomplish the same purposes. Representatives of the testing industry have indicated that the…

This document contains the codebooks and data record layouts for the 1985-86 Public-Use Data Tapes, Version 1.0. The Public-Use data tapes are created by the National Assessment of Educational Progress (NAEP) to provide outside researchers access to NAEP data. This book provides coded data collected on and from third graders and 9-year-olds via…

The National Assessment of Educational Progress (NAEP) has completed two assessments of mathematics, the first conducted in 1972-73 and the second during 1977-78. Each assessment surveyed the mathematics achievement of American 9-, 13-, and 17-year-olds, using a deeply stratified, multi-stage probability sample design. This report documents…

This guide begins with a series of questions and answers that introduce Pennsylvania's Educational Quality Assessment (EQA) Inventory as a 188- to 190-item multiple-choice test for grades 5, 8, and 11. Items are selected from a 400-item bank using matrix sampling procedures. Test results are analyzed at the school level; no individual student…

In 1982, the Educational Testing Service (ETS) proposed to implement a new, complex design for the National Assessment of Educational Progress (NAEP). The major features of this design are described in "A New Design for a New Era" (Messick, Beaton, and Lord, 1983). The purpose of this document is to describe the actual implementation of…

This guide begins with a series of questions and answers which introduce Pennsylvania's Educational Quality Assessment (EQA) Inventory as a 188-to 190-item multiple choice test for fifth, eighth, and eleventh grades. Items are selected from a 400-item bank using matrix sampling procedures. Test results are analyzed at the school level; no…

An achievement testing framework is being developed by the Los Angeles Unified School District to assess the educational progress of 14,000 secondary level compensatory education students with something other than standardized tests. The technique of multiple matrix sampling was applied to the use of large item domains in the subject area of…

For the past six years the National Assessment of Educational Progress has sponsored a national Conference on Large-Scale Assessment, designed to promote and improve communications among educational assessment personnel in State Departments of Education and other agencies. This volume contains most of the papers that were accepted for presentation…