Some form of a short interrupted time series (ITS) is often used to evaluate state and national programs. An ITS design with a single treatment group assumes that the pretest functional form can be validly estimated and extrapolated into the postintervention period where it provides a valid counterfactual. This assumption is problematic. Ambiguous…

This paper discusses different perspectives associated with the use of multiple measures in educational assessment and explores some of the technical considerations for selecting and combining multiple measures. The paper concludes with an example of the use of multiple measures with regard to the recent No Child Left Behind (NCLB) Act of 2002.…

Rigorous research provides information that will allow other schools and districts to select interventions that have a scientifically based track record of effectiveness. All Striving Reader grants include the mandate to evaluate literacy intervention(s) targeted to adolescents who are reading significantly below grade level. Although all studies…

The effects of certain characteristics (e.g., sample design) of National Assessment of Educational Progress (NAEP) data on statistical analysis techniques are considered. Ignoring special features of NAEP data and proceeding with a standard analysis can produce inferences that underestimate the true variability and overestimate the true degrees of…

This chapter gives an overview of the design and statistical and psychometric analysis methods of the National Assessment of Educational Progress (NAEP). The evolution of the NAEP continues to provide challenges to statisticians and psychometricians, as demonstrated in the expansion of the NAEP to the assessment of individual states. (SLD)

To fulfill its mandate and its potential the National Assessment of Educational Progress (NAEP) must be a multi-content annual assessment of priority learning areas. This report describes a procedure under which measures of attainment can be charted over time in a broad range of skill domains, on item-invariant item response theory scales, using…

Procedures for obtaining student samples for the National Assessment of Educational Progress (NAEP) and deriving survey weights for analysis of survey data are described. Sample designs are economically and operationally feasible, and weighting procedures result in increased precision of estimates as they account for the probabilities of student…

With the view that social scientists conduct too little exploratory research, the author outlines the methods and attitudes of the exploratory analyst and describes the National Assessment of Educational Progress (NAEP) as a source of research data. Exploratory analysis, in which no paradigm is used, is contrasted with confirmatory research, which…

Several developments growing out of the National Assessment of Educational Progress (NAEP) are reviewed, with a discussion of their extension and application to other projects. These developments include: (1) complex matrix item sampling designs; (2) performance-based items in large-scale assessments; (3) vertical scaling; and (4) innovative…

The material provides a comprehensive look at the design, goals, and methods to be used in the 1998 National Assessment of Educational Progress (NAEP) Civics Assessment. This assessment will attempt to gauge the civic knowledge and skills of the nation's 4th, 8th, and 12th grade students. To do well on the assessment, students will have to show…