Some form of a short interrupted time series (ITS) is often used to evaluate state and national programs. An ITS design with a single treatment group assumes that the pretest functional form can be validly estimated and extrapolated into the postintervention period where it provides a valid counterfactual. This assumption is problematic. Ambiguous…

At present, although the percentages of students with disabilities (SDs) and/or students who are English language learners (ELL) excluded from a NAEP administration are reported, no statistical adjustment is made for these excluded students in the calculation of NAEP results. However, the exclusion rates for both SD and ELL students vary…

Findings on alternative procedures for evaluating measures of achievement in individual data packages at the National Assessment of Educational Progress are presented with their methodological implications. The need for secondary analysts to be aware of the organization of the data, and positive and negative features are discussed. (Author/CM)

Developed an empirical strategy for studying broad-based educational change using quantitative methods and an extensive database consisting of state policy indicators and data on schools and classrooms from states participating in the National Assessment of Educational Progress. Results of a Rasch analysis indicate that patterns of state policy…

Discusses Educational Testing Service (ETS) findings that teachers matter most for student achievement. ETS used 1996 National Assessment of Educational Progress (NAEP) data to examine relationships between student achievement and teacher characteristics. This paper critiques the research, noting serious flaws in the study design, highlighting…

Documenting the design and data analysis procedures behind the 1992 Trial State Assessment in reading, this book also provides insight into the rationale behind the technical decisions made about the program. Chapters in the book are: (1) "Overview: The Design, Implementation, and Analysis of the 1992 Trial State Assessment Program in…

The National Assessment of Educational Progress (NAEP) uses item response theory (IRT) based scaling methods to summarize information in complex data sets. The necessity of global scores or more detailed subscores, creation of developmental scales for different ages, and use of scale anchoring for scale interpretation are discussed. (SLD)

To fulfill its mandate and its potential the National Assessment of Educational Progress (NAEP) must be a multi-content annual assessment of priority learning areas. This report describes a procedure under which measures of attainment can be charted over time in a broad range of skill domains, on item-invariant item response theory scales, using…

With the view that social scientists conduct too little exploratory research, the author outlines the methods and attitudes of the exploratory analyst and describes the National Assessment of Educational Progress (NAEP) as a source of research data. Exploratory analysis, in which no paradigm is used, is contrasted with confirmatory research, which…

The research reported in this paper was conducted to gain information to guide the selection of standard setting procedures for use with polytomous items to set achievement levels on the National Assessment of Educational Progress (NAEP) assessments in U.S. History and geography. Standard-setting procedures were evaluated to determine the relative…

The National Assessment is a plan for a systematic survey of knowledges, skills, understandings, and attitudes. The development of citizenship assessment described in this paper typifies the processes undergone for each subject--art, career and occupational development, citizenship, literature, mathematics, music, reading, science, social studies,…

Changes that have been made in reporting and disseminating information from the National Assessment of Educational Progress are discussed, focusing on those changes brought about by the first Trial State Assessment (TSA). The TSA collected data on the mathematics performance of students in grades 4, 8, and 12 in public and private schools in 40…