Comparability studies between CBT and PPT studied if there were score differences between these two versions. However, there were no consistent results across studies and time. The purpose of this study is to summarize the testing mode effects for K-12 mathematics tests by conducting a meta-analysis. 103 effect sizes of 32 studies were collected…

The evaluation strategies and results for Title I programs were compared and contrasted. Two systems of evaluation to examine the nature and effectiveness of Elementary Secondary Education Act (ESEA) Title I programs nationwide were implemented. The first involved sponsoring national studies, including the Study of the Sustaining Effects of…

This paper describes (1) the procedures developed to set achievement levels for the National Assessment of Educational Progress (NAEP) that contribute to establishing the validity of the levels and (2) the research studies designed to collect information related to the validity of the achievement levels and the outcomes of the process. The central…

A set of methods is proposed for the analysis of opportunity to learn (OTL) in relation to achievement in large-scale educational assessments. Methods are illustrated with data from the National Assessment of Educational Progress and the National Education Longitudinal Study. Implications for large-scale educational assessment are discussed. (SLD)

Large-scale educational assessments differ from familiar educational measurements by attempting to provide information about the levels and natures of skills in populations rather than in individuals. That the distinct purposes of assessment require different methodologies than individual measurement was recognized by the development of…

Standard setting is a fairly widespread activity in educational and psychological measurement, but there is no formal psychometric theory to guide the development of standard setting methodology. This paper presents a conceptual framework for such a psychometric theory and uses the conceptual framework to analyze a number of methods for setting…

Performance in assessing the unidimensionality of tests was examined for four methods: (1) W. F. Stout's procedure (1987); (2) the approach of P. W. Holland and P. R. Rosenbaum (1986); (3) linear factor analysis; and (4) non-linear factor analysis. Each method was examined and compared with the others using simulated and real test data. Seven data…

This paper discusses different perspectives associated with the use of multiple measures in educational assessment and explores some of the technical considerations for selecting and combining multiple measures. The paper concludes with an example of the use of multiple measures with regard to the recent No Child Left Behind (NCLB) Act of 2002.…

The Achievement-Levels Setting (ALS) process for the National Assessment of Educational Progress (NAEP) resulted in numerical cutscores on the NAEP score scale representing the performance standards for three achievement levels: Basic, Proficient, and Advanced. This paper focuses on an important, but less researched, aspect of the standard setting…

A systematic analysis of student assessment data from Maine and Kentucky, using National Assessment of Educational Progress (NAEP) data and state and local assessment results, was conducted to address issues of measurement and attribution involved in evaluating systemic school reform. This paper (1) examines ways to cope with the challenges of…

James A. Hazlett, NAEP Administrative Director, discusses the National Assessment program as an accountability tool. The strengths and weaknesses of the program as an evaluative method in the cognitive areas are discussed at length. (MPJ) Aspect of National Assessment (NAEP) dealt with in this document: Program Description (Historical Background).

This study introduced a probabilistic model utilizing item response theory (IRT) for dealing with a variety of misconceptions. The model can be used for evaluating the transition behavior of error types, advancement of learning stages, or the stability and persistence of particular misconceptions. Moreover, it apparently can be used for relating…

The process of developing a state education indicators system using the model proposed by the Council of Chief State School Officers (CCSSO) is described. The CCSSO model has three components--indicators of educational outcomes; indicators of school policies and practices; and indicators of contextual factors, or state characteristics. The goal is…

Roles suggested for the National Assessment of Educational Progress (NAEP) in proposed national testing are discussed. Recent proposals center around monitoring, evaluation and accountability, and serving as a benchmark for other tests. Monitoring has been the traditional function of the NAEP, and in this role, its influence has been substantial.…

The role of technology and the measurement of educational progress are discussed, with an emphasis on the development of school report cards. Comments are made from the perspectives of (1) a member of a university community and (2) a researcher in evaluation and measurement. Efforts to formulate school report cards in Florida, California, and…