Recent methods to improve generalizations from nonrandom samples typically invoke assumptions such as the strong ignorability of sample selection, which is challenging to meet in practice. Although researchers acknowledge the difficulty in meeting this assumption, point estimates are still provided and used without considering alternative…

Randomized experiments are commonly used to evaluate if particular interventions improve student achievement. While these experiments can establish that a treatment actually "causes" changes, typically the participants are not randomly selected from a well-defined population and therefore the results do not readily generalize. Three…

As education systems around the world move towards increased accountability based on performance measures, it is important to investigate the unintended effects of accountability systems. This article seeks to explore the extent to which head teachers in a large Norwegian municipality may resort to gaming the incentive system to boost their…

Data have long been considered a key factor in organizational decision-making (Simon, 1955; Lindblom & Cohen, 1979). Data offer perspective, guidance, and insights that inform policy and practice (Newell & Simon, 1972; Kennedy, 1984). Recently, education policymakers have invested in the use of data for organizational improvement in states…

This report summarizes the complex research literature on quantitative methods for assessing how impacts of educational interventions on instructional practices and student learning differ across students, educators, and schools. It also provides technical guidance about the use and interpretation of these methods. The research topics addressed…

The "PISA 2009 Technical Report" describes the methodology underlying the PISA 2009 survey. It examines additional features related to the implementation of the project at a level of detail that allows researchers to understand and replicate its analyses. The reader will find a wealth of information on the test and sample design,…

In large-scale assessments, such as state-wide testing programs, national sample-based assessments, and international comparative studies, there are many steps involved in the measurement and reporting of student achievement. There are always sources of inaccuracies in each of the steps. It is of interest to identify the source and magnitude of…

Comprehensive Achievement Monitoring (CAM), a new evaluation method, systematically and comprehensively measures student achievement. The unique characteristics of this model are described in detail, as are those of two more familiar models, classroom testing, and curriculum program evaluation. The amount and quality of information available from…

Describes the evolution of national educational assessment in the United States, its present stage of development, methodological and reporting problems, and approaches applicable to assessment data. Examines how proposed approaches can be applied in the California Assessment Program and in the National Assessment of Educational Progress. (MJL)

The Voluntary Public School Choice (VPSC) Program supports the emergence and growth of choice initiatives across the country, by assisting states and local school districts in developing innovative strategies to expand public school choice options for students. This report contains the final assessment of the first five years of the VPSC Program…

Demands for more complete information on educational programs have emanated from national, state and local sources. Their focus is on the processes that are occurring in individual classrooms. The information that is collected to provide insight into educational programs is customarily summative in nature, answering, for example, questions…

In the Spring, 1996, issue of "CRESST Line," E. Baker and R. Linn commented that, in efforts to measure the progress of schools, "the fluctuations due to differences in the students themselves could conceal differences in instructional effects." This is particularly true in the context of the evaluation of adequate yearly…

As part of the overall study of the development of behavioral objectives for preschool children, the three parts of this report discusses matching procedures, survey instruments, and sampling guidelines. Part C, Matching Objectives and Recommendations for Development of Measures, contains discussions of the following: A Matching of Existing…

This evaluation handbook was written for school district personnel who may be either novices or experts in the use of educational research techniques. Tailored to categorically aided projects, the handbook describes techniques that will isolate specific activity effects as reflected by pupil achievement. Chapters are devoted to the construction of…