Educational evaluation in too many states and school districts, if done at all across an institution, has been reduced to crunching test scores in only the subject areas and grade levels tested. While measuring student achievement in reliable and valid ways is an important part of the mission of education, it is not enough. What's also needed is a…

E-learning researchers face considerable challenges in creating meaningful and generalizable studies due to the complex nature of this dynamic training medium. Our experience in conducting workplace e-learning research led us to create this guide for planning research on e-learning. We share the unanticipated complications we encountered in our…

In this article we give a systematic review of the nature and design of earlier research into the impact of instructional development in higher education. Studies are clustered on the basis of the level of outcome that was measured, meaning that another synthesis technique is used than in prior reviews related to the same topic. In addition, we…

This guide is designed to help districts and schools evaluate reform efforts related to the American Recovery and Reinvestment Act (ARRA), including School Improvement Grants. It includes practical, how-to tips for conducting program evaluations, whether one takes a "do-it-yourself" approach or seeks outside assistance to evaluate reform…

Evaluations of practice have often adopted experimental or quasi-experimental approaches and have focused heavily on outcomes of innovative programmes. This creates problems in replicating studies and effectively creates obstacles to the sharing of good practice. In this paper, ideas from Pawson and Tilley's (1997) approach to Realistic Evaluation…

A design for evaluation studies of educational programs should provide valid and defensible inferences. Goals of evaluation are the identity of major components of inferences and specific validity concerns. Design problems may be resolved by creatively utilizing features of specific evaluations in designing unique conditions that permit valid…

Important findings are often a balance between the rigor of the experimental design and innovativeness of the experimental question. One broad topic area that has received a great deal of discussion, but little empirical study, is the evaluation of educational systems. Experimental designs that permit the analysis of practices used by state…

This article presents evidence from a national evaluation of the effects of 21st Century Community Learning Center after-school programs. The study was conducted in 12 school districts and 26 after-school centers, at which 2,308 elementary school students who were interested in attending a center were randomly assigned either to the treatment or…

This investigation examines two contrasting evaluations of early childhood education: the Westinghouse-Ohio University study and the Ypsilanti Perry Preschool Project. The Westinghouse study concluded that Head Start experience had very little, if any, academic value for participating children, whereas the Perry Preschool Project concluded that…

A series of papers was developed by the authors as part of their deliberations as members of the National Research Council's Committee of Program Evaluation in Education. The papers provide a broad range of present evaluative thinking. The conflict between preferences in evaluation methodology comes through in these papers. The selections include:…

This paper discusses how an experimental design can be applied to a large-scale evaluation. The purpose of the study described is to assess the success of the National Science Foundation (NSF) in achieving its goal for five comprehensive projects. This paper is divided into four main sections: (1) design; (2) sampling procedures; (3) data…

Presents various options for schools to consider when shopping for education reform approaches. The Social Research and Demonstration Corporation was established to identify and test social programs and innovative social policy ideas that can improve the wellbeing of disadvantaged Canadians. Examples are presented of random assignment evaluations…

A consumer-oriented program evaluation of a master's level clinical psychology training program is reported. A goal attainment model of evaluation was employed, requiring specification of program goals, development of goal-related objectives and hypotheses, and formulation of recommendations for program modification based on results achieved.…

The concept of quality is analyzed with special attention to its different senses in two contexts: qualitative evaluation and assessment of educational program quality. The view that evaluation of program quality cannot be validly expressed in quantitative terms and that qualitative evaluation methodologies have special relevance for the…

Guidelines are presented for project evaluation techniques with emphasis on the management of a project or an educational program. Part One--general considerations--discusses the four basic concepts of the decision-making evaluation model: objectives, evaluative criteria, analysis/synthesis, and evaluation. Stages in the evaluation of a system are…