In this commentary, we examine Braverman's insights into the trade-offs between feasibility and rigor in evaluation measures and reject his assessment of the trade-off as a zero-sum game. We, argue that feasibility and policy salience are, like reliability and validity, intrinsic to the definition of a good measure. To reduce the tension between…

Multiple matrix sampling is a data collection technique that ensures accuracy and efficiency in group performance. It has been widely used in large-scale curriculum evaluation since the 1980s. However, the design does not always fully embrace the dynamics of local evaluation demands. The purpose of this study is to introduce a modified matrix…

Preschool classrooms generate a host of data that can be used to inform program improvement. This article suggests ways to document child-level data and mentoring protocols in a way that makes the information usable to teachers and can positively influence child outcomes.

The current review addresses the validity of published meta-analyses in education that determines the credibility and generalizability of study findings using a total of 56 meta-analyses published in education in the 2000s. Our objectives were to evaluate the current meta-analytic practices in education, identify methodological strengths and…

The second year of effort in the development of a process evaluation system for the programs of the National Center for the Improvement of Educational Systems (NCIES) is reported. The activity involved the full-scale implementation of the system in 438 projects in 12 NCIES programs. Programs included are: Bilingual Education, Early Childhood,…

Higher education experts tout learning outcomes assessment as a vehicle for program improvement. To this end the authors share a rubric designed explicitly to evaluate the quality of assessment and how it leads to program improvement. The rubric contains six general assessment areas, which are further broken down into 14 elements. Embedded within…

A case study approach was used to evaluate the statewide reading improvement project in Louisiana called Special Plan Upgrading Reading (SPUR). SPUR emphasized a change agent approach by providing Technical Assistants and small stipends (approximately $5,000) to participating school districts. During the first two years of the evaluation,…

A manual for trainers of development workers and change agents in the third world provides guidelines for evaluation of development training programs. Chapters define evaluation, present a general model of development, and discuss evaluation in development training programs. Designing training systems for development workers and change agents,…

This commentary highlights the work of the Collaborative for Building After-School Systems (CBASS), a collaborative of mature, city and county-wide nonprofit OST intermediaries, to develop and adopt common youth-, program- and system-level measures that are easy and cost-effective for local systems to implement. By agreeing to adopt and publicly…

A successful data collection system used to gather evaluative information about agricultural technician programs involves five components: an enrollee's report, second year student report, survey of early leavers, employer's report of graduates, and a graduate survey. A schematic diagram of the data collection process is included. (KP)

With a focus on the use of technology when evaluating programs for Aboriginal people, this article explores the possibility of using visual and oral computer technology to enhance the incorporation of Aboriginal worldviews in program evaluation. The author situates Aboriginal worldviews, including methods of communication and transmission of…

Strengths and weaknesses of prevailing evaluation models are analyzed, with attention to the role of feedback in each paradigm. A framework is then presented for analyzing issues faced by evaluators of interactive instructional technologies. The current practice of evaluation relies heavily on 3 models developed over 20 years ago: (1) the…

A collaborative publication between Harvard Family Research Project and The Finance Project, this brief offers guidance in documenting progress and demonstrating results in local out-of-school-time programs. Following introductory remarks providing a rationale for program evaluation, discussing principles of program evaluation, and clarifying key…

Discusses associations between levels of service point use, population information and library inputs, based on data from two London boroughs in respect of individual static service points. Outlines methods of deriving catchment area data using membership information or postcode data from a user survey. Describes a method for librarians seeking to…

Ethical issues relating to naturalistic evaluation are addressed, focusing on the role of the evaluator, problems of privacy and data gathering techniques, and issues relating to the use/abuse of findings. Benefits and costs of the naturalistic approach to program evaluation are also identified. (EGS)