Written as four scenes that span a five-year period, this article examines the summative evaluation of an organized school reform programme in the United States from the teacher perspective. The study provides fine-grained details of how the evaluation of the particular reform effort went "awry" in the view of some teachers who directly…

Current economic conditions and the increasing competition for government funding are leading to an increased focus on the impact of research. Measuring the impact of research is difficult because not all impacts are direct and some can be negative or result from the identification of problems that require a non-research response. The time between…

Although the Delphi is widely used, research on certain methodological issues is somewhat limited. After a brief introduction to the strengths, limitations, and methodological challenges of the technique, we share our experiences (as well as problems encountered) with an electronic Delphi of educational program evaluation (EPE) in the Asia-Pacific…

Rapidly declining response rates and the associated threat of nonresponse bias call into question the validity of data obtained through telephone surveys, a tool often used in evaluation. This article explores changes in nonresponse bias over time by examining three data points (1991, 1996, and 2002) from an annual household telephone survey…

A report from the School Choice Demonstration Project examines issues concerning the funding formula used for the Milwaukee Parental Choice Program (MPCP). It finds that the program generates a net saving to taxpayers in Wisconsin but imposes a significant fiscal burden on taxpayers in Milwaukee. However, these findings depend significantly on how…

Treatment integrity (also referred to as "treatment fidelity," "intervention integrity," and "procedural reliability") is an important methodological concerning both research and practice because treatment integrity data are essential to making valid conclusions regarding treatment outcomes. Despite its relationship to validity, treatment…

This special series focused on treatment integrity in the child mental health and education field is timely. The articles do a laudable job of reviewing (a) the current status of treatment integrity research and measurement, (b) existing conceptual models of treatment integrity, and (c) the limitations of prior research. Overall, this thoughtful…

Purpose: The use of e-learning is largely predicated upon the assumption that it can facilitate improvements in student learning and therefore can be more effective than conventional techniques. This assumption has been supported by some in the literature but has been questioned by a continuing body of contrary or indifferent evidence. The purpose…

"How New York City's Charter Schools Affect Achievement" estimates the effects on student achievement of attending a New York City charter school rather than a traditional public school and investigates the characteristics of charter schools associated with the most positive effects on achievement. Because the report relies on an…

With the myriad challenges facing parents in recent years, formal parenting training programs have been developed to address parent needs. However, few studies have been conducted to assess the actual effects of participating in such programs. An attempt to develop a better understanding of how informal parental support networks and formal parent…

Documenting treatment integrity is an important issue in research and practice in any discipline concerned with prevention and intervention. However, consensus concerning the dimensions of treatment integrity and how they should be measured has yet to emerge. Advances from three areas in which significant treatment integrity work has taken…

The concept of treatment integrity cuts across a diversity of fields involved with providing treatments or interventions to individuals. In medical treatments, the concept of "treatment compliance" or "treatment adherence" is an important and problematic issue. In the field of nutrition, the concept of "dietary adherence" is important for…

Six potential problems with the use of randomized experiments to evaluate programs in the field are addressed. Problems include treatment dilution, treatment contamination or confounding, inaccurate case flow and power estimates, violations of the random assignment processes, changes in the environmental context, and changes in the treatment…

A new report published by the Manhattan Institute for Education Policy, "The Effect of Special Education Vouchers on Public School Achievement: Evidence from Florida's McKay Scholarship Program," attempts to examine the complex issue of how competition introduced through school vouchers affects student outcomes in public schools. The…

Program impact methodology--usually referred to as evaluation research--is described as it is developing in the United States. Several problems face the field of evaluation research. First, those issues grouped as "meta-scientific" include: (1) the distinction between qualitative and quantitative studies; (2) the separation of implementation and…