A randomized experiment that measures the impact of a social policy in a sample of the population reveals whether the policy will work on average with universal application. An experiment that includes only the subset of the population that volunteers for the intervention generates narrower "proof-of-concept" evidence of whether the…

Funded grant projects all involve some form of evaluation, and Advanced Technological Education (ATE) grants are no exception. Program evaluation serves as a critical component not only for evaluating if a project has met its intended and desired outcomes, but the evaluation process is also a central feature of the grant application itself.…

Tiered evidence initiatives are an important federal strategy to incentivize and accelerate the use of rigorous evidence in planning, implementing, and assessing social service investments. The Social Innovation Fund (SIF), a program of the Corporation for National and Community Service, adopted a public-private partnership approach to tiered…

Federal social programs are rarely evaluated to determine whether they are actually accomplishing their intended purposes. As part of its obligation to spend taxpayers' dollars wisely, Congress should mandate that experimental evaluations of every federal social program be conducted. The evaluations should be large-scale, multisite studies to…

The Race to the Top Assessment program was authorized as part of the American Recovery and Reinvestment Act of 2009 (ARRA). In September 2010, the U.S. Department of Education (Department) awarded competitive, four-year grants to two consortia of states, the Partnership for Assessment of Readiness for College and Careers (PARCC) and the Smarter…

This article examines the power analyses for the first wave of group-randomized trials funded by the Institute of Education Sciences. Specifically, it assesses the precision and technical accuracy of the studies. The authors identified the appropriate experimental design and estimated the minimum detectable standardized effect size (MDES) for each…

This article reports on the results of a study undertaken by the U.S. Department of Education on the impact of the Reading First program, which offer little insight into which parts of the program are worth saving and which need revamping. The study found that the $6 billion spent on the program has helped students with basic decoding but not with…

The purpose of this Guide is to advise researchers, policymakers, and others on when it is possible to conduct a high-quality randomized controlled trial in education at reduced cost. Well-designed randomized controlled trials are recognized as the gold standard for evaluating the effectiveness of an intervention (i.e., program or practice) in…

Four aspects of the relationship between auditing and evaluation in their approaches to program assessment are examined: (1) their different origins; (2) the definitions and purposes of both, and the questions they seek to answer; (3) contrasting viewpoints and emphases of auditors and evaluators; and (4) commonalities of interest and potential…

Shows how evaluation design and federal involvement in Youth Employment Demonstration Projects unintentionally cause a negative appraisal. Indicates the problem stems from interaction of the contract research corporation, the educational research establishment, and the federal bureaucracy, rather than a specific methodology or bureaucratic…

During 1973-75, the Rand Corporation conducted a study of federally funded change agent programs to determine the factors for successful change in the schools at the local, state, and federal levels. The programs included the Elementary and Secondary Education Act (ESEA), Title III, Innovation Projects; ESEA Title VII, Bilingual Projects;…

Looks at past evaluation of compensatory education, suggests new approaches to evaluation, and makes recommendations about program implementation. (IRT)

Using examples from government program evaluation studies, six areas where research synthesis is more effective than individual studies are presented. Research synthesis can: (1) help match treatment type with recipient type; (2) explain which treatment features matter; (3) explain conflicting results; (4) determine critical outcomes; (5) assess…

A series of papers was developed by the authors as part of their deliberations as members of the National Research Council's Committee of Program Evaluation in Education. The papers provide a broad range of present evaluative thinking. The conflict between preferences in evaluation methodology comes through in these papers. The selections include:…

The Rand change agent study was designed to determine the factors for successful implementation and continuation of selected federal programs--the Elementary and Secondary Education Act, Title III, Innovation Projects; ESEA Title VII, Bilingual Projects; Vocational Education Part D, Exemplary Programs; and the Right-to-Read program. This report…