The present article considers a fundamental question in evaluation research: "By how much do program effects vary across sites?" The article first presents a theoretical model of cross-site impact variation and a related estimation model with a random treatment coefficient and fixed site-specific intercepts. This approach eliminates…

A cross-case analysis of four National Science Foundation (NSF) case studies identified both unique details and common themes related to promoting the use and influence of multisite evaluations. The analysis provided evidence of diverse evaluation use by stakeholders and suggested that people taking part in the multisite evaluations perceived…

This sequential mixed methods study analyzed how program evaluation was used to assess educational administration and examined thematic trends in educational evaluation published over 10 years (2001-2010). First, qualitative content analysis examined the articles in eight peer-reviewed evaluation journals. This analysis revealed that numerous…

The goal of this article is to contribute to the validation of a self-evaluation method, which can be used by schools to evaluate the quality of their Competence Assessment Program (CAP). The outcomes of the self-evaluations of two schools are systematically compared: a novice school with little experience in competence-based education and…

This paper reviews empirical research on the use of evaluation from 1986 to 2005 using Cousins and Leithwood's 1986 framework for categorizing empirical studies of evaluation use conducted since that time. The literature review located 41 empirical studies of evaluation use conducted between 1986 and 2005 that met minimum quality standards. The…

Educators have long debated the usefulness (or otherwise) of final examinations; a debate that has typically revolved around the relative merits of closed-book exams, open-book exams, take-home exams or their substitution by some other assessment format (e.g., project work). This paper adds a new dimension to the debate by considering how the…

The Comparing Options in Secondary Mathematics: Investigating Curriculum (COSMIC) project is a longitudinal study of student learning from two types of mathematics curricula: integrated and subject-specific. Previous large-scale research studies such as the National Assessment of Educational Progress (NAEP) indicate that numerous variables are…

Increasing demands for evidence-based early childhood services and the need by policymakers to know whether a program is effective or whether it warrants a significant investment of public and/or private funds--coupled with the often-politicized debate around these topics--make it imperative for policymakers and civic leaders to have independent…

This guide is designed as a resource for leaders and evaluators of K-12 online learning programs. In this guide, the term "online learning" is used to refer to a range of education programs and resources in the K-12 arena, including distance learning courses offered by universities, private providers, or teachers at other schools;…

This article focuses on the use of statistics in policy and evaluation research and the need to present statistical information in a form that is meaningful to mixed audiences. Three guidelines for formulating and presenting meaningful statistics are outlined. Understandability ensures that knowledge of statistical methods is not required for…

The Randomized Control Trials (RCT) design and its quasi-experimental kissing cousin, the Comparison Group Trials (CGT), are golden to some and not even silver to others. At the center of the affection, at the vortex of the discomfort, are beliefs about what it takes to establish causality. These designs are considered primarily when the purpose…

The Office of Management and Budget has recommended the termination of numerous federal programs, citing a lack of program results as the primary reason for this decision. In response to this recommendation, several federal agencies have turned to logic modeling to demonstrate that programs are on the path to results accountability. However,…

Education researchers, practitioners, and policymakers alike are committed to identifying interventions that teach students more effectively. Increased emphasis on evaluation and accountability has increased desire for sound evaluations of these interventions; and at the same time, school-level data have become increasingly available. This article…

From 1932 to 1940, the Progressive Education Association (PEA) conducted its Eight-Year Study. At first, the study appeared to be a poorly funded comparison of two groups of students in secondary schools. During the last four years, as more financial support became available, the Eight-Year Study became a broadly based demonstration of a wide…

There has been increasing concern about the accountability of federal programs in complying with laws and regulations and in achieving desired results. At the request of Congress, the Government Accounting Office examined how the Department of Health and Human Services (HHS) ensures Head Start accountability. Data were collected by means of…