This guide will help researchers design and implement impact studies in education so that the findings are more generalizable to the study's target population. Guidance is provided on key steps that researchers can take, including defining the target population, selecting a sample of schools--and replacement schools, when needed--managing school…

This article presents a content analysis of the 2013 School Leadership Program (SLP) grants. SLP projects provide a unique opportunity for participants in the field to explore innovative leadership preparation and development and their impact on program participants, schools, school districts, and students. The article begins with an overview of…

This research gathered data from a sample of university students with disabilities (and a control group without disabilities) from two university campuses (one urban and the other suburban). The project examined the current work and school experiences, plans upon graduation, and understanding of major legislation designed to protect against…

Legitimate knowledge claims about causation have been a central concern among evaluators and applied researchers for several decades and often have been the subject of heated debates. In recent years these debates have resurfaced with a renewed intensity, due in part to the priority currently being given to randomized experiments by many funders…

This study addresses the central question "How do evaluators' background characteristics relate to their evaluation design choices?" Evaluators were provided with a fictitious description of a school-based program and asked to design an evaluation of that program. Relevant background characteristics such as level of experience,…

How do evaluation practitioners make choices when they evaluate a program? What function do evaluation theories play in practice? In this article, I report on an exploratory study that examined evaluation practices in France. The research began with observations of practitioners' activities, with a particular focus on the phases of evaluation…

A debate is emerging concerning the use of deception in social science research (especially when it employs experimental methods), driven primarily by the relatively recent move by many economists into experimental work. These economists generally argue that deception should be banned. Deception includes a variety of practices in social science…

Throughout history human subject research has been plagued by researchers whose studies were riddled with danger and/or risk to their subjects. Indeed, one may cite numerous situations where human subjects were injured and/or killed in experiments that were designed with no thought or anticipation of risk to the subjects. As a result, the federal…

In this paper, the authors describe the benefits and pitfalls of mixed methods designs. They argue that mixed methods designs may be preferred when evaluating professional development programs for p-K-12 education given the new call for accountability in making data-driven decisions. They summarize and critique the studies in terms of limitations…

Rigorous research provides information that will allow ECE (Early Childhood Educator) programs to select interventions that have a scientifically based track record of effectiveness in increasing teachers' skills and teaching quality. This paper shares implementation and impact results as well as the lessons learned in conducting an evaluation of…

The purpose of this Guide is to advise researchers, policymakers, and others on when it is possible to conduct a high-quality randomized controlled trial in education at reduced cost. Well-designed randomized controlled trials are recognized as the gold standard for evaluating the effectiveness of an intervention (i.e., program or practice) in…

Two methods for dividing an interrupted time-series study between baseline and experimental phases when study resources are limited are compared. In fixed designs, the baseline duration is predetermined. In flexible designs the baseline duration is contingent on remaining resources and the match of results to prior expectations of the evaluator.…

Focuses on ethical issues that can arise in the collection and analysis of data in evaluations. Addresses four issues: (1) the application of cost-benefit thinking to judgments about research ethics, (2) the quality of research design, (3) the need to explore the data, and (4) the censoring of data. (Author/SLD)

Explores the role of theory in strengthening causal interpretations in nonexperimental research. Evaluators must conduct theory-driven research, concentrating on "small theory," in that the focus is on the explanation of processes specific to the program being evaluated. Theory-guided treatment research must be programmatic and…

Evaluation as a field has moved beyond debating whether certain designs are good while others are bad to begin to appreciate that each of a group of designs may have its uses. The importance of good designs that reflect real-world complexities is increasingly being recognized. (SLD)