This paper reviews literature on the rationale, challenges, and recommendations for choosing a nonequivalent comparison (NEC) group design when evaluating intervention effects. After reviewing frequently addressed threats to validity, the paper describes recommendations for strengthening the research design and how the recommendations were…

This "What Works Clearinghouse Procedures and Standards Handbook (Version 3.0)" provides a detailed description of the standards and procedures of the What Works Clearinghouse (WWC). The remaining chapters of this Handbook are organized to take the reader through the basic steps that the WWC uses to develop a review protocol, identify…

The use of educational technology in K-12 classrooms has been gaining tremendous momentum across the country since the 1990s. Many school districts have been investing heavily in various types of technology, such as computers, mobile devices, internet access, and interactive whiteboards. Almost all public schools have access to the internet and…

This paper introduces an empirical strategy to estimate dynamic treatment effects in randomized trials that provide treatment in multiple stages and in which various noncompliance problems arise such as attrition and selective transitions between treatment and control groups. Our approach is applied to the highly influential four year randomized…

The present review examines research on the effects of technology use on reading achievement in K-12 classrooms. Unlike previous reviews, this review applies consistent inclusion standards to focus on studies that met high methodological standards. In addition, methodological and substantive features of the studies are investigated to examine the…

Evaluating school-based mental health services for children and youth with emotional disturbance (ED) has been a challenge for researchers. One particular challenge is the study design of using the student as the statistical unit of analysis, which in certain cases may lead to a violation of the "independence of error" assumption.…

This note suggests a solution to the problem of achieving randomization in experimental settings where units deemed eligible for treatment "trickle in," that is, appear at any time. The solution permits replication of the experiment in order to test for time-dependent effects. (Author/CTM)

Introspective interviewing can often determine the magnitude of relations more meaningfully than statistical analysis. Deduction from empirically validated premises avoids many research design problems. Guesswork can be combined with sensitivity analysis to determine the effects of guesses and missing information on conclusions. (Author/DWH)

Based on an independent analysis of the data used to evaluate the Developmental Economic Education Program, questions are raised about the methodology and conclusions reached by Walstad and Soper in an article published in the Winter 1982 issue of the Journal. The original study is also defended in two replies. (Author/RM)

Trade-offs are implicit in choosing a subjective or objective method for evaluating social programs. The differences between Bayesian and traditional statistics, decision and cost-benefit analysis, and anthropological and traditional case systems illustrate trade-offs in choosing methods because of limited resources. (SLD)

A new report published by the Manhattan Institute for Education Policy, "The Effect of Special Education Vouchers on Public School Achievement: Evidence from Florida's McKay Scholarship Program," attempts to examine the complex issue of how competition introduced through school vouchers affects student outcomes in public schools. The…

Program impact methodology--usually referred to as evaluation research--is described as it is developing in the United States. Several problems face the field of evaluation research. First, those issues grouped as "meta-scientific" include: (1) the distinction between qualitative and quantitative studies; (2) the separation of implementation and…

Educational outcomes are affected by student level, classroom level, and school level characteristics. The fact that educational data are multilevel in nature poses serious analysis questions. Though strong arguments can be made for focusing on a single level of analysis, such studies have several basic limitations: the choice of analytic level…

Third in a series, the monograph summarizes the key evaluation issues, design approaches, and statistical techniques used in conducting the 1980-1983 impact evaluation of Indian Education Act Title IV Part A programs. The monograph describes the major problems in evaluating the program to determine the degree of its positive contribution to…

A proposal to monitor and summarize the experience that will be gained during the course of the ATS-F Education Satellite Communications Demonstration is described. The goals of the demonstration and the context in which it must be evaluated are discussed in an introduction. Subsequent sections deal with specific analytic approaches proposed, the…