Realist evaluation (RE) is a research design increasingly used in program evaluation, that aims to explore and understand the influence of context and underlying mechanisms on intervention or program outcomes. Several methodological challenges, however, are associated with this approach. This article summarizes RE key principles and examines some…

Treatment effect estimates from a "regression discontinuity design" (RDD) have high internal validity. However, the arguments that support the design apply to a subpopulation that is narrower and usually different from the population of substantive interest in evaluation research. The disconnect between RDD population and the…

This report from the Career and Technical Education (CTE) Research Network Lead team provides final results from an evaluability assessment of CTE programs that feasibly could be evaluated using a rigorous experimental design. Evaluability assessments (also called feasibility studies) are used in education and other fields, such as international…

Card and Giuliano conducted a regression discontinuity study in a large Florida school district to investigate the magnitude of academic benefits of the district's gifted program. They found that for children identified as gifted through an intelligence test, the program provided few or no benefits. But children who were admitted to the gifted…

The effective evaluation of correctional programs is critically important. However, research in corrections rarely allows for the randomization of offenders to conditions of the study. This limitation compromises internal validity, and thus, causal conclusions can rarely be drawn. Increasingly, researchers are employing propensity score matching…

The report of the national response to intervention (RTI) evaluation study, conducted during 2011-2012, was released in November 2015. Anyone who has read the lengthy report can attest to its complexity and the design used in the study. Both these factors can influence the interpretation of the results from this evaluation. In this commentary, we…

This paper reviews literature on the rationale, challenges, and recommendations for choosing a nonequivalent comparison (NEC) group design when evaluating intervention effects. After reviewing frequently addressed threats to validity, the paper describes recommendations for strengthening the research design and how the recommendations were…

HIV/AIDS is one of the most important public health challenges facing Nigeria today. Recent evidence has revealed that the adolescent population make up a large proportion of the 3.7% reported prevalence rate among Nigerians aged 15-49 years. School-based sexual health education has therefore become an important tool towards fighting this problem.…

Background/Context: School finance litigation has often prompted funding reforms, but what happens as a result is the subject of considerable dispute. Purpose: This article explores design problems encountered in studies examining the nature and effects of those reforms. Analysis: After describing the development and current status of school…

This article discusses the use of propensity scoring in experimental program evaluations to estimate impacts for subgroups defined by program features and participants' program experiences. The authors discuss estimation issues and provide specification tests. They also discuss the use of an overlooked data collection design--obtaining predictions…

This paper focuses on transition programs for youth to postsecondary education, broadly considered. It addresses the following questions: (1) What models or programs of transition exist? (2) On what basis can we say one transition program is more effective than another? In other words, how is successful transition defined? (3) How are…

This paper describes the analysis of the National Science Foundation (NSF) Math and Science Partnership (MSP) program's January 2008 Learning Network Conference (LNC). This study focuses on the methodologies used by the MSP community to generate evidence and seeks to understand topics of interest to the 320 LNC participants. The data set for this…

LISREL V computer program is applied to a weak quasi-experimental design involving the Head Start program, as a multiple analysis attempt to assure that differences between nonequivalent control groups do not confound interpretation of a posteriori differences. (PN)

A model is described which illustrates two general uses of multiple analyses to evaluate quasi-experiments: obtaining estimates of the treatment effect, and checking the validity of the experiment. (MH)

Faculty collaboratively developed an on-line Bachelor of Science degree in Consumer Science and Merchandising (CSM). Part-time faculty and technical support services supported the four-member team. Small grants assisted in the creation and redesign of all CSM major courses for on-line delivery. Issues of appropriate learning strategies, student…