Propensity score matching and weighting methods are applied to balance covariates and reduce selection bias in the analysis of observational study data, and ultimately estimate a treatment effect. We wish to evaluate the impact of a Supplemental Instruction (SI) program on student success in an Introductory Statistics course. In such student…

This article reviews important statistical methods for estimating the impact of interventions on outcomes in education settings, particularly programs that are implemented in field, rather than laboratory, settings. We begin by describing the causal inference challenge for evaluating program effects. Then four research designs are discussed that…

At least sixteen US states have taken steps toward holding teacher preparation programs (TPPs) accountable for teacher value-added to student test scores. Yet it is unclear whether teacher quality differences between TPPs are large enough to make an accountability system worthwhile. Several statistical practices can make differences between TPPs…

This brief provides a summary of the validity of Istation ISIP Early Reading scores in predicting students' performance levels on the Idaho Standards Achievement Test (ISAT) in English language arts (ELA). This correlational study analyzed how well third grade students' winter performance on the ISIP Early Reading test predicted their spring…

Researchers addressing posttreatment complications in randomized trials often turn to principal stratification to define relevant assumptions and quantities of interest. One approach for the subsequent estimation of causal effects in this framework is to use methods based on the "principal score," the conditional probability of belonging…

This case study examines the research process used to assess the intermediate effects of a community-based college preparation program in a fast growth, high needs exurban school district in Texas. Because it was not possible to randomly assign participants to treatment or control conditions, a nonexperimental design using propensity score…

The effective evaluation of correctional programs is critically important. However, research in corrections rarely allows for the randomization of offenders to conditions of the study. This limitation compromises internal validity, and thus, causal conclusions can rarely be drawn. Increasingly, researchers are employing propensity score matching…

The National Math + Science Initiative's (NMSI's) College Readiness Program (CRP) is an established program whose goal is to promote science, technology, engineering, and mathematics education in high schools to improve students' readiness for college. The program provides teacher, student, and school supports to promote high school students'…

The National Math + Science Initiative (NMSI) is a nonprofit organization committed to improving educational outcomes that traces its roots back to the early 1990s. NMSI's College Readiness Program (CRP) is a long-standing program with the goal of promoting STEM education in high schools to improve students' preparation for college. The three-year…

Online instructional videos are becoming increasingly common within education. This study adopts a quasi-experimental 2 × 2 crossover design (control and experimental groups) to evaluate the efficacy of instructional videos to teach practical rehabilitation skills. The students performed practical sessions in class and were formatively assessed by…

The cost of attending college has risen steadily over the past 30 years, making financial aid an important determinant of college choice for many students and a subject of concern for colleges and state governments. In this paper, we estimate the effect of rule-based merit aid assignment on students' enrollment decisions at the University of Iowa.…

This study aims to study households' expectations for their children's academic performance in Khyber Pakhtunkhwa, Pakistan. Education has a significant role in increasing the productivity and income level of an individual in a society. Household education, income, distance from school, gender discrimination within household and cost of education…

Recently, the authors have been exploring the use of propensity score methods for developing evidence of program impact. Specifically, they have been developing evidence (after one year of implementation) of the effects of the Math Science Partnership in New York City ("MSPinNYC2") on high school students' achievement--both in terms of…

This presentation focuses on the quasi-experimental methods used to select comparison schools for an evaluation of a federal investing in innovation (i3) validation grant. The Northeast Tennessee College and Career Ready Consortium (NETCO) consists of 29 high schools participating in a five-year program to expand students' access to rigorous…

The main goal of this study was to illustrate and provide some direction for dealing with the complexities of propensity score matching within different multilevel contexts. Special attention is given to how procedures typically applied in a non-hierarchical setting may be modified to properly reduce the expected bias in the estimated treatment…