As evidence from evaluation and experimental studies continue to influence decision and policymaking, applied researchers and practitioners require tools to derive valid and credible inferences. Over the past several decades, research in causal inference has progressed with the development and application of propensity scores. Since their…

Background: The generalizability of a study's results continues to be at the forefront of concerns in evaluation research in education (Tipton & Olsen, 2018). Over the past decade, statisticians have developed methods, mainly based on propensity scores, to improve generalizations in the absence of random sampling (Stuart et al., 2011; Tipton,…

This article reviews important statistical methods for estimating the impact of interventions on outcomes in education settings, particularly programs that are implemented in field, rather than laboratory, settings. We begin by describing the causal inference challenge for evaluating program effects. Then four research designs are discussed that…

Two approaches to causal inference in the presence of non-random assignment are presented: The Propensity Score approach which pseudo-randomizes by balancing groups on observed propensity to be in treatment, and the Endogenous Treatment Effects approach which utilizes systems of equations to explicitly model selection into treatment. The three…

Purpose: This paper aims to introduce matching in propensity score analysis (PSA) as an alternative statistical approach for researchers looking to make causal inferences using intact groups. Design/methodology/approach: An illustrative example demonstrated the varying results of analysis of variance, analysis of covariance and PSA on a heuristic…

Recent methods to improve generalizations from nonrandom samples typically invoke assumptions such as the strong ignorability of sample selection, which is challenging to meet in practice. Although researchers acknowledge the difficulty in meeting this assumption, point estimates are still provided and used without considering alternative…

Fully updated to reflect the most recent changes in the field, the Second Edition of "Propensity Score Analysis" provides an accessible, systematic review of the origins, history, and statistical foundations of propensity score analysis, illustrating how it can be used for solving evaluation and causal-inference problems. With a strong…

When randomized experiments are infeasible, quasi-experimental designs can be exploited to evaluate causal treatment effects. The strongest quasi-experimental designs for causal inference are regression discontinuity designs, instrumental variable designs, matching and propensity score designs, and comparative interrupted time series designs. This…

Making causal inferences from a quasi-experiment is difficult. Sensitivity analysis approaches to address hidden selection bias thus have gained popularity. This study serves as an introduction to a simple but practical form of sensitivity analysis using Monte Carlo simulation procedures. We examine estimated treatment effects for a school-based…

The purpose of this study was to investigate the impact of inference making, decoding, memory, and vocabulary on reading comprehension among 7th- through 12th-grade struggling readers with varying levels of inattention and hyperactivity. We categorized a group of 414 struggling readers into 3 groups based on results from factor mixture modeling:…

Recent research suggests that the use of student search services is an effective part of a college's student marketing and recruitment strategy. What is not clear, however, is whether participating in a search service is an effective part of a student's college search strategy. To address this question, we exploit a recent change in the choice…

Most questions in social and biomedical sciences are causal in nature: what would happen to individuals, or to groups, if part of their environment were changed? In this groundbreaking text, two world-renowned experts present statistical methods for studying such questions. This book starts with the notion of potential outcomes, each corresponding…

Northwest Evaluation Association™ (NWEA™) is committed to providing partners with useful tools to help make inferences from the Measures of Academic Progress® (MAP®) interim assessment scores. Recently, NWEA completed a concordance study to connect the scales of the Pennsylvania System of School Assessment (PSSA) reading and math with those of the…

Northwest Evaluation Association™ (NWEA™) is committed to providing partners with useful tools to help make inferences from the Measures of Academic Progress® (MAP®) interim assessment scores. Recently, NWEA completed a concordance study to connect the scales of the New York State Testing Program (NYSTP) reading and math with those of the MAP…

Northwest Evaluation Association™ (NWEA™) is committed to providing partners with useful tools to help make inferences from the Measures of Academic Progress® (MAP®) interim assessment scores. One important tool is the concordance table between MAP and state summative assessments. Concordance tables have been used for decades to relate scores on…