Evidence-based health interventions are frequently translated into real-world settings where practical needs drive changes to intervention protocols. Due to logistical and resource constraints, these naturally arising adaptations are rarely assessed for comparative effectiveness using a randomized trial. Nevertheless, when observational data are…

Evaluators are often called upon to assess the effects of programs. To assess a program effect, evaluators need a clear understanding of how a program effect is defined. Arguably, the most widely used definition of a program effect is the counterfactual one. According to the counterfactual definition, a program effect is the difference between…

A randomized experiment that measures the impact of a social policy in a sample of the population reveals whether the policy will work on average with universal application. An experiment that includes only the subset of the population that volunteers for the intervention generates narrower "proof-of-concept" evidence of whether the…

Policymakers and institutional leaders in higher education too often make decisions based on descriptive data analyses or even anecdote when better analysis options could produce more nuanced and more valuable results. Employing the setting of higher education program evaluation at a midwestern regional public university, for this study we…

In multisite trials, learning about treatment effect variation across sites is critical for understanding where and for whom a program works. Unadjusted comparisons, however, capture "compositional" differences in the distributions of unit-level features as well as "contextual" differences in site-level features, including…

New Hampshire's Performance Assessment of Competency Education (PACE) innovative assessment system uses student scores from classroom performance assessments as well as other classroom tests for school accountability purposes. One concern is that not having annual state testing may incentivize schools and teachers away from teaching the breadth of…

Purpose: As new technologies such as immersive and augmented platforms emerge, training approaches are also transforming. The virtual reality (VR) platform provides a completely immersive learning experience for simulated training. Despite increased prevalence of these technologies, the extent literature is lagging behind in terms of evaluating…

In multisite trials, learning about treatment effect variation across sites is critical for understanding where and for whom a program works. Unadjusted comparisons, however, capture "compositional" differences in the distributions of unit-level features as well as "contextual" differences in site-level features, including…

The desire to support student learning and professional development, in combination with accreditation requirements, necessitates the need to evaluate the learning environment of educational programs. The Health Education Learning Environment Survey (HELES) is a recently-developed global measure of the learning environment for health professions…

Objective: After the discontinuation of School Health Policies and Practices Study (SHPPS) in 2016, the US Centers for Disease Control and Prevention (CDC) began exploring innovative ways to gather school health information using existing surveillance systems. School Health Profiles (Profiles) is a school-based system of surveys that monitors…

The Upward Bound Math and Science (UBMS) program at Temple University (TU) seeks to guide first-generation, low-income high school students from Philadelphia in their quest to obtain postsecondary STEM degrees that lead to related careers. This study aims to evaluate the efficacy of the program by analyzing its students' performance in light of…

School discipline disproportionality has long been documented in educational research, primarily impacting Black/African American and non-White Hispanic/Latinx students. In response, federal policymakers have encouraged educators to change their disciplinary practice, emphasizing that more proactive support is critical to promoting students'…

School discipline disproportionality has long been documented in educational research, primarily impacting Black/African American and non-White Hispanic/Latinx students. In response, federal policymakers have encouraged educators to change their disciplinary practice, emphasizing that more proactive support is critical to promoting students'…

This technical supplement to the Supplemental Nutrition Assistance Program (SNAP) Employment and Training (E&T) evaluation final reports presents details of the technical approach used for creating analysis variables and estimating impacts of treatment group services on outcomes, including employment, earnings, and SNAP participation. It also…