Evidence-based health interventions are frequently translated into real-world settings where practical needs drive changes to intervention protocols. Due to logistical and resource constraints, these naturally arising adaptations are rarely assessed for comparative effectiveness using a randomized trial. Nevertheless, when observational data are…

In multisite trials, learning about treatment effect variation across sites is critical for understanding where and for whom a program works. Unadjusted comparisons, however, capture "compositional" differences in the distributions of unit-level features as well as "contextual" differences in site-level features, including…

Purpose: As new technologies such as immersive and augmented platforms emerge, training approaches are also transforming. The virtual reality (VR) platform provides a completely immersive learning experience for simulated training. Despite increased prevalence of these technologies, the extent literature is lagging behind in terms of evaluating…

In multisite trials, learning about treatment effect variation across sites is critical for understanding where and for whom a program works. Unadjusted comparisons, however, capture "compositional" differences in the distributions of unit-level features as well as "contextual" differences in site-level features, including…

Among econometricians, instrumental variable (IV) estimation is a commonly used technique to estimate the causal effect of a particular variable on a specified outcome. However, among applied researchers in the social sciences, IV estimation may not be well understood. Although there are several IV estimation primers from different fields, most…

This paper reviews the use of multiple trials, defined as multiple sites or multiple arms in a single evaluation and replications, in evaluating social programs. After defining key terms, the paper discusses the rationales for conducting multiple trials, which include increasing sample size to increase statistical power; identifying the most…

The "Tutors@UQ" programme provides an example of a formalised, institution-wide, cross-discipline, academic development programme to enhance the quality of teaching that has been maintained for seven years despite a pattern of substantial organisational change. We present a case study of the programme framed around a four-phase model of…

Integrating evaluation into an Extension program offers multiple opportunities to understand program success through achieving program goals and objectives, delivering programming using the most effective techniques, and refining program audiences. It is less common that evaluation is used to guide and validate the effectiveness of program…

The purpose of this paper is to review and critique a groundbreaking research report. The research article under review in this paper is: "A Big Apple for Educators: New York City's Experiment with Schoolwide Performance Bonuses: Final Evaluation Report" (see ED521920). In 2007-08, a program called the Schoolwide Performance Bonus…

A campus-wide program to highlight Catholic identity and mission, Community Day has been celebrated at Notre Dame of Maryland University every year since 1993. Featuring a keynote speaker, followed by an array of thematic workshops led by faculty, staff, and students, the event invites participants to reflect on Catholic Social Teaching, embedded…

We consider the challenges and implications of controlling for school contextual bias when modeling teacher preparation program effects. Because teachers are not randomly distributed across schools, failing to account for contextual factors in achievement models could bias preparation program estimates. Including school fixed effects controls for…

Educators increasingly need to evaluate schoolwide reform efforts; however, complex program evaluations often are not feasible in schools. Through a case example, we provide a heuristic for program evaluation that is easily replicated in schools. Criterion-referenced interpretations of schoolwide screening data were used to evaluate outcomes…

To answer "what works?" questions about policy interventions based on an experimental design, Peck (2003) proposes to use baseline characteristics to symmetrically divide treatment and control group members into subgroups defined by endogenously determined postrandom assignment events. Symmetric prediction of these subgroups in both…

The research-based pedagogical strategy of flipped classes has been shown to be effective for increasing student achievement and retention in postsecondary chemistry classes. The purpose of flipped classes is to move content delivery (e.g., lecture) outside of the classroom, freeing more face-to-face time for active learning strategies. The…

We evaluated a self-instructional training manual for conducting multiple-stimulus without replacement stimulus preference assessment. The manual was compared to a method description, adapted from the method sections of previous published research articles, in an unbalanced crossover design with 18 undergraduate university students, randomly…