A randomized experiment that measures the impact of a social policy in a sample of the population reveals whether the policy will work on average with universal application. An experiment that includes only the subset of the population that volunteers for the intervention generates narrower "proof-of-concept" evidence of whether the…

The Benefit Offset National Demonstration (BOND) tested a $1 reduction in benefits per $2 earnings increase above the level at which Social Security Disability Insurance benefits drop from full to zero under current law. BOND included a rare and large "population-representative" experiment: It applied the rule to a nationwide, random…

Funded grant projects all involve some form of evaluation, and Advanced Technological Education (ATE) grants are no exception. Program evaluation serves as a critical component not only for evaluating if a project has met its intended and desired outcomes, but the evaluation process is also a central feature of the grant application itself.…

Tiered evidence initiatives are an important federal strategy to incentivize and accelerate the use of rigorous evidence in planning, implementing, and assessing social service investments. The Social Innovation Fund (SIF), a program of the Corporation for National and Community Service, adopted a public-private partnership approach to tiered…

Heterogeneity between and within people necessitates the need for sequential personalized interventions to optimize individual outcomes. Personalized or adaptive interventions (AIs) are relevant for diseases and maladaptive behavioral trajectories when one intervention is not curative and success of a subsequent intervention may depend on…

Federal social programs are rarely evaluated to determine whether they are actually accomplishing their intended purposes. As part of its obligation to spend taxpayers' dollars wisely, Congress should mandate that experimental evaluations of every federal social program be conducted. The evaluations should be large-scale, multisite studies to…

Researchers have become increasingly interested in programs' main and interaction effects of two variables (A and B, e.g., two treatment variables or one treatment variable and one moderator) on outcomes. A challenge for estimating main and interaction effects is to eliminate selection bias across A-by-B groups. I introduce Rubin's causal model to…

Use of multivariate analysis (e.g., multivariate analysis of variance) is common when normally distributed outcomes are collected in intervention research. However, when mixed responses--a set of normal and binary outcomes--are collected, standard multivariate analyses are no longer suitable. While mixed responses are often obtained in…

The present study utilized a short-term longitudinal research design to model the relationship between shyness-anxiousness and receptive language skills. Hypotheses regarding the direction of the causal relationship, mediation, and moderation were evaluated. Subjects included 340 Head Start attendees from primarily English- and Spanish-speaking…

This article examines the power analyses for the first wave of group-randomized trials funded by the Institute of Education Sciences. Specifically, it assesses the precision and technical accuracy of the studies. The authors identified the appropriate experimental design and estimated the minimum detectable standardized effect size (MDES) for each…

Purpose: By examining data from interviews with students in the Upward Bound (UB) Program (a federally sponsored program that provides academic support to students at risk preparing for college entrance), this study seeks to strengthen understanding of the stages of trust in educational organizations that serve urban youth and in so doing provoke…

This article reports on the results of a study undertaken by the U.S. Department of Education on the impact of the Reading First program, which offer little insight into which parts of the program are worth saving and which need revamping. The study found that the $6 billion spent on the program has helped students with basic decoding but not with…

The National Center for Supported eText (NCSeT) has a research agenda designed to investigate the effects of supported electronic text on the ability of students with disabilities to read and learn from text. This article introduces four studies conducted by research teams who have collaborated with NCSeT to implement that agenda. It describes…

A major tenet of both the Individuals with Disabilities Education Act and the No Child Left Behind Act is the identification and use of evidence-based practices, or those instructional techniques shown by research as most likely to improve student outcomes meaningfully. However, much confusion exists regarding the meaning and potential…

Using evidence-based practices, or those instructional techniques shown by research to improve student outcomes meaningfully, increases the performance of students with disabilities and should therefore be a priority for special educators. But how does a practice come to be considered evidence based? The unique characteristics of group…