In experimental research, internal validity refers to what extent researchers can conclude that changes in dependent variable (i.e. outcome) are caused by manipulations in independent variable. The causal inference permits researchers to meaningfully interpret research results. This article discusses (a) internal validity threats in social and…

Meta-analyses are syntheses of effect-size estimates obtained from a collection of studies to summarize a particular field or topic (Hedges, 1992; Lipsey & Wilson, 2001). These reviews are used to integrate knowledge that can inform both scientific inquiry and public policy, therefore it is important to ensure that the estimates of the effect…

Randomized controlled trials (RCTs) and regression discontinuity (RD) studies both provide estimates of causal effects. A major difference between the two is that RD only estimates local average treatment effects (LATE) near the cutoff point of the forcing variable. This has been cited as a drawback to RD designs (Cook & Wong, 2008).…

This article discusses the difficulties in conducting intervention research or evaluating intervention programs in a school setting. In particular, the problems associated with randomization and obtaining control groups are examined. The use of quasi-experimental designs, specifically a paired comparison design using the individual as his or her…

Describes problems in implementing placebo controls in psychotherapy including the difficulty of ensuring that therapists and their patients will view both the experimental and placebo treatments as comparably credible. Considers six research issues stemming from the definitional requirement that the placebo control for the critical and specific…

An examination of the scientific basis of both control group designs and comparative designs in outcome research reveals that comparative designs generally have fewer threats to validity and provide a more efficient means of control for nonspecific treatment factors suggesting that comparative studies warrant a larger role in the study of…

This article discusses the use of propensity scoring in experimental program evaluations to estimate impacts for subgroups defined by program features and participants' program experiences. The authors discuss estimation issues and provide specification tests. They also discuss the use of an overlooked data collection design--obtaining predictions…

The Hawthorne and placebo effects in educational experiments were examined over a 20-year period, as reported in the ERIC, Dissertation Abstracts International, and Psych Info data bases. The literature searches identified 325 studies in education employing the relevant control procedures, and each study was coded for the type of placebo…

Many problems confront researchers who are developing appropriate methods to determine the effects of sexual abuse. Particularly serious problems arise when the victims are children who were abused while attending a day care center. Several methods for studying child sexual abuse have been recommended. This paper discusses problems researchers…

The use of analysis of variance was examined under the assumption that the treatment had been randomly assigned to students, when in fact, the class had been the unit. Data support the idea that if one can randomly assign treatments to intact classes, consideration should certainly be given to doing so. (Author/GK)

Is it ethical to select clients at random for a beneficial social service, then deny the benefits to a control group for the sake of science? Participation of control groups in planning, implementation and evaluation of social programs may resolve ethical issues. (Author/CP)

A persistent problem in the evaluation of field-based projects is the lack of no-treatment comparison groups. Frequently, potential comparison groups are confounded by socioeconomic, racial, or other factors. Among the possible methods for dealing with this problem are various matching procedures, but they are cumbersome to use with multiple…

A relationship between experimental and statistical controls is considered in an effort to show how this relationship affects the power of an experiment. Statistical rather than experimental controls are often used to reduce the effect of subjects at the extremes of individual difference. This results in attrition of actual experimental subjects.…

The need to attend to both methodological and social concerns is demonstrated in two examples: one where the unchecked effects of the social consequences of a methodologically rigorous research design invalidated the evaluation and a second where explicit trade-offs between methodological rigor and social costs enabled the evaluation to proceed.…

Criteria for suitable research designs for use with large databases are suggested and analyzed. The advantages and disadvantages of several types of quasi-experimental designs are compared. Examples are taken from the authors' research with data from the Manitoba Health Services Commission. (Author/AL)