A problem for program evaluators involves a search for ways to maximize internal validity and inferential power of research designs while being able to assess long-term effects of social programs. A multimethodological research strategy combining a delayed control group true experiment with a multiple time series and switching replications design…

This article discusses the use of propensity scoring in experimental program evaluations to estimate impacts for subgroups defined by program features and participants' program experiences. The authors discuss estimation issues and provide specification tests. They also discuss the use of an overlooked data collection design--obtaining predictions…

The use of analysis of variance was examined under the assumption that the treatment had been randomly assigned to students, when in fact, the class had been the unit. Data support the idea that if one can randomly assign treatments to intact classes, consideration should certainly be given to doing so. (Author/GK)

The need to attend to both methodological and social concerns is demonstrated in two examples: one where the unchecked effects of the social consequences of a methodologically rigorous research design invalidated the evaluation and a second where explicit trade-offs between methodological rigor and social costs enabled the evaluation to proceed.…

A method is presented for analyzing data collected in a multiple group time-series design. This consists of testing linear hypotheses about the experimental and control group-means. Both a multivariate and a univariate procedure are described. (Author/GK)

Criteria for suitable research designs for use with large databases are suggested and analyzed. The advantages and disadvantages of several types of quasi-experimental designs are compared. Examples are taken from the authors' research with data from the Manitoba Health Services Commission. (Author/AL)

Evaluation research on vocational counseling in substance dependency treatment should distinguish between the effects of counselors and counseling methods on clients' employment outcomes. Three experimental designs permit investigation of possible confounds between these types of effects: (a) nested designs (each counselor delivers one counseling…

The problem of the appropriate comparison group for bereavement studies is discussed. The use of the retrospective pretest design is illustrated through a longitudinal study of bereavement outcome following pregnancy loss, involving 138 bereaved women and a comparison group of 215 currently pregnant women. (SLD)

A solution to the problem of specification error due to excluded variables in statistical models of treatment effects in nonrandomized (nonequivalent) control group designs is presented. It involves longitudinal observation with at least two pretests. A maximum likelihood estimation program such as LISREL may provide reasonable estimates of…

Difficulties experienced in implementing a randomized design in a school-based adolescent health promotion program are explained, and a solution using a randomized control group design nested within two treatment options is proposed. The Health for Life project involved 21 middle schools in Wisconsin. (SLD)

Evidence is presented that strengthens the argument against comparison group research strategies. Results from a study of youth attitudes related to the California Conservation Corps involving 528 participants and 668 comparisons indicate that the comparison group may not adequately represent the nontreatment state. (SLD)