This study explores the possibility that any drug prevention program might be considered "evidence-based" given the use of data analysis procedures that optimize the chance of producing statistically significant results by reanalyzing data from a Drug Abuse Resistance Education (DARE) program evaluation. The analysis produced a number of…

Findings from an evaluation of a model system for delivering mental health services to youth were reassessed to determine the robustness of key findings to the use of methodologies unavailable to the original analysts. These analyses address a key concern about earlier findings--that the quasi-experimental design involved the comparison of two…

Advantages of using logistic and hazards regression techniques in assessing the overall impact of a treatment program and the differential impact on client subgroups are examined and compared using data from a juvenile court program for status offenders. Implications are drawn for management and effectiveness of intensive supervision programs.…

A broad set of ethical principles is examined to determine their relevance for evaluation. The five principles of K. S. Kitchener (autonomy, nonmaleficience, beneficence, justice, and fidelity) are relevant to evaluation and are fully represented in the evaluation standards of the Joint Committee on Standards for Educational Evaluation. (SLD)

In this article, the authors discuss program evaluation of intervention studies when the outcome of interest is collected routinely at equally spaced intervals of time. They illustrate concepts using data from the American Stop Smoking Intervention Study, where the outcome is state per capita tobacco consumption. States differ widely in mean…

Effectiveness of a new 72-hour delivery system (USXPRESS) for pharmaceuticals purchased by the Department of Veterans Affairs (VA) from VA depots was evaluated by comparing 33 test sites with 11 matched sites using a pretest posttest quasiexperimental design. The USXPRESS system reduced inventory, decreased space needs, and satisfied service…

A Monte Carlo study compared performance of linear and logistic mixed-model analyses of simulated community trials having specific event rates, intraclass correlations, and degrees of freedom. Results indicate that in studies with adequate denominator degrees of freedom, the researcher may use either method of analysis, with certain cautions. (SLD)

This article documents evaluations of the New Primary Grades Reading System (NRS). NRS was implemented in a variety of settings. Evaluations track implementation from early stages of pilot testing through large-scale adoption of NRS. Accumulation of evidence regarding success of innovation across settings allows greater confidence in conclusions…

The Teaching Family Model (TFM) of delinquency treatment has evolved into a national network of community-based behaviorally oriented group homes. A longitudinal summative evaluation of TFM homes and comparison groups from the same or neighboring communities was mounted, and the results of a five year cost-effectiveness study are presented.…

The efficacy of an Assertive Community Treatment approach in preventing jail recidivism for the mentally ill was compared with that of forensic specialist case management or referral to mental health centers. More experimental subjects (total final sample of 94) returned to jail within a year. (SLD)

To explore the treatment fidelity of employee safety-belt programs, 614 organizations were divided into 2 groups: those that had (n=183), and those that had not (n=431) conducted an evaluation. Multiple regression analysis of associated data indicated that evaluation-active organizations were generally more active in other program areas. (TJH)

Survey items representing 30 standards of the Joint Committee on Standards for Educational Evaluation were used to examine violations in program evaluations. Twenty-nine novice graduate students, 57 graduate students with moderate knowledge about evaluation, and 61 expert evaluators viewed propriety and accuracy standards as suffering the most…

Recommendations for feasible and rigorous evaluation design formulated by a 1991 conference on the evaluation of comprehensive community interventions are summarized and illustrated through the example of a program for drug risk reduction. Implications of a multisite design with matched pairs randomized into intervention and comparison conditions…

This article demonstrates a novel application of propensity score matching techniques: to estimate nonexperimental impacts on program participants within the context of an experimental research design. The author examines the relationship between program participation, defined as qualifying for an earnings supplement by working full-time, and…

Use of performance gaps analysis in program evaluation is examined by analyzing and critiquing the gap reduction design as an experimental design and an analytic tool in program outcome evaluation. The example of the Bilingual Education Evaluation System illustrates the contention that disadvantages of the method outweigh its contributions. (SLD)