Research synthesis of evaluation findings is a multistep process. An investigator identifies a research question, acquires the relevant literature, codes findings from that literature, and analyzes the coded data to estimate the average treatment effect and its distribution in a population of interest. The process of estimating the average…

Complex survey data are collected by means other than simple random samples. This creates two analytical issues: nonindependence and unequal selection probability. Failing to address these issues results in underestimated standard errors and biased parameter estimates. Using data from the nationally representative Head Start Family and Child…

The determinants of sentencing are of much interest in criminal justice and legal research. Understanding the determinants of sentencing decisions is important for ensuring transparent, consistent, and justifiable sentencing practice that adheres to the goals of sentencing, such as the punishment, rehabilitation, deterrence, and incapacitation of…

Many Medicaid programs have either fully or partially carved out mental health services. The evaluation of carve-out plans requires a case-mix model that accounts for differing health status across Medicaid managed care plans. This article develops a diagnosis-based case-mix adjustment system specific to Medicaid behavioral health care. Several…

In multilevel regression, centering the model variables produces effects that are different and sometimes unexpected compared with those in traditional regression analysis. In this article, the main contributions in terms of meaning, assumptions, and effects underlying a multilevel centering solution are reviewed, emphasizing advantages and…

The problem of drawing causal inferences from retrospective case-controlled studies is considered. A model for causal inference in prospective studies is applied to retrospective studies. Limitations of case-controlled studies are formulated concerning relevant parameters that can be estimated in such studies. A coffee-drinking/myocardial…

A meta-program theory is proposed to overcome the limitations and improve the use of program theory as an approach to faith-based program evaluation. The essentials for understanding religious organizations, their various programs, and faith and spirituality are discussed to support a rationale for developing a faith-based program theory that…

The problem associated with evaluating an econometric model using values outside those used in the model estimation is illustrated in the evaluations of a residential load management program during each of two successive years. Analysis reveals that attention must be paid to this problem. (Author/TJH)

A technique to estimate variances of weighted estimates in program evaluation is discussed. BRR estimates of variance are constructed from orthogonally weighted subsamples and can provide better estimates from complex samples than can other methods. Use of BRR is illustrated in a government evaluation of a food service program. (SLD)

Five causal metaphors running throughout youth employment and education programs are reviewed: (1) the functionalist economic model; (2) the institutional socialization model; (3) a psychosocial view of youth development; (4) the subcultural model; and (5) a self-determined community development vision. (Author/GK)

The role of program evaluation is examined in relation to policy analysis. Program evaluation is most persuasive as a component of the policy process when its role is clearly understood. Clarity can be achieved by using the program mandate to guide evaluation and by including practitioners in evaluation model developments. (SLD)

The authors contribute to the development of empirical methods for measuring the impacts of place-based local development strategies by introducing the adjusted interrupted time-series (AITS) approach. It estimates a more precise counterfactual scenario, thus offering a stronger basis for drawing causal inferences about impacts. The authors…

A group of defenses of qualitative evaluation methods is examined, based on a hard relativistic interpretation of the work of Thomas Kuhn. A promising defense of qualitative evaluation may be found in a soft-relativist interpretation of Kuhn's analysis of the nature of scientific discovery. (Author/LMO)

The authors demonstrate how exponential smoothing can play a role in the identification of the intervention component of an interrupted time-series design model that is analogous to the role that the sample autocorrelation and partial autocorrelation functions serve in the identification of the noise portion of such a model. (Author/BW)

After a brief discussion of the terms implementation and monitoring, this article illustrates a strategy for monitoring program implementation. The strategy, called model-guided method, offers a way to develop conceptual and empirical specifications for testing the adequacy of a program implementation. It also provides feedback to program…