The purpose of this paper is to introduce the Five-Level Qualitative Data Analysis (5LQDA) method for ATLAS.ti as a way to intentionally design methodological approaches applicable to the field of evaluation. To demonstrate my analytical process using ATLAS.ti, I use examples from an existing evaluation of a STEM Peer Learning Assistant program.…

Realist evaluation (RE) is a research design increasingly used in program evaluation, that aims to explore and understand the influence of context and underlying mechanisms on intervention or program outcomes. Several methodological challenges, however, are associated with this approach. This article summarizes RE key principles and examines some…

The primary objective of this article is to evaluate the design variables of a newly developed teaching intervention, "The Amazing Tax Race". It comprises a race against time in which accounting students participate within teams in multiple tax-related activities so that they are exposed to pervasive skills. The findings provide…

The Success By 6® (SB6) initiative is designed to support early care and education centers in improving and sustaining quality in Pennsylvania's Keystone STARS Quality Rating and Improvement System (QRIS). The SB6 evaluation report examines implementation and outcomes. The findings have implications for SB6 continous quality improvement process…

This qualitative study identified key factors associated with exemplar service-learning programs at six community colleges across the United States. The purpose of this case study was to determine why these programs have sustained over time and to better understand how administrators and faculty meet shifting demands for service-learning. The…

This response to Slavin's article (2008) explores the issues of transparency, representation, and warranting of claims in Slavin's descriptions of the work of others and his suggestions for program evaluation syntheses. Through contrastive analyses between Slavin's representations of the program evaluation synthesis efforts of five organizations…

Written by a social worker for social work students, this is a nuts and bolts guide to statistics that presents complex calculations and concepts in clear, easy-to-understand language. It includes numerous examples, data sets, and issues that students will encounter in social work practice. The first section introduces basic concepts and terms to…

The Greater Avenues for Independence (GAIN) welfare-to-work program in California was evaluated by the Manpower Demonstration Research Corporation (MDRC). The evaluation relied on involvement of stakeholders, examination of program implementation, flexibility, the adaptation of evaluation questions and designs during the study, use of random…

A review of qualitative methods used in a predominantly quantitative evaluation indicates a variety of roles for such a mixing of methods, including framing and revising research questions, assessing the validity of measures and adaptations to program implementation, and gauging the degree of uncertainty and generalizability of conclusions.…

Ten evaluation models are examined in terms of their applicability to investigative, emergent design programs: Stake's portrayal, Wolf's adversary, Patton's utilization, Guba's investigative journalism, Scriven's goal-free, Scriven's modus operandi, Eisner's connoisseurial, Stufflebeam's CIPP, Tyler's objective based, and Levin's cost…

Some specific opportunities and techniques are described for combining and integrating qualitative and quantitative methods from the design stage of a substance abuse program evaluation through implementation and reporting. The multiple problems and requirements of such an evaluation make integrated methods essential. (SLD)

Considers the workings and effects of The Innovation and New Technology Programme in Norway, describing the design of the evaluation and the use of a centrally established control group to assess programs results. Uses results from innovation theory to appraise the aims and working methods of the program and a combination of quantitative and…

Five authors representing diverse perspectives comment on the revised "Program Evaluation Standards" approved by the American National Standards Institute (ANSI). Standards are considered in light of their development; measurement issues; program evaluation; evaluation in the local education agency; and the context of evaluation…

Seven qualitative issues associated with randomization that have the potential to weaken or destroy otherwise sound experimental designs are reviewed and illustrated via actual field experiments. Issue areas include ethics and legality, liability risks, manipulation of randomized outcomes, hidden bias, design intrusiveness, case flow, and…

The 9 articles of this theme issue stem from a project on alcohol and drug abuse that involved 14 projects, 10 of which began with randomized clinical trials. These papers describe implementation problems associated with experimentation in field research and the focus on ensuring internal validity. (SLD)