The authors respond to the commentaries on a series of articles (in the March 2009 issue of "Educational Researcher") that discuss the challenges associated with measuring reading instruction. The authors argue that the teaching of reading is a complex, multifaceted phenomenon best studied through a variety of overlapping and complementary…

This article is a response to the growing criticisms of the British Educational Research Association (BERA) and Economic and Social Research Council (ESRC) ethical guidelines on anonymity and pseudonymity as default positions for participants in qualitative educational research. It discusses and responds to those criticisms under four…

Lecoutre, Lecoutre, and Poitevineau (2010) have provided sophisticated grounding for "p[subscript rep]." Computing it precisely appears, fortunately, no more difficult than doing so approximately. Their analysis will help move predictive inference into the mainstream. Iverson, Wagenmakers, and Lee (2010) have also validated…

Productive dialogue is informed best by multiple and diverse voices. Five seasoned evaluators, representing a range of evaluation perspectives, offer their views in two- to three-page discussant contributions. These individuals were asked to reflect and comment on the previous chapters in the spirit of critical review as a key source of evidence…

This article describes the design and evaluation approaches to address the complexity posed by systems change initiatives. The role of evaluations in addressing the following issues is briefly reviewed: moving from strategic planning to implementation, impacts on system-level coordination, anticipated timeline of impact, and individual level…

D'Eon et al. concluded that change in performance self-assessment means from before to after a workshop can detect workshop success in their and other situations. In this commentary, their recommendation is refuted by showing that (a) self-assessments with balanced over- and underestimations are still biased and should not be used to evaluate…

"Theory as Method: Small Theories of Treatments," by Mark W. Lipsey, is one of the most influential and highly cited articles to appear in "New Directions for Evaluation." It articulated an approach in which methods for studying causation depend, in large part, on what is known about the theory underlying the program. Lipsey discussed the benefits…

Suggests that evaluation research is relatively and, unfortunately, atheoretical in approach. The incorporation of empiricism is a philosophical and methodological strategy for evaluation research. As the functions of theory have defined the growth and utility of science, it will characterize the growth and usefulness of evaluation research. (JAC)

Argues that evaluations of games are often inadequate due to poor experimental design. In order to correct this flaw, particular attention must be given to controlling the confounding variables and obtaining equivalent treatment groups. References are included. (LLS)

This article started as a debate between the two authors. Both authors present a series of propositions about quality standards in education research. Cook's propositions, as might be expected, not only concern the importance of experimental trials for establishing the security of causal evidence, but also include some important practical and…

The current literature is replete with instances of imprecise and conflicting usage of "qualitative" and "quantitative" evaluation. This paper presents a framework for making distinctions among several related concepts to eliminate this potential source of confusion. In the process, the terms "qualitative" and…

Written in response to George Fairweather's position concerning the use of experimental studies in evaluation, this paper highlights considerations in deciding when experimental procedures are feasible. The current debate on the use of experimental methods in evaluation is reviewed. (Author/GK)

Provides a condensed description of the criteria and choices for selecting a particular assessment, evaluation, or research design. Strengths and weaknesses of each design are presented. (Author/JEG)

Discusses training costs and methods for evaluating the effectiveness of training. Topics addressed include tracking training investments; tracking individual and business performance, including measurement techniques; research designs for analyzing performance; demonstrating return on investment; reasons for low return on investment; and ways to…

Discusses two areas for improving consultation research: defining "boundary" conditions for consultation and analogue research and expanding measures used to evaluate consultation outcome research. Defines consultation as egalitarian relationship between consultant and consultee focusing on problem of third party client and resulting in…