Methodological rigor consists of a series of elements that, in combination, determine the confidence with which conclusions can be drawn from the evaluation results. These elements include evaluation design, conceptualization of constructs, measurement strategies, time frames, program integrity, and others. The authors examine the factors that…

This chapter describes the adaptation of the case study method to assessing increasingly complex, comprehensive reform initiatives that highlight the blurring of the boundaries between phenomenon and context and the concurrence of multiple interventions. Completed studies of two education reform programs illustrate the ongoing challenges of…

This chapter presents the findings from a review of the practice of evaluation in federal agencies as an attempt to inform policies on method choice. The author explores whether federal agencies differ in their approaches to evaluation design and the factors that influence these differences. The nature of the programs, agency culture, evaluator…

Describes activity theory, which provides a conceptual framework for understanding the relationships among evaluation inquiry, technology, and the social contexts of technologically mediated evaluation practice. In activity theory, the activity provides the fundamental unit of analyses. The focus is on the use of tools by a subject to achieve an…

Crude mental models should take priority over designs in guiding evaluative inquiry. Three illustrative mental models are shown to have implications for how method mixing is carried out in practice. One of the models, a complex and contextually contingent model, is applied to the Arizona Student Assessment Program. (SLD)

The editors give each of the theorists a chance to respond to questions posed about the context of the situation in relation to their own experience in the field, exploring how the exercise had an impact on their evaluation designs.

This final chapter summarizes the areas of consensus in the debate on method choice, including considering the nature of the primary evaluation questions, the nature of the phenomenon being evaluated, the constraints on the evaluation, and ethical issues. Pragmatic suggestions based on these areas as well as areas still in contention are offered.…

Illustrative case studies support a contingency approach to mixed-method evaluation in which the evaluation team bases its selection of methods on the information to be provided, the availability of data, and the degree to which the program environment is an open or closed system. (SLD)

The past two decades have witnessed a significant growth in the number of school improvement programs and in the accompanying efforts to evaluate such programs. Passage of the No Child Left Behind (NCLB) Act in 2002 has intensified the need for evaluations to assess and understand the quality and value of educational interventions. Well over a…

There is concern that the nation's schools are not preparing students to excel in science, technology, engineering, and mathematics (STEM) fields. At both precollege and postsecondary levels, much effort is needed to create and implement powerful STEM curricula, prepare and support highly qualified teachers, deliver effective instruction, and give…