This research aims to make a methodological contribution to the mixed methods literature by investigating the value-added of empirical studies that are explicitly framed with the purpose of interrogating divergent findings. Case studies introduce the role philosophical paradigms can play in reconciliation process and reveal the potential to…

This article discusses the nonuse, misuse, and proper use of pilot studies in experimental evaluation research. The authors first show that there is little theoretical, practical, or empirical guidance available to researchers who seek to incorporate pilot studies into experimental evaluation research designs. The authors then discuss how pilot…

The purpose of this article is to develop guidelines to assist practitioners and researchers in evaluating and developing rigorous case studies. The main concern in evaluating a case study is to accurately assess its quality and ultimately to offer clients social work interventions informed by the best available evidence. To assess the quality of…

Donald W. Compton, the first director of evaluation services at the National Home Office (Atlanta) of the American Cancer Society, tells the story of building the unit in conditions of high demand and a limited budget. Along the way, evaluation was brought to regional divisions and to local offices in part as a response to United Way and to his…

This article describes the design and evaluation approaches to address the complexity posed by systems change initiatives. The role of evaluations in addressing the following issues is briefly reviewed: moving from strategic planning to implementation, impacts on system-level coordination, anticipated timeline of impact, and individual level…

Qualitative case study methodology provides tools for researchers to study complex phenomena within their contexts. When the approach is applied correctly, it becomes a valuable method for health science research to develop theory, evaluate programs, and develop interventions. The purpose of this paper is to guide the novice researcher in…

Action research can play an important role in institutional research endeavors to improve the quality of organizational performance in higher education. This article outlines how the action research perspective informed a research design that integrated fragmented and isolated student success initiatives at a large urban two-year institution where…

This study examined the impact of rater agreement on decisions concerning the alignment between the "Golden State Examination in High School Mathematics" (California Department of Education, 2001a) and the University of California (UC) "Statement on Competencies in Mathematics Expected of Entering College Students" (Academic…

Determining the potential of educational information and communications technology for enhancing teaching and learning is the subject of a great deal of research activity. This paper argues that the technologies employed to enhance learning can also be exploited for the enhancement of research. Recorded learner-system interactions, dynamically…

Not all policy questions can be addressed by social experiments. Nonexperimental evaluation methods provide an alternative to experimental designs but their results depend on untestable assumptions. This paper presents evidence on the reliability of propensity score matching (PSM), which estimates treatment effects under the assumption of…

One effective method to improve the learning of science is through case study teaching. Teachers use realistic or true narratives to provide opportunities for students to integrate multiple sources of information in an authentic context, and may engage students with ethical and societal problems related to their discipline. However, little…

Program theory-driven evaluation science uses substantive knowledge, as opposed to method proclivities, to guide program evaluations. It aspires to update, clarify, simplify, and make more accessible the evolving theory of evaluation practice commonly referred to as theory-driven or theory-based evaluation. The evaluator in this chapter provides a…

In 2001, the Bunche Academy was chosen by its district to join in partnership with the Da Vinci Learning Corporation to embark on an ambitious whole-school reform initiative, especially designed by the corporation for low-performing schools. In this chapter, the author describes how, as illustrated in the Bunche-Da Vinci Learning Academy context,…

This paper is an "evaluation" study in the form of an extensive "case study" of one particular business school in Greece. Its alleged contribution focuses on the conclusions it hopes to draw in relation to the evaluation research methodology, based on the case evaluation research, experimenting with the use of students' ratings…

The suggestions offered in this paper are based on the experiences of two scientific researchers of how evaluations undertaken in a competitive arena, in which true experimental designs are not viable, can be designed in such a way that meaningful comparative data can be examined. Case studies of the Science and Engineering Research Semester and…