Contemporary case studies rely on verbal arguments and set theory to build or evaluate theoretical claims. While existing procedures excel in the use of qualitative information (information about kind), they ignore quantitative information (information about degree) at central points of the analysis. Effectively, contemporary case studies rely on…

The work presented in this case study results from a study conducted in 2012-2014 examining a newly created teacher evaluation system to determine the inter-rater reliability of the classroom observation instrument. The teacher evaluation system was the result of a partnership between the school district and the university in the same city…

We present a case study of the process through which a methodology was developed and applied to a quasi-experimental research study that employed propensity score matching. Methodological decisions are discussed and summarized, including an explanation of the approaches selected for each step in the study as well as rationales for these…

The purpose of this paper is to summarize some of the author's logical and empirical reflections on the practice of using naturalistic techniques to conduct evaluations. The main assertion to be supported by these reflections is that naturalistic evaluation is possible and in some cases powerful. This assertion is supported through a review of (1)…

Reviews elements of research that need to be monitored by funding agencies and/or organizations carrying out research, and examines the problems and possibilities of research into the impact of research. Presents a case study of a project that focused on three strands of research undertaken at the University of Sheffield (United Kingdom). (Author)

The author feels that the stakeholder approach unavoidably pushes the evaluation toward technical compromises and accommodations that diminish the long-term gains in knowledge. (PN)

The four papers in these proceedings provide a comprehensive picture of the rationale, methodology, strengths, and limitations of case studies. In "Doing Justice in Evaluation Research," David Hamilton argues that the application of scientific methods to the study of social systems has given us a false illusion of certainty. He offers an…

Stakeholder-based studies modify the relationship between evaluator and user, producing conflict that can cause evaluators to regress to familiar, traditional patterns that leave interactive evaluation strategies only partially implemented. (Author/PN)

Trade-offs are implicit in choosing a subjective or objective method for evaluating social programs. The differences between Bayesian and traditional statistics, decision and cost-benefit analysis, and anthropological and traditional case systems illustrate trade-offs in choosing methods because of limited resources. (SLD)

Analysis of the assumptions underlying the stakeholder approach to evaluation combined with the limited experience in testing the approach reported in this volume, suggests that some claims are cogent and others problematical. (Author)

The evaluators of Push/Excel assumed that it was a systematically developed program with measurable outcomes, not a charismatically inspired movement whose effects would be hard to pin down. As a result, neither the program nor the evaluation approach used were adequately tested. (Author)

Critiquing the stakeholder idea, the author states: if government chooses to take account of competing views in social program evaluation, it can get a better result if it encourages the competing views to find a voice of their own, not to speak through the government's chosen instrument. (Author/PN)

Discusses the use of focus groups for conducting research with children, and presents a case study that used the focus group method in a child sexual abuse prevention program. Critiques the advantages and disadvantages of the use of focus groups with children for evaluative purposes. (Author/CR)

Deliberately involving urban education stakeholders in the Cities-in-Schools evaluation study improved the utility of the findings for formative evaluation stakeholders but not for summative evaluation stakeholders. (Author/PN)

This paper discusses how assessment practices in higher education can improve or hinder learning. An example is given to illustrate some common educational practices that may be contributing to underpreparation and underperformance of students. Elements of effective learning environments that may better address underlying metacognitive issues are…