H. Abikoff, M. Courtney, W. E. Pelham, and H. S. Koplewicz (1993) presented elementary school teachers with a videotape of a 4th-grade male child exhibiting behavior associated with either Attention Deficit Hyperactivity Disorder (ADHD) or Oppositional Defiant Disorder (ODD). Comparisons with ratings generated from a control tape (same child…

During workshops held in March and April of 2005, Michigan educators got a lesson in how to benchmark their work against promising practices in higher-performing schools with similar socioeconomic profiles. Teams of teachers, educational supervisors, and principals from 13 elementary and middle schools that had been struggling to make adequate…

This interview with Dr. Hallie Preskill focuses on the organizational change method known as appreciative inquiry (AI), which is described as a process that builds on past successes (and peak experiences) in an effort to design and implement future actions. Preskill takes the philosophy and principles put forth by organizational change AI…

The paper discusses two common scenarios in which evaluators must conduct impact evaluations when working under budget, time, or data constraints. Under the first scenario the evaluator is not called in until the project is already well advanced, and there is a tight deadline for completing the evaluation, frequently combined with a limited budget…

This paper considers the role of tests in evaluating the effectiveness of a government-funded initiative to teach heritage languages in selected Australian schools. External evaluations were commissioned by the central educational authority and included a pre-and post-test requirement, although the nature and purpose of the tests was not…

In this article, the author shares insights derived from her journey as an evaluation practitioner and as a capacity-building evaluation facilitator committed to demystifying evaluation as a core development resource. The author weaves in some foundational stories that have shaped and informed her path. Through her work and praxis, she has been…

Program theory plays a prominent role in many evaluations, not only in theory-driven evaluations. This paper presents a case study of the process of developing and refining a program's theory within a user-oriented evaluation. In user-oriented (or utilization-focused) evaluations, primary users can play a role in defining their own program theory.…

This article reviews the evolution of American Evaluation Association's (AEA) Independent Consulting Topical Interest Group (IC TIG). The TIG goes back a joint meeting held in San Francisco in 1984 of the Evaluation Network (ENet) and the Evaluation Research Society (ERS), two years before the organizations merged to become the AEA. On the fringes…

This fact sheet summarizes data gathered in the 2009 survey of National Science Foundation (NSF) Advanced Technological Education (ATE) grant recipients. Conducted by The Evaluation Center at Western Michigan University, this was the tenth annual survey of ATE projects and centers. Included here are statistics about the program's grantees and…

How is the quality of teaching portfolios being assessed in summative assessment contexts? This question is of special importance in the growing debate on standards and criteria in assessment. In our study we looked for contexts in which portfolios are used for summative assessment purposes and gauge the issues raised by assessors with respect to…

In 1993, the Accrediting Commission for Community and Junior Colleges initiated a comprehensive evaluation of the standards and procedures it uses to accredit colleges. The Commission asked the Research and Planning Group for California Community Colleges to carry out the survey. A total of 1,361 questionnaires were mailed to representative…

This paper reports the results of an analysis of the relationship between scorer behaviors and score variability. Thirty-six essay scorers were interviewed and asked to perform a think-aloud task as they scored 24 essays. Each comment made by a scorer was coded according to its content focus (i.e. appearance, assignment, mechanics, communication,…

This paper reports the development and evaluation of an observation guide that draws on qualitative techniques of participant observation and semistructured interviewing to help anyone interested in visiting and understanding schools to develop a "big picture" profile quickly and efficiently. The guide reflects the philosophy of E.…

"Feedback to Managers" is a two-volume report. Volume 2 compares 16 of the better feedback instruments available. The following are the instruments: (1) ACUMEN Group Feedback; (2) BENCHMARKS; (3) the Campbell Leadership Index; (4) COMPASS: the Managerial Practices Survey; (5) the Executive Success Profile; (6) Leader Behavior Analysis…

Three case studies are presented demonstrating the application of straight-forward Rasch techniques to rank order data. Paired comparisons are the simplest form of rank ordering. A consumer preference test with 56 pairs of cups of coffee tasted by each of 26 consumers illustrates analysis of these rankings. When subjects are allowed the option of…