Evaluations are practiced in political contexts, posing ethical dilemmas to evaluators. It is important, therefore, to prepare evaluation students for ethical decision-making in their future evaluative work. This study explores the use of scenario-based moral reasoning and ethical argumentation as an instructional strategy for teaching evaluation…

A large data set of L1 psycholinguistic norms (Balota "et al." 2007) was used to assess spoken L2 English lexical proficiency in cross-sectional and longitudinal learner corpora. Behavioral norms included lexical decision and word naming latencies (i.e. reaction times) and accuracies for 40,481 English words. A frequency measure was…

There has been a worldwide increase in the number of postgraduate students over the last few years and therefore some examiners struggle to maintain high standards of consistency, accuracy and fairness. This is especially true in developing countries where the increase is supervision capacity is not on a par with the growth in student numbers. The…

The importance of engaging in high quality program evaluation is a generally accepted principle underscored by external pressure from funders. High quality evaluation necessarily begins with good evaluation planning. This paper outlines Evolutionary Evaluation and specifically the Systems Evaluation Protocol, an approach that emphasizes…

Research on evaluation theories, methods, and practices has increased considerably in the past decade. Even so, little is known about whether published findings from research on evaluation are read by evaluators and whether such findings influence evaluators' thinking about evaluation or their evaluation practice. To address these questions, and…

In 2009, the New Teacher Project's "The Widget Effect" documented the failure of U.S. public school districts to recognize and act on differences in teacher effectiveness. We revisit these findings by compiling teacher performance ratings across 24 states that adopted major reforms to their teacher evaluation systems. In the vast…

The Research Excellence Framework (REF) is a new system for assessing the quality of research in UK higher education institutions (HEIs). For the first time, part of the assessment included the wider impact of research. RAND Europe was commissioned to evaluate the assessment process of the impact element of REF submissions, and to explore the…

In this paper we share our reflections, as evaluators, on an evaluation where we encountered Excessive Evaluation Anxiety (XEA). The signs of XEA which we discerned were particularly evident amongst the program head and staff who were part of a new training program. We present our insights on the evaluation process and its difficulties, as well as…

Making the program theory explicit is an essential first step in Theory Driven Evaluation (TDE). Once explicit, the program logic can be established making necessary links between the program theory, activities, and outcomes. Despite its importance evaluators often encounter situations where the program theory is not explicitly stated. Under such…

In 1999, the National Science Foundation (NSF) awarded funds to the Evaluation Center at Western Michigan University to conduct an external evaluation of the Advanced Technological Education (ATE) program. ATE, a federally mandated program designed to increase the number and quality of skilled technicians in the U.S. workforce, has funded over 346…

Targeting Outcomes of Programs (TOP) is a seven-step hierarchical programming model in which the program development and performance sides are mirror images of each other. It served as a framework to identify a simple method for targeting photographic events in nonformal education programs, indicating why, when, and how photographs would be useful…

Written in a refreshing conversational style, this text thoroughly prepares students, program administrators, and new evaluators to conduct evaluations or to use them in their work. The book's question-driven focus and clear discussions about the importance of fostering evaluation use by building collaborative relationships with stakeholders set…

In 1995, the National Science Foundation (NSF) contracted with principal investigator Iris Weiss and an evaluation team at Horizon Research, Inc. (HRI) to conduct a national evaluation of the Local Systemic Change for Teacher Enhancement program (LSC). HRI conducted the core evaluation under a $6.25 million contract with NSF. This program…

As Americans shift their work and leisure activities online, Extension seeks to remain viable by delivering programs through a website known as eXtension. eXtension is predicated on the voluntary labor of Extension specialists and educators who form Communities of Practice to create and deliver content through the website. Evaluation of eXtension…

Background: A music education program with a goal of enhancing cognitive development of preschool-aged children enrolled in local preschools is evaluated by The Evaluation Center at Western Michigan University. The budget for the evaluation was small, and therefore presented several challenges to the evaluation team. Purpose: Through a case study…