This article describes an alternative approach to setting standards for performance assessments. The procedure was designed for use in low-budget, relatively low-stakes contexts where it is not possible to bring expert judges together. The procedure that allowed participant judges to work individually throughout the process was an effort to…

This paper looked into the interplay between language and culture based on the metatext categories utilized by the writers in three speech communities: Philippines, Iran, and Taiwan. Specifically, it answered the following subproblems: (1) What metatext categories exist in the results-and-discussion section of ESL/EFL research articles written in…

Using our experience of a work-related learning project, run jointly between Newcastle and Northumbria Universities, in which 600 students annually undertake work placements, this paper focuses on the inherent challenges of using over 30 assessors to assess students. The assessors come from a wide variety of backgrounds and we have had to find…

In the context of a project developing software for pronunciation practice and feedback for Mandarin-speaking learners of English, a key issue is how to decide which features of pronunciation to focus on in giving feedback. We used naive and experienced native speaker ratings of comprehensibility and nativeness to establish the key features…

Evaluators face challenges when programs consistently fail to meet expectations for performance or improvement and consequently, evaluators may recommend that closing a program is the most prudent course of action. However, the evaluation literature provides little guidance regarding when an evaluator might recommend program closure. Given…

This study is the first to demonstrate that features of psychopathy can be reliably and validly detected by lay raters from "thin slices" (i.e., small samples) of behavior. Brief excerpts (5 s, 10 s, and 20 s) from interviews with 96 maximum-security inmates were presented in video or audio form or in both modalities combined. Forty raters used…

This fact sheet summarizes data gathered in the 2012 survey of National Science Foundation (NSF) Advanced Technological Education (ATE) grant recipients. Conducted by EvaluATE, the evaluation resource center for the ATE program located at The Evaluation Center at Western Michigan University, this was the thirteenth annual survey of ATE projects…

Education Law Section 3012-c requires a new performance evaluation system for classroom teachers ("teachers") and building principals ("principals"). New York State will implement a statewide comprehensive evaluation system for school districts and boards of cooperative educational services (BOCES). The evaluation system is…

Research perspectives on teacher evaluation present evaluators with a set of possible acts. Local evaluation systems, on the other hand, specify a permissible set of acts from the total universe. The acts specified within a given locality act as conditions for teacher action. Using the sampling and analytical procedures of grounded theory, this…

Background: Readability tests are indicators that measure how easy a document can be read and understood. Simple, but very often ignored, readability statistics cannot only provide information about the level of difficulty of the readability of particular documents but also can increase an evaluator's credibility. Purpose: The purpose of this…

The clash between Western and Indigenous ways of knowing has been epitomized by the "parachuting model" of the Western researcher who drops onto the reservation, collects data, and leaves, never to be heard from again. The strengths of indigenous science, for example, observation and contextual factors, are either ignored or…

This article is concerned with the underpinnings of practical participatory evaluation (PPE). Evaluation approaches have long been criticized because their results are often not used. It is believed that PPE addresses this drawback. The article focuses on the mechanisms underlying the links between activities and consequences in PPE. A PPE theory…

Meta-evaluations reported in the literature, although rare, often have focused on retrospective assessment of completed evaluations. Conducting a meta-evaluation concurrently with the evaluation modifies this approach. This method provides the opportunity for the meta-evaluators to advise the evaluators and provides the basis for a summative…

The examination of doctoral theses controls an important academic threshold, yet practices are often private, codes non-specific, and individuals isolated. This article adds to recent investigation of the examination culture by reporting informal panel discussion amongst a total of 23 University of Auckland (New Zealand) faculty members as to…

The field of evaluation faces a number of serious challenges in light of No Child Left Behind legislation, among them feasibility, resources, and blurring lines among research, evaluation, and assessment. At the same time, these challenges open the door for opportunities in evaluation. Now more than ever, the expertise of evaluators is needed and…