This study explores which features of Spanish as a foreign language (SFL) pronunciation most impact raters' evaluations. Native Spanish speakers (NSSs) from regions with different pronunciation norms were polled: 147 evaluators from northern Mexico and 99 evaluators from central Argentina. These evaluations were contrasted with ratings from…

Human raters are normally involved in L2 performance assessment; as a result, rater behavior has been widely investigated to reduce rater effects on test scores and to provide validity arguments. Yet raters' cognition and use of rubrics in their actual rating have rarely been explored qualitatively in L2 speaking assessments. In this study three…

As part of a mixed-methods doctoral project, this study drew on an interview, bracketed by two surveys, with experienced Australian reviewers. It explored questions about the contributions of peer review, the need and possibility to improve it and the driving factors for reviewers to contribute voluntarily. The respondents emphasised the essential…

This article presents the results of a survey conducted in an attempt to examine evaluators' perceptions of the contributions that different educational experiences make to the development of their evaluation skill sets. Self-identified new and graduate student evaluators were asked to describe their evaluation training experiences and to indicate…

We systematically evaluated the peer-reviewed Rorschach validity literature for the 65 main variables in the popular Comprehensive System (CS). Across 53 meta-analyses examining variables against externally assessed criteria (e.g., observer ratings, psychiatric diagnosis), the mean validity was r = 0.27 (k = 770) as compared to r = 0.08 (k = 386)…

The "indigenous assessment practices" (Jacoby & McNamara, 1999) in selected health professions were investigated to inform a review of the scope of assessment in the speaking sub-test of a specific-purpose English language test for health professionals, the Occupational English Test (OET). The assessment criteria in current use on…

This study reports the development of a scale to evaluate the speech act performance by intermediate-level Chinese learners of English. A qualitative analysis of the American raters' comments was conducted on learner scripts in response to a total of 16 apology and request written discourse completion task (WDCT) situations. The results showed…

Recent approaches to assessment and feedback in higher education stress the importance of students' involvement in these processes, where effective reception of feedback is as important as effective delivery. Many interventions have been developed to support students' active use of feedback; however, students' engagement will be influenced by…

While much literature has considered feedback and professional growth in formative peer reviews of teaching, there has been little empirical research conducted on these issues in the context of summative peer reviews. This article explores faculty members' perceptions of feedback practices in the summative peer review of teaching and reports on…

How do evaluators make decisions about how to approach an evaluation in their everyday practice? What are the bases for evaluators' approach choices? In what ways do evaluators think about evaluation models? The evaluation literature remains unclear about what specific information evaluators consider when making decisions in response to everyday…

This paper addresses the issue of the knowledge gap between evaluators and the entity being evaluated: the dilemma of the knowledge of professional evaluators vs. the in-depth knowledge of the evaluated subjects. In order to optimize evaluative outcomes, the author suggests an approach based on ideas borrowed from the science of cybernetics as a…

This engaging text takes an evenhanded approach to major theoretical paradigms in evaluation and builds a bridge from them to evaluation practice. Featuring helpful checklists, procedural steps, provocative questions that invite readers to explore their own theoretical assumptions, and practical exercises, the book provides concrete guidance for…

As states, districts, and schools transition toward more rigorous educator evaluation systems, they are placing additional weight on judgments about educator practice. Since teacher and principal observation ratings inherently rely on evaluators' professional judgment, there is always a question of how much the ratings depend on the particular…

Objective: Independent evaluators (IE) are used widely in clinical trials to make unbiased determinations of treatment response. By virtue of being kept blind to treatment condition, however, IEs are also kept unaware of many pertinent clinical details that are relevant for decisions about clinical improvement. In this study, agreement among…

All social and professional practices are historically situated, evolving forms of acting and interacting. Evaluation, as a practice, is shaped by and shapes the practice evaluated. This article contributes to responsive and values-engaged evaluation approaches by reflecting on the space where these two practices intersect. The evaluative task was…