Publication Date
In 2025 | 1 |
Since 2024 | 1 |
Since 2021 (last 5 years) | 4 |
Since 2016 (last 10 years) | 10 |
Since 2006 (last 20 years) | 14 |
Descriptor
Source
Author
Abbasi, Abbas | 1 |
Attali, Yigal | 1 |
Barkaoui, Khaled | 1 |
Berson, Nancy | 1 |
Biancarosa, Gina | 1 |
Blair, William O. | 1 |
Brull, Harry | 1 |
Collins, Kathleen M. | 1 |
Crowson, Mary | 1 |
Cummings, Kelli D. | 1 |
Davis, Lawrence Edward | 1 |
More ▼ |
Publication Type
Journal Articles | 18 |
Reports - Research | 13 |
Reports - Evaluative | 4 |
Speeches/Meeting Papers | 2 |
Dissertations/Theses -… | 1 |
Information Analyses | 1 |
Reports - Descriptive | 1 |
Tests/Questionnaires | 1 |
Education Level
Higher Education | 6 |
Postsecondary Education | 5 |
Adult Education | 1 |
Elementary Education | 1 |
Grade 5 | 1 |
Grade 6 | 1 |
Intermediate Grades | 1 |
Middle Schools | 1 |
Audience
Laws, Policies, & Programs
Assessments and Surveys
International English… | 2 |
Test of English as a Foreign… | 1 |
What Works Clearinghouse Rating
Makiko Kato – Journal of Education and Learning, 2025
This study aims to examine whether differences exist in the factors influencing the difficulty of scoring English summaries and determining scores based on the raters' attributes, and to collect candid opinions, considerations, and tentative suggestions for future improvements to the analytic rubric of summary writing for English learners. In this…
Descriptors: Writing Evaluation, Scoring, Writing Skills, English (Second Language)
Attali, Yigal – Educational Measurement: Issues and Practice, 2019
Rater training is an important part of developing and conducting large-scale constructed-response assessments. As part of this process, candidate raters have to pass a certification test to confirm that they are able to score consistently and accurately before they begin scoring operationally. Moreover, many assessment programs require raters to…
Descriptors: Evaluators, Certification, High Stakes Tests, Scoring
Heidari, Nasim; Ghanbari, Nasim; Abbasi, Abbas – Language Testing in Asia, 2022
It is widely believed that human rating performance is influenced by an array of different factors. Among these, rater-related variables such as experience, language background, perceptions, and attitudes have been mentioned. One of the important rater-related factors is the way the raters interact with the rating scales. In particular, how raters…
Descriptors: Evaluators, Rating Scales, Language Tests, English (Second Language)
Thai, Thuy; Sheehan, Susan – Language Education & Assessment, 2022
In language performance tests, raters are important as their scoring decisions determine which aspects of performance the scores represent; however, raters are considered as one of the potential sources contributing to unwanted variability in scores (Davis, 2012). Although a great number of studies have been conducted to unpack how rater…
Descriptors: Rating Scales, Speech Communication, Second Language Learning, Second Language Instruction
Han, Chao; Xiao, Xiaoyan – Language Testing, 2022
The quality of sign language interpreting (SLI) is a gripping construct among practitioners, educators and researchers, calling for reliable and valid assessment. There has been a diverse array of methods in the extant literature to measure SLI quality, ranging from traditional error analysis to recent rubric scoring. In this study, we want to…
Descriptors: Comparative Analysis, Sign Language, Deaf Interpreting, Evaluators
Hsu, Tammy Huei-Lien – Language Testing in Asia, 2019
Background: A strong interest in researching World Englishes (WE) in relation to language assessment has become an emerging theme in language assessment studies over the past two decades. While research on WE has highlighted the status, function, and legitimacy of varieties of English language, it remains unclear how raters respond to the results…
Descriptors: Language Attitudes, Language Variation, Language Tests, Second Language Learning
Sahan, Özgür; Razi, Salim – Language Testing, 2020
This study examines the decision-making behaviors of raters with varying levels of experience while assessing EFL essays of distinct qualities. The data were collected from 28 raters with varying levels of rating experience and working at the English language departments of different universities in Turkey. Using a 10-point analytic rubric, each…
Descriptors: Decision Making, Essays, Writing Evaluation, Evaluators
Han, Qie – Working Papers in TESOL & Applied Linguistics, 2016
This literature review attempts to survey representative studies within the context of L2 speaking assessment that have contributed to the conceptualization of rater cognition. Two types of studies are looked at: 1) studies that examine "how" raters differ (and sometimes agree) in their cognitive processes and rating behaviors, in terms…
Descriptors: Second Language Learning, Student Evaluation, Evaluators, Speech Tests
Reed, Deborah K.; Cummings, Kelli D.; Schaper, Andrew; Lynn, Devon; Biancarosa, Gina – Reading and Writing: An Interdisciplinary Journal, 2019
Informal reading inventories (IRI) and curriculum-based measures of reading (CBM-R) have continued importance in instructional planning, but raters have exhibited difficulty in accurately identifying students' miscues. To identify and tabulate scorers' mismarkings, this study employed examiners and raters who scored 15,051 words from 108 passage…
Descriptors: Accuracy, Miscue Analysis, Grade 5, Grade 6
Han, Turgay; Huang, Jinyan – PASAA: Journal of Language Teaching and Learning in Thailand, 2017
Using generalizability (G-) theory and rater interviews as both quantitative and qualitative approaches, this study examined the impact of scoring methods (i.e., holistic versus analytic scoring) on the scoring variability and reliability of an EFL institutional writing assessment at a Turkish university. Ten raters were invited to rate 36…
Descriptors: English (Second Language), Second Language Learning, Second Language Instruction, Scoring
Yeates, Peter; O'Neill, Paul; Mann, Karen; Eva, Kevin – Advances in Health Sciences Education, 2013
Assessors' scores in performance assessments are known to be highly variable. Attempted improvements through training or rating format have achieved minimal gains. The mechanisms that contribute to variability in assessors' scoring remain unclear. This study investigated these mechanisms. We used a qualitative approach to study…
Descriptors: Performance Based Assessment, Scores, Evaluators, Scoring
Everson, Mark D.; Sandoval, Jose Miguel; Berson, Nancy; Crowson, Mary; Robinson, Harriet – Journal of Child Sexual Abuse, 2012
In the absence of photographic or DNA evidence, a credible eyewitness, or perpetrator confession, forensic evaluators in cases of alleged child sexual abuse must rely on psychosocial or "soft" evidence, often requiring substantial professional judgment for case determination. This article offers a three-part rebuttal to Herman's (2009) argument…
Descriptors: Evidence, Evaluators, Persuasive Discourse, Sexual Abuse
Davis, Lawrence Edward – ProQuest LLC, 2012
Speaking performance tests typically employ raters to produce scores; accordingly, variability in raters' scoring decisions has important consequences for test reliability and validity. One such source of variability is the rater's level of expertise in scoring. Therefore, it is important to understand how raters' performance is influenced by…
Descriptors: Evaluators, Expertise, Scores, Second Language Learning
Barkaoui, Khaled – Language Assessment Quarterly, 2010
Various factors contribute to variability in English as a second language (ESL) essay scores and rating processes. Most previous research, however, has focused on score variability in relation to task, rater, and essay characteristics. A few studies have examined variability in essay rating processes. The current study used think-aloud protocols…
Descriptors: Protocol Analysis, Holistic Evaluation, Evaluation Criteria, Rating Scales

Lunz, Mary E.; And Others – Educational and Psychological Measurement, 1994
In a study involving eight judges, analysis with the FACETS model provides evidence that judges grade differently, whether or not scores correlate well. This outcome suggests that adjustments for differences among judges should be made before student measures are estimated to produce reproducible decisions. (SLD)
Descriptors: Correlation, Decision Making, Evaluation Methods, Evaluators
Previous Page | Next Page »
Pages: 1 | 2