Publication Date
| In 2025 | 0 |
| Since 2024 | 0 |
| Since 2021 (last 5 years) | 1 |
| Since 2016 (last 10 years) | 1 |
| Since 2006 (last 20 years) | 6 |
Descriptor
| Protocol Analysis | 6 |
| English (Second Language) | 4 |
| Evaluators | 4 |
| Foreign Countries | 4 |
| Second Language Learning | 4 |
| Evaluation Criteria | 3 |
| Questionnaires | 3 |
| Scoring | 3 |
| Classification | 2 |
| Comparative Analysis | 2 |
| Decision Making | 2 |
| More ▼ | |
Source
| Language Assessment Quarterly | 6 |
Author
| Baker, Beverly Anne | 1 |
| Barkaoui, Khaled | 1 |
| Cai, Hongwen | 1 |
| He, Lianzhen | 1 |
| Li, Hang | 1 |
| Liu, Xiaohua | 1 |
| Llosa, Lorena | 1 |
| Read, John | 1 |
| Wei, Jing | 1 |
Publication Type
| Journal Articles | 6 |
| Reports - Research | 6 |
| Tests/Questionnaires | 3 |
Education Level
| Adult Education | 1 |
| Higher Education | 1 |
| Postsecondary Education | 1 |
Audience
Location
| China | 2 |
| Canada | 1 |
| New Zealand | 1 |
Laws, Policies, & Programs
Assessments and Surveys
| Test of English as a Foreign… | 1 |
What Works Clearinghouse Rating
Peer reviewed
Direct linkLiu, Xiaohua; Read, John – Language Assessment Quarterly, 2021
Expert judgement has been frequently employed with reading assessments to gauge the skills potentially measured by test tasks, for purposes such as construct validation or producing diagnostic information. Despite the critical role it plays in such endeavours, few studies have triangulated its results with other types of data such as reported…
Descriptors: Reading Tests, Reading Skills, Test Items, Expertise
Cai, Hongwen – Language Assessment Quarterly, 2015
This study is an attempt to classify raters according to their weighting patterns and explore systematic differences between rater types in the rating process. In the context of an EFL speaking test, 126 raters were classified into three types--form-oriented, balanced, and content-oriented--through cluster analyses of their weighting patterns…
Descriptors: Classification, Language Tests, English (Second Language), Second Language Learning
Li, Hang; He, Lianzhen – Language Assessment Quarterly, 2015
This study used think-aloud protocols to compare essay-rating processes across holistic and analytic rating scales in the context of China's College English Test Band 6 (CET-6). A group of 9 experienced CET-6 raters scored the same batch of 10 CET-6 essays produced in an operational CET-6 administration twice, using both the CET-6 holistic…
Descriptors: Protocol Analysis, English (Second Language), Second Language Learning, Classification
Wei, Jing; Llosa, Lorena – Language Assessment Quarterly, 2015
This article reports on an investigation of the role raters' language background plays in raters' assessment of test takers' speaking ability. Specifically, this article examines differences between American and Indian raters in their scores and scoring processes when rating Indian test takers' responses to the Test of English as a Foreign…
Descriptors: North Americans, Indians, Evaluators, English (Second Language)
Baker, Beverly Anne – Language Assessment Quarterly, 2012
Researchers of high-stakes, subjectively scored writing assessments have done much work to better understand the process that raters go through in applying a rating scale to a language performance to arrive at a score. However, there is still unexplained, systematic variability in rater scoring that resists rater training (see Hoyt & Kerns,…
Descriptors: Individual Differences, Decision Making, Scoring, High Stakes Tests
Barkaoui, Khaled – Language Assessment Quarterly, 2010
Various factors contribute to variability in English as a second language (ESL) essay scores and rating processes. Most previous research, however, has focused on score variability in relation to task, rater, and essay characteristics. A few studies have examined variability in essay rating processes. The current study used think-aloud protocols…
Descriptors: Protocol Analysis, Holistic Evaluation, Evaluation Criteria, Rating Scales
