NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
ERIC Number: EJ1343617
Record Type: Journal
Publication Date: 2022
Pages: 21
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1755-6031
EISSN: N/A
Available Date: N/A
A Summary of OCR's Pilots of the Use of Comparative Judgement in Setting Grade Boundaries
Benton, Tom; Gill, Tim; Hughes, Sarah; Leech, Tony
Research Matters, n33 p10-30 Spr 2022
The rationale for the use of comparative judgement (CJ) to help set grade boundaries is to provide a way of using expert judgement to identify and uphold certain minimum standards of performance rather than relying purely on statistical approaches such as comparable outcomes. This article summarises the results of recent trials of using CJ for this purpose in terms of how much difference it might have made to the positions of grade boundaries, the reported precision of estimates and the amount of time that was required from expert judges. The results show that estimated grade boundaries from a CJ approach tend to be fairly close to those that were set (using other forms of evidence) in practice. However, occasionally, CJ results displayed small but significant differences with existing boundary locations. This implies that adopting a CJ approach to awarding would have a noticeable impact on awarding decisions but not such a large one as to be implausible. This article also demonstrates that implementing CJ using simplified methods (described by Benton, Cunningham et al, 2020) achieves the same precision as alternative CJ approaches, but in less time. On average, each CJ exercise required roughly 30 judge-hours across all judges.
Cambridge University Press & Assessment. Shaftesbury Road Cambridge CB2 8EA. Tel: 44-1223-553311; e-mail: directcs@cambridge.org; Web site: https://www.cambridgeassessment.org.uk/our-research/all-published-resources/research-matters/
Publication Type: Journal Articles; Information Analyses
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A