ERIC Number: EJ1262653
Record Type: Journal
Publication Date: 2020
Pages: 11
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0895-7347
EISSN: N/A
Available Date: N/A
Applying Cognitive Theory to the Human Essay Rating Process
Finn, Bridgid; Arslan, Burcu; Walsh, Matthew
Applied Measurement in Education, v33 n3 p223-233 2020
To score an essay response, raters draw on previously trained skills and knowledge about the underlying rubric and score criterion. Cognitive processes such as remembering, forgetting, and skill decay likely influence rater performance. To investigate how forgetting influences scoring, we evaluated raters' scoring accuracy on TOEFL and GRE essays. We used binomial linear mixed effect models to evaluate how the effect of various predictors such as time spent scoring each response and days between scoring sessions relate to scoring accuracy. Results suggest that for both nonoperational (i.e., calibration samples completed prior to a scoring session) and operational scoring (i.e., validity samples interspersed among actual student responses), the number of days in a scoring gap negatively affects performance. The findings, as well as other results from the models are discussed in the context of cognitive influences on knowledge and skill retention.
Descriptors: Epistemology, Essay Tests, Evaluators, Cognitive Processes, Scoring, Accuracy, English (Second Language), Language Tests, Second Language Learning, Graduate Study, College Entrance Examinations, Predictor Variables, Skill Development
Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: Test of English as a Foreign Language; Graduate Record Examinations
Grant or Contract Numbers: N/A
Author Affiliations: N/A