ERIC Number: EJ1386392
Record Type: Journal
Publication Date: 2022
Pages: 17
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: EISSN-2375-5636
Available Date: N/A
Evaluating Coherence in Writing: Comparing the Capacity of Automated Essay Scoring Technologies
Shin, Jinnie; Gierl, Mark J.
Journal of Applied Testing Technology, v23 spec iss 1 p4-20 2022
Automated Essay Scoring (AES) technologies provide innovative solutions to score the written essays with a much shorter time span and at a fraction of the current cost. Traditionally, AES emphasized the importance of capturing the "coherence" of writing because abundant evidence indicated the connection between coherence and the overall writing quality yet, limited studies have been conducted to investigate the capacity of the modern and traditional automated essay scoring technologies in capturing the sequential information (i.e., cohesion). In this study, we investigate the performance of traditional and modern AES systems in attribute-specific scoring. Traditional AES focuses on holistic scoring with limited application for the attribute-specific scoring. Hence, the current study focuses on understanding whether a deep-neural AES system using a convolutional neural networks approach could provide better performance in attribute-specific essay scoring compared to a traditional feature-based AES system in capturing coherence scores in essays. Our finding indicated that a deep-neural AES model showed improved accuracy in predicting coherence-related score categories. Implications for the scoring capacity of the two models are also discussed.
Descriptors: Computer Assisted Testing, Scoring, Essays, Automation, Artificial Intelligence, Scores, Models, Prompting, Prediction
Association of Test Publishers. 601 Pennsylvania Avenue NW, South Building Suite 900, Washington DC 20004. Tel: 866-240-7909; Fax: 717-755-8962; e-mail: wgh.atp@att.net; Web site: https://jattjournal.net/index.php/atp/index
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A