Generative artificial intelligence technologies, such as ChatGPT, bring an unprecedented change in education by leveraging the power of natural language processing and machine learning. Employing ChatGPT to assist with marking written assessment presents multiple advantages including scalability, improved consistency, eliminating biases associated…

Aim/Purpose: This article investigates the process of identifying and correcting hallucinations in ChatGPT-4's recall of student-written responses as well as its evaluation of these responses, and provision of feedback. Effective prompting is examined to enhance the pre-evaluation, evaluation, and post-evaluation stages. Background: Advanced Large…

One of the challenges of discussing validity arguments for machine scoring of essays centers on the absence of a commonly held definition and theory of good writing. At best, the algorithms attempt to measure select attributes of writing and calibrate them against human ratings with the goal of accurate prediction of scores for new essays.…

Teachers need instructionally useful data to make timely and appropriate decisions to meet their students with intensive needs (Filderman et al., 2019). Teachers have still experienced difficulty in instructional decision making in response to students' CBM data (Gesel et al., 2021). This is because data itself that was used for simply determining…

Unfolding models are frequently used to develop scales for measuring attitudes. Recently, unfolding models have been applied to examine rater severity and accuracy within the context of rater-mediated assessments. One of the problems in applying unfolding models to rater-mediated assessments is that the substantive interpretations of the latent…

This study examined Chinese EFL researchers' English abstract writing in language education. Using open-ended questionnaires, it first investigated 24 Chinese EFL researchers' perceptions of their challenges in writing English abstracts. Using generalizability theory and follow-up interviews, it then invited 16 experienced English journal…

The study highlights ChatGPT-4's potential in educational settings for the evaluation of university students' open-ended written examination responses. ChatGPT-4 evaluated 54 written responses, ranging from 24 to 256 words in English. It assessed each response using five criteria and assigned a grade on a six-point scale from fail to excellent,…

Writing is generally assessed internationally using rubric-based approaches, but there is a growing body of evidence to suggest that the reliability of such approaches is poor. In contrast, comparative judgement studies suggest that it is possible to assess open ended tasks such as writing with greater reliability. Many previous studies, however,…

The primary purpose of this study is to investigate the degree to which register knowledge, register-specific motivation, and diverse linguistic features are predictive of human judgment of writing quality in three registers--narrative, informative, and opinion. The secondary purpose is to compare the evaluation metrics of register-partitioned…

Automated writing evaluation (AWE) plays an important role in writing pedagogy and has received considerable research attention recently; however, few reviews have been conducted to systematically analyze the recent publications arising from the many studies in this area. The present review aims to provide a comprehensive analysis of the…

Although there are claims that ChatGPT, an AI-based language model, is capable of assessing the writing of L2 learners accurately and consistently in the classroom, a number of recent studies have shown discrepancies between AI and human raters. Furthermore, there is a lack of studies investigating the intrareliability of ChatGPT scores.…

Automated Essay Scoring (AES) tools aim to improve the efficiency and consistency of essay scoring by using machine learning algorithms. In the existing research work on this topic, most researchers agree that human-automated score agreement remains the benchmark for assessing the accuracy of machine-generated scores. To measure the performance of…

Corrective feedback (CF) has long been a hot topic in language education circles and has received extensive research attention. However, there is still controversy over the effectiveness of CF use and error correction in language classes. To address this discrepancy, the current study probed the effectiveness of different CF types in improving…

This quasi-experimental study investigates the effects of pre-writing (Pre-EI) and post-writing explicit instruction (Post-EI) on L2 learners' overall writing accuracy and errors at different severity levels. Situated within process-genre-oriented writing classrooms, a total of three intact groups (N = 101) were designated as two experimental…

Although, written corrective feedback (hereafter referred to as CF) is applauded in many writing courses for fostering students' quality writing, its impact on grammatical accuracy in L2 students' writing remains a debated topic. Thus, this study looked into the effect of CF types on L2 students' grammatical accuracy in writing. To achieve this…