This study explores the writing quality of two AI chatbots, OpenAI ChatGPT and Google Gemini. The research assesses the quality of the generated texts based on five essay models using the T.E.R.A. software, focusing on ease of understanding, readability, and reading levels using the Flesch-Kincaid formula. Thirty essays were generated, 15 from…

As the development and application of large language models (LLMs) in physics education progress, the well-known AI-based chatbot ChatGPT4 has presented numerous opportunities for educational assessment. Investigating the potential of AI tools in practical educational assessment carries profound significance. This study explored the comparative…

Testing English writing skills could be multi-dimensional; thus, the study aimed to compare students' writing scores calculated according to Classical Test Theory (CTT) and Multi-Facet Rasch Model (MFRM). The research was carried out in 2019 with 100 university students studying at a foreign language preparatory class and four experienced…

Vocabulary knowledge plays an important role in writing. Previous research has examined the relationship between various aspects of vocabulary knowledge and independent writing performance and limited attention has been given to integrated writing such as summary writing. Our study investigated the contribution of two aspects of vocabulary…

This study conducts a comprehensive quantitative evaluation of OpenAI's language model, ChatGPT 4, for grading Task 2 writing of the IELTS exam. The objective is to assess the alignment between ChatGPT's grading and that of official human raters. The analysis encompassed a multifaceted approach, including a comparison of means and reliability…

Studies on automatic scoring systems in writing assessments have also evaluated the relationship between human and machine scores for the reliability of automated essay scoring systems. This study investigated the magnitudes of indices for inter-rater agreement and discrepancy, especially regarding human and machine scoring, in writing assessment.…

This study investigated the impact of generative AI-assisted writing feedback, specifically using Grammarly, on English as a Foreign Language (EFL) learners' writing proficiency, revision practices, and writing quality. Sixty postgraduate EFL students were randomly assigned to either an experimental group using Grammarly as an AI-enhanced feedback…

The first purpose of this empirical study was to assess and validate the Academic Writing Self-Efficacy Belief Questionnaire (AWSEBQ) framed by social cognitive theory. The second purpose was to evaluate the predictive effects of different aspects of self-efficacy beliefs on academic writing performance. Data were collected from 743 learners at a…

The quality of writing in a second language (L2) is one of the indicators of the level of proficiency for many college students to be eligible for departmental studies. Although certain software programs, such as Intelligent Essay Assessor or IntelliMetric, have been introduced to evaluate second-language writing quality, an overall assessment of…

Metaphors are well-known tools for teaching statistics to novices. However, educators might overlook metaphor theoretical developments that offer nuanced and testable perspectives on their pedagogical applications. This article introduces the notion of metaphor types--"correspondence" (CO) and "class inclusion" (CI)--as…

Research into metacognition has found it to facilitate self-regulation and correlate to learners' L2 writing level. Following Lee & Mak's (2018) framework of Metacognitive Instruction (MI) for L2 writing classrooms, this study applies Dynamic Assessment (DA) to writing MI (MI-DA) in a rural middle school EFL class in China. A one-semester…

This quasi-experimental research investigates the employment of a formative assessment platform aided by artificial intelligence in an English public speaking course. The platform integrates deep learning, automatic speech recognition, and automatic writing evaluation. It provides automated assessment and immediate feedback on speakers' public…

Development of reliable rubrics for educational intervention studies that address reading and writing skills is labor-intensive, and could benefit from an automated approach. We compare a main ideas rubric used in a successful writing intervention study to a highly reliable wise-crowd content assessment method developed to evaluate…

This study explored how task complexity, writing behaviors (i.e., pausing and revision behaviors), and writing performance (i.e., task completion, coherence and cohesion, language use, and expression and tone) influence and relate to each other. Thirty advanced-level Korean EFL undergraduates completed writing tasks differing in complexity. A…

In this article, we compare holistic, analytic and cross-linguistic measures of assessment in writing considering the whole linguistic repertoire of multilingual writers. We explore different ways to look at their three languages by focusing on the relationships between these languages rather than looking at each language in isolation. We analyse…