ChatGPT has been demonstrated to possess significant capabilities in generating intricate human-like text, and recent studies have established that its performance in theory of mind (ToM) tasks is strikingly comparable to a nine-year-old child's. However, it remains unknown whether ChatGPT outperforms children of this age group in Chinese writing,…

This study explores the writing quality of two AI chatbots, OpenAI ChatGPT and Google Gemini. The research assesses the quality of the generated texts based on five essay models using the T.E.R.A. software, focusing on ease of understanding, readability, and reading levels using the Flesch-Kincaid formula. Thirty essays were generated, 15 from…

Automatic essay scoring (AES) is an essential educational application in natural language processing. This automated process will alleviate the burden by increasing the reliability and consistency of the assessment. With the advances in text embedding libraries and neural network models, AES systems achieved good results in terms of accuracy.…

Background: Generative artificial intelligence (AI) represents a potentially powerful, time-saving tool for grading student essays. However, little is known about how AI-generated essay scores compare to human instructor scores. Objective: The purpose of this study was to compare the essay grading scores produced by AI with those of human…

Reflective practice holds critical importance, for example, in higher education and teacher education, yet promoting students' reflective skills has been a persistent challenge. The emergence of revolutionary artificial intelligence technologies, notably in machine learning and large language models, heralds potential breakthroughs in this domain.…

Self-Regulated Strategy Development (SRSD) is a method designed to explicitly teach students self-regulation skills and help them implement them. While SRSD's beneficial effect on writing quality has been confirmed in a large body of literature, the questions of why, how, and for whom SRSD works remain unclear. To investigate the underlying…

Writing is hard for students who are learning English; they often find it challenging to transform what is on their mind in writing. Therefore, this study examined the relationship between the writing test and assessment writing through argumentative writing. Data was analyzed using the correlation test to determine the close relationship between…

As the development and application of large language models (LLMs) in physics education progress, the well-known AI-based chatbot ChatGPT4 has presented numerous opportunities for educational assessment. Investigating the potential of AI tools in practical educational assessment carries profound significance. This study explored the comparative…

The impact of self- and peer-assessment strategies (assessment as learning) on the development of writing skill of EFL learners is a research area requiring more research studies. The present study aimed to investigate the impact of language assessment literacy enhancement (LALE) on Iranian high school English as Foreign Language (EFL) students'…

This study investigates the validity and reliability of generative large language models (LLMs), specifically ChatGPT and Google's Bard, in grading student essays in higher education based on an analytical grading rubric. A total of 15 experienced English as a foreign language (EFL) instructors and two LLMs were asked to evaluate three student…

Comparative Judgement (CJ) is an evaluation method, typically conducted online, whereby a rank order is constructed, and scores calculated, from judges' pairwise comparisons of performances. CJ has been researched in various educational contexts, though only rarely in English as a Foreign Language (EFL) writing settings, and is generally agreed to…

Manually scoring and revising student essays has long been a time-consuming task for educators. With the rise of natural language processing techniques, automated essay scoring (AES) and automated essay revising (AER) have emerged to alleviate this burden. However, current AES and AER models require large amounts of training data and lack…

While extensive research exists on peer feedback and its effects on writing, there are few experimental studies that rigorously investigate the effects of guided dialogic peer feedback on students' argumentative writing performance. This study, adopting a mixed-methods approach, examined the influence of guided dialogic peer feedback within a…

In Chinese, writing is a basic competency that pupils should possess. But it is still challenging for teachers to improve pupils' writing abilities. Therefore, this study proposes an intelligence-based cognitive diagnostic feedback strategy to improve pupils' writing ability and writing learning quality by analyzing their writing performance,…

In a randomized trial that collects text as an outcome, traditional approaches for assessing treatment impact require that each document first be manually coded for constructs of interest by human raters. An impact analysis can then be conducted to compare treatment and control groups, using the hand-coded scores as a measured outcome. This…