This study explores the writing quality of two AI chatbots, OpenAI ChatGPT and Google Gemini. The research assesses the quality of the generated texts based on five essay models using the T.E.R.A. software, focusing on ease of understanding, readability, and reading levels using the Flesch-Kincaid formula. Thirty essays were generated, 15 from…

One of the challenges of discussing validity arguments for machine scoring of essays centers on the absence of a commonly held definition and theory of good writing. At best, the algorithms attempt to measure select attributes of writing and calibrate them against human ratings with the goal of accurate prediction of scores for new essays.…

As the development and application of large language models (LLMs) in physics education progress, the well-known AI-based chatbot ChatGPT4 has presented numerous opportunities for educational assessment. Investigating the potential of AI tools in practical educational assessment carries profound significance. This study explored the comparative…

Background: The emergence of artificial intelligence (AI) in higher education has sparked numerous discussions about its implications. ChatGPT, a prominent AI conversational model, has attracted significant attention for its ability to generate essays and formulate responses. Objective: The current study sought to explore how and why students are…

The proliferation of large language models represents a paradigm shift in the landscape of automated essay scoring (AES) systems, fundamentally elevating their accuracy and efficacy. This study presents an extensive examination of large language models, with a particular emphasis on the transformative influence of transformer-based models, such as…

To test the predictive power of the SSARC (stabilize, simplify, automatize, reconstruct, and complexify) model of pedagogic task sequencing in second language (L2) writing development, the present study explores the performance of written decision-making tasks with varied levels of cognitive complexity in a simple-to-complex sequence in comparison…

Second language writing researchers have examined the affordances of Automated Writing Evaluation programs in providing immediate feedback that helps improve students' writing outputs. However, a little is known about tracking learners' process during writing essays and whether much/less pauses made by learners could predict good/poor quality of…

Contract cheating is currently a concern for universities and the higher education (HE) sector. It has been brought into the spotlight in recent years through the growth of online essay mills, where students can easily commission and purchase written assessment responses. This study contributes to the wider literature on academic integrity in HE…

Cohesion features were calculated for a corpus of 960 essays by 480 test-takers from the Test of English as a Foreign Language (TOEFL) in order to examine differences in the use of cohesion devices between integrated (source-based) writing and independent writing samples. Cohesion indices were measured using an automated textual analysis tool, the…

Since researchers investigated automatic scoring systems in writing assessments, they have dealt with relationships between human and machine scoring, and then have suggested evaluation criteria for inter-rater agreement. The main purpose of my study is to investigate the magnitudes of and relationships among indices for inter-rater agreement used…

The assessment of argumentative writing generally includes analyses of the specific linguistic and rhetorical features contained in the individual essays produced by students. However, researchers have recently proposed that an individual's ability to flexibly adapt the linguistic properties of their writing may more accurately capture their…

However influential the interdependence hypothesis has become in bilingual research, it still lacks full empirical support. This longitudinal study explores the parallels in the biliteracy development (L1 Spanish and L2 English) of 20 students in a European immersion programme (i.e. CLIL) over a two-year period. A bilingual learner corpus of…

No significant body of research examines writing achievement and the specific skills and knowledge in the writing domain for postsecondary (college) students in the U.S., even though many at-risk students lack the prerequisite writing skills required to persist in their education. This paper addresses this gap through a novel…

In this research, we investigated the feasibility of implementing the "e-rater"® scoring engine as a check score in place of all-human scoring for the "Graduate Record Examinations"® ("GRE"®) revised General Test (rGRE) Analytical Writing measure. This report provides the scientific basis for the use of e-rater as a…

Automated essay scoring tools are often criticized on the basis of construct validity. Specifically, it has been argued that computational scoring algorithms may be unaligned to higher-level indicators of quality writing, such as writers' demonstrated knowledge and understanding of the essay topics. In this paper, we consider how and whether the…