This study explores the writing quality of two AI chatbots, OpenAI ChatGPT and Google Gemini. The research assesses the quality of the generated texts based on five essay models using the T.E.R.A. software, focusing on ease of understanding, readability, and reading levels using the Flesch-Kincaid formula. Thirty essays were generated, 15 from…

Automatic essay scoring (AES) is an essential educational application in natural language processing. This automated process will alleviate the burden by increasing the reliability and consistency of the assessment. With the advances in text embedding libraries and neural network models, AES systems achieved good results in terms of accuracy.…

Background: Generative artificial intelligence (AI) represents a potentially powerful, time-saving tool for grading student essays. However, little is known about how AI-generated essay scores compare to human instructor scores. Objective: The purpose of this study was to compare the essay grading scores produced by AI with those of human…

Generative AI has become a quotidian discussion topic in many writing departments, and the conversations often focus on the negative aspects or the disruptions it has caused. A growing number of teachers and scholars, though, have embraced the new technology and welcomed it into their classrooms. In the Spring 2024 semester, students in my…

As the development and application of large language models (LLMs) in physics education progress, the well-known AI-based chatbot ChatGPT4 has presented numerous opportunities for educational assessment. Investigating the potential of AI tools in practical educational assessment carries profound significance. This study explored the comparative…

Background: The emergence of artificial intelligence (AI) in higher education has sparked numerous discussions about its implications. ChatGPT, a prominent AI conversational model, has attracted significant attention for its ability to generate essays and formulate responses. Objective: The current study sought to explore how and why students are…

This study investigates the validity and reliability of generative large language models (LLMs), specifically ChatGPT and Google's Bard, in grading student essays in higher education based on an analytical grading rubric. A total of 15 experienced English as a foreign language (EFL) instructors and two LLMs were asked to evaluate three student…

Broadly defined, academic language (AL) is a set of lexical-grammatical norms and registers commonly used in educational and academic discourse. Mastery of academic language in writing is an important aspect of writing instruction and assessment. The purpose of this study was to use Natural Language Processing (NLP) tools to examine the extent to…

This article examined student experiences before and after an essay writing assignment that required the use of ChatGPT within an undergraduate engineering course. Utilizing a pre-post study design, we gathered data from 24 participants to evaluate ChatGPT's support for both completing and grading an essay assignment, exploring its educational…

The aim of this project was to understand student perspectives on generative artificial intelligence (GAI) technologies such as Chat generative Pre-Trained Transformer (ChatGPT), in order to inform changes to the University of Liverpool Academic Integrity code of practice. The survey for this study was created by a library student team and vetted…

Manually scoring and revising student essays has long been a time-consuming task for educators. With the rise of natural language processing techniques, automated essay scoring (AES) and automated essay revising (AER) have emerged to alleviate this burden. However, current AES and AER models require large amounts of training data and lack…

In a randomized trial that collects text as an outcome, traditional approaches for assessing treatment impact require that each document first be manually coded for constructs of interest by human raters. An impact analysis can then be conducted to compare treatment and control groups, using the hand-coded scores as a measured outcome. This…

E-learning systems are transforming the educational sector and making education more affordable and accessible. Recently, many e-learning systems have been equipped with advanced technologies that facilitate the roles of educators and increase the efficiency of teaching and learning. One such technology is Automatic Essay Grading (AEG) or…

Despite the growing interest in AI for language assessment, there remains a significant research gap regarding its usefulness for assessing less proficient language skills, particularly those of learners of English as a second or foreign language (S/FL). AI models often prioritize proficient writing, neglecting the intricacies of learner language.…

In university settings, writing argumentative essays from reading conflicting source texts is a common task for students. In performing this synthesis task, they must deal with conflicting claims about a controversial issue as they develop their own positions. Argumentative synthesis is characterized by writers' back-and-forth moves between…