With the generative artificial intelligence (AI) tool's remarkable capabilities in understanding and generating meaningful content, intriguing questions have been raised about its potential as an automated essay scoring (AES) system. One such tool is ChatGPT, which is capable of scoring any written work based on predefined criteria. However,…

Technological Advances in Artificial Intelligence (AI) have Brought forth the Potential for Models to Assist in Academic Writing. However, Concerns Regarding the Accuracy, Reliability, and Impact of AI in Academic Writing have been Raised. This Study Examined the Capabilities of GPT-4, a state-of-the-art AI Language Model, in Writing an American…

Automatic essay scoring (AES) is an essential educational application in natural language processing. This automated process will alleviate the burden by increasing the reliability and consistency of the assessment. With the advances in text embedding libraries and neural network models, AES systems achieved good results in terms of accuracy.…

Rater comments tend to be qualitatively analyzed to indicate raters' application of rating scales. This study applied natural language processing (NLP) techniques to quantify meaningful, behavioral information from a corpus of rater comments and triangulated that information with a many-facet Rasch measurement (MFRM) analysis of rater scores. The…

This study investigates the potential of automated classification using prompt-based learning approaches with transformer models (large language models trained in an unsupervised manner) for a domain-specific classification task. Prompt-based learning with zero or few shots has the potential to (1) make use of artificial intelligence without…

This paper presents a multidimensional model of variation in writing quality, register, and genre in student essays, trained and tested via confirmatory factor analysis of 1.37 million essay submissions to ETS' digital writing service, Criterion®. The model was also validated with several other corpora, which indicated that it provides a…

This literature review delves into the applications and implications of ChatGPT, an artificial intelligence-driven chatbot, in educational settings. As technology continues to permeate the field of education, ongoing debates persist regarding the potential impact on learners' critical thinking skills and the evolving role of educators. Instead of…

Background: When learning causal relations, completing causal diagrams enhances students' comprehension judgements to some extent. To potentially boost this effect, advances in natural language processing (NLP) enable real-time formative feedback based on the automated assessment of students' diagrams, which can involve the correctness of both the…

This study conducts a comprehensive quantitative evaluation of OpenAI's language model, ChatGPT 4, for grading Task 2 writing of the IELTS exam. The objective is to assess the alignment between ChatGPT's grading and that of official human raters. The analysis encompassed a multifaceted approach, including a comparison of means and reliability…

We used multivariate generalizability theory to examine the reliability of hand-scoring and automated essay scoring (AES) and to identify how these scoring methods could be used in conjunction to optimize writing assessment. Students (n = 113) included subsamples of struggling writers and non-struggling writers in Grades 3-5 drawn from a larger…

At present, ChatGPT is penetrating all spheres of human activities, and especially education is no exception. The purpose of this exploratory study is to examine the potentials and pitfalls of using ChatGPT for academic purposes among university students, as well as provide relevant pedagogical implications for its use in academia. The methodology…

The purpose of this study is to examine the impact of artificial intelligence (AI) on online assessment in the context of opportunities and threats based on the literature. To this end, 19 articles related to the AI tool ChatGPT and online assessment were analysed through rapid literature review. In the content analysis, the themes of "AI's…

Children's oral language proficiency (OLP) is integral for developing literacy skills. Storytelling or retelling is often used by parents and educators to elicit children's OLP, yet it is less commonly used for assessment purposes. Leveraged by natural language processing and machine learning, this study examined the extent to which computational…

In this investigation we propose new machine learning methods for automated scoring models that predict the vocabulary acquisition in science and social studies of second grade English language learners, based upon free-form spoken responses. We evaluate performance on an existing dataset and use transfer learning from a large pre-trained language…

This study presents a new approach for transforming the latent representation derived from a Latent Semantic Analysis (LSA) space into one where dimensions have nonlatent meanings. These meanings are based on lexical descriptors, which are selected by the LSA user. The authors present three analyses that provide examples of the utility of this…