An increasing number of language testing companies are developing and deploying deep learning-based automated essay scoring systems (AES) to replace traditional approaches that rely on handcrafted feature extraction. However, there is hesitation to accept neural network approaches to automated essay scoring because the features are automatically…

This study explores the potential of OpenAI's ChatGPT-4 (gpt-4-0613) as an automated essay scoring (AES) tool in a trial involving 300 essays from an American university's academic English program placement test. Three prompting strategies (minimal/detailed rubric, require/not require rationale, and with/without scoring examples) were tested for…

Information units (IUs) have been proposed as a way of grading students' notes. However, little work has been done to validate and check the reliability of such an approach. This study explores how teacher-researchers (TRs) use information units (IUs) to rate content items in students' listening notes using a rubric. Data were collected from six…

Machine learning-based automatic scoring faces challenges with imbalanced student responses across scoring categories. To address this, we introduce a novel text data augmentation framework that leverages GPT-4, a generative large language model specifically tailored for imbalanced datasets in automatic scoring. Our experimental dataset consisted…

This column describes the game of Royal Pickles -- a "King of the Court'' variation of pickleball that removes the bored spectator element from the class and creates a solution for the challenge of having too many students and not enough court space.

The purpose of this study was to investigate which of the available Spanish sentence tests U.S. audiologists currently utilize to evaluate Spanish-speaking cochlear implant candidates. An online questionnaire was created and distributed nationwide. A total of 25 audiologists reported using the Spanish HINT and Spanish AzBio. Limitations regarding…

This paper describes the development of the "Flourishing Classroom System Observation Framework and Rubric," which provides a framework and practical approach to defining and describing multiple interconnected observable characteristics of a classroom system that individually and together can be targeted to cultivate collective…

Generative artificial intelligence technologies, such as ChatGPT, bring an unprecedented change in education by leveraging the power of natural language processing and machine learning. Employing ChatGPT to assist with marking written assessment presents multiple advantages including scalability, improved consistency, eliminating biases associated…

It is widely recognised that feedback is an important part of learning: effective feedback should result in a meaningful change in student behaviour (Morris et al., 2021). However, individual feedback takes time to produce, and for large cohorts -- typified by the North of 300 challenge in computing (CPHC, 2019), it can be difficult to do so in a…

Multiple-choice questions have become ubiquitous in educational measurement because the format allows for efficient and accurate scoring. Nonetheless, there remains continued interest in constructed-response formats. This interest has driven efforts to develop computer-based scoring procedures that can accurately and efficiently score these items.…

Quantitative measures of systematic differences in OSCE scoring across examiners (often termed examiner stringency) can threaten the validity of examination outcomes. Such effects are usually conceptualised and operationalised based solely on checklist/domain scores in a station, and global grades are not often used in this type of analysis. In…

Concept mapping is widely used as a tool for assessing students' understanding of science. To fully realize the diagnostic potential of concept mapping, a scoring method that not only provides an objective and accurate assessment of students' drawn concept maps but also provides a detailed understanding of students' proficiency and deficiencies in…

We investigated the relationship between the scores assigned by an Automated Essay Scoring (AES) system, the Intelligent Essay Assessor (IEA), and grades allocated by trained, professional human raters to English essay writing by instigating two procedures novel to written-language assessment: the logistic transformation of AES raw scores into…

In automated essay scoring (AES) systems, similarity techniques are used to compute the score for student answers. Several methods to compute similarity have emerged over the years. However, only a few of them have been widely used in the AES domain. This work shows the findings of a ten-year review on similarity techniques applied in AES systems…

In this article, we argue that automated scoring engines should be transparent and construct relevant--that is, as much as is currently feasible. Many current automated scoring engines cannot achieve high degrees of scoring accuracy without allowing in some features that may not be easily explained and understood and may not be obviously and…