Recent developments in the use of large-language models have led to substantial improvements in the accuracy of content-based automated scoring of free-text responses. The reported accuracy levels suggest that automated systems could have widespread applicability in assessment. However, before they are used in operational testing, other aspects of…

Online assessment is an essential part of online education, and if conducted properly, has been found to effectively gauge student learning. Generally, textbased questions have been the cornerstone of online assessment. Recently, however, the emergence of generative artificial intelligence has added a significant challenge to the integrity of…

The analysis of spoken language has been integral to a breadth of research in social science and beyond. However, for analyses to occur with efficiency, language must be in the form of computer-readable text. Historically, the speech-to-text process has occurred manually using human transcriptionists. Automated speech recognition (ASR) is…

Purpose: ParAlg (Paraphasia Algorithms) is a software that automatically categorizes a person with aphasia's naming error (paraphasia) in relation to its intended target on a picture-naming test. These classifications (based on lexicality as well as semantic, phonological, and morphological similarity to the target) are important for…

This study explores using transfer learning in machine learning for natural language processing (NLP) to create generic automated essay scoring (AES) models, providing instant online scoring for statewide writing assessments in K-12 education. The goal is to develop an instant online scorer that is generalizable to any prompt, addressing the…

This study seeks to clarify whether teachers are able to distinguish between essays written by English L2 students or generated by ChatGPT. 47 instructors who hold experience teaching English to native speakers of Japanese in universities or other higher education institutions were tested on whether they could identify between human written essays…

Students' writing proficiency is measured through holistic and analytical ratings in writing assessment; however, recent studies suggest that measurement of syntactic complexity in second language writing research has become an effective measure of writing proficiency. Within this paradigm, we investigated how automated measurement of syntactic…

Massive pretrained language models have garnered attention and controversy due to their ability to generate humanlike responses: Attention due to their frequent indistinguishability from human-generated phraseology and narratives and controversy due to the fact that their convincingly presented arguments and facts are frequently simply false. Just…

This study explores how Korean-to-English machine translation (MT) systems (e.g., Google Translator, NAVER Papago) deal with Korean passive structures. Cross-linguistically, Korean and English passives show different ways to construct passive-voice sentences from active structure. English passives including with [to be + past participle] may have…

We investigated spelling errors in English and Spanish essays by Spanish-English dual language learners in Grades 1, 2, and 3 (N = 278; 51% female) enrolled in either English immersion or English-Spanish dual immersion programs. We examined what types of spelling errors students made, whether they made spelling errors that could be due to…

Mass adoption of neural machine translation (NMT) tools in the translation workflow has exerted a significant impact on the language services industry over the last decade. There are claims that with the advent of NMT, automated translation has reached human parity for translating news (see, e.g. Popel et al. 2020). Moreover, some machine…

Despite considerable typographical innovations over the past twenty years that have enabled and facilitated typing capabilities for many Indigenous language orthographies, typographical errors continue to disproportionately affect Indigenous languages. These include errors in glyph shapes, which impact legibility, and issues with glyph…

This report sets out the findings of the technical development work completed as part of the Use Cases for Generative AI in Education project, commissioned by the Department for Education (DfE) in September 2023. It has been published alongside the User Research Report, which sets out the findings from the ongoing user engagement activity…

Large language models (LLMs) offer an opportunity to make large-scale changes to educational content that would otherwise be too costly to implement. The work here highlights how LLMs (in particular GPT-4) can be prompted to revise educational math content ready for large scale deployment in real-world learning environments. We tested the ability…

This research study aims to explore students' perceptions and reflections on the effectiveness of project-based learning (PBL) in a Thai-English translation course in the EFL context. The participants of this study included forty-nine students who were enrolled in the Thai-English translation course. The participants were asked to produce a…