The current educational system relies heavily on manual grading, posing challenges such as delayed feedback and grading inaccuracies. Automated grading tools (AGTs) offer solutions but come with limitations. To address this, "GRAD-AI" is introduced, an advanced AGT that combines automation with teacher involvement for precise grading,…

Over the past decade, social scientists have adapted computational methods for qualitative text analysis, with the hope that they can match the accuracy and reliability of hand coding. The emergence of GPT and open-source generative large language models (LLMs) has transformed this process by shifting from programming to engaging with models using…

Background: Analyzing student short-answer written justifications to conceptually challenging questions has proven helpful to understand student thinking and improve conceptual understanding. However, qualitative analyses are limited by the burden of analyzing large amounts of text. Purpose: We apply dense and sparse Large Language Models (LLMs)…

The ability of large language models (LLMs) to generate code has raised concerns in computer science education, as students may use tools like ChatGPT for programming assignments. While much research has focused on higher education, especially for languages like Java and Python, little attention has been given to K-12 settings, particularly for…

Background: Assessing spoken discourse during aphasia clinical examination is crucial for diagnostic and rehabilitation purposes. Recent approaches have been developed to quantify content word fluency (CWF) and informativeness of spoken discourse without the need to perform time-consuming transcription and coding. However, the accuracy of these…

Background: High-quality feedback is crucial for academic success, driving student motivation and engagement while research explores effective delivery and student interactions. Advances in artificial intelligence (AI), particularly natural language processing (NLP), offer innovative methods for analyzing complex qualitative data such as feedback…

Background: In the context of large-scale educational assessments, the effort required to code open-ended text responses is considerably more expensive and time-consuming than the evaluation of multiple-choice responses because it requires trained personnel and long manual coding sessions. Aim: Our semi-supervised coding method eco (exploring…

Background: Non-word repetition (NWR) tests are an important way speech and language therapists (SaLTs) assess language development. NWR tests are often scored whilst participants make their responses (i.e., in real time) in clinical and research reports (documented here via a secondary analysis of a published systematic review). Aims: The main…

Telephone survey interviewers need to be able to accurately record answers to questions. While straightforward for closed questions, this task can be complicated for open questions. We examine interviewer recording accuracy rates from a national landline random digit dial telephone survey. We find that accuracy rates are over 90% for numeric…

This study aims to evaluate ChatGPT's capabilities in certain numerical analysis problem: solving ordinary differential equations. The methodology which is developed in order to conduct this research takes into account the following mathematical abilities (defined according to National Centre for Education Statistics): Conceptual Understanding,…

The effectiveness of written corrective feedback, WCF, has been much disputed even till the present day. Various strategies of WCF are still being developing with the aim to enhance students' writing performance especially in the English language. Coded corrective feedback, CCF, is classified as an indirect WCF method, and the studies of CCF are…

A methodologically sound systematic review is characterized by transparency, replicability, and a clear inclusion criterion. However, little attention has been paid to reporting the details of interrater reliability (IRR) when multiple coders are used to make decisions at various points in the screening and data extraction stages of a study. Prior…

Purpose: A major barrier to the wider use of language sample analysis (LSA) is the fact that transcription is very time intensive. Methods that can reduce the required time and effort could help in promoting the use of LSA for clinical practice and research. Method: This article describes an automated pipeline, called Batchalign, that takes raw…

Researchers in second language (L2) and education domain use different statistical methods to assess their constructs of interest. Many L2 constructs emerge from elements/parts, i.e., the elements "define" and "form" the construct and not the other way around. These constructs are referred to as emergent variables (also called…

Advances in computer science and computational linguistics have yielded new, and faster, computational approaches to structuring and analyzing textual data. These approaches perform well on tasks like information extraction, but their ability to identify complex, socially constructed, and unsettled theoretical concepts--a central goal of…