This study investigates the validity and reliability of generative large language models (LLMs), specifically ChatGPT and Google's Bard, in grading student essays in higher education based on an analytical grading rubric. A total of 15 experienced English as a foreign language (EFL) instructors and two LLMs were asked to evaluate three student…

[This paper is part of the Focused Collection in Artificial Intelligence Tools in Physics Teaching and Physics Education Research.] Using a high-stakes thermodynamics exam as the sample (252 students, four multipart problems), we investigate the viability of four workflows for AI-assisted grading of handwritten student solutions. We find that the…

Due to the introduction and rapid ubiquity of artificial intelligence (AI) and AI-integrated programs that can be used by students and teachers, educational scholarship evaluating the capabilities of AI is needed. This study evaluates the abilities of three prominent AI programs--ChatGPT, Microsoft's Bing, and Google's Bard--to create high school…

This study aims to develop a robust rubric for evaluating artificial intelligence (AI)--assisted essay writing in English as a Foreign Language (EFL) contexts. Employing a modified Delphi technique, we conducted a comprehensive literature review and administered Likert scale questionnaires. This process yielded nine key evaluation criteria,…

This article outlines the a educational design for Digital Ensemble, an innovative approach to English assessment integrating drama pedagogy with mobile computing (e.g. ad). a represents the key themes that framed and informed the research: ensemble, narrative, collaboration and technology. Starting with a as a prototype concept design for the…

Current literature provides many examples of rubrics that are used to evaluate the quality of web-quest designs. However, reliability of these rubrics has not yet been researched. This is the first study to fully characterize and assess the reliability of a webquest evaluation rubric. The ZUNAL rubric was created to utilize the strengths of the…

Over the past several decades, a wider range of assessment strategies has gained prominence in classrooms, including complex assessment items such as individual or group projects, student journals and other creative writing tasks, graphic/artistic representations of knowledge, clinical interviews, student presentations and performances, peer- and…