In a randomized trial that collects text as an outcome, traditional approaches for assessing treatment impact require that each document first be manually coded for constructs of interest by human raters. An impact analysis can then be conducted to compare treatment and control groups, using the hand-coded scores as a measured outcome. This…

This study explores using transfer learning in machine learning for natural language processing (NLP) to create generic automated essay scoring (AES) models, providing instant online scoring for statewide writing assessments in K-12 education. The goal is to develop an instant online scorer that is generalizable to any prompt, addressing the…

English writing and critical thinking are not only fundamental to academic success but also play a crucial role for the holistic development of primary students. This study investigated the effects of a cloud-based gamified instruction integrating peer assessment on primary student English writing and critical thinking. A pre- and post-test with a…

We investigated spelling errors in English and Spanish essays by Spanish-English dual language learners in Grades 1, 2, and 3 (N = 278; 51% female) enrolled in either English immersion or English-Spanish dual immersion programs. We examined what types of spelling errors students made, whether they made spelling errors that could be due to…

Automated Writing Evaluation (AWE) provides automatic writing feedback and scoring to support student writing and revising. The purpose of the present study was to analyze a statewide implementation of an AWE software (n = 114,582) in grades 4-11. The goals of the study were to evaluate: (1) to what extent AWE features were used; (2) if equity and…

Automated Essay Scoring (AES) can reliably grade essays at scale and reduce human effort in both classroom and commercial settings. There are currently three dominant supervised learning paradigms for building AES models: feature-based, neural, and hybrid. While feature-based models are more explainable, neural network models often outperform…

This report sets out the findings of the technical development work completed as part of the Use Cases for Generative AI in Education project, commissioned by the Department for Education (DfE) in September 2023. It has been published alongside the User Research Report, which sets out the findings from the ongoing user engagement activity…

This study examined a naturalistic, districtwide implementation of an automated writing evaluation (AWE) software program called "MI Write" in elementary schools. We specifically examined the degree to which aspects of MI Write were implemented, teacher and student attitudes towards MI Write, and whether MI Write usage along with other…

This study examined a naturalistic, districtwide implementation of an automated writing evaluation (AWE) software program called "MI Write" in elementary schools. We specifically examined the degree to which aspects of MI Write were implemented, teacher and student attitudes towards MI Write, and whether MI Write usage along with other…

We investigated spelling errors in English and Spanish essays by Spanish-English dual language learners in Grades 1, 2, and 3 (N = 278; 51% female) enrolled in either English immersion or English-Spanish dual immersion programs. We examined what types of spelling errors students made, whether they made spelling errors that could be due to…

This study reports the integrated findings of a mixed-methods sequential explanatory study that investigated the feasibility and effectiveness of mobile-based writing tools (MBWTs) as well as gender differences that emerged from these learning effects in the narrative writing skills of a group of sixth to seventh grade Latinx English Learners…

The assessment of writing competency in French as a Foreign Language presents significant challenges due to the multidimensional nature of writing and heterogeneous evaluation practices. This research introduces an innovative framework integrating artificial intelligence and interactive visualization to support the formative assessment of writing…

We investigate students' implementation of the feedback messages they received in an automated writing evaluation system ("eRevise") that aims to improve students' use of text evidence in their writing. Seven 5th and 6th-grade teachers implemented "eRevise" (n = 143 students). Qualitative analysis of students' essays across…

Storytelling is universal and is popular in many cultures where it is a medium people use to preserve their beliefs, social values, wisdom, and cultural experiences as well as transfer them from one generation to another. In American Samoa, over 90% of students are classified as English Language Learners (ELLs) and literacy rates are very low,…

The present study used a focus group methodology to qualitatively explore elementary writing teachers' attitudes and experiences using an automated writing evaluation (AWE) system called MI Write as part of a districtwide implementation of MI Write in Grades 3-5 in 14 elementary schools. We used activity theory as a theoretical framework to answer…