Existing approaches to measuring writing performance are insufficient in terms of both technical adequacy as well as feasibility for use as a screening measure. This study examined the validity and diagnostic accuracy of several approaches to automated text evaluation as well as written expression curriculum-based measurement (WE-CBM) to determine…

Compared to other content areas, there is a dearth of research examining curriculum-based measurement of writing (CBM-W). This study conducted a conceptual replication examining the reliability, stability, and sensitivity to growth of slopes produced from CBM-W. Eighty-nine (N = 89) eighth-grade students responded to one CBM-W probe weekly for 11…

In automated essay scoring (AES), scores are automatically assigned to essays as an alternative to grading by humans. Traditional AES typically relies on handcrafted features, whereas recent studies have proposed AES models based on deep neural networks to obviate the need for feature engineering. Those AES models generally require training on a…

Existing approaches to measuring writing performance are insufficient in terms of both technical adequacy as well as feasibility for use as a screening measure. This study examined the validity and diagnostic accuracy of several approaches to automated text evaluation as well as written expression curriculum-based measurement (WE-CBM) to determine…

While much research supports the benefits of computer-mediated collaborative writing (CW) in second language (L2) classrooms, the assessment of CW has received scant attention. This study proposed an assessment scheme considering both the products and processes when assessing online synchronous CW, and explored its effects on learners'…

As integrated writing tasks in large-scale and classroom-based writing assessments have risen in popularity, research studies have increasingly concentrated on providing validity evidence. Given the fact that most of these studies focus on adult second language learners rather than younger ones, this study examined the relationship between written…

With the burgeoning popularity and swift advancements of automated writing evaluation (AWE) systems in language classrooms, scholarly and practical interest in this area has noticeably increased. This systematic review aims to comprehensively investigate current research on three prominent AWE systems: Grammarly, Pigai, and Criterion. Objectives…

While studies on teacher written feedback and automated writing evaluation (AWE) feedback have proliferated in recent decades, little attention has been paid to how AWE-teacher integrated feedback would influence students' engagement and their writing performance in second language (L2) writing. Against this backdrop, a quasi-experimental design…

The purpose of this study is to investigate how the US-based First-Year Composition (FYC) instructors understand and facilitate metacognition in their classes and assess students' metacognition through exploratory, mixed methods approaches. I argue that even if we understand the importance of metacognition generally for student populations, we…

The purpose of this study was to explore the nature of interactions among second/foreign language (L2) writing subskills. Two types of relationships were investigated: subskill-item and subskill-subskill relationships. To achieve the first purpose, using writing data obtained from the writing essays of 500 English as a foreign language (EFL)…

Manually scoring and revising student essays has long been a time-consuming task for educators. With the rise of natural language processing techniques, automated essay scoring (AES) and automated essay revising (AER) have emerged to alleviate this burden. However, current AES and AER models require large amounts of training data and lack…

What do writers do with the feedback they receive? While the answer will vary depending on the writer's experience and the rhetorical situation, understanding what writers do can provide important information for course redesign and professional development of tutors and instructors. In this first of two manuscripts, the authors examine how…

This paper explores the integration of Generative Artificial Intelligence (GenAI) feedback into higher education. Specifically, it examines the views of 11 experienced instructors on fine-tuned GenAI formative feedback of student works in an online graduate program in the United States. The participants assessed sample GenAI reviews, and their…

Checking essays written by students is a very time consuming task. Besides spellings and grammar, they also need to be evaluated on their semantic content such as cohesion, coherence, etc. In this study we focus on one such aspect of semantic content which is the topic of the essay. Putting it formally, given a prompt or essay-statement and an…

Testing English writing skills could be multi-dimensional; thus, the study aimed to compare students' writing scores calculated according to Classical Test Theory (CTT) and Multi-Facet Rasch Model (MFRM). The research was carried out in 2019 with 100 university students studying at a foreign language preparatory class and four experienced…