Evaluating the impact of Academic Literacies teaching (Lea and Street [1998. "Student Writing in Higher Education: An Academic Literacies Approach." "Studies in Higher Education" 23 (2): 157-72. doi:10.1080/03075079812331380364]) is difficult, as it involves gauging whether writers: (1) gain better understanding of what…

The proliferation of large language models represents a paradigm shift in the landscape of automated essay scoring (AES) systems, fundamentally elevating their accuracy and efficacy. This study presents an extensive examination of large language models, with a particular emphasis on the transformative influence of transformer-based models, such as…

There is significant overlap between reading and writing, but no known standardized measure assesses these jointly. The goal of the present study is to evaluate the properties of a novel measure, the Assessment of Writing, Self-Monitoring, and Reading (AWSM Reader), that simultaneously evaluates both reading comprehension and writing. In doing so,…

Research into metacognition has found it to facilitate self-regulation and correlate to learners' L2 writing level. Following Lee & Mak's (2018) framework of Metacognitive Instruction (MI) for L2 writing classrooms, this study applies Dynamic Assessment (DA) to writing MI (MI-DA) in a rural middle school EFL class in China. A one-semester…

Development of reliable rubrics for educational intervention studies that address reading and writing skills is labor-intensive, and could benefit from an automated approach. We compare a main ideas rubric used in a successful writing intervention study to a highly reliable wise-crowd content assessment method developed to evaluate…

Little research has been done on the effects of peer raters' quality characteristics on peer rating qualities. This study aims to address this gap and investigate the effects of key variables related to peer raters' qualities, including content knowledge, previous rating experience, training on rating tasks, and rating motivation. In an experiment…

In this research report, we present an empirical argument for the use of a contributory scoring approach for the 2-essay writing assessment of the analytical writing section of the "GRE"® test in which human and machine scores are combined for score creation at the task and section levels. The approach was designed to replace a currently…

This paper presents a method for equating writing assessments using pairwise comparisons which does not depend upon conventional common-person or common-item equating designs. Pairwise comparisons have been successfully applied in the assessment of open-ended tasks in English and other areas such as visual art and philosophy. In this paper,…

The importance of functional adequacy as an essential component of L2 proficiency has been observed by several authors (Pallotti, 2009; De Jong, Steinel, Florijn, Schoonen, & Hulstijn, 2012a, b). The rationale underlying the present study is that the assessment of writing proficiency in L2 is not fully possible without taking into account the…

In this report, we examine the feasibility of characterizing writing performance using process features derived from a keystroke log. Using data derived from a set of "CBAL"™ writing assessments, we examine the following research questions: (a) How stable are the keystroke timing and process features across testing occasions?; (b) How…

Writing assessments have taken two primary forms in the past two decades: direct and indirect. Irrespective of type, either form needs to be anchored to making decisions in the classroom and predicting performance on high-stakes tests, particularly in a high-stakes environment with serious consequences. In this study, 11th-grade students were…

Scoring models for the "e-rater"® system were built and evaluated for the "TOEFL"® exam's independent and integrated writing prompts. Prompt-specific and generic scoring models were built, and evaluation statistics, such as weighted kappas, Pearson correlations, standardized differences in mean scores, and correlations with…

This paper presents a socio-cognitive framework for connecting writing pedagogy and writing assessment with modern social and cognitive theories of writing. It focuses on providing a general framework that highlights the connections between writing competency and other literacy skills; identifies key connections between literacy instruction,…

An objectively scored assessment of basic writing skills was used systematically in several courses throughout the marketing curriculum at a private university. Results indicate that repeated testing on editing skills with feedback will lead to repeated improvements in skills and that these skills tend to be recalled for substantial periods of…

Congenitally, profoundly deaf children whose first language is British Sign Language (BSL) and whose speech is largely unintelligible need to be literate to communicate effectively in a hearing society. Both spelling and writing skills of such children can be limited, to the extent that no currently available assessment method offers an adequate…