Multilingual summative assessments face significant challenges due to tensions that exist between multiple language provision and comparability. Yet, conventional approaches for investigating comparability in multilingual assessments fail to accommodate assessments that comprise extended responses that target complex constructs. This article…

In a randomized trial that collects text as an outcome, traditional approaches for assessing treatment impact require that each document first be manually coded for constructs of interest by human raters. An impact analysis can then be conducted to compare treatment and control groups, using the hand-coded scores as a measured outcome. This…

E-learning systems are transforming the educational sector and making education more affordable and accessible. Recently, many e-learning systems have been equipped with advanced technologies that facilitate the roles of educators and increase the efficiency of teaching and learning. One such technology is Automatic Essay Grading (AEG) or…

Comparative Judgement (CJ) is an increasingly widely investigated method in assessment for creating a scale, for example of the quality of essays. One area that has attracted attention in CJ studies is the optimisation of the selection of pairs of objects for judgement. One approach is known as adaptive comparative judgement (ACJ). It has been…

In this article, I report on examiners' views and experiences of using Pairwise Comparative Judgement (PCJ) and Rank Ordering (RO) as alternatives to traditional analytical marking for GCSE English Language essays. Fifteen GCSE English Language examiners took part in the study. After each had judged 100 pairs of essays using PCJ and eight packs of…

Writing comprehensibly, dynamically, and persuasively in a foreign language is a significant challenge for learners. Written communication assesses language and writing progress for various summative and developmental goals. Scenario-based assessment (SBA), as one of the methodologies of classroom-based assessment (CBA), aims to elicit both…

In this work, we explore the use of digital technologies and statistical analysis to monitor how Italian secondary school students' writing changes over time and how comparisons can be made across different high school types. We analyzed more than 2,000 exam essays written by Italian high school students over 13 years and in five different school…

The purpose of the present study was to investigate the comparative effect of dynamic and negotiated assessment on EFL learners' writing complexity and fluency. To this end, 72 female intermediate EFL participants, selected from a larger group of 103 learners based on their performances on a piloted PET, in Tak language institute in Dezfoul, Iran…

Recent years have witnessed an increased interest in automated writing evaluation (hereafter AWE). However, few studies have examined the use of AWE with apprehensive writers. Hence, this study extends research in this area, investigating the effect of using AWE on reducing writing apprehension and enhancing grammatical knowledge. The participants…

Little research has been done on the effects of peer raters' quality characteristics on peer rating qualities. This study aims to address this gap and investigate the effects of key variables related to peer raters' qualities, including content knowledge, previous rating experience, training on rating tasks, and rating motivation. In an experiment…

Automated Essay Scoring (AES) has garnered a great deal of attention from the rhetoric and composition/writing studies community since the Educational Testing Service began using e-rater[R] and the "Criterion"[R] Online Writing Evaluation Service as products in scoring writing tests, and most of the responses have been negative. While the…

After passage of the Every Student Succeeds Act (ESSA) in 2015, states assumed greater responsibility for designing their own accountability and assessment systems. ESSA requires states to measure "higher order thinking skills and understanding" and encourages the use of open-ended performance assessments, which are essential for…

The Every Student Succeeds Act (ESSA) opened up new possibilities for how student and school success are defined and supported in American public education. States have greater responsibility for designing and building their assessment and accountability systems. These new opportunities to develop performance assessments are critically important…

We implemented two different approaches of using peer review to support undergraduate essay assignments for students taking large second-year courses in life sciences and biology: a web-based online peer review (OPR) approach and a more traditional face-to-face peer review (FPR) approach that was conducted in tutorial settings. The essays…

This study examined the effects of marking method and rater experience on ESL (English as a Second Language) essay test scores and rater performance. Each of 31 novice and 29 experienced raters rated a sample of ESL essays both holistically and analytically. Essay scores were analysed using a multi-faceted Rasch model to compare test-takers'…