This study examines the potential to use non-expert, crowd-sourced raters to score essays by comparing expert raters' and crowd-sourced raters' assessments of writing quality. Expert raters and crowd-sourced raters scored 400 essays using a standardised holistic rubric and comparative judgement (pairwise ratings) scoring techniques, respectively.…

Generative artificial intelligence technologies, such as ChatGPT, bring an unprecedented change in education by leveraging the power of natural language processing and machine learning. Employing ChatGPT to assist with marking written assessment presents multiple advantages including scalability, improved consistency, eliminating biases associated…

Depending on the nature of comments made during peer review of academic writing, students may be able to evaluate and revise their performance. Therefore, it is essential to explore how the content of comments affects student writing. Since peer review is a process of interaction, it is critical to understand how comments affect student academic…

This paper introduces REFLECTSUMM, a novel summarization dataset specifically designed for summarizing students' reflective writing. The goal of REFLECTSUMM is to facilitate developing and evaluating novel summarization techniques tailored to real-world scenarios with little training data, with potential implications in the opinion summarization…

Students are expected to interpret the complexities and nuances of literary texts yet might struggle with interpreting texts in ways that are valued in literary studies. Examining students' language choices can support instructors and students with developing concrete, explicit understandings of the ways language creates meanings in discourse.…

Background: A common approach to assessing students' integrated understanding of multiple documents is to analyse their post-reading written reports. This study investigated to what extent writing self-efficacy directly and indirectly (via integrative processing) contributed to multiple-document comprehension as assessed with an integrative…

Evaluating the quality of university exam papers is crucial for universities seeking institutional and program accreditation. Currently, exam papers are assessed manually, a process that can be tedious, lengthy, and in some cases, inconsistent. This is often due to the focus on assessing only the formal specifications of exam papers. This study…

This paper reports the findings of a study of student writing that assessed the skill needs of a cohort of developmental writing students enrolled in a co-requisite composition course at a community college that is part of the City University of New York (CUNY). This credit-bearing course fulfills the first half of the required composition…

Aim/Purpose: This article investigates the process of identifying and correcting hallucinations in ChatGPT-4's recall of student-written responses as well as its evaluation of these responses, and provision of feedback. Effective prompting is examined to enhance the pre-evaluation, evaluation, and post-evaluation stages. Background: Advanced Large…

Dynamic assessment (DA) integrates assessment and instruction between the instructor and learners. It emphasizes the learning process. DA was introduced to 24 undergraduates and 9 graduate students in two academic writing courses in Taiwan. The analysis of the pre- and post-tests, writing portfolios, observations, and interviews reached the…

This case study investigated first-year composition students' perceptions of instructors' written feedback. Participants reported on their understanding of the role of feedback, how they processed feedback, and if and how they used it. Thirty-one participants completed a short online survey; the responses were analyzed to identify students who…

Performance standards have important consequences for all the stakeholders in the assessment of L2 academic writing. These standards not only describe the level of writing performance but also provide a basis for making evaluative decisions on the academic writing. Such a high-stakes role of the performance standards requires the enhancement of…

Despite the growing movement to embrace sociomaterial approaches to feedback practices (e.g. Gravett, 2020), dialogicity remains the prominent and dominant approach, especially in the teaching of introductory or compulsory writing courses at the tertiary level. To examine this in our own practice, we reflected on and compared our written…

This project discusses the creation and implementation of labor logs in multimodal curricula in two levels of writing courses and how these labor logs support students' development of meta-awareness through reflection-in-action (Yancey, 1998). Labor logs create a space for students to focus on in the moment recognition, or monitoring, of what…

This dissertation compares the impact of instructor screencast feedback and instructor written feedback on the writing of college students in freshman composition courses. A mixed methods study was used to assess the level of improvement between drafts of essays, the final scores of essays, and the level of improvement over time for both types of…