In the process of measuring and assessing high-level cognitive skills, interference of rater errors in measurements brings about a constant concern and low objectivity. The main purpose of this study was to investigate the impact of rater training on rater errors in the process of assessing individual performance. The study was conducted with a…

This research paper employs an exploratory framework to evaluate the potential of ChatGPT as an Automated Writing Evaluation (AWE) tool in teaching English as a Foreign Language (EFL) in Thailand. The main objective is to investigate how well ChatGPT can assess students' writing using prompts and pre-defined rubrics compared to human raters.…

Written communication skills are one of the most assessed criteria in higher education contexts, especially in humanities disciplines, including teacher education. There is a need to research and develop an assessment grading tool (i.e. criteria sheet or rubric) that would assist students in pre-service teacher education programs to better…

The aim of this study is to examine the variability in and reliability of scores assigned to different quality EFL compositions by EFL instructors and their rating behaviors. Using a mixed research design, quantitative data were collected from EFL instructors' ratings of 30 compositions of three different qualities using a holistic scoring rubric.…

This paper investigates the performance of holistic and analytic scoring rubrics in the context of EFL writing. Specifically, the paper compares EFL students' scores on a writing task using holistic and analytic scoring rubrics. The data for the study was collected from 30 participants attending an English undergraduate program in a Yemeni…

An examination of the scores given using analytic rating scales can provide insight into how well the scales are measuring what they are supposed to be measuring. Many analytic rating scales include separate scales for lexis and grammar, yet the distinction between lexis and grammar has not been widely researched or defined. It is therefore…

"Voice" is widely considered to be a feature of effective writing. It's no surprise, then, that voice criteria frequently appear on rubrics used to score student essays in large-scale writing assessments. However, composition theorists hold vastly different views regarding voice and how it should be applied in the evaluation of student writing, if…

Examined the relationship between students' composition scores and the writers' and raters' perceptions of a given topic: does a student receive a higher score if the rater agrees with the student's thesis? Findings indicated that there are other factors, such as students' attitude and cognitive ability that may affect raters' judgments of student…

The New Standards Project is a joint effort of the Learning Research and Development Center (LRDC) at the University of Pittsburgh (Pennsylvania) and the National Center on Education and the Economy toward creation of a national examination system based on performance assessments. This study explored the feasibility of comparing performance on…

Ten states participated in a cross-state scoring workshop in 1991, evaluating writing from elementary school, middle school, and high school students. Correlation of scores assigned by readers from one state with those from readers from another state were generally quite high. Implications for defining common standards are discussed. (SLD)

The use of portfolio assessment as a method of evaluating the writing competence of elementary school students was studied. The study contained two components: (1) an empirical study of the utility and meaningfulness of using an analytic rubric developed for the evaluation of traditional writing samples to score student portfolios; and (2) a…

The writing examination package is divided into three volumes: Examination Handbook (present volume), Training Manual, and a Ratings Manual. The volumes are interdependent, and all three must be understood before using the Training Manual to train raters. The Examination Handbook provides a context for the other two volumes. It gives a general…