Recent developments in the use of large-language models have led to substantial improvements in the accuracy of content-based automated scoring of free-text responses. The reported accuracy levels suggest that automated systems could have widespread applicability in assessment. However, before they are used in operational testing, other aspects of…

Forensic science practitioners compare visual evidence samples (e.g. fingerprints) and decide if they originate from the same person or different people (i.e. fingerprint 'matching'). These tasks are perceptually and cognitively complex--even practising professionals can make errors--and what limited research exists suggests that existing…

In this practice paper I examine the evaluation of one project and ask whether or not the evaluation failed. I suggest that because I was unaware of the larger connections between the grant agency and the grantee I made tactical errors in the presentation of findings, which resulted in not simply disagreement but anger and resentment. I conclude…

Despite the increased use of integrated tasks in high-stakes academic writing assessment, research on rating criteria which reflect the unique construct of integrated summary writing skills is comparatively rare. Using a mixed-method approach of expert judgement, text analysis, and statistical analysis, this study examines writing features that…

Researchers have explored a variety of topics related to identifying and distinguishing among specific types of rater effects, as well as the implications of different types of incomplete data collection designs for rater-mediated assessments. In this study, we used simulated data to examine the sensitivity of latent trait model indicators of…

Writing support programs for students in writing-intensive, disciplinary courses are well established and take many forms, including communication centers, web-based skill development programs, and embedded writing consultants. This paper assesses the effectiveness of a program that embeds a writing grader, who assesses only the grammar of…

In the process of measuring and assessing high-level cognitive skills, interference of rater errors in measurements brings about a constant concern and low objectivity. The main purpose of this study was to investigate the impact of rater training on rater errors in the process of assessing individual performance. The study was conducted with a…

Purpose: The aim of this study was to advance the use of structured, monologic discourse analysis by validating an automated scoring procedure for core lexicon (CoreLex) using transcripts. Method: Forty-nine transcripts from persons with aphasia and 48 transcripts from persons with no brain injury were retrieved from the AphasiaBank database. Five…

This study is a descriptive-qualitative study with an embedded-case study research design. The purpose of the study is to describe the quality of the translation on the accuracy in content, acceptability, and readability of translated text of English Idioms Errors Made by Jordanian EFL, Undergraduate Students abstract. The data of the research is…

Previous research on pronunciation errors observed that the most frequently mispronounced English phonemes are word-initial consonants. The findings of the current study invalidate those observations by presenting the data analysis of 40 Turkish EFL learners' speech samples, which illustrates that segmental speech errors are position-independent…

Successive UK government policies have strengthened the phonics element of the National Curriculum for English in England. The policies have included inviting publishers to submit completed self-assessments of their systematic phonics programmes. The self-assessment criteria focus on what is deemed to be 'high-quality provision', as defined in the…

Informal reading inventories (IRI) and curriculum-based measures of reading (CBM-R) have continued importance in instructional planning, but raters have exhibited difficulty in accurately identifying students' miscues. To identify and tabulate scorers' mismarkings, this study employed examiners and raters who scored 15,051 words from 108 passage…

As a result of the fact that judgments of non-native speech are closely tied to social biases, oral proficiency ratings are susceptible to error because of rater background and social attitudes. In the present study we seek first to estimate the variance attributable to rater background and attitudinal variables on novice raters' assessments of L2…

Throughout the dissertation process, the chair and committee members provide feedback regarding quality to help the doctoral candidate to produce the highest-quality document and become an independent scholar. Nevertheless, results of previous research suggest that overall dissertation quality generally is poor. Because much of the feedback about…

Amongst extensive research on thesis writing quality, few provide a detailed account of common writing challenges/errors as perceived by examiners vs. students. In this study, eight recent English theses marked-up by examiners and defended in Iranian universities were selected randomly among those thesis made available (n=45). Based on an…