This study is the first attempt to explore the relationship between rater variables focusing on raters' language aptitude and their judgments of second language (L2) speech. Thirty-four English listeners rated 65 spontaneous native and nonnative speech samples for comprehensibility, accentedness, and fluency. They also completed the LLAMA language…

Purpose: The ongoing COVID-19 pandemic has prompted changes to child assessment procedures in schools such as the use of face masks by assessors. Research with adults suggests that face masks diminish performance on speech processing and comprehension tasks, yet little is known about how assessor masking affects child performance. Therefore, we…

According to expectation violation theory, job applicants can be upgraded or downgraded during an interview when their accent does not match employers' speech expectations. Focusing on the employment of second language French job candidates in Québec, this study explored this issue dynamically in terms of how expectations may impact the trajectory…

Whereas many scholars have emphasized the relative importance of "comprehensibility" as an ecologically valid goal for L2 speech training, testing, and development, eliciting listeners' judgments is time-consuming. Following calls for research on more efficient L2 speech rating methods in applied linguistics, and growing attention toward…

Sign languages present particular challenges to language assessors in relation to variation in signs, weakly defined citation forms, and a general lack of standard-setting work even in long-established measures of productive sign proficiency. The present article addresses and explores these issues via a mixed-methods study of a human-rated…

Purpose: This study builds a new system for automatically assessing learners' speech elicited from an oral discourse completion task (DCT), and evaluates the prediction capability of the system with a view to better understanding factors deemed influential in predicting speaking proficiency scores and the pedagogical implications of the system.…

Previous research that explores the effect of ethnicity in the perception of speaker accentedness and personality traits often finds that Asian appearance contributes to a more accented and less competent impression. Importantly, most of the work done to date employed only Caucasian first language-speaking listeners; moreover, ethnicity and gender…

When students present writing tasks that require higher order thinking skills to work, one of the most important problems is scoring these writing tasks objectively. The fact that raters give scores below or above their performance based on several environmental factors affects the consistency of the measurements. Inconsistencies in scoring…

It is widely believed that human rating performance is influenced by an array of different factors. Among these, rater-related variables such as experience, language background, perceptions, and attitudes have been mentioned. One of the important rater-related factors is the way the raters interact with the rating scales. In particular, how raters…

While listeners tend to downgrade speakers' accent and comprehensibility when they perceive them to be from a different language community--a process known as reverse linguistic stereotyping (RLS)--research has generally relied solely on quantitative data such as Likert scale ratings. The current study sought to extend the analysis further by…

This study compared holistic and analytic marking methods for their effects on parameter estimation (of examinees, raters, and items) and rater cognition in assessing speech act production in L2 Chinese. Seventy American learners of Chinese completed an oral Discourse Completion Test assessing requests and refusals. Four first-language (L1)…

Purpose: There are many fewer speech-language pathologists (SLPs) who can administer Spanish language testing than there are children in the United States who need such testing. Although there are precautions against language testing by foreign speakers, results from testers using native- versus foreign-accented speech have not been compared using…

The current study examined the degree to which collocation use (i.e., meaningful co-occurrences of multiple words) is related to first language (L1) raters' intuitive judgments of second language (L2) speech. Speech samples from a picture description task performed by 85 Japanese learners of English with varied L2 proficiency profiles were…

Pronunciation teaching is an important stage in language learning activities. This article tackles the pronunciation scoring problem where research has demonstrated relatively low human-human and low human-machine agreement rates, which makes teachers skeptical about their relevance. To overcome these limitations, a fuzzy combination of two…

The quality of sign language interpreting (SLI) is a gripping construct among practitioners, educators and researchers, calling for reliable and valid assessment. There has been a diverse array of methods in the extant literature to measure SLI quality, ranging from traditional error analysis to recent rubric scoring. In this study, we want to…