This study is the first attempt to explore the relationship between rater variables focusing on raters' language aptitude and their judgments of second language (L2) speech. Thirty-four English listeners rated 65 spontaneous native and nonnative speech samples for comprehensibility, accentedness, and fluency. They also completed the LLAMA language…

Purpose: The ongoing COVID-19 pandemic has prompted changes to child assessment procedures in schools such as the use of face masks by assessors. Research with adults suggests that face masks diminish performance on speech processing and comprehension tasks, yet little is known about how assessor masking affects child performance. Therefore, we…

Purpose: The ongoing COVID-19 pandemic has prompted changes to child assessment procedures in schools such as the use of face masks by assessors. Research with adults suggests that face masks diminish performance on speech processing and comprehension tasks, yet little is known about how assessor masking affects child performance. Therefore, we…

Automated assessment using Natural Language Processing (NLP) has the potential to make English speaking assessments more reliable, authentic, and accessible. Yet without careful examination, NLP may exacerbate social prejudices based on gender or native language (L1). Current NLP-based assessments are prone to such biases, yet research and…

Sign languages present particular challenges to language assessors in relation to variation in signs, weakly defined citation forms, and a general lack of standard-setting work even in long-established measures of productive sign proficiency. The present article addresses and explores these issues via a mixed-methods study of a human-rated…

Purpose: This study builds a new system for automatically assessing learners' speech elicited from an oral discourse completion task (DCT), and evaluates the prediction capability of the system with a view to better understanding factors deemed influential in predicting speaking proficiency scores and the pedagogical implications of the system.…

It is widely believed that human rating performance is influenced by an array of different factors. Among these, rater-related variables such as experience, language background, perceptions, and attitudes have been mentioned. One of the important rater-related factors is the way the raters interact with the rating scales. In particular, how raters…

This study compared holistic and analytic marking methods for their effects on parameter estimation (of examinees, raters, and items) and rater cognition in assessing speech act production in L2 Chinese. Seventy American learners of Chinese completed an oral Discourse Completion Test assessing requests and refusals. Four first-language (L1)…

Purpose: There are many fewer speech-language pathologists (SLPs) who can administer Spanish language testing than there are children in the United States who need such testing. Although there are precautions against language testing by foreign speakers, results from testers using native- versus foreign-accented speech have not been compared using…

Assessing language skills in an integrative form has drawn the attention of assessment experts in recent years. While some research data exists on integrative listening/reading-to-write assessment, there is comparatively little research literature on listening-to-speak integrated assessment. Also, little attention has been devoted to the role of…

This study examined the impact of the manipulated task complexity (Robinson 2001a, 2001b, 2007, 2011; Robinson & Gilabert, 2007) on second language (L2) speech comprehensibility. I examined whether manipulated task complexity (a) impacts L2 speech comprehensibility, (b) aligns with L2 speakers' perception of task difficulty (cognitive…

Many studies that have investigated the educational value of online machine translation (MT) in second language (L2) writing generally report significant improvements after MT use, but no study as of yet has comprehensively analyzed the effectiveness of MT use in terms of various measures in syntactic complexity, accuracy, lexical complexity, and…

This study compares the language proficiency gains of two groups of students taking a business English course module in a bilingual university in Catalonia (Spain). Whereas one of these groups followed a 'translanguaging' or 'plurilingual' pedagogy, the other followed a strictly monolingual approach. Participants were 54 mostly Catalan/Spanish…

There is emerging evidence that collocation use plays a primary role in determining various dimensions of L2 oral proficiency assessment and development. The current study presents the results of three experiments which examined the relationship between the degree of association in collocation use (operationalized as t scores and mutual…

Because English is widely used as a lingua franca, language testers have started to consider the introduction of non-native accents into English listening tests. This study investigates how accents influence test-takers' performance, and also elicits test-takers' subjective perception of accents. Eighty adolescent L1-Mandarin test takers were…