A challenge of large-scale oral communication assessments is to feasibly assess a broad construct that includes interactional competence. One possible approach in addressing this challenge is to use a spoken dialog system (SDS), with the computer acting as a peer to elicit a ratable speech sample. With this aim, an SDS was built and four trained…

The rating of proficiency tests that use the Inter-agency Roundtable (ILR) and American Council on the Teaching of Foreign Languages (ACTFL) guidelines claims that each major level is based on hierarchal linguistic functions that require mastery of multidimensional traits in such a way that each level subsumes the levels beneath it. These…

Nonverbal behavior can impact language proficiency scores in speaking tests, but there is little empirical information of the size or consistency of its effects or whether language proficiency may be a moderating variable. In this study, 100 novice raters watched and scored 30 recordings of test takers taking an international, high stakes…

This study compared holistic and analytic marking methods for their effects on parameter estimation (of examinees, raters, and items) and rater cognition in assessing speech act production in L2 Chinese. Seventy American learners of Chinese completed an oral Discourse Completion Test assessing requests and refusals. Four first-language (L1)…

This study examined the impact of the manipulated task complexity (Robinson 2001a, 2001b, 2007, 2011; Robinson & Gilabert, 2007) on second language (L2) speech comprehensibility. I examined whether manipulated task complexity (a) impacts L2 speech comprehensibility, (b) aligns with L2 speakers' perception of task difficulty (cognitive…

This study compares the language proficiency gains of two groups of students taking a business English course module in a bilingual university in Catalonia (Spain). Whereas one of these groups followed a 'translanguaging' or 'plurilingual' pedagogy, the other followed a strictly monolingual approach. Participants were 54 mostly Catalan/Spanish…

Pronunciation features are not equal in how they affect listeners' ability to understand. Some are low value, while others are high value. This study explores whether contrastive stress is high value. Previous research has shown that identification of contrastive stress is learnable (Pennington & Ellis, 2000), and that explicit teaching about…

The current study examined the extent to which native speakers of North American and Singapore English differentially perceive the comprehensibility (ease of understanding) of second language (L2) speech. Spontaneous speech samples elicited from 50 Japanese learners of English with various proficiency levels were first rated by 10 Canadian and 10…

The current study examined how two groups of native speakers--monolingual Canadians and multilingual Singaporeans--differentially perceive foreign accentedness in spontaneous second language (L2) speech. The Singaporean raters, who had exposure to various models of English and also spoke multiple L2s on a daily basis, demonstrated more lenient…

In early stages of classroom language learning, many adult second language (L2) learners communicate primarily with one another, yet we know little about which speech stream characteristics learners tune into or the extent to which they understand this lingua franca communication. In the current study, 25 native English speakers learning German as…

The "indigenous assessment practices" (Jacoby & McNamara, 1999) in selected health professions were investigated to inform a review of the scope of assessment in the speaking sub-test of a specific-purpose English language test for health professionals, the Occupational English Test (OET). The assessment criteria in current use on…

This study aimed to investigate the characteristics and developmental phenomena apparent in Japanese learners of English with their raters' perspectives in terms of fluency which has been recognized as a major factor in judging non-native speakers' proficiency. The following temporal indices demonstrated strong relationships with the raters'…

This paper reports a case study of the nature and extent of progress in speaking skills made by a group of upper intermediate instructed learners, and also assessors' perceptions of that progress. Initial and final interview data were analysed using several measures of Grammatical and Lexical Complexity, Language Accuracy and Fluency. These…

This study employed Bachman and Palmer's (2010) Assessment Use Argument framework to investigate to what extent the use of a second language oral test as an exit test in a Hong Kong university can be justified. It also aimed to help test developers of this oral test identify the most critical areas in the current test design that might need…

The definition and operationalization of interactional competence in speaking tests that entail co-construction of discourse is an area of language testing requiring further research. This article explores the reactions of four trained raters to paired candidates who oriented to asymmetric patterns of interaction in a discussion task. Through an…