This controlled experimental study investigated the interaction of variables associated with rating the pronunciation component of high-stakes English-language-speaking tests such as IELTS and TOEFL iBT. One hundred experienced raters who were all either familiar or unfamiliar with Brazilian-accented English or Papua New Guinean Tok Pisin-accented…

This paper extends the use of Generalizability Theory to the measurement of extemporaneous L2 speech through the lens of speech perception. Using six datasets of previous studies, it reports on "G studies"--a method of breaking down measurement variance--and "D studies"--a predictive study of the impact on reliability when…

According to expectation violation theory, job applicants can be upgraded or downgraded during an interview when their accent does not match employers' speech expectations. Focusing on the employment of second language French job candidates in Québec, this study explored this issue dynamically in terms of how expectations may impact the trajectory…

Comprehensibility has emerged as a useful and intuitive means of globally evaluating second language (L2) speakers in many research and instructional contexts. In most cases, L2 speakers' comprehensibility is assessed by external listeners who do not engage in extensive communication with the speakers, even though the degree to which a speaker is…

This paper provides an experimental study of interlanguage phonological characteristics of Chinese students learning Thai as a foreign language and the accentedness perceived by native Thai speakers. Both production and perception experiments were designed to see how Chinese students acoustically produced Thai final nasal consonants and how Thai…

Whereas many scholars have emphasized the relative importance of "comprehensibility" as an ecologically valid goal for L2 speech training, testing, and development, eliciting listeners' judgments is time-consuming. Following calls for research on more efficient L2 speech rating methods in applied linguistics, and growing attention toward…

Previous research that explores the effect of ethnicity in the perception of speaker accentedness and personality traits often finds that Asian appearance contributes to a more accented and less competent impression. Importantly, most of the work done to date employed only Caucasian first language-speaking listeners; moreover, ethnicity and gender…

This study examined the effect of benchmarking and peer-assessment activities on second language (L2) French learners' self-assessments of accentedness, comprehensibility, and fluency. The learners, who included 25 L2 French students enrolled in a 15-week university-level French course, recorded two oral presentations at the beginning and the end…

This study examined how L2 English speakers interpreted the notion of native English speakers (NESs) and nonnative English speakers (NNESs) and whether nativeness would influence their self-perception and speech production. It aimed at filling the following research gaps. First, limited studies have explored how L2 English speakers view the other…

The rating of proficiency tests that use the Inter-agency Roundtable (ILR) and American Council on the Teaching of Foreign Languages (ACTFL) guidelines claims that each major level is based on hierarchal linguistic functions that require mastery of multidimensional traits in such a way that each level subsumes the levels beneath it. These…

Nonverbal behavior can impact language proficiency scores in speaking tests, but there is little empirical information of the size or consistency of its effects or whether language proficiency may be a moderating variable. In this study, 100 novice raters watched and scored 30 recordings of test takers taking an international, high stakes…

Nonverbal behavior is essential in human interaction (Gullberg, de Bot, & Volterra, 2008; McNeill, 1992, 2005). For second language speakers, nonverbal features can be helpful for successful and efficient communication (e.g., Dahl & Ludvigsen, 2014). However, due to the complexity of nonverbal features, language testing institutions have…

This study examined the use of a popular Automatic Speech Recognition (ASR), Google Voice Typing (GVT), to automatically assess English as second language pronunciation. It aimed to answer the following question: What is the relationship between GVT-rated scores and human-rated scores? To answer this question, we compared audio recordings of 56…

The practice of second language (L2) speech perception has traditionally relied on equal-interval perceptual scales and novice listeners' (NLs) impressionistic judgments of constructs such as accentedness and comprehensibility (Munro & Derwing, 2011). However, issues have surfaced with respect to how well NLs can use these scales, whether they…

The factors that influence rater scoring have been a subject of great interest to researchers in second language assessment. However, the research on the impact of test-takers' speech profiles (e.g., a jagged or a flat profile reflecting analytic subscores) on raters' scoring behaviors remains to be seen. To investigate the role of speech profiles…