Drawing on the framework of invariant measurement from Rasch measurement theory, the purpose of this study is to psychometrically evaluate the 20 language and teaching skill domains of the International Teaching Assistant (ITA) Test using the many-facet Rasch model and to empirically explore performance differences between females and males in…

This mixed methods study compared IELTS examiners' scores when assessing spoken performances under live and two 'non-live' testing conditions using audio and video recordings. Six IELTS examiners assessed 36 test-takers' performances under the live, audio, and video rating conditions. Scores in the three rating modes were calibrated using the…

This paper presents a methodology for directly aligning 'can do' frameworks to each other. The methodology, inspired by the manual for relating examinations to the "Common European Framework of Reference for Languages: Learning, teaching, assessment" (CEFR) (Council of Europe, 2009) and Kane's (2004, 2013) interpretative argument, takes…

Researchers debate whether listening tasks should be supported by visuals. Most empirical research in this area has been conducted on the effects of visual support on listening comprehension tasks employing multiple-choice questions. The present study seeks to expand this research by investigating the effects of video listening passages (vs.…

This study investigates how accurate one item writer can be on item difficulty estimates and whether factors affecting item writer judgments correspond to predictors of actual item difficulty. The items were based on conversational dialogs (presented as videos online) that focus on pragmatic functions. Thirty-five 2nd-, 3rd-, and 4th-year learners…

The use of video technology has become widespread in the teaching and testing of second-language (L2) listening, yet research into how this technology affects the learning and testing process has lagged. The current study investigated how the channel of input (audiovisual vs. audio-only) used on an L2 listening test affected test-taker…

To what extent are the discourse behaviors of examiners salient to participants of an oral performance test? This exploratory study employs a grounded ethnographic approach to investigate the perceptions of the verbal, paralinguistic and nonverbal discourse behaviors of an examiner in a one-on-one role-play task that is one of four tasks in an…

This article examines how eight advanced ESL learners attend to and utilize the nonverbal information found in a video text. Using verbal report methodology in which test-takers verbalized aloud their thought processes while taking a video listening test, the results suggested that the test-takers varied in how they attended to and utilized the…