This study addressed the question of the consistency of rater characteristics over time, and investigated the potential of multifaceted Rasch measurement in exploring this question. The purpose of the research is to investigate the use of these analytical techniques in rater training for the speaking subtest of the Occupational English Test, an…

Investigates potential problems with training native speaker interlocutors to carry out a series of oral interactions with a candidate for overseas employment with the Australian government. Findings reveal the effects of interlocutor variables and audiotape quality on ratings. Evaluates the overall feasibility of the procedure and examines…

This study investigated potential problems associated with one method of testing English-as-a-Second-Language (ESL) skills in work settings in foreign countries. The method, used by the Australian government to assess speech skills of individuals in locations outside Australia, involves having native English-speakers carry out a series of oral…

Discusses the role of the Rasch model IRT in evaluating two subtests of the Occupational English test and argues for its use in exploring test constructs and in considering the implications of the empirical analysis presented for the validity of communicative language tests involving speaking and writing skills. (39 references) (Author/JL)

The role of item response theory (IRT) in determining the validity of second language tests is examined in the case of one specific test, the listening subtest of the Occupational English Test (OET), used in Australia to measure the language skills of non-native English-speaking health professionals. First, the listening subtest is described. Then…