This paper addresses the intersection of testing and policy, situating test-driven impact and validation within the context of policy-led educational reform in Korea. I will briefly review the existing validation models. Then, arguing for an expansion of the conventional conceptualization of consequential validity research, I use Fairclough's…

Shifting focus of asocial language tests toward social considerations recalls their ideological basis (Mirhosseini, De Costa, 2020). The recurrent exposure to and involvement in discursive constructions of high stakes' contents, may bring along certain sociocultural conceptualizations and values by test audiences (van Dijk, 1998). Thus, unless a…

Conversation analysis (CA) has exerted influence both on second language (L2) classroom pedagogy (e.g., Barraja-Rohan A, Pritchard R, Beyond talk: a course in communication and conversation skills for intermediate adult learners of English. Western Metropolitan Institute of TAFE, Melbourne, 1997; Wong J, Waring HZ, Conversation analysis and second…

Many English language proficiency (ELP) tests used for university admissions and placement now include integrated writing tasks that require examinees to use external sources when writing. Integrated writing tasks improve test authenticity and impact, but they raise several validity questions, such as what academic language skills they engage and…

Whereas previous research revealed the interactional variability occurring in oral assessments, demonstrating how it could undermine test validity (e.g., A. Brown, 2003), little has been published regarding how language programs can mine and analyze video recordings of in-house oral placement tests for validation purposes. Addressing this need,…

Oral Proficiency Interviews (OPIs) are widely used to measure speaking ability in a second or foreign language. The Michigan English Language Assessment Battery (MELAB) Speaking Test is an OPI used for academic and professional purposes around the world. However, little research on this or other OPIs has quantitatively compared test takers' speech…

In this article, we report on our development of a translanguaged French/English listening task as part of the revision of a test for professors in a bilingual Canadian university. The primary objective in revising the test was to more authentically represent the target language use domain, which regularly includes translanguaging. We describe the…

Investigations of the validity of a number of high-stakes language assessments are conducted using an argument-based approach, which requires evidence for inferences that are critical to score interpretation (Chapelle, Enright, & Jamieson, 2008b; Kane, 2013). The current study investigates the extrapolation inference for a high-stakes test of…

In this study we compare test takers' performance on the Speaking section of the TOEFL iBT™and their performances during their real-life academic studies. Thirty international graduate students from mixed language backgrounds in two different disciplines (Sciences and Social Sciences) responded to two independent and four integrated speaking tasks…

One of the major innovations of the "TOEFL iBT"® test is the incorporation of integrated tasks complementing the independent tasks to which examinees respond. In addition, examinees must produce discourse in both modes (speech and writing). The validity argument for the TOEFL iBT includes the claim that examinees vary their discourse in…

A number of scholars have questioned the practice of assessing academic writing in the context of a one-off language test, claiming that the time restrictions imposed in the test environment, when compared to the writing conditions typical at university, may prevent learners from displaying the kinds of writing skills required in academic…

Examination of oral proficiency interview discourse suggests that the extent of interviewer accommodation is an overlooked criterion for gauging the authenticity of the interview as simulated conversational interaction and that misplaced accommodation could threaten interview validity and the rating process. (61 references) (Author/CB)

This article reports on a case study of negotiation that occurred in peer group oral interactions under assessment conditions. Discourse analysis was used to illustrate how participants negotiated and co-constructed the assessment format itself as well as meaning exchange sequences. Analyses of the data point to the advantage of using peer group…

A study investigated the rating scale used in the Interagency Language Roundtable (ILR) oral interview, focusing on the concept of vocabulary underlying assessment. The study examined the differences in strategies used by fluent native speakers and non-fluent non-native speakers to avoid disruption of communication in real conversation when…

Presents study designed to examine trait structure of a cloze test using confirmatory factor analysis. Results suggest that a modified cloze passage, using rational deletions, is capable of measuring syntactic- and discourse-level relationships in a text, and this advantage may outweigh considerations of reduced redundancy which underlie random…