This study aimed to answer an ongoing validity question related to the use of nonstandard English accents in international tests of English proficiency and associated issues of test fairness. More specifically, we examined (1) the extent to which different or shared English accents had an impact on listeners' performances on the Duolingo listening…

This study details the development of a local test designed to place university Spanish students (n = 719) into one of the four different course levels and to distinguish between traditional L2 learners and early bilinguals on the basis of their linguistic knowledge, regardless of the variety of Spanish they were exposed to. Early bilinguals…

Although there is abundant evidence for the use of first-language (L1) knowledge by bilinguals when using a second language (L2), investigation into the impact of L1 knowledge in large-scale L2 language assessments and discussion of how such impact may be controlled has received little attention in the language assessment literature. This study…

In view of the ubiquitous increase in the use of C-tests, which are almost unanimously believed to measure general language proficiency, this study investigates whether the aspects of language proficiency tapped into by the C-test format are the same when the test is taken by a learner population other than that of foreign language learners.…

Celpe-Bras, is the exam for the certification of proficiency in Portuguese as a foreign language. It, is the only Portuguese proficiency test recognized by the Brazilian government (Ministério da Educação, 2013). Given the recent growth of interest and also its unique design as a large-scale proficiency test, this article provides a general…

There is a strong international demand for official certification of French competence. A range of tests are available to meet this goal. Among the recognized tests available for this purpose is the "DELF" ("Diplôme d'études en langue française"). The "DELF" has been awarded the Association of Language Testers in…

There is still relatively little research on how well the CEFR and similar holistic scales work when they are used to rate L2 texts. Using both multifaceted Rasch analyses and qualitative data from rater comments and interviews, the ratings obtained by using a CEFR-based writing scale and the Finnish National Core Curriculum scale for L2 writing…

This paper presents the author's critique on Xiaoming Xi's article, "How do we go about investigating test fairness?," which lays out a broad framework for studying fairness as comparable validity across groups within the population of interest. Xi proposes to develop a fairness argument that would identify and evaluate potential fairness-based…

This paper reports on an investigation of the potential for a shared-L1 advantage on an academic English listening test featuring speakers with L2 accents. Two hundred and twelve second-language listeners (including 70 Mandarin Chinese L1 listeners and 60 Japanese L1 listeners) completed three versions of the University Test of English as a Second…

This paper presents the author's response to Xiaoming Xi's article titled "How do we go about investigating test fairness?" In this response, the author focuses on test fairness and Toulmin's model of argument structure, Xi's proposal, and the challenges the proposal brings. Xi proposes an approach to investigating test fairness to guide…

This study tracked gender differential item functioning (DIF) on the English subtest of the Korean College Scholastic Aptitude Test (KCSAT) over a nine-year period across three data points, using both the Mantel-Haenszel (MH) and item response theory likelihood ratio (IRT-LR) procedures. Further, the study identified two factors (i.e. reading…

The present study employed multi-faceted Rasch measurement (MFRM) to explore the rater bias patterns of native English-speaker (NES) raters when they rate EFL essays. Forty NES raters rated 40 essays written by female Japanese university students on a single topic adapted from the TOEFL Test of Written English (TWE). The essays were assessed using…

In this article, I describe a practical application of the Roussos and Stout (1996) multidimensional analysis framework for interpreting group performance differences on an ESL reading proficiency test. Although a variety of statistical methods have been developed for flagging test items that function differentially for equal ability examinees…

Analyzes simulated performance ratings on a six-point scale by two independent raters to account for nonsystematic error in performance ratings. Results suggest that rater agreement or covariance is not always a dependable estimate of score reliability and that the practice of seeking additional raters for adjudication of discrepant ratings is not…

Argues that language tests employing methods not fair to all test takers are unethical. Ways of reducing sources of unfairness in language testing is discussed. (15 references) (Author/CK)