In this contribution, I argue for a radical reappraisal of accepted ideas about how English is taught and tested. I am not alone in questioning the validity of the current orthodoxy. Others have expressed views that correspond or are consistent with the points I make here. What I have sought to do is to synthesize them in the formulation of a…

This commentary discusses the issue of small samples in instructed second language acquisition research. We discuss the current state of affairs, and consider the disadvantages of small samples. We also explore other considerations regarding sample size, such as research ethics and ecological validity. We present a range of recommendations for…

When the COVID-19 pandemic made it impossible to do in-person, on campus testing, we were forced to create a new system to screen International Teaching Assistants (ITA) for Temple university. We used this opportunity to address many of the concerns and problems that we had identified with the previous test, and created a new test that could be…

In recent years, continuing advances in technology have increased the capacity to automate the extraction of a range of linguistic features of texts and thus have provided the impetus for the substantial growth of corpus linguistics. While corpus linguistic tools and methods have been used extensively in second language learning research, they…

There is a growing theoretical, empirical, and methodological debate in the SLA field as to whether unawareness plays a role during attention to or processing of new incoming second language (L2) data. Indeed, studies that have methodologically addressed the construct of unawareness in their research designs offer both empirical support (e.g.,…

In this interview, JD Brown reflects on language testing/assessment. He suggests that language testing can be seen as a continuum with hard core positivist approaches at one end and post modernist interpretive perspectives at the other, and also argues that norm referencing (be it proficiency, placement, or aptitude testing) and criterion…

Those who work in second- and foreign-language testing often find Koretz's concern for validity inferences under high-stakes (VIHS) conditions both welcome and familiar. While the focus of the article is more narrowly on the potential for two instructional responses to test-based accountability, "reallocation" and "coaching,"…

In this article, the author begins by discussing four challenges on the concept of validity. These challenges are: (1) the appeal to logic and syllogistic reasoning; (2) the claim of reliability; (3) the local and the universal; and (4) the unitary and the divisible. In language testing validity cannot be achieved directly but only through a…

Incidental vocabulary learning has attracted a great deal of attention in ELT research. However, it is important that teacher and researcher exploitation of vocabulary developments be guided by more than replication of previous research designs. For conclusions based on empirical research to be valid, it is important to be clear about exactly what…

This paper presents the author's critique on Xiaoming Xi's article, "How do we go about investigating test fairness?," which lays out a broad framework for studying fairness as comparable validity across groups within the population of interest. Xi proposes to develop a fairness argument that would identify and evaluate potential fairness-based…

In this response I discuss several points that Bruton made in his own response paper. I first argue that his main theme, that design factors should be emphasized over statistical analysis, has never been a genuine issue; it is two independent issues, and for each of them I have some concerns about his discussion. I then consider his comments on…

Just like buildings, tests are designed and built for specific purposes, people, and uses. However, both buildings and tests grow and change over time as the needs of their users change. Sometimes, they are also both used for purposes other than those intended in the original designs. This paper explores architecture as a metaphor for language…

In this speech, the author covers a lot of ground. In the first half of his speech, the author gives a brief summary of the last 40 years of the history of language testing, from his perspective. The author reviews these years more or less by decade. Additionally, he discusses the evolution of the profession of language testing during this period,…

Using the British "Index to Theses", we found forty-seven Ph.D.s relating to second and foreign language learning and/or teaching defended in English universities in 2006. Objective criteria led us to fourteen theses which had investigated both teaching and learning. Over half of these adopted a process-product research design with the…

Discusses the problem of measuring the validity of interview ratings in the American Council on the Teaching of Foreign Languages (ACTFL) Oral Proficiency Interviews (OPI), proposes frameworks to distinguish abilities from testing methods, and considers factors affecting test performance. Suggestions for research and development on the ACTFL OPI…