This article offers a critique of the validity argument put forward by Camara, Mattern, Croft, and Vispoel (2019) regarding the use of college-admissions tests in high school assessment systems. We challenge their argument in two main ways. First, we illustrate why their argument fails to address broader issues related to consequences of using…

Maul's paper, "Rethinking Traditional Methods of Survey Validation," is a clever and pointed indictment of a set of specific but widespread practices in psychological measurement and the social sciences at large. Through it, Maul highlights central issues in the way to approach theory building and theory testing, bringing to mind the…

The six primary papers in this issue of "Assessment in Education" emphasise a single primary point: the concept of validity is a complex one. Essentially, validity is a collective noun. That is, just as a group of players may be called a team and a group of geese a flock, so too does validity represent a variety of processes and…

As the number of candidates who repeat English language proficiency tests more than once to meet a certain cutscore (e.g., for university admission) or to demonstrate progress (e.g., after instruction) continues to increase dramatically, there is a need for more research on the attributes and test performance of test repeaters. This article…

Naomi Gafni, director of Research and Development, National Institute for Testing and Evaluation, Jerusalem, Israel, has devoted a substantial part of her career to the development of admissions tests and other educational tests and to the investigation of their validity. As such she is keenly aware of the complexities involved in this process.…

Paul E. Newton's article on the consensus definition of validity tackles a number of big issues and makes a number of strong claims. I agreed with much of what he said, and I disagreed with a number of his claims, but I found his article to be consistently interesting and thought provoking (whether I agreed or not). I will focus on three general…

Kane distinguishes between two kinds of argument: the interpretation/use argument and the validity argument. This commentary considers whether there really are two kinds of argument, two arguments, or just one. It concludes that there is just one argument: the validity argument. (Contains 2 figures and 5 notes.)

As researchers at a testing organization concerned with the appropriate uses and validity evidence for our assessments, we provide an applied perspective related to the issues raised in the focus article. Newton's proposal for elaborating the consensus definition of validity is offered with the intention to reduce the risks of inadequate…

In Shepard's (1997) discussion on the importance of test use and consequences in a validity argument for educational assessments, she reflected on Cronbach and Meehl's (1955) perspective on the role of test developers in providing consequential evidence. In the following year, a special issue in "Educational Measurement: Issues and Practice"…

In validating uses of testing, it is helpful to distinguish those that rely directly on the information provided by scores or score distributions ("direct" uses and consequences) versus those that instead capitalize on the motivational effects of testing, or use testing and test reporting to shape public opinion ("indirect" uses and consequences).…

In the context of using tests for educational improvement, Haertel's article--"How Is Testing Supposed to Improve Schooling?"--highlights how the measurement field falls short on validation efforts and demonstrates how by posing the question of how testing is supposed to improve schooling one may start testing different hypotheses about the…

Kane (this issue) presents a comprehensive review of validity theory and reminds us that the focus of validation is on test score interpretations and use. In reacting to his article, I support the argument-based approach to validity and all of the major points regarding validation made by Dr. Kane. In addition, I call for a simpler, three-step…

According to Kane (this issue), "the validity of a proposed interpretation or use depends on how well the evidence supports" the claims being made. Because truth and evidence are distinct, this means that the validity of a test score interpretation could be high even though the interpretation is false. As an illustration, we discuss the case of…

At the outset of his thoughtful and thought-provoking article, Haertel (this issue) clearly identifies the issue with which he will be dealing: The disjunct, or gap, in current approaches to evaluating the merits of a given test, between the intended uses of that test and the validity of its score-based interpretations. The author thinks that…

Kane's paper "Validating the Interpretations and Uses of Test Scores" is the most complete and clearest discussion yet available of the argument-based approach to validation. At its most basic level, validation as formulated by Kane is fundamentally a simply-stated two-step enterprise: (1) specify the claims inherent in a particular interpretation…