Paul E. Newton's article on the consensus definition of validity tackles a number of big issues and makes a number of strong claims. I agreed with much of what he said, and I disagreed with a number of his claims, but I found his article to be consistently interesting and thought provoking (whether I agreed or not). I will focus on three general…

The argument-based approach to validation involves two steps; specification of the proposed interpretations and uses of the test scores as an interpretive argument, and the evaluation of the plausibility of the proposed interpretive argument. More ambitious interpretations and uses tend to involve an extended network of inferences and assumptions…

This paper presents the author's critique on Xiaoming Xi's article, "How do we go about investigating test fairness?," which lays out a broad framework for studying fairness as comparable validity across groups within the population of interest. Xi proposes to develop a fairness argument that would identify and evaluate potential fairness-based…

The 12th annual William H. Angoff Memorial Lecture was presented by Dr. Michael T. Kane, ETS's (Educational Testing Service) Samuel J. Messick Chair in Test Validity and the former Director of Research at the National Conference of Bar Examiners. Dr. Kane argues that it is important for policymakers to recognize the impact of errors of measurement…

Given the changing landscape of educational accountability at the local, state, and national levels, and the changes in the uses of the National Assessment of Educational Progress (NAEP), including the evolving uses of NAEP as a policy tool to interpret state assessment and accountability systems, an explicit statement of the current and potential…

The theories of validity developed over the past 60 years are quite sophisticated, but the methodology of validity is not generally very effective. The validity evidence for major testing programs is typically much weaker than the evidence for more technical characteristics such as reliability. In addition, most validation efforts have a strong…

The commentaries include a wealth of insightful and interesting observations and suggestions, and I appreciate each author taking the time to comment on my efforts. In responding to their suggestions, I am inclined to develop a few general points raised in the commentaries a bit further.

The current status of teacher testing practices and materials in the public schools of the District of Columbia was evaluated. This document reports the findings and recommendations of eight professionals who convened in Washington, D.C., on November 17-19, 1987. These consultants, via interviews with practitioners, found a high degree of…

The National Academy of Education (NAE) Panel has drawn two major conclusions in its evaluation of the National Assessment Governing Board's (NAGB) efforts to set achievement levels for the National Assessment of Educational Progress (NAEP). They are that the Angoff procedure is fundamentally flawed for the setting of achievement levels, and that…