The authors describe challenges of following professional standards for educational achievement testing due to the complexity of gathering appropriate evidence to support demanding test interpretation and use. Validity evidence has been found to be low for some individual testing standards, leading to the possibility of faulty or impoverished test…

Despite the call for an argument-based approach to validity over 25 years ago, few examples exist in the published literature. One possible explanation for this outcome is that the complexity of the argument-based approach makes implementation difficult. To counter this claim, we propose that the Assessment Triangle can serve as the overarching…

The 1999 "Standards for Educational and Psychological Testing" defines validity as the degree to which evidence and theory support the interpretations of test scores entailed by proposed uses of tests. Although quite explicit, there are ways in which this definition lacks precision, consistency, and clarity. The history of validity has taught us…

In this article we rethink validation within the complex contexts of high-stakes assessment. We begin by considering the utility of existing models for validation and argue that these models tend to overlook some of the complexities inherent to assessment use, including the multiple interpretations of assessment purposes and the potential…

The intervention validity of cognitive assessment batteries is considered within an historical context to identify what the evidence supports (knowns), what cannot be known (unknowables), and what is not yet known (unknowns). Two ways cognitive batteries could inform intervention are identified: a disordinal (i.e., aptitude-treatment interaction)…

Lissitz and Samuelsen (2007) have proposed an operational definition of "validity" that shifts many of the questions traditionally considered under validity to a separate category associated with the utility of test use. Operational definitions support inferences about how well people perform some kind of task or how they respond to some kind of…

This report describes the purposes of benchmark assessments and provides recommendations for selecting and using benchmark assessments--addressing validity, alignment, reliability, fairness and bias and accessibility, instructional sensitivity, utility, and reporting issues. We also present recommendations on building capacity to support schools'…

The articles in this issue, written from the perspectives of academics, practitioners, and publishers, show that examining the consequences of assessment is an important, large, and difficult task. Collaborative action by assessment developers, users, and the educational measurement community is needed if progress is to be made. (SLD)

This paper addresses three key topics related to making state National Assessment of Educational Progress (NAEP) assessments more efficient: (1) reducing the burden for the states; (2) stabilizing the assessment schedule; and (3) facilitating and promoting the use of state NAEP data. The paper recommends promoting the use of state NAEP data for…

This paper describes the Myers-Briggs Type Indicator (MBTI), developed by I. Myers and K. Briggs (1940s) to assess personality type. Based on Jungian theory, the MBTI has become a tool for identifying the 16 different patterns of action into which every person fits. The 16 personality types are based on patterns of: (1) extraversion-introversion;…

Provides an argument for incorporating consideration of consequences into validity theory that is grounded in the reflexive nature of social knowledge. It also calls for the consideration of evidence of validity based on the actual discourse surrounding the practices and products of testing. (SLD)

Presents conclusions of an independent measurement expert that the Texas Assessment of Academic Skills (TAAS) was constructed according to acceptable professional standards and tests curricular material considered by the Texas Board of Education important for graduates to have mastered. Also supports the validity and reliability of the TAAS and…

Discusses issues of validity in high-stakes testing, beginning with some purposes of a testing program and proceeding to some underlying assumptions about testing. Suggests four possible studies to address assumptions often ignored by asking various groups of people about testing. (SLD)

The fields of language testing and educational and psychological measurement have not, as yet, developed a set of principles and procedures for linking test scores and score-based inferences to test use and the consequences of test use. Although Messick (1989) discusses test use and consequences, his framework provides virtually no guidance on how…

Almost every state has in place a state assessment and accountability system. These systems vary greatly in their characteristics but share a common global purpose of improving teaching and learning. Some of the variations in the state systems are discussed and illustrated with examples from selected states. Issues that are critical to the value…