Despite agreement about the central importance of validity for educational and psychological testing, consensus regarding the definition of validity remains elusive. Differences in the definition of validity are examined and reveals that a potential cause of disagreement stems from differences in word use and meanings given to key terms commonly…

Aberrant behavior refers to any type of unusual behavior that would not be expected under normal circumstances. In educational and psychological testing, such behaviors have the potential to severely bias the aberrant examinee's test score while also jeopardizing the test scores of countless others. It is therefore crucial that aberrant examinees…

This dissertation aims to bridge the gap between validity theory and the practice of validation. The dissertation employs a three-article approach. Following the introduction in Chapter I, three independent manuscripts representing three empirical studies are presented (i.e., Chapters II - IV). Each chapter is a stand-alone publishable manuscript,…

Quality assessment is an essential component of preschool education. The Standards for Educational and Psychological Testing provide benchmarks for evaluating the validity of inferences made from assessment or test results (AERA, APA & NCME, 2014). According to the Standards, test developers should investigate and document information related…

What is assessment literacy? It is a handful of fundamental understandings about the testing concepts and procedures that influence educational decisions. And it just might be the most cost-effective means of real school improvement. With characteristic humor and aplomb, assessment expert W. James Popham strips away the psychometrician-speak and…

In this digital ITEMS module, Dr. Robert [Bob] Mislevy and Dr. Maria Elena Oliveri introduce and illustrate a sociocognitive perspective on educational measurement, which focuses on a variety of design and implementation considerations for creating fair and valid assessments for learners from diverse populations with diverse sociocultural…

This article reviews four interrelated approaches to reducing an inequitable gap in cognitive and educational test scores between individuals of a dominant culture and individuals of other cultures or subcultures. These approaches include (a) use of broader measures, (b) performance- and project-based assessments, (c) direct measurement of…

One of the substantive changes in the 2014 Standards for Educational and Psychological Testing was the elevation of fairness in testing as a foundational element of practice in addition to validity and reliability. Previous research indicates that testing practices often do not align with professional standards and guidelines. Therefore, to raise…

The ability to convey shared meaning with minimal ambiguity is highly desirable for technical terms within disciplines and professions. Unfortunately, there is no widespread professional consensus over the meaning of the word "validity" as it pertains to educational and psychological testing. After illustrating the nature and extent of…

A misconception exists that validity may refer only to the "interpretation" of test scores and not to the "uses" of those scores. The development and evolution of validity theory illustrate test score interpretation was a primary focus in the earliest days of modern testing, and that validating interpretations derived from test…

Accountability has become a primary function of large-scale testing in the United States. The pressure on educators to raise scores is vastly greater than it was several decades ago. Research has shown that high-stakes testing can generate behavioral responses that inflate scores, often severely. I argue that because of these responses, using…

In Shepard's (1997) discussion on the importance of test use and consequences in a validity argument for educational assessments, she reflected on Cronbach and Meehl's (1955) perspective on the role of test developers in providing consequential evidence. In the following year, a special issue in "Educational Measurement: Issues and Practice"…

Given worldwide prevalence of low-stakes testing for monitoring educational quality and students' progress through school (e.g., Trends in International Mathematics and Science Study, Program for International Student Assessment), interpretability of resulting test scores is of global concern. The nonconsequential nature of low-stakes tests…

Haertel's argument is that one must "expand the scope of test validation to include indirect testing effects" because these effects are often the "rationale for the entire testing program." The author strongly agrees that this is essential. However, he maintains that Haertel's argument does not go far enough and that there are two additional…

In the context of using tests for educational improvement, Haertel's article--"How Is Testing Supposed to Improve Schooling?"--highlights how the measurement field falls short on validation efforts and demonstrates how by posing the question of how testing is supposed to improve schooling one may start testing different hypotheses about the…