Despite agreement about the central importance of validity for educational and psychological testing, consensus regarding the definition of validity remains elusive. Differences in the definition of validity are examined and reveals that a potential cause of disagreement stems from differences in word use and meanings given to key terms commonly…

Multidimensional forced-choice (MFC) items have been found to be useful to reduce response biases in personality assessments. However, conventional scoring methods for the MFC items result in ipsative data, hindering the wider applications of the MFC format. In the last decade, a number of item response theory (IRT) models have been developed,…

Since it was formalized by Kane, the argument-based approach to validation has been promoted as the preferred method for validating interpretations and uses of test scores. Because validation is discussed in terms of arguments, and arguments are both interactive and social, the present review systematically examines the scholarly arguments which…

One of the substantive changes in the 2014 Standards for Educational and Psychological Testing was the elevation of fairness in testing as a foundational element of practice in addition to validity and reliability. Previous research indicates that testing practices often do not align with professional standards and guidelines. Therefore, to raise…

With the 2014 publication of the 5th revision of the "Standards for Educational and Psychological Testing," the cochairs of the Joint Committee for the revision process were asked to consider the role and importance of the "Standards" for the educational testing community, and in particular for members of the National Council…

The purpose of this ITEMS module is to provide an introduction to subscores. First, examples of subscores from an operational test are provided. Then, a review of methods that can be used to examine if subscores have adequate psychometric quality is provided. It is demonstrated, using results from operational and simulated data, that subscores…

Drawing on experience between 2000 and 2007 in developing a validity argument for the high-stakes Test of English as a "Foreign Language[TM]" (TOEFL[R]), this paper evaluates the differences between the argument-based approach to validity as presented by "Kane (2006)" and that described in the 1999 "AERA/APA/NCME Standards for Educational and…

Some of the challenges that test publishers face in constructing educational assessments that meet high technical quality as prescribed in the "Standards for Educational and Psychological Testing" (AERA, APA, NCME, 1999) are examined. Federal educational initiatives are used to illustrate demands on technical quality that challenge the efforts of…

The "Standards for Educational and Psychological Testing" have evolved in the breadth and depth of coverage of issues in educational testing and measurement since their first publication in 1954. There were a number of substantive changes in the 1999 revision that addressed validity, fairness, accommodations, and compliance with the…

The psychometric literature is replete with comprehensive discussions of test validity, test validation, and the characteristics of quality assessment programs. The most authoritative source for guidance regarding sound test development and evaluation practices is the Standards for Educational and Psychological Testing. However, the Standards are…

The goal of the Standards for Educational and Psychological Testing is to improve testing practices, but their impact on practice appears spotty. Self-regulation clearly fails in some instances. The establishment of an external agency to oversee testing practices and adherence to the Standards would face substantial hurdles, and the ambiguity of…