Researchers who work with ordinal rating scales sometimes encounter situations where the scale categories do not function in the intended or expected way. For example, participants' use of scale categories may result in an empirical difficulty ordering for the categories that does not match what was intended. Likewise, the level of distinction…

Koretz, in his article published in this issue, provides compelling arguments that the high stakes currently associated with accountability testing lead to behavioral changes in students, teachers, and other stakeholders that often have negative consequences, such as inflated scores. Koretz goes on to argue that these negative consequences require…

Paul E. Newton provides an insightful and scholarly overview of central issues in validity theory. As he notes, many of the conceptual problems in validity theory derive from the fact that the word "validity" has two meanings. First, it indicates "whether a test measures what it purports to measure." This is a factual claim about the psychometric…

This article is a commentary of a paper by Derek C. Briggs and Frederick A. Peck, "Using Learning Progressions to Design Vertical Scales That Support Coherent Inferences about Student Growth," which describes an elegant potential framework for at least beginning to address three priorities in large-scale assessment that have not been…

In their paper "Unique Characteristics of Diagnostic Classification Models: A Comprehensive Review of the Current State-of-the-Art," Andre Rupp and Jonathan Templin (2008) provide a comparative analysis of selected psychometric models useful for the analysis of multidimensional data for purposes of diagnostic score reporting. Recent assessment…

One promising application of diagnostic classification models (DCM) is in the area of cognitive diagnostic assessment in education. However, the successful application of DCM in educational testing will likely come with a price--and this price may be in the form of new test development procedures and practices required to yield data that satisfy…

According to Schilling, Blunk, and Hill, the set of papers presented in this journal issue had two main purposes: (1) to use an argument-based approach to evaluate the validity of the tests of mathematical knowledge for teaching (MKT), and (2) to critically assess the author's version of an argument-based approach to validation (Kane, 2001, 2004).…

Operating from multiple literature bases in cognitive psychology, mathematics education, and theoretical and applied psychometrics, Schilling, Hill and their colleagues provide a systemic approach to studying the validity of scores of mathematical knowledge for teaching. This system encompasses an array of task formats and methodologies. The…