Standard setting is arguably one of the most subjective techniques in test development and psychometrics. The decisions when scores are compared to standards, however, are arguably the most consequential outcomes of testing. Providing licensure to practice in a profession has high stake consequences for the public. Denying graduation or forcing…

In Programme for International Student Assessment (PISA), item response theory (IRT) scaling is used to examine the psychometric properties of items and scales and to provide comparable test scores across participating countries and over time. To balance the comparability of IRT item parameter estimations across countries with the best possible…

The field of educational measurement has evolved considerably since the first doctoral programs were established. In response, programs have typically tacked on courses that address newly developed theories, methods, tools, and techniques. As our review of current programs evidences, this approach produces artificial distinctions among topics and…

Drawing upon his experience, more than 60 years ago, as a psychometric support person to a very special teacher of brain damaged children, the author of this article reflects on the productive use of educational assessments and data from them to educate - assessment in the service of learning. Findings from the Gordon Commission on the Future of…

The Programme for International Student Assessment (PISA) is an important international study of 15-olds' knowledge and skills. New results are released every 3 years, and have a substantial impact upon education policy. Yet, despite its influence, the methodology underpinning PISA has received significant criticism. Much of this criticism has…

This work systematically reviews teacher assessment literacy measures within the context of contemporary teacher evaluation policy. In this study, the researchers collected objective tests of assessment knowledge, teacher self-reports, and rubrics to evaluate teachers' work in assessment literacy studies from 1991 to 2012. Then they evaluated…

Views on testing--its purpose and uses and how its data are analyzed--are related to one's perspective on test takers. Test takers can be viewed as learners, examinees, or contestants. I briefly discuss the perspective of test takers as learners. I maintain that much of psychometrics views test takers as examinees. I discuss test takers as a…

As part of Nebraska's assessment and accountability system, districts' local assessment systems are evaluated for their psychometric quality. This article provides an overview of a two-stage evaluation strategy, discusses how it was applied in Nebraska, and presents results from the first three years of the evaluation process. Benefits of the…

Presents two approaches to evaluating gender measurement bias in mathematics testing, and discusses how to assess these approaches. The two approaches are the study of internal test structure and external test relationships. Ensuring a gender-fair test requires attention to both areas. (SLD)

The history of generalizability theory (G theory) is told from the perspective of one researcher's experiences, describing psychometric and scientific perspectives that influenced the development of G theory and its adoption. Work that remains to be done in the field is outlined. (SLD)

The historic origins of measurement are traced, and the basic requirements for fundamental measurement are outlined. Studying the history of social science measurement shows specific pitfalls the researcher must avoid. Applications of item response theory to social science measurement are discussed. (SLD)

A conceptual framework is proposed for a psychometric theory of standard setting. The framework suggests that participants in a standard setting process (panelists) develop an internal, intended standard as a result of training and the participant's background. The goal of a standard setting process is to convert panelists' intended standards to…

This brief history traces the development of item response theory (IRT) from concepts originating in 19th-century mathematics and psychology to present-day principles drawn from statistical estimation theory. Connections to other fields and current trends in IRT are outlined. (SLD)

Research on psychometric methods is heavily dependent on software. The quality, availability, and documentation of such software are critical to the advancement of the field. In 2000, an ad hoc committee of NCME recommended that NCME adopt policies that promote greater availability and better documentation of software. This article follows the ad…

Classical test theory is founded on the proposition that measurement error, a random latent variable, is a component of the observed score random variable. This article traces the history of the development of classical test theory, beginning in the early 20th century. (SLD)