Background/Context: This article explains the idea of a neopragmatic postmodernist test theory and offers some thoughts about what changing notions concerning the nature of and meanings assigned to knowledge imply for educational assessment, present and future. Purpose: Advances in the learning sciences--particularly situative and sociocognitive…

This article presents the author's observations on Neil Dorans's NCME Career Award Address: "The Contestant Perspective on Taking Tests: Emanations from the Statue within." He calls attention to some points that Dr. Dorans made in his address, and offers his thoughts in response.

Standardizing aspects of assessments has long been recognized as a tactic to help make evaluations of examinees fair. It reduces variation in irrelevant aspects of testing procedures that could advantage some examinees and disadvantage others. However, recent attention to making assessment accessible to a more diverse population of students…

People use external knowledge representations (EKRs) to identify, depict, transform, store, share, and archive information. Learning how to work with EKRs is central to becoming proficient in virtually every discipline. As such, EKRs play central roles in curriculum, instruction, and assessment. Five key roles of EKRs in educational assessment are…

The models of standard test theory, having evolved under a trait-oriented psychology, do not reflect the knowledge structures and the problem-solving strategies now seen as central to understanding performance and learning. In some applications, however, key qualitative distinctions among persons as to structures and strategies can be expressed…

In educational assessment, what students say, do, and sometimes make is observed, and assessors attempt to infer what students know, can do, or have accomplished more generally. Some links in the chain of inference depend on statistical models and probability-based reasoning, and it is with these links that terms such as validity, reliability, and…

Psychologist Andrea diSessa coined the term "phenomenological primitives", or p-prims, to talk about nonexperts' reasoning about physical situations. P-prims are primitive in the sense that they stand without significant explanatory substructure or explanation. Examples are "Heavy objects fall faster than light objects" and "Continuing force is…

This article addresses issues in model building and statistical inference in the context of student modeling. The use of probability-based reasoning to explicate hypothesized and empirical relationships and to structure inference in the context of proportional reasoning tasks is discussed. Ideas are illustrated with an example concerning…

Educational test theory consists of statistical and methodological tools to support inferences about examinees' knowledge, skills, and accomplishments. The evolution of test theory has been shaped by the nature of users' inferences which, until recently, have been framed almost exclusively in terms of trait and behavioral psychology. Progress in…

Developments in cognitive and developmental psychology have broadened the inferences researchers want to make about students' learning and the nature and acquisition of knowledge. The principles of inference that led to standard test theory can support inference in the broader context of the cognitive revolution. (SLD)

Relationships between Bayesian ability estimates and the parameters of a normal population distribution are derived in the context of classical test theory. Analogies are provided for use as approximations in work with item response theory (IRT). The following issues are addressed: (1) the relationship between the distribution of the latent…

Large-scale educational assessments differ from familiar educational measurements by attempting to provide information about the levels and natures of skills in populations rather than in individuals. That the distinct purposes of assessment require different methodologies than individual measurement was recognized by the development of…

Relationships between Bayesian ability estimates and the parameters of a normal population distribution are derived in the context of classical test theory. Formulas are presented for practical work with Bayesian ability estimates, and a numerical illustration is provided. (SLD)

Test theory encompasses models and methods for drawing inferences about what students know and can do, cast in a framework of ideas from measurement, education, and psychology. The emerging paradigm of cognitive psychology prompts new considerations about collecting and interpreting evidence, suggesting alternative models for the nature,…

Educational assessments that exploit advances in technology and cognitive psychology can produce observations and pose student models that outstrip familiar test-theoretic models and analytic methods. Bayesian inference networks (BINs), which include familiar models and techniques as special cases, can be used to manage belief about students'…