A new variant of the iterative "data = fit + residual" data-analytical approach described by Mosteller and Tukey is proposed and implemented in the context of item response theory psychometric models. Posterior probabilities from a Bayesian mixture model of a Rasch item response theory model and an unscalable latent class are expressed…

Paul E. Newton's "Clarifying the Consensus Definition of Validity" addresses the single most important, yet stubbornly protean, value in educational and psychological assessment. "Standards for Educational and Psychological Testing" (American Educational Research Association, American Psychological Association, & National Council on Measurement in…

From a contemporary perspective on cognition, the between-persons variables in trait-based arguments in educational assessment are absurd over-simplifications. Yet, for a wide range of applications, they work. Rather than seeing such variables as independently-existing characteristics of people, we can view them as summaries of patterns in…

Advances in cognitive research increase the need for assessment that can address the processes and the strategies by which persons solve problems. Several psychometric models have been introduced to handle claims cast in information-processing terms, explicitly modeling performance in terms of theory-based predictions of performance. Cognitively…

This article addresses issues in model building and statistical inference in the context of student modeling. The use of probability-based reasoning to explicate hypothesized and empirical relationships and to structure inference in the context of proportional reasoning tasks is discussed. Ideas are illustrated with an example concerning…

An active area in psychometric research is coordinated task design and statistical analysis built around cognitive models. Compared with classical test theory and item response theory, there is often less information from observed data about the measurement-model parameters. On the other hand, there is more information from the grounding…

Probability-based inference in complex networks of interdependent variables is an active topic in statistical research, spurred by such diverse applications as forecasting, pedigree analysis, troubleshooting, and medical diagnosis. This paper concerns the role of Bayesian inference networks for updating student models in intelligent tutoring…

Recent developments in cognitive psychology suggest models for knowledge and learning that often fall outside the realm of standard test theory. This paper concerns probability-based inference in terms of such models. The essential idea is to define a space of "student models"--simplified characterizations of students' knowledge, skill,…

Educational assessment concerns inference about students' knowledge, skills, and accomplishments. Because data are never so comprehensive and unequivocal as to ensure certitude, test theory evolved in part to address questions of weight, coverage, and import of data. The resulting concepts and techniques can be viewed as applications of more…

Standard item response theory (IRT) models posit latent variables to account for regularities in students' performance on test items. They can accommodate learning only if the expected changes in performance are smooth, and, in an appropriate metric, uniform over items. Wilson's "Saltus" model extends the ideas of IRT to development that…

A model is presented for item responses when different subjects use different strategies, but only responses--not choice of strategy--can be observed. Substantive theory is used to differentiate the likelihoods of response vectors under a fixed set of strategies, and response probabilities are modeled via item parameters for each strategy. (TJH)

The challenges of modeling students' performance in simulation-based assessments include accounting for multiple aspects of knowledge and skill that arise in different situations and the conditional dependencies among multiple aspects of performance in a complex assessment. This paper describes a Bayesian approach to modeling and estimating…

Educational assessment concerns inference about student knowledge, skills, and accomplishments. Test theory has evolved in part to address questions of weight, coverage, and import of data. Resulting concepts and techniques can be viewed as applications of more general principles for inference in the presence of uncertainty. (SLD)