In educational and psychological research, we are often interested in discrete latent states of individuals responding to an assessment (e.g., proficiency or non-proficiency on educational standards, the presence or absence of a psychological disorder). Diagnostic classification models (DCMs; also called cognitive diagnostic models [CDMs]) are a…

In this digital ITEMS module, Dr. Roy Levy describes Bayesian approaches to psychometric modeling. He discusses how Bayesian inference is a mechanism for reasoning in a probability-modeling framework and is well-suited to core problems in educational measurement: reasoning from student performances on an assessment to make inferences about their…

Despite the importance of both response probability and response time for testing models of choice, there is a dearth of chronometric studies examining systematic asymmetries that occur over time- and space-orders in the method of paired comparisons. In this study, systematic asymmetries in discriminating the magnitude of paired visual stimuli are…

Pseudo Bayes probability estimates are weighted averages of raw and modeled probabilities; these estimates have been studied primarily in nonpsychometric contexts. The purpose of this study was to evaluate pseudo Bayes probability estimates as applied to the estimation of psychometric test score distributions and chained equipercentile equating…

This paper provides a history of ETS's role in developing assessment instruments and psychometric procedures for measuring change in large-scale national assessments funded by the Longitudinal Studies branch of the National Center for Education Statistics. It documents the innovations developed during more than 30 years of working with…

Presents a fully Bayesian analysis for the Probability Matrix Decomposition (PMD) model using the Gibbs sampler. Identifies the advantages of this approach and illustrates the approach by applying the PMD model to opinions of respondents from different countries concerning the possibility of contracting AIDS in a specific situation. (SLD)

Advances in cognitive research increase the need for assessment that can address the processes and the strategies by which persons solve problems. Several psychometric models have been introduced to handle claims cast in information-processing terms, explicitly modeling performance in terms of theory-based predictions of performance. Cognitively…

I. Klugkist, O. Laudy, and H. Hoijtink (2005) presented a Bayesian approach to analysis of variance models with inequality constraints. Constraints may play 2 distinct roles in data analysis. They may represent prior information that allows more precise inferences regarding parameter values, or they may describe a theory to be judged against the…

Standard item response theory (IRT) models posit latent variables to account for regularities in students' performance on test items. They can accommodate learning only if the expected changes in performance are smooth, and, in an appropriate metric, uniform over items. Wilson's "Saltus" model extends the ideas of IRT to development that…

Educational assessment concerns inference about student knowledge, skills, and accomplishments. Test theory has evolved in part to address questions of weight, coverage, and import of data. Resulting concepts and techniques can be viewed as applications of more general principles for inference in the presence of uncertainty. (SLD)

This paper describes two procedures for making binary classification decisions using tailored testing: the sequential probability ratio test (SPRT) and a Bayesian decision procedure. The first procedure described, the SPRT, was developed by Wald for quality control work. It has not been widely applied for testing applications because the…

The assumptions of the Criterion-Referenced Test (CRT) model proposed by Kriewall are compared to those of Emrick and Adam's Mastery-Learning (ML) model. Testing, in the context of instructional management, serves three general purposes: performance evaluation (achievement of objectives), placement (classification of students for instruction), and…

Summarizing work which is part of an Army research program on Methodological Issues in the Construction of Criterion Referenced Tests, the focus of this paper is on a Bayesian model, which gives the probability of correctly classifying an examiner as a master or as a nonmaster while taking into consideration the test length and the mastery cut-off…

This primer is an introduction to item response theory (also called item characteristic curve theory, or latent trait theory) as it is used most commonly--for scoring multiple choice achievement or aptitude tests. Written for the testing practitioner with minimum training in statistics and psychometrics, it presents and illustrates the basic…