Knowledge space theory (KST) is a mathematical framework for modeling and assessing student knowledge. While KST has successfully served as the foundation of several learning systems, recent advancements in machine learning provide an opportunity to improve on purely KST-based approaches to assessing student knowledge. As such, in this work we…

We confronted 151, 5th and 6th elementary grade pupils with a quantitative problem in a mathematics or religion class, to examine the influence of the context on pupils' understanding and solution of such problems inside and outside the mathematics class. Pupils were first asked to solve a problem about fair sharing either during a mathematics or…

The interest in the application of large-scale computerized adaptive testing has served to focus attention on issues that arise when theoretical advances are made operational. Some of these issues stem less from changes in testing conditions and more from changes in testing paradigms. One such issue is that of the order in which questions are…

A polychotomous nominal response model-based computerized adaptive test (CAT) was simulated using data from 1,093 University of Texas students. The ability estimation of this model and its overall performance were compared with those of a dichotomous three-parameter logistic model-based CAT. Advantages and drawbacks of nominal response CAT are…

Latent trait model calibration procedures were used on data obtained from a group testing program. The one-parameter model of Wright and Panchapakesan and the three-parameter logistic model of Wingersky, Wood, and Lord were selected for comparison. These models and their corresponding estimation procedures were compared, using actual and simulated…

The ultimate goal of physics education research (PER) is to develop a theoretical framework to understand and improve the learning process. In this journey of discovery, assessment serves as our headlamp and alpenstock. It sometimes detects signals in student mental structures, and sometimes presents the difference between expert understanding and…

Computerized adaptive testing procedures (CATPs) based on the graded response method (GRM) of F. Samejima (1969) and the partial credit model (PCM) of G. Masters (1982) were developed and compared. Both programs used maximum likelihood estimation of ability, and item selection was conducted on the basis of information. Two simulated data sets, one…

A detailed description of five latent structure models of achievement measurement is presented. The first project paper, by David L. McArthur, analyzes the history of mental testing to show how conventional item analysis procedures were developed, and how dissatisfaction with them has led to fragmentation. The range of distinct conceptual and…

We concern ourselves with the hypothesis that two variables have a perfect disattenuated correlation, hence measure the same trait except for errors of measurement. This hypothesis is equivalent to saying, within the adopted model, that true scores of two psychological tests satisfy a linear relation. Statistical tests of this hypothesis are…

A study involving 1,000 simulated examinees compared the partial credit and graded response models in computerized adaptive testing (CAT). The graded response model fit the data well and provided slightly more accurate ability estimates than those of the partial credit model. Benefits of polytomous model-based CATs are discussed. (SLD)

Using one simulated and two real data sets, the effects of the systematic variation of the item-selection procedure and the stepsize method on the operating characteristics of computerized adaptive testing (CAT) for instruments with polychotomously scored rating scale items were studied. The six rating scale CAT procedures used performed well.…

Two methods of interactive, computer-assisted testing methods for multiple-choice items were compared with each other and with conventional multiple-choice tests. The interactive testing methods compared were tailored testing and the respond-until-correct (RUC) item response method. In tailored testing, examinee ability is successively estimated…

Challenge studies can be used to see whether there is a causal relationship between an agent of interest and a response. An approach based on union-intersection testing is presented that allows researchers to examine observations on a single subject and test the hypothesis of interest. An application using psychological data is presented. (SLD)

The sequential probability ratio test (SPRT), developed by Abraham Wald, is one statistical model available for making mastery decisions during computer-based criterion referenced tests. The predictive validity of the SPRT was empirically investigated with two different and relatively large item pools with heterogeneous item parameters. Graduate…

The reliability of four branching algorithms commonly used in computer adaptive testing (CAT) was examined. These algorithms were: (1) maximum likelihood (MLE); (2) Bayesian; (3) modal Bayesian; and (4) crossover. Sixty-eight undergraduate college students were randomly assigned to one of the four conditions using the HyperCard-based CAT program,…