A test battery with two different levels of adaptation is presented: a within-subtest level for the selection of the items in the subtests and a between-subtest level to move from one subtest to the next. The battery runs on a two-level model consisting of a regular response model for each of the subtests extended with a second level for the joint…

Designing a theory-based assessment with sound psychometric qualities to measure a higher-order cognitive construct is a highly desired yet challenging task for many practitioners. This paper proposes a framework for designing a theory-based assessment to measure a higher-order cognitive construct. This framework results in a modularized yet…

Twelve structural models of similarity were fitted to data from conventional and computer adaptive test (CAT) batteries measuring the same aptitude in a double cross-validation design. Three of the 12 models, including a multiplicative structure model, performed well, providing support for using CATs as replacements for conventional tests. (NSF)

An adaptive test can usually match or exceed the measurement precision of conventional tests several times its length. This increased efficiency is not without costs, however, as the models underlying adaptive testing make strong assumptions about examinees and items. Most troublesome is the assumption that item pools are unidimensional. Truly…

Test validity is a concept that has often been ignored in the context of latent trait models and in modern test theory, particularly as it relates to computerized adaptive testing. Some considerations about the validity of a test and of a single item are proposed. This paper focuses on measures that are population-free and that will provide local…

Extensions of unidimensional appropriateness indices are developed for multiunidimensional tests (multidimensional tests composed of unidimensional subtests). Simulated and real data (scores of 2,978 students on the Armed Services Vocational Aptitude Battery) were used to evaluate the indices' effectiveness in determining individuals who are…

Recent experience with the Computerized Adaptive Test (CAT) has raised a number of concerns about its practical applications. The concerns are principally involved with the concept of having the computer construct the test from a precalibrated item pool, and substituting statistical characteristics for the test developer's skills. Problems with…

A theorem is developed to show that the Kruskal-Wallis test (W. Kruskal and W. Wallis, 1952) (KWt) can be used validly to test a hypothesis about location under much less restrictive conditions than required by the recently accepted, mathematically correct shift model. The specific null hypothesis for which the KWt can be used validly is…

TAILOR is a computer program that uses the implied orders concept as the basis for computerized adaptive testing. The basic characteristics of TAILOR, which does not involve pretesting, are reviewed here and two studies of it are reported. One is a Monte Carlo simulation based on the four-parameter Birnbaum model and the other uses a matrix of…

Hierarchical (adaptive) and linear methods of testlet construction were compared. The performance of 2,080 ninth and tenth graders on a 4-item testlet was used to predict performance on the entire test. The adaptive test was slightly superior as a predictor, but the cost of obtaining that superiority was considerable. (SLD)

This paper is the final report of a multi-year project sponsored by the Office of Naval Research (ONR) in 1987 through 1990. The main objectives of the research summarized were to: investigate the non-parametric approach to the estimation of the operating characteristics of discrete item responses; revise and strengthen the package computer…

An elementary survey of item characteristic curve theory, centered around the problems of individualized (tailored) testing, is presented. Following the introduction, discussions are provided of the following: Test Theory for Itemized Tests; The Guttman Scale; Item Characteristic Curve Theory; An Alternative Model; Specialization, Application, and…

This report is the Proceedings of the third conference of its type. Included are 23 of the 25 papers presented at the conference, discussion of these papers by invited discussants, and symposium papers by a group of leaders in adaptive testing and latent trait test theory research and applications. The papers are organized into the following…

Tailored testing procedures for achievement testing were applied in a situation that failed to meet some of the specifications generally considered to be necessary for tailored testing. Discrepancies from the appropriate conditions included the use of small samples for calibrating items, and the use of an item pool that was not designed to be…

The strategy for investigating convergent and discriminant test validity, known as the multitrait-multimethod matrix, is investigated. A nonparametric significance testing procedure is suggested and demonstrated. (JKS)