When individuals are given tests that are too hard or too easy, the resulting scores are likely to be poor estimates of their performance. To get valid and accurate test scores that provide meaningful results, one should use functional-level testing (FLT). FLT is the practice of administering to an individual a version of a test with a difficulty…

Chapters in this book present the challenges and dilemmas faced by researchers as they created new computerized assessments, focusing on issues addressed in developing, scoring, and administering the assessments. Chapters are: (1) "Beyond Bells and Whistles; An Introduction to Computerized Assessment" (Julie B. Olson-Buchanan and Fritz Drasgow);…

Issues concerning the cost and feasibility of using Formulating Hypotheses (FH) test item types for the Graduate Record Examinations have slowed research into their use. This project focused on two major issues that need to be addressed in considering FH items for operational use: the costs of scoring and the assignment of scores along a range of…

A multiple choice test item is identified as flawed if it has no single best answer. In spite of extensive quality control procedures, the administration of flawed items to test-takers is inevitable. Common strategies for dealing with flawed items in conventional testing, grounded in the principle of fairness to test-takers, are reexamined in the…

Two-stage and multistage test designs provide a way of roughly adapting item difficulty to test-taker ability. All test takers take a parallel stage-one test, and, based on their scores, they are routed to tests of different difficulty levels in subsequent stages. These designs provide some of the benefits of standard computerized adaptive testing…

The purpose of this book is to make it possible for measurement specialists to solve practical testing problems through the use of item response theory (IRT). The topics, organization, and presentation are those used in a 4-week seminar held each summer for the past several years. The material is organized to facilitate understanding; all related…

Major efforts of this project fall into four categories: (1) investigations were performed on the relationship between the dimensionality of a dataset and its underlying cognitive processes; (2) two approaches for diagnosing erroneous rules of operation were developed (an "error vector" system for constructing error diagnostic programs…

Computerized adaptive testing (CAT), while well-grounded in psychometric theory, has had few large-scale applications for high-stakes, secure tests in the past. This is now changing as the cost of computing has declined rapidly. As is always true where theory is translated into practice, many practical issues arise. This paper discusses a number…

Issues that must be addressed in the large-scale application of computerized adaptive testing are explored, including considerations of test design, scoring, test administration, item and item bank development, and other aspects of test construction. Possible solutions and areas in which additional work is needed are identified. (SLD)

A computerized flexilevel test was implemented and its ability estimates were compared with those of a Bayesian estimation based computerized adaptive test (CAT) as well as with known true ability estimates. Results showed that when the flexilevel test was terminated according to Lord's criterion, its ability estimates were highly and…

In the summer of 1996, the U.S. Armed Services will implement the computerized adaptive testing version of the Armed Services Vocational Aptitude Battery (CAT-ASVAB). When conversion is completed at the 65 targeted military entrance processing stations, about half the applicants will take this test version. This paper describes the efforts that…

Dichotomous search strategies (DSSs) for computerized adaptive testing are similar to golden section search strategies (GSSSs). Each middle point of successive search regions is a testing point. After each item is administered, the subject's obtained score is compared with the expected score at successive testing points. If the subject's obtained…

Computerized adaptive testing (CAT) has become increasingly common in large-scale testing programs. This paper considers relevant practical issues that are likely to be faced by the developers and managers of a CAT program. The first cluster of issues is that of item pool development and maintenance. It includes such considerations as item pool…

During a three-year project (1977-1980) on computerized adaptive achievement testing, item characteristic curve theory (ICC) and adaptive testing strategies designed almost exclusively for ability testing were applied to achievement testing. Adaptive techniques substantially reduced test length without reducing quality, when applied to three…

This symposium consists of five papers and presents some recent developments in adaptive testing which have applications to several military testing problems. The overview, by James R. McBride, defines adaptive testing and discusses some of its item selection and scoring strategies. Item response theory, or item characteristic curve theory, is…