This article discusses the main purposes of reading comprehension assessment and identifies the key features of good assessment. The article also identifies pitfalls that clinicians and educators should avoid to conduct valid assessments of reading comprehension, such as the degree to which the measure taps the constructive and integrative…

With advances in computerbased technologies and the emergence of elearning, there are unprecedented opportunities to reconsider assessment of learning (and, axiomatically, of teaching) and how this can be undertaken. One approach is adaptive assessment. Although it has existed in the tertiary environment since the time of the oral examination,…

A multiple choice test item is identified as flawed if it has no single best answer. In spite of extensive quality control procedures, the administration of flawed items to test-takers is inevitable. Common strategies for dealing with flawed items in conventional testing, grounded in the principle of fairness to test-takers, are reexamined in the…

The interest in the application of large-scale computerized adaptive testing has served to focus attention on issues that arise when theoretical advances are made operational. Some of these issues stem less from changes in testing conditions and more from changes in testing paradigms. One such issue is that of the order in which questions are…

Two-stage and multistage test designs provide a way of roughly adapting item difficulty to test-taker ability. All test takers take a parallel stage-one test, and, based on their scores, they are routed to tests of different difficulty levels in subsequent stages. These designs provide some of the benefits of standard computerized adaptive testing…

In computerized adaptive testing, updating parameter estimates using adaptive testing data is often called online calibration. In this paper, how to evaluate whether the adaptive testing model used for online calibration fits the item response model used sufficiently is studied. Three approaches are investigated, based on a Lagrange multiplier…

R. J. Owen (1975) proposed an approximate empirical Bayes procedure for item selection in adaptive testing. The procedure replaces the true posterior by a normal approximation with closed-form expressions for its first two moments. This approximation was necessary to minimize the computational complexity involved in a fully Bayesian approach, but…

This paper synthesizes ideas from the fields of graphical modeling and education testing, particularly item response theory (IRT) applied to computerized adaptive testing (CAT). Graphical modeling can offer IRT a language for describing multifaceted skills and knowledge, and disentangling evidence from complex performances. IRT-CAT can offer…

The use of computers to build diagnostic inferences is explored in two contexts. In computerized monitoring of liquid oxygen systems for the space shuttle, diagnoses are exact because they can be derived within a world which is closed. In computerized classroom testing of reading comprehension, programs deliver a constrained form of adaptive…

The purpose of this paper is to identify and discuss some of the problems presented by the use of computerized adaptive testing (CAT) in an instructional programs environment versus large scale testing applications, and to describe an actual implementation of CAT in an instructional programs setting. This particular application is in the…

The relationship between selected subtests from the Armed Services Vocational Aptitude Battery (ASVAB) and corresponding subtests administered as computerized adaptive tests (CAT) was investigated using a sample of Marine recruits. Results showed that the CAT subtest scores correlated as well with initial ASVAB scores as did ASVAB retest scores,…

Computerized adaptive testing procedures (CATPs) based on the graded response method (GRM) of F. Samejima (1969) and the partial credit model (PCM) of G. Masters (1982) were developed and compared. Both programs used maximum likelihood estimation of ability, and item selection was conducted on the basis of information. Two simulated data sets, one…

The specification of a computerized adaptive test, like the specification of computer-assisted instruction, is easier and can be done by personnel who are not proficient in computer programming if an authoring language is provided. The Minnesota Computerized Adaptive Testing Language (MCATL) is an authoring language specifically designed for…

In connection with the development of item pools for the Navy's experimental Computerized Adaptive Testing (CAT) system, items in five different content areas (General Science, Paragraph Composition, Word Knowledge, Math Knowledge, and Arithmetic Reasoning) have been calibrated using item response theory (IRT) methodology. Operational Armed…

A description is provided of the experiences of three community colleges in implementing computerized adaptive testing to assess the entry-level skills of students. Chapter 1 provides background information on the project, which utilized The College Board's Computerized Placement Tests (CPT's), a battery of untimed, individualized tests of reading…