This study investigated specific conditions under which out-of-scale information improves measurement precision and the factors that influence the degree of reliability gains and the amount of bias induced in the reported scores when out-of-scale information is used. In-scale information is information that an item provides for a composite trait…

Procedures used to establish the comparability of scores derived from the College Board Admissions Testing Program (ATP) computer adaptive Scholastic Aptitude Test (SAT) prototype and the paper-and-pencil SAT are described in this report. Both the prototype, which is made up of Verbal and Mathematics computer adaptive tests (CATs), and a form of…

The purpose of this study was to compare and evaluate five online pretest item calibration/scaling methods in computerized adaptive testing (CAT): (1) the marginal maximum likelihood estimate with one-EM cycle (OEM); (2) the marginal maximum likelihood estimate with multiple EM cycles (MEM); (3) Stocking's Method A (M. Stocking, 1988); (4)…

Two operational item pools were used to investigate the performance of stratum computerized adaptive tests (CATs) when items were assigned to strata based on empirical estimates of item difficulty or human judgments of item difficulty. Items from the first data set consisted of 54 5-option multiple choice items from a form of the ACT mathematics…

Error indices (bias, standard error of estimation, and root mean square error) obtained on different scales of measurement under different test termination rules in a computerized adaptive test (CAT) context were examined. Four ability estimation methods were studied: (1) maximum likelihood estimation (MLE); (2) weighted likelihood estimation…

Setting standards on educational tests is extremely challenging. The psychometric literature is replete with methods and guidelines for setting standards on educational tests; however, little attention has been paid to the process of setting standards on computerized adaptive tests (CATs). This lack of attention is unfortunate because CATs are…

The present study showed that Rasch-based adaptive tests--when item domains were finite and specifiable--had greater precision in domain score estimation than test forms created by random sampling of items. Results were replicated across four data sources representing a variety of criterion-referenced, domain-based tests varying in length.…

Computerized adaptive testing allows us to create a unique personalized test that matches the ability and knowledge of the test taker. (Author)

Scoring accuracy and item functioning were studied for an open-ended response type test in which correct answers can take many different surface forms. Results with 1,864 graduate school applicants showed automated scoring to approximate the accuracy of multiple-choice scoring. Items functioned similarly to other item types being considered. (SLD)

A group of covariance structure models was examined to ascertain the similarity between conventionally administered and computerized adaptive versions of the Differential Aptitude Test (DAT). Results for 332 students indicate that the computerized version of the DAT is an adequate representation of the conventional test battery. (TJH)

An extension of the multidimensional Rasch model for learning and change is presented that permits theories of processes and knowledge structures to be incorporated into the item response model. The extension resolves basic problems in measuring change and permits adaptive testing. The method is illustrated in a study of mathematical problem…

Simulated data were used to investigate the performance of modified versions of the Mantel-Haenszel method of differential item functioning (DIF) analysis in computerized adaptive tests (CAT). Results indicate that CAT-based DIF procedures perform well and support the use of item response theory-based matching variables in DIF analysis. (SLD)

Discusses the development and use of a computer-adaptive test (CAT) for learners of Chinese. Each time the examinee answers a question, the CAT program registers the answer and, based on the examinee's response, decides whether the next question should be harder or easier. (23 references) (MDM)

Fifth and sixth grade children were assigned to three pretesting conditions: a full-length pretest, an adaptive pretest that exited learners when nonmastery was indicated, and a no pretest control. Learners in the adaptive treatment demonstrated higher levels of performance on the posttest and had greater motivation to continue the instruction.…

Discusses computerized adaptive testing, which is designed to provide students with an individualized examination adjusted to their skill levels. Indicates that scores are derived by comparing correct answers with the answers' difficulty rating. Highlights the advantages and disadvantages of the system and provides guidelines for successful…