It has been widely reported that in computerized adaptive testing some examinees may get much lower scores than they would normally if an alternative paper-and-pencil version were given. The main purpose of this investigation is to quantitatively reveal the cause for the underestimation phenomenon. The logistic models, including the 1PL, 2PL, and…

When power tests include a time limit, it is important to assess the possibility of "speededness" for examinees. Research on differential speededness in the past has included looking at gender and ethnic subgroups in the United States on paper and pencil tests. The needs of a global audience necessitated, and the availability of computer…

This report includes three papers that address critical "next generation" issues in assessment policy that can help to guide the choices made about system design: computer adaptive assessments, assessment of English learners and assessing science. None of these topics has received the attention that it deserves in the current debate on assessment…

Computerized adaptive testing (CAT) is a mode of testing which enables more efficient and accurate recovery of one or more latent traits. Traditionally, CAT is built upon Item Response Theory (IRT) models that assume unidimensionality. However, the problem of how to build CAT upon latent class models (LCM) has not been investigated until recently,…

An adaptive testing method is presented that controls the speededness of a test using predictions of the test takers' response times on the candidate items in the pool. Two different types of predictions are investigated: posterior predictions given the actual response times on the items already administered and posterior predictions that use the…

The purpose of this study was to answer the question: Are principals good at identifying effective teachers? Some studies have suggested they are not, but the evidence is not consistent. It is troubling that research results are inconsistent regarding principals' abilities to identify effective teachers. Why is there a disconnect between…

Technical adequacy and information/cost return were examined for four early reading measures: the Dynamic Indicators of Basic Early Literacy Skills (DIBELS), STAR Early Literacy (SEL), Group Reading Assessment and Diagnostic Evaluation (GRADE), and the Texas Primary Reading Inventory (TPRI). All four assessments were administered to the same…

This study investigated the effects of using modified items in achievement tests to enhance accessibility. An experiment determined whether tests composed of modified items would reduce the performance gap between students eligible for an alternate assessment based on modified achievement standards (AA-MAS) and students not eligible, and the…

The number right (NR) method, in which students pick one option as the answer, is the conventional method for scoring multiple-choice tests that is heavily criticized for encouraging students to guess and failing to credit partial knowledge. In addition, computer technology is increasingly used in classroom assessment. This paper investigates the…

There is a great need to help test designers determine how to make tests that are accessible to individuals with disabilities. This report takes "design patterns", which were developed at SRI for assessment design, and uses them to clarify issues related to accessibility features for individuals with disabilities--such as low-vision and…

Although computerized adaptive tests have enjoyed tremendous growth, solutions for important problems remain unavailable. One problem is the control of item exposure rate. Because adaptive algorithms are designed to select optimal items, they choose items with high discriminating power. Thus, these items are selected more often than others,…

Computer-based testing (CBT) to assess second language ability has undergone remarkable development since Garret (1991) described its purpose as "the computerized administration of conventional tests" in "The Modern Language Journal." For instance, CBT has made possible the delivery of more authentic tests than traditional paper-and-pencil tests.…

Test security has been a concern for computerized adaptive tests (CAT) due to the nature of continuous testing. This concern becomes unprecedentedly severe with increasingly easy access to the World-Wide-Web where some examinees post on the internet their recollections of items they are administered, leaving future examinees with opportunities to…

After a brief review of recent advances in sequential analysis involving sequential generalized likelihood ratio tests, we discuss their use in psychometric testing and extend the asymptotic optimality theory of these sequential tests to the case of sequentially generated experiments, of particular interest in computerized adaptive testing. We…

This article presents an application of Monte Carlo methods for developing and assembling multistage adaptive tests (MSTs). A major advantage of the Monte Carlo assembly over other approaches (e.g., integer programming or enumerative heuristics) is that it provides a uniform sampling from all MSTs (or MST paths) available from a given item pool.…