This report includes three papers that address critical "next generation" issues in assessment policy that can help to guide the choices made about system design: computer adaptive assessments, assessment of English learners and assessing science. None of these topics has received the attention that it deserves in the current debate on assessment…

The purpose of this study was to answer the question: Are principals good at identifying effective teachers? Some studies have suggested they are not, but the evidence is not consistent. It is troubling that research results are inconsistent regarding principals' abilities to identify effective teachers. Why is there a disconnect between…

Technical adequacy and information/cost return were examined for four early reading measures: the Dynamic Indicators of Basic Early Literacy Skills (DIBELS), STAR Early Literacy (SEL), Group Reading Assessment and Diagnostic Evaluation (GRADE), and the Texas Primary Reading Inventory (TPRI). All four assessments were administered to the same…

This study investigated the effects of using modified items in achievement tests to enhance accessibility. An experiment determined whether tests composed of modified items would reduce the performance gap between students eligible for an alternate assessment based on modified achievement standards (AA-MAS) and students not eligible, and the…

When power tests include a time limit, it is important to assess the possibility of "speededness" for examinees. Research on differential speededness in the past has included looking at gender and ethnic subgroups in the United States on paper and pencil tests. The needs of a global audience necessitated, and the availability of computer…

The number right (NR) method, in which students pick one option as the answer, is the conventional method for scoring multiple-choice tests that is heavily criticized for encouraging students to guess and failing to credit partial knowledge. In addition, computer technology is increasingly used in classroom assessment. This paper investigates the…

Computer-based testing (CBT) to assess second language ability has undergone remarkable development since Garret (1991) described its purpose as "the computerized administration of conventional tests" in "The Modern Language Journal." For instance, CBT has made possible the delivery of more authentic tests than traditional paper-and-pencil tests.…

Test security has been a concern for computerized adaptive tests (CAT) due to the nature of continuous testing. This concern becomes unprecedentedly severe with increasingly easy access to the World-Wide-Web where some examinees post on the internet their recollections of items they are administered, leaving future examinees with opportunities to…

After a brief review of recent advances in sequential analysis involving sequential generalized likelihood ratio tests, we discuss their use in psychometric testing and extend the asymptotic optimality theory of these sequential tests to the case of sequentially generated experiments, of particular interest in computerized adaptive testing. We…

This article presents an application of Monte Carlo methods for developing and assembling multistage adaptive tests (MSTs). A major advantage of the Monte Carlo assembly over other approaches (e.g., integer programming or enumerative heuristics) is that it provides a uniform sampling from all MSTs (or MST paths) available from a given item pool.…

Computer adaptive testing (CAT) is theoretically sound and efficient, and is commonly seen in larger testing programs. It is, however, rarely seen in a smaller-scale scenario, such as in classrooms or business daily routines, because of the complexity of most adopted Item Response Theory (IRT) models. While the Sequential Probability Ratio Test…

The popularity of intelligent tutoring systems (ITSs) is increasing rapidly. In order to make learning environments more efficient, researchers have been exploring the possibility of an automatic adaptation of the learning environment to the learner or the context. One of the possible adaptation techniques is adaptive item sequencing by matching…

This study compares the amount of test anxiety experienced on a computerized adaptive test (CAT) to a paper-and-pencil test (P&P), as well as the state test anxiety experienced between males and females. Ninety-four middle school CAT examinees were compared to 65 middle school P&P examinees on their responses to the State-Trait Anxiety…

Individuals with moderate to severe TBI often need extensive rehabilitation. To verify the effectiveness of intervention and design rehabilitation programs that meet individual's needs, precise and efficient outcome measures are crucial. Current assessments for TBI either focus on measuring impairments, such as neuropsychological tests or lack of…

This paper provides a history of ETS's role in developing assessment instruments and psychometric procedures for measuring change in large-scale national assessments funded by the Longitudinal Studies branch of the National Center for Education Statistics. It documents the innovations developed during more than 30 years of working with…