In large-scale educational assessment data consistent with a simple-structure multidimensional item response theory (MIRT) model, where every item measures only one latent variable, separate unidimensional item response theory (UIRT) models for each latent variable are often calibrated for practical reasons. While this approach can be valid for…

Many large-scale educational surveys have moved from linear form design to multistage testing (MST) design. One advantage of MST is that it can provide more accurate latent trait [theta] estimates using fewer items than required by linear tests. However, MST generates incomplete response data by design; hence, questions remain as to how to…

Reviewed methods of estimating theta suitable for computerized adaptive testing (CAT) and discussed the differences between Fisher and Kullback-Leibler information criteria for selecting items. Examined the accuracy of different CAT algorithms using samples from the National Assessment of Educational Progress. Results show when correcting for…

If examinees are permitted to choose to answer a subset of the questions on a test, just knowing which questions were chosen can provide a measure of proficiency that may be as reliable as would have been obtained from the test graded traditionally. This new method of scoring is much less time consuming and expensive for both the examinee and the…

A conceptual framework and heuristic model for considering the existence, magnitude, and consequences of context effects are presented through an extension of some generalizability theory concepts. Context effects are often misunderstood, and current measurement models have serious limitations for examining them. Their importance needs to be…

This report is a summary of a March 1994 meeting to agree on guidelines for making inclusion and accommodation decisions concerning students with disabilities in national and state large-scale assessments. Much of the discussion at the meeting focused on the National Assessment of Educational Progress. Factors that lead to the exclusion of…

This study introduced a probabilistic model utilizing item response theory (IRT) for dealing with a variety of misconceptions. The model can be used for evaluating the transition behavior of error types, advancement of learning stages, or the stability and persistence of particular misconceptions. Moreover, it apparently can be used for relating…

The current technology of computerized testing is discussed, and a few comments are made on how such technology might be used for assessing school-related skills as part of the National Assessment of Educational progress (NAEP). The critical feature of computerized assessment procedures is that the test items are presented in interactive fashion,…

This report examines the potential of adaptive testing, two?-stage testing in particular, for improving the data quality of the National Assessment of Educational Progress (NAEP). Following a discussion of the rationale for adaptive testing in assessment and a review of previous studies of two-?stage testing, this report describes a 1993 Ohio…

This paper discusses the rationale for enhancing the current National Assessment of Educational Progress (NAEP) design by adding a capacity for adaptive testing. Items are tailored to the achievement level of the student in adaptive testing. The report describes a 1993 Ohio field trail of two-stage assessment carried out by the National Opinion…

This paper reviews the salient characteristics of the current inventory of large-scale achievement tests, analyzes two recent proposals for creating a national testing program, and describes a new approach to statewide and nationwide testing that RAND is examining. Section 2 discusses the criteria typically used for evaluating large-scale testing…

Four researchers at the Educational Testing Service describe what they consider some of the most vexing research problems they face. While these problems are not completely statistical, they all have major statistical components. Following the introduction (section 1), in section 2, "Problems with the Simultaneous Estimation of Many True…

This report examines the potential of adaptive testing, two-stage testing in particular, for improving the data quality of the National Assessment of Educational Progress (NAEP). Following a discussion of the rationale for adaptive testing in assessment and a review of previous studies of two-stage testing, this report describes a 1993 Ohio field…

The value of statistical research depends on valid comparisons which can usefully influence educational policy. Educational research needs to extend the measures of learning (such as the National Assessment of Educational Progress) through nationally-calibrated absolute measures and through computer-assisted and adaptive testing. Direct sampling…