Many large-scale educational surveys have moved from linear form design to multistage testing (MST) design. One advantage of MST is that it can provide more accurate latent trait [theta] estimates using fewer items than required by linear tests. However, MST generates incomplete response data by design; hence, questions remain as to how to…

This study investigates how different rounding rules and ways of providing Angoff standard-setting judgments affect cut-scores. A simulation design based on data from the National Assessment of Education Progress was used to investigate how rounding judgments to the nearest whole number (e.g., 0, 1, 2, etc.), nearest 0.05, or nearest two decimal…

Trend estimation in international comparative large-scale assessments relies on measurement invariance between countries. However, cross-national differential item functioning (DIF) has been repeatedly documented. We ran a simulation study using national item parameters, which required trends to be computed separately for each country, to compare…

This dissertation addresses two research questions: 1. Do states misrepresent their progress on their own state assessments? 2. If states do distort their progress, are their predictors to suggest why this distortion occurs? The first research question requires that distortion be defined. For the purposes of this dissertation I calculated the…

To find population proficiency distributions, a two-level hierarchical linear model may be applied to large-scale survey assessments such as the National Assessment of Educational Progress (NAEP). The model and parameter estimation are developed and a simulation was carried out to evaluate parameter recovery. Subsequently, both a hierarchical and…

"Harvard Education Letter" is published bimonthly at the Harvard Graduate School of Education. This issue of "Harvard Education Letter" contains the following articles: (1) Online Testing, Version 1.0: Oregon's Adaptive Computer-Based Accountability Test Offers a Peek at a Brave New Future (Robert Rothman); (2) Beyond…

Extended an Item Response Theory (IRT) method for detection of differential item functioning to the partial credit model and applied the method to simulated data using a stepwise procedure. Then applied the stepwise DIF analysis based on the multiple-group partial credit model to writing trend data from the National Assessment of Educational…

A study was conducted to extend the sequential probability ratio testing (SPRT) procedure with the polytomous model under some practical constraints in computerized classification testing (CCT), such as methods to control item exposure rate, and to study the effects of other variables, including item information algorithms, test difficulties, item…

The purposes of this study were to: (1) extend the sequential probability ratio testing (SPRT) procedure to polytomous item response theory (IRT) models in computerized classification testing (CCT); (2) compare polytomous items with dichotomous items using the SPRT procedure for their accuracy and efficiency; (3) study a direct approach in…

The partial credit model with a varying slope parameter is developed and called the generalized partial credit model (GPCM). Analysis results for simulated data by this and other polytomous item-response models demonstrate that the rating formulation of the GPCM is adaptable to the analysis of polytomous item responses. (SLD)

The extent to which properties of the ability scale and the form of the latent trait distribution influence the estimated item parameters of item response theory (IRT) was investigated using real and simulated data. Simulated data included 5,000 ability values randomly drawn from the standard normal distribution. Real data included the results for…

Develops and evaluates the linkage of the Armed Services Vocational Aptitude Battery to the mathematics scale of the National Assessment of Educational Progress. The accuracy of the proficiency distribution estimated from the projection was close to the accuracy of the distribution estimated from the large scale assessment. (SLD)

Studied the impact of changes in item difficulty on National Assessment of Educational Progress (NAEP) estimates over time through a Monte Carlo study that simulated the responses of 1990 NAEP mathematics respondents to 1990 and 1996 NAEP mathematics items. Results support the idea that these changes have not affected the NAEP trend line.…

Performance in assessing the unidimensionality of tests was examined for four methods: (1) W. F. Stout's procedure (1987); (2) the approach of P. W. Holland and P. R. Rosenbaum (1986); (3) linear factor analysis; and (4) non-linear factor analysis. Each method was examined and compared with the others using simulated and real test data. Seven data…

This paper proposes a new Information-Time index as the basis for item selection in computerized classification testing (CCT) and investigates how this new item selection algorithm can help improve test efficiency for item pools with mixed item types. It also investigates how practical constraints such as item exposure rate control, test…