In this study we compared five item selection procedures using three ability estimation methods in the context of a mixed-format adaptive test based on the generalized partial credit model. The item selection procedures used were maximum posterior weighted information, maximum expected information, maximum posterior weighted Kullback-Leibler…

Scoring rubrics are routinely used to evaluate the quality of writing samples produced for writing performance assessments, with anchor papers chosen to represent score points defined in the rubric. Although the careful selection of anchor papers is associated with best practices for scoring, little research has been conducted on the role of…

Proposed a definition of selection bias and studied the empirical relation between prediction bias and selection bias with respect to prominent social groups. Results show that, although the two biases are related, the relation is not isomorphic. It is mediated by the selection ratio, and for most selection ratios, it is only moderate. (SLD)

Explored how allowing examinees to select test questions affected examinee performance and test characteristics for a measure of ability to generate hypotheses about a situation. Results with 2,429 examinees who elected the choice condition on the Graduate Record Examination suggest that items are differentially attractive to examinees. (SLD)

This simulation investigated two procedures that reduce differences between paper-and-pencil testing and computerized adaptive testing (CAT) by making CAT content sensitive. Results indicate that the price in terms of additional test items of using constrained CAT for content balancing is much smaller than that of using testlets. (SLD)

A procedure to detect differential item functioning (DIF) is introduced that is suitable for tests with a cutoff score. DIF is assessed on a limited closed interval of thetas in which a cutoff score falls. How this approach affects the identification of DIF items is demonstrated with real data sets. (SLD)

An evolved form of the Edwards and Beckworth (1989) model for probability selection for Scholastic Achievement Test takers using truncated normal distributions is presented. It is shown that the arguments of Holland and Wainer are not sufficient to reject this model. (SLD)

The impact of capitalizing on chance in item selection on the accuracy of test information functions was studied through simulation, focusing on examinee sample size in item calibration and the ratio of item bank size to test length. (SLD)

The effects of presenting a choice of writing tasks on the quality of essays produced by eleventh graders were studied with 34,200 students in Georgia. The choice condition had no substantive effect on the quality of essays, but race, gender, and the writing task variable did. (SLD)

This paper discusses how to determine whether a test was developed in a legally defensible manner, reviewing general issues, specific cases bearing on different types of test use, some evaluative dimensions, and evidence of test quality. Tests constructed and used according to existing standards will generally stand legal scrutiny. (SLD)