The conventional two-group differential item functioning (DIF) analysis is extended to an analysis of variance-like (ANOVA-like) DIF analysis where multiple factors with multiple groups are compared simultaneously. Moreover, DIF is treated as a parameter to be estimated rather than simply a sign to be detected. This proposed approach allows the…

This study is a critical evaluation of the roles for coding and scoring of missing responses to multiple-choice items in educational tests. The focus is on tests in which the test-takers have little or no motivation; in such tests omitting and not reaching (as classified by the currently adopted operational rules) is quite frequent. Data from the…

Many arguments have been made against allowing examinees to review and change their answers after completing a computer adaptive test (CAT). These arguments include: (1) increased bias; (2) decreased precision; and (3) susceptibility of test-taking strategies. Results of simulations suggest that the strength of these arguments is reduced or…

This study investigated the effect on examinees' ability estimate under item response theory (IRT) when they are presented an item, have ample time to answer the item, but decide not to respond to the item. Simulation data were modeled on an empirical data set of 25,546 examinees that was calibrated using the 3-parameter logistic model. The study…

To ensure fairness, it is of critical importance that testing programs make sure that essay items given to examinees are equivalent in difficulty. The purpose of this study was to evaluate the stability and accuracy of a logistic regression based polytomous essay difficulty index. Preliminary results from a simulation study (9 conditions with a…

This paper applies specific information item selection using a method developed by T. Davey and M. Fan (2000) to a multiple-choice passage-based reading test that is being developed for computer administration. Data used to calibrate the multidimensional item parameters for the simulation study consisted of item responses from randomly equivalent…

Data from a 50-item translated test used for certification were used to assess the percentage and type of agreement between the Mantel-Haenszel (MH) and Differential Functioning of Items and Tests (DFIT) techniques for the detection of differential item functioning (DIF). The DFIT procedure flagged 10 of 30 items as exhibiting significant DIF…

With the advent of computer-based testing (CBT) and the need to increase the number of items available in computer adaptive test pools, the idea of item variants was conceived. An item variant can be defined as an item with content based on an existing item to a greater or lesser degree. Item variants were first proposed as a way to enhance test…

A simulation study was performed to determine whether a group's average percent correct in a content domain could be accurately estimated for groups taking a single test form and not the entire domain of items. Six Item Response Theory (IRT)-based domain score estimation methods were evaluated, under conditions of few items per content area per…

A method based on 0-1 linear programming (LP) is presented to stratify an item pool optimally for use in "alpha"-stratified adaptive testing. Because the 0-1 LP model belongs to the subclass of models with a network-flow structure, efficient solutions are possible. The method is applied to a previous item pool from the computerized…

Louisiana Educational Assessment Program for the 21st Century (LEAP 21) is an integral part of the Louisiana school and district accountability system passed by the state legislature and signed into law in 1997. In March 2004, students in grade 8 took LEAP 21 English Language Arts, Mathematics, Science, and Social Studies tests. This document…

Five types of multiple-choice items that can be used to assess higher-order educational objectives are examined. The item types do not exhaust the possibilities, but they are standard forms found helpful in writing items to measure more than recognitive memory. (Author/CM)

Ratings given to questionnaire items on four types of rating scales were compared. This study shows that the differences between scales are contingent upon the particular anchors used for the intermediate options. The results suggest that the mean score is predictably influenced by changes in the intermediate anchors. (Author/PN)

Assessed the validity of short forms that reduce the number of items within subtests rather than the number of subtests. Used data from the standardization samples for the Wechsler Intelligence Scale for Children, Wechsler Adult Intelligence Scale, Wechsler Preschool and Primary Scale of Intelligence, WISC-Revised, and WAIS-Revised. (Author)