In typical differential item functioning (DIF) assessments, an item's DIF status is not influenced by its status in previous test administrations. An item that has shown DIF at multiple administrations may be treated the same way as an item that has shown DIF in only the most recent administration. Therefore, much useful information about the…

Differential item functioning (DIF) analysis is a key component in the evaluation of the fairness and validity of educational tests. Although it is often assumed that refinement of the matching criterion always provides more accurate DIF results, the actual situation proves to be more complex. To explore the effectiveness of refinement, we…

This study demonstrates how the stability of Mantel-Haenszel (MH) DIF (differential item functioning) methods can be improved by integrating information across multiple test administrations using Bayesian updating (BU). The authors conducted a simulation that showed that this approach, which is based on earlier work by Zwick, Thayer, and Lewis,…

Several researchers (e.g., Klein, Hamilton, McCaffrey, & Stecher, 2000; Koretz & Barron, 1998; Linn, 2000) have asserted that test-based accountability, a crucial component of U.S. education policy, has resulted in score inflation. This inference has relied on comparisons with performance on other tests such as the National Assessment of…

Developed an empirical Bayes enhancement to Mantel-Haenszel (MH) analysis of differential item functioning (DIF) in which it is assumed that the MH statistics are normally distributed and that the prior distribution of underlying DIF parameters is also normal. (Author/SLD)

Studied a method for flagging differential item functioning (DIF) based on loss functions. Builds on earlier research that led to the development of an empirical Bayes enhancement to the Mantel-Haenszel DIF analysis. Tested the method through simulation and found its performance better than some commonly used DIF classification systems. (SLD)

Differential item functioning (DIF) assessment procedures for items with more than two ordered score categories, referred to as polytomous items, were evaluated. Three descriptive statistics (standardized mean difference and two procedures based on the SIBTEST computer program) and five inferential procedures were used. Conditions under which the…

Recent simulations have shown that, for a given sample size, the Mantel-Haenszel (MH) variances tend to be larger when items are administered to randomly selected examinees than when they are administered adaptively. Results suggest that adaptive testing may lead to more efficient application of MH differential item functioning analyses. (SLD)

The mixture index of fit (T. Rudas et al, 1994) is used to estimate the fraction of a population for which differential item functioning (DIF) occurs, and this approach is compared to the Mantel Haenszel test of DIF. The proposed noniterative procedure provides information about data portions contributing to DIF. (SLD)

Used a simulation to investigate the applicability to computerized adaptive test data of a differential item functioning (DIF) analysis method. Results show the performance of this empirical Bayes enhancement of the Mantel Haenszel DIF analysis method to be quite promising. (SLD)

The Mantel Haenszel (MH; 1959) approach of Holland and Thayer (1988) is a well-established method for assessing differential item functioning (DIF). The formula for the variance of the MH DIF statistic is based on work by Phillips and Holland (1987) and Robins, Breslow, and Greenland (1986). Recent simulation studies showed that the MH variances…

Two possible standard error formulas for the polytomous differential item functioning index proposed by N. J. Dorans and A. P. Schmitt (1991) were derived. These standard errors, and associated hypothesis-testing procedures, were evaluated through simulated data. The standard error that performed better is based on N. Mantel's (1963)…

Compared four mapping item methods using data from the physical science test of the National Assessment of Educational Progress and studied the opinions of science content area experts about the difficulty of the items through a survey completed by 148 science teachers or scientists. Results of model-based mapping methods were more concordant with…

Use of the Mantel-Haenszel procedure as a test for differential item functioning under the Rasch model of item-response theory is examined. Results of the procedure cannot be generalized to the class of items for which item-response functions are monotonic and local independence holds. (TJH)

Although the belief has been expressed that performance assessments are intrinsically more fair than multiple-choice measures, some forms of performance assessment may in fact be more likely than conventional tests to tap construct-irrelevant factors. As performance assessment grows in popularity, it will be increasingly important to monitor the…