This study examined the extent to which log-linear smoothing could improve the accuracy of differential item functioning (DIF) estimates in small samples of examinees. Examinee responses from a certification test were analyzed using White examinees in the reference group and African American examinees in the focal group. Using a simulation…

Differential item functioning (DIF) analysis is a statistical technique used for ensuring the equity and fairness of educational assessments. This study formulates a new DIF analysis method using the information similarity index (ISI). ISI compares item information functions when data fits the Rasch model. Through simulations and an international…

One method to correct for selection bias in comparing Scholastic Aptitude Test (SAT) scores among states is presented, which is a modification of J. J. Heckman's Selection Bias Correction (1976, 1979). Empirical results suggest that sample selection bias is present in SAT score regressions. (SLD)

It is suggested that some of the technology applied to state Scholastic Aptitude Test scores to measure states' educational performance (particularly use of a truncated Gaussian model) may make it possible to adjust National Assessment of Educational Progress (NAEP) scores to make inferences about state educational progress possible. (SLD)

Test bias detection methods based on item response theory (IRT) are reviewed. Five such methods are commonly used: (1) equality of item parameters; (2) area between item characteristic curves; (3) sums of squares; (4) pseudo-IRT; and (5) one-parameter-IRT. A table compares these and six newer or less tested methods. (SLD)

The extent to which performance assessments can have unintended and undesirable consequences is not yet clear, but preliminary evidence suggests several issues of bias and fairness. Among these is the sheer difficulty of scoring complex performance assessments, as Vermont's experiences with portfolio assessment illustrate. (SLD)

Barbara Jordan served as the hearing officer for three-day adversary evaluation hearings about the pros and cons of minimum competency testing (MCT). This report is the complete transcript of the first day of proceedings. James Popham and George Madaus presented the opening arguments for the pro team and con team, respectively. Michael Scriven,…

Unsupervised summative assessment has become a feature of the educational landscape in various educational jurisdictions around the world, including the state of Queensland in Australia. However, I suggest it is an invalid and unnecessary practice that can impact negatively on the affect of students, call for a reconsideration of its use, and…

This study was patterned after a previous study by Skaggs and Lissitz (1992) in which inconsistency of differential item functioning (DIF) was reported across test administrations. They suggested multidimensionality of test data as one possible reason for inconsistency. Therefore, in this study, DIF indices which were developed recently with a…

This study compares logistic discriminant function analysis for differential item functioning (DIF) with a technique for the detection of DIF that is based on item response theory rather than the Mantel-Haenszel procedure. In this study, the areas between the two item characteristic curves, also called the item characteristic curve method is…

Recent investigations of the extent to which students with disabilities are allowed to participate in major national data collections used in measurement-driven education reform suggest that 40 to 50 percent of students with disabilities are typically excluded from major assessments, although they are included to a greater degree in assessments…

Although testing accommodations are commonly provided to students with disabilities within large-scale testing programs, research findings on how well accommodations allow for comparable measurement of student knowledge and skill remain inconclusive. The purpose of this study was to examine the extent to which 1 commonly held belief about testing…

While studying testing programs in juvenile incarceratory facilities, the authors found that there may be racial bias in the tests, test items, and in the interpretation of the tests. This article presents new data that have emerged from their original study. The focus is on performance on standardized norm-referenced tests. (CT)

This article provides a brief primer overview of Differential Item Functioning (DIF) analysis. DIF analysis investigates a differential characteristic of a test item between subpopulations of examinees and is useful in detecting possibly biased items toward a particular subpopulation. As demonstration, a dataset from a 40-item math test in a…

Barbara Jordan served as the hearing officer for three-day adversary evaluation hearings about the pros and cons of minimum competency testing (MCT). This report is the complete transcript of the second day of proceedings. The pro team, lead by James Popham, began by presenting representatives of four states (Florida, California, Texas, and…