ERIC Number: ED575929
Record Type: Non-Journal
Publication Date: 2016
Pages: 174
Abstractor: As Provided
ISBN: 978-1-3696-5799-9
ISSN: N/A
EISSN: N/A
Available Date: N/A
Differential Item Functioning for Accommodated Students with Disabilities: Effect of Differences in Proficiency Distributions
Quesen, Sarah
ProQuest LLC, Ph.D. Dissertation, University of Pittsburgh
When studying differential item functioning (DIF) with students with disabilities (SWD) focal groups typically suffer from small sample size, whereas the reference group population is usually large. This makes it possible for a researcher to select a sample from the reference population to be similar to the focal group on the ability scale. Doing so may improve the accuracy of DIF detection. Using scores from a 60-item multiple choice math assessment administered statewide to eighth graders, this study examined the effect of similar "vs." dissimilar proficiency distributions on uniform DIF detection. Results from testing the similar- and different-ability reference groups with a SWD focal group were compared for four models: logistic regression, hierarchical generalized linear model, the Wald-1 IRT-based test, and the Mantel-Haenszel procedure. A DIF-free-then-DIF strategy, using a Wald-2 test to identify DIF-free anchor items, was used with these methods. The rate of DIF detection was examined for both similar and dissimilar distribution groups among all accommodated scores and the most common accommodation subcategories (extended time, frequent breaks, some/all items read aloud) to see if type of accommodation changed the rate of items flagged for DIF. The difference in item difficulty (delta DIF) was used to compare results among models, using the ETS' classification system, which served both as a measure of effect size and allowed for comparisons between models. No items were detected for DIF using the similar distribution reference group, regardless of method. No items were detected for DIF with either reference group when the IRT-based Wald-1 test was used. With the dissimilar reference group, logistic regression had the lowest rate of items flagged for DIF (<5%), Mantel Haenszel flagged 8-15% of items, and hierarchical generalized linear model flagged 23-38% of items for DIF. Forming focal groups by accommodation type did not alter the pattern of DIF detection observed among models. This study found that creating a reference group to be similar in ability to the focal group by purposefully sampling from the reference population might control the rate of erroneous DIF detection for SWD. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]
Descriptors: Test Items, Academic Accommodations (Disabilities), Testing Accommodations, Disabilities, Statistical Distributions, Sample Size, Accuracy, Scores, Multiple Choice Tests, Mathematics Tests, Grade 8, Middle School Students, Regression (Statistics), Hierarchical Linear Modeling, Item Response Theory, Error of Measurement, Test Bias
ProQuest LLC. 789 East Eisenhower Parkway, P.O. Box 1346, Ann Arbor, MI 48106. Tel: 800-521-0600; Web site: http://www.proquest.com/en-US/products/dissertations/individuals.shtml
Publication Type: Dissertations/Theses - Doctoral Dissertations
Education Level: Grade 8
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A