ERIC Number: ED654568
Record Type: Non-Journal
Publication Date: 2021
Pages: 243
Abstractor: As Provided
ISBN: 979-8-5699-6624-0
ISSN: N/A
EISSN: N/A
Available Date: N/A
Effect Size Measures for Differential Item Functioning in Cognitive Diagnostic Models
Yanan Feng
ProQuest LLC, Ph.D. Dissertation, Indiana University
This dissertation aims to investigate the effect size measures of differential item functioning (DIF) detection in the context of cognitive diagnostic models (CDMs). A variety of DIF detection techniques have been developed in the context of CDMs. However, most of the DIF detection procedures focus on the null hypothesis significance test. Few studies investigated the practical significance of DIF items. The probability-based effect size indices (signed and unsigned probability difference: SPD and UPD) were proposed in this study. The estimation accuracy of SPD and UPD, as well as the classification of DIF items, were examined under various conditions. The accuracy of the SPD and UPD estimates was largely satisfactory in the studied conditions. The SPD and UPD indices were used together with the Wald test in detecting DIF in the CDM context. The performance of three DIF detection procedures were compared: the Wald method, the Mantel-Haenszel method matched on the total score (MH-T), and the Mantel-Haenszel method matched on the attribute mastery profiles (MH-P). This study also examined the impact of including effect size measures on the Type I error and power rates of DIF detection. The results showed that the Wald method outperformed the two MH procedures in most conditions. The two MH procedures yielded inflated Type I error when group distributions were unequal, and the power rates were also impaired. Including the effect size measures could substantially reduce the Type I error rates of all three methods. In the meantime, including effect size measures also led to a reduction in power for detecting smaller DIF, while the power rates of large DIF were not affected. Moreover, the Trends of International Mathematics and Science Study (TIMSS) 2007 fourth grade mathematics assessment was used to illustrate how the effect size measures and the significance test are combined in detecting DIF. Implications and recommendations for detecting DIF in the CDM context are provided. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]
Descriptors: Effect Size, Item Response Theory, Cognitive Measurement, Models, Test Items, Error of Measurement, Statistical Analysis, Achievement Tests, Elementary Secondary Education, Foreign Countries, Science Achievement, International Assessment, Mathematics Tests, Mathematics Achievement, Science Tests
ProQuest LLC. 789 East Eisenhower Parkway, P.O. Box 1346, Ann Arbor, MI 48106. Tel: 800-521-0600; Web site: http://www.proquest.com/en-US/products/dissertations/individuals.shtml
Publication Type: Dissertations/Theses - Doctoral Dissertations
Education Level: Elementary Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: Trends in International Mathematics and Science Study
Grant or Contract Numbers: N/A
Author Affiliations: N/A