ERIC Number: EJ1482661
Record Type: Journal
Publication Date: 2025
Pages: 21
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1543-4303
EISSN: EISSN-1543-4311
Available Date: 0000-00-00
Informing Q-Matrix Specification with Exploratory Factor Analysis in Cognitive Diagnostic Assessment of Language Comprehension
Language Assessment Quarterly, v22 n3 p306-326 2025
This paper advocates the use of exploratory factor analysis (EFA) for Q-matrix specification in cognitive diagnostic assessment (CDA) of language comprehension. We propose that integrating EFA results into Q-matrix specification improves the psychometric quality of CDA classification results with reference to Q-matrices based solely on human judgment. To test this hypothesis, we conducted EFA on the response data of 3,607 test takers in a large-scale EFL reading test. We then specified two Q-matrices, one combining EFA results with subject matter expert judgment (SME+EFA Q-matrix) and the other based solely on expert judgment (SME Q-matrix). We fit the G-DINA models to the response data using the two Q-matrices. The results showed that the G-DINA model based on the SME+EFA Q-matrix fit the data better and yielded clearer attribute distinction and higher classification accuracy. To examine the generalizability of the results, the same models were fitted to a second dataset with 1,577 test takers. The SME+EFA Q-matrix again yielded better results than the SME Q-matrix, but heterogeneity between the samples resulted in inferior classification accuracy relative to the first dataset. The findings made a strong case for using EFA to complement expert judgement in Q-matrix specification for CDA studies in language assessment.
Descriptors: Factor Analysis, Cognitive Measurement, Comprehension, English (Second Language), Reading Tests, Foreign Countries
Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: China
Grant or Contract Numbers: N/A
Author Affiliations: 1Guangdong University of Foreign Studies, Guangzhou, China; 2Zhejiang University, Hangzhou, China

Peer reviewed
Direct link
