ERIC Number: EJ1317195
Record Type: Journal
Publication Date: 2021-Dec
Pages: 36
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1076-9986
EISSN: N/A
Available Date: N/A
Detecting Noneffortful Responses Based on a Residual Method Using an Iterative Purification Process
Liu, Yue; Liu, Hongyun
Journal of Educational and Behavioral Statistics, v46 n6 p717-752 Dec 2021
The prevalence and serious consequences of noneffortful responses from unmotivated examinees are well-known in educational measurement. In this study, we propose to apply an iterative purification process based on a response time residual method with fixed item parameter estimates to detect noneffortful responses. The proposed method is compared with the traditional residual method and noniterative method with fixed item parameters in two simulation studies in terms of noneffort detection accuracy and parameter recovery. The results show that when severity of noneffort is high, the proposed method leads to a much higher true positive rate with a small increase of false discovery rate. In addition, parameter estimation is significantly improved by the strategies of fixing item parameters and iteratively cleansing. These results suggest that the proposed method is a potential solution to reduce the impact of data contamination due to severe low test-taking effort and to obtain more accurate parameter estimates. An empirical study is also conducted to show the differences in the detection rate and parameter estimates among different approaches.
Descriptors: Response Style (Tests), Reaction Time, Test Items, Accuracy, Achievement Tests, Foreign Countries, International Assessment, Secondary School Students
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Publication Type: Journal Articles; Reports - Research
Education Level: Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: Program for International Student Assessment
Grant or Contract Numbers: N/A
Author Affiliations: N/A