ERIC Number: EJ1241852
Record Type: Journal
Publication Date: 2020
Pages: 21
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1530-5058
EISSN: N/A
Available Date: N/A
The (Non)Impact of Differential Test Taker Engagement on Aggregated Scores
Wise, Steven L.; Soland,, James; Bo, Yuanchao
International Journal of Testing, v20 n1 p57-77 2020
Disengaged test taking tends to be most prevalent with low-stakes tests. This has led to questions about the validity of aggregated scores from large-scale international assessments such as PISA and TIMSS, as previous research has found a meaningful correlation between the mean engagement and mean performance of countries. The current study, using data from the computer-based version of the PISA-Based Test for Schools, examined the distortive effects of differential engagement on aggregated school-level scores. The results showed that, although there was considerable differential engagement among schools, the school means were highly stable due to two factors. First, any distortive effects of disengagement in a school were diluted by a high proportion of the students exhibiting no non-effortful behavior. Second, and most interestingly, disengagement produced both positive and negative distortion of individual student scores, which tended to cancel out much of the net distortive effect on the school's mean.
Descriptors: Foreign Countries, International Assessment, Achievement Tests, Secondary School Students, Student Motivation, Scores, Computer Assisted Testing, Guessing (Tests)
Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Research
Education Level: Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: Program for International Student Assessment
Grant or Contract Numbers: N/A
Author Affiliations: N/A