ERIC Number: EJ1199457
Record Type: Journal
Publication Date: 2018-Dec
Pages: 9
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-1531-7714
EISSN: N/A
Available Date: N/A
Addressing the Shortcomings of Traditional Multiple-Choice Tests: Subset Selection without Mark Deductions
Otoyo, Lucia; Bush, Martin
Practical Assessment, Research & Evaluation, v23 n18 Dec 2018
This article presents the results of an empirical study of "subset selection" tests, which are a generalisation of traditional multiple-choice tests in which test takers are able to express partial knowledge. Similar previous studies have mostly been supportive of subset selection, but the deduction of marks for incorrect responses has been a cause for concern. For the present study, a novel marking scheme based on Akeroyd's "dual response system" was used instead. In Akeroyd's system, which assumes that every question has four answer options, test takers are able to split their single 100% bet on one answer option into two 50% bets by selecting two options, or into four 25% bets by selecting no options. To achieve full subset selection, this idea was extended so that test takers could also split their 100% bet equally between three options. The results indicate increased test reliability (in the sense of measurement consistency), and also increased satisfaction on the part of the test takers. Furthermore, since the novel marking scheme does not in principle lead to either inflated or deflated marks, this makes it easy for educators who currently use traditional multiple-choice tests to switch to using subset selection tests.
Descriptors: Multiple Choice Tests, Grading, Test Reliability, Test Format, Undergraduate Students, Summative Evaluation, Student Attitudes, Preferences, Guessing (Tests), Scores, Test Use
Center for Educational Assessment. 813 North Pleasant Street, Amherst, MA 01002. e-mail: pare@umass.edu; Tel: 413-577-2180; Web site: https://scholarworks.umass.edu/pare
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A