Testwiseness may introduce construct-irrelevant variance to multiple-choice test scores. Presenting response options sequentially has been proposed as a potential solution to this problem. In an experimental validation, we determined the psychometric properties of a test based on the sequential presentation of response options. We created a strong…

The central purpose of this paper is to examine validity problems arising from the multiple-choice items and technical passages in the Test of English as a Foreign Language Internet-based Test (TOEFL iBT) reading section, primarily concentrating on construct-irrelevant variance (Messick, 1989). My personal TOEFL iBT experience, along with my…

The reading comprehension items for the Nelson Reading Skills Test, a revision of a widely used standardized reading test, were administered to several hundred fourth- and sixth-grade students in order to determine whether the student's ability to answer correctly actually depended on his comprehension of the accompanying passage. All the…

The general purpose of this study was to determine the effects of similarities between stems and keyed choices on test difficulty. Unlike previous investigations of this undesirable characteristic of some multiple-choice items, the present study employed items that were unintentionally faulty and samples of examinees who were highly experienced…

This article investigates positional response bias, testwiseness, and guessing strategy as components of variance in test responses on multiple-choice tests. University students responded to two content exams, a testwiseness measure, and a guessing strategy measure. The proportion of variance in test scores accounted for by positional response…

Two studies examined the effect of making the correct answer of a multiple choice test item grammatically consistent with the item. American College Testing Assessment experimental items were constructed to investigate grammatical compliance to investigate grammatical compliance for plural-singular and vowel-consonant agreement. Results suggest…

A critique of a variety of formats used in combined-response test items (those in which the respondent must choose the correct combination of options: a and b, all of the above, etc.) illustrates why this kind of testing is inherently flawed and should not be used in optometry examinations. (MSE)

The design of a valid and reliable test of test-wiseness is reported: a 33-item multiple-choice instrument with 4 subscales trialed with several groups of English-as-a-Second-Language students. Findings indicate differential skills in test-taking; some learner scores are influenced by skills that are not the focus of the test. (13 references)…

The effects of violating four item construction principles were examined to assess the validity of the principles and the importance of students' test wiseness. While flawed items were significantly less difficult than sound items, differences in item discrimination, test reliability, and concurrent validity were not observed. (Author/BW)

Two studies examined the extent to which test takers use plausibility as a method for locating correct responses when guessing and the extent to which scores can be improved by teaching test takers this approach. Results confirm that this aspect of multiple choice items merits further consideration by test constructors. (Author/BW)

The objectives for this study were to: (1) develop a valid, reliable measure of test-wiseness with equivalent forms for use with students in the health sciences; and (2) determine the level of test-wiseness of entering medical students. The test-wiseness areas included in this study were: similar options, umbrella term, item give-away, convergence…

Instructions are provided on how to create and take computer aided admissible probability measurement (CAAPM) tests using programs available on PLATO IV. Admissible probability measurement is a testing procedure that permits a user to express a degree of uncertainty as to the correctness of alternative answers. Section II describes PLATO IV, an…

Although the "Don't Know" (DK) option has received telling criticism in maximum performance summative tests, its potential use in formative evaluation was considered and judged to be more promising. The pretest of an instructional module was administered with DK options. Examinees were then required to answer each question to which they had…

Corrected-for-guessing scores on multiple-choice tests depend upon the ability and willingness of examinees to guess when they have some basis for answering, and to avoid guessing when they have no basis. The present study determined the extent to which college students were able and willing to comply with formula-scoring directions. (Author/CTM)