Over the last five years, tremendous strides have been made in advancing the AIG methodology required to produce items in diverse content areas. However, the one content area where enormous problems remain unsolved is language arts, generally, and reading comprehension, more specifically. While reading comprehension test items can be created using…

We examine the effects of computer-based versus paper-based assessment of critical thinking skills, adapted from English (in the U.S.) to Chinese. Using data collected based on a random assignment between the two modes in multiple Chinese colleges, we investigate mode effects from multiple perspectives: mean scores, measurement precision, item…

Writing multiple-choice test items to measure student learning in higher education is a challenge. Based on extensive scholarly research and experience, the author describes various item formats, offers guidelines for creating these items, and provides many examples of both good and bad test items. He also suggests some shortcuts for developing…

This paper represents the third installment of the Reading for Understanding (RfU) assessment framework. This paper builds upon the two prior installments (Sabatini & O'Reilly, 2013; Sabatini, O'Reilly, & Deane, 2013) by discussing the role of performance moderators in the test design and how scenario-based assessment can be used as a tool…

This article describes and reviews the Singapore-Cambridge General Certificate of Education Advanced Level General Paper (GP) examination. As a written test that is administered to preuniversity students, the GP examination is internationally recognised and accepted by universities and employers as proof of English competence. In this article, the…

As the ongoing national indicator of what American students know and can do, the National Assessment of Educational Progress (NAEP) in Reading regularly collects achievement information on representative samples of students in grades 4, 8, and 12. The information that NAEP provides about student achievement helps the public, educators, and…

In the last 15 years or so, language testing practitioners have increasingly favored assessing vocabulary in context. The discrete-point vocabulary measure used in the old version of the Test of English as a Foreign Language (TOEFL) has long been criticized for encouraging test candidates to memorize wordlists out of context although test items…

Reports on two studies of the effect of different typographic settings on the speed and accuracy of responses to cloze procedure reading tests. Concludes that in-text responding and dashes produce significantly higher scores. (SRT)

A Monte Carlo design examined how the effects of guessing and item dependence influence test characteristics and student scores. Although validity for cloze variants was high, multiple-choice cloze had significantly lower reliabilities than did true score equivalents. (Author/PN)

The research project reported in this paper had two aims: to demonstrate what standard multiple-choice questions and two new varieties (multiple-choice cloze and guided summary test and to assess whether the new formats test the same skills. To find out what the various question formats test, the study looked at a battery of tests, each of which…

Whether the shaping of test items can itself result in qualitative differences in examinees' comprehension of reading passages was studied using the Pearson-Johnson item classification system. The specific practice studied incorporated, within an item stem line, references that point the examinee to a specific location within a reading passage.…

A study investigated whether or not a discrete-item test of English affixes could be used to measure second language reading comprehension. Subjects were 1,254 mostly first-year university students in four academic departments studying to pass the first half of a foreign language requirement, an English reading comprehension test. Results from two…

This study provided an exploratory investigation of item features that might contribute to a lack of invariance of item parameters for the Test of English as a Foreign Language (TOEFL). Data came from seven forms of the TOEFL administered in 1989. Subjective and quantitative measures developed for the study provided consistent information related…

Using informal, objectives-based, or linguistic methods, three elementary school teachers and three experienced item writers developed criterion-referenced pretests-posttests to accompany a prose passage. Item difficulites were tabulated on the responses of 364 elementary students. The informal-subjective method, used by many achievement test…

Two frequently cited guidelines for writing multiple choice test item stems are: (1) the stem can be written in either a question or statement-to-be-completed format; and (2) only positively worded stems should be used. These guidelines were evaluated in a survey of the test item banks of 13 nationally administered examinations in the physician…