Item difficulty modeling studies involve (a) hypothesizing item features, or item response demands, that are likely to predict item difficulty with some degree of accuracy; and (b) entering the features as independent variables into a regression equation or other statistical model to predict difficulty. In this review, we report findings from 13…

The Multiple-choice (MC) item format is commonly used in educational assessments due to its economy and effectiveness across a variety of content domains. However, numerous studies have examined the quality of MC items in high-stakes and higher-education assessments and found many flawed items, especially in terms of distractors. These faulty…

Teachers' professional knowledge is assumed to be a key variable for effective teaching. As teacher education has the goal to enhance professional knowledge of current and future teachers, this knowledge should be described and assessed. Nevertheless, only a limited number of studies quantitatively measures physics teachers' professional…

Theories of self-regulated study assume that learners monitor item difficulty when making decisions about which items to select for study. To complement such theories, the authors propose an agenda-based regulation (ABR) model in which learners' study decisions are guided by an agenda that learners develop to prioritize items for study, given…

This study compared the item difficulty, item discrimination, and test reliability of three forms of multiple-choice items: (1) one correct answer; (2) "none of the above" as a foil; and (3) "none of the above" as the correct answer. Twelve items in the three formats were administered in a college statistics examination. (BS)

Two forms of an undergraduate examination were constructed. Tests varied with respect to item truth value (true, false) and method of phrasing (positive, negative). Negatively stated items were more difficult but not more discriminating than positively stated items. False items were not more difficult but were more discriminating than true items.…

Five sets of paired comparison judgments were made concerning test item difficulty, in order to identify the most probable source of intrasensitivity in the data. The paired comparisons method was useful in providing information about sensitivity to stimulus differences, but less useful for assessing dimensionality of judgment criteria.…

The reliability and validity of a priori estimates of item characteristics are assessed. Results suggest that judges can make a modest contribution to estimation prior to actual administration. (Author/GK)

Content characteristics of 1,400 Graduate Record Examination (GRE) analytical reasoning items were coded for item difficulty and discrimination. The results provide content characteristics for consideration in extending specifications for analytical reasoning items and a better understanding of the construct validity of these items. (TJH)

The accuracy with which testees perceived the difficulty of ability-test items was investigated by administering two 41-item conventional tests on verbal ability. High- and low-ability groups responded to test items by choosing the correct alternative and then rating each item's difficulty relative to their levels of ability. Least-squares…

This study was concerned with the reliability and validity of subjective judgments about five characteristics of multiple-choice test items from an introductory college-level astronomy test: (1) item difficulty, (2) language complexity, (3) content importance or relevance, (4) response set convergence, and (5) process complexity. (Author)

This study compared item difficulty and item discrimination indices for parallel multiple-choice items in three content areas: measurement concepts, statistical terminology, and synonyms. The statistics and measurement items were administered in classes where graduate students taking the test were studying the content. Vocabulary items represented…

A standard method for exploring item bias is the intergroup comparison of item difficulties. This paper describes a refinement and generalization of this technique. In contrast to prior approaches, the proposed method deletes outlying items from the formulation of a criterion for identifying items as deviant. It also extends the mathematical…

Two different formats for multiple-choice test items were compared in an experimental test given in a college class in introductory psychology. In one format, a question or incomplete statement was followed by four answers or completions, only one of which was correct. In the other format, the double multiple-choice version, the same questions…

Providing examinees with advanced knowledge of the difficulty of an item led to an increase in test performance with no loss of reliability. This finding was consistent across several test formats. ( Author/JKS)