The problems of validity and fairness involved in multiple-choice critical thinking tests can be lessened by using verbal reports of examinees' thinking during the process of developing such tests in order to retain only those items which rely on critical thinking skills to obtain the correct answer. Multiple-choice testing can lead to unfair…

The results of five studies into the characteristics of wrong answers as a class of divergent behavior are presented. The evidence from these studies, when taken in combination, suggests that the tendency of researchers to ignore wrong answers has been a fundamental procedural error of broad scope and serious consequences. Instead of the straight…

Brief explanations are presented of some of the different methods used to score multiple-choice tests; and some studies of partial information, guessing strategies, and test-taking behaviors are reviewed. Studies are grouped in three categories of effort to improve scoring: (1) those that require extra effort from the examinee to answer…

The viewpoint is expressed that adding to test reliability by either selecting a more homogeneous set of items, restricting the range of item difficulty as closely as possible to the most efficient level, or increasing the number of items will not add to test validity and that there is considerable danger that efforts to increase reliability may…

These studies in test adequacy focus on two problems: procedures for estimating reliability, and techniques for identifying ineffective distractors. Fourteen papers are presented on recent advances in measuring achievement (a response to Molenaar); "an extension of the Dirichlet-multinomial model that allows true score and guessing to be…

Current Scoring practices for multiple-choice tests are rooted in early Associationist Theory and are based on a two-step procedure: (1) right answers counted as ones and wrong answers are zeros, and (2) number of right answers form a total-correct score. The author contends that if either step is invalid, the use of the general linear model (GLM)…

This paper deals with test fairness regarding a test consisting of two parts: (1) a "common" section, taken by all students; and (2) a "variable" section, in which some students may answer a different set of questions from other students. For example, a test taken by several thousand students each year contains a common multiple-choice portion and…

Some examinees' test-taking behavior may be so idiosyncratic that their scores are not comparable to the scores of more typical examinees. Appropriateness indices, which provide quantitative measures of response-pattern atypicality, can be viewed as statistics for testing a null hypothesis of normal test-taking behavior against an alternative…

Presented in this symposium reader are nine papers, four of which deal with the theory and impact of the Rasch model on language testing and five of which discuss final examinations in secondary schools in both general and specific terms. The papers are: "Introduction to Rasch Measurement: Some Implications for Language Testing" (J. J.…