In this article, the author begins by discussing four challenges on the concept of validity. These challenges are: (1) the appeal to logic and syllogistic reasoning; (2) the claim of reliability; (3) the local and the universal; and (4) the unitary and the divisible. In language testing validity cannot be achieved directly but only through a…

Students were observed directly, in a usability laboratory, and indirectly, by means of an extensive evaluation of responses, as they attempted interactive computer-marked assessment questions that required free-text responses of up to 20 words and as they amended their responses after receiving feedback. This provided more general insight into…

Weiss, Keith, Zhu, and Chen (2013a) and Weiss, Keith, Zhu, and Chen (2013b), this issue, report examinations of the factor structure of the Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV) and Wechsler Intelligence Scale for Children-Fourth Edition (WISC-IV), respectively; comparing Wechsler Hierarchical Model (W-HM) and…

Client ethnicity has been shown to affect clinicians' diagnostic impressions. However, it is not known whether interpretation of the Minnesota Multiphasic Personality Inventory-2 (MMPI-2) clinical scales is affected by ethnic bias. In this study, clinicians (82 males, 60 females) provided severity ratings for six symptoms based on three MMPI-2…

Universal screening in elementary schools often includes administering curriculum-based measurement in reading (CBM-R); but in first grade, nonsense word fluency (NWF) and, to a lesser extent, word identification fluency (WIF) are used because of concerns that CBM-R is too difficult for emerging readers. This study used Kane's argument-based…

Since its release in 1979 the TOEIC® (Test of English for International Communication) has been consistently and widely used by educational institutions and companies of Japan despite criticisms that it provides little useable information about language ability. In order to both reduce the extreme focus on and also aid with the practical…

Student examinees are key stakeholders in large-scale, high-stakes, public examination systems. How they perceive the purpose, comprehend the technical characteristics of testing and how they interpret scores influence their response to the system demands and their preparation for the examinations; this information relates to intended and…

To validate an interpretation or use of test scores is to evaluate the plausibility of the claims based on the scores. An argument-based approach to validation suggests that the claims based on the test scores be outlined as an argument that specifies the inferences and supporting assumptions needed to get from test responses to score-based…

Background/Context: There is little question that education is changing, seemingly quickly and in some cases dramatically. The mechanisms through which individuals learn are shifting from paper-based ones to electronic media. Simultaneously, the nature of what individuals must learn is evolving, in good part due to an exponential accumulation of…

This paper examines the utilization of construct validity in formative assessment for classroom-based purposes. Construct validity pertains to the notion that interpretations are made by educators who analyze test scores during formative assessment. The purpose of this paper is to note the challenges that educators face when interpreting these…

Supplementary methods for the analysis of the Repeatable Battery for the Assessment of Neuropsychological Status are made available, including (a) quantifying the number of abnormally low Index scores and abnormally large differences exhibited by a case and accompanying this with estimates of the percentages of the normative population expected to…

The measurement community needs to better understand how to interact with the media to effectively disseminate important findings from educational testing efforts. To this end, the current paper will review media coverage of educational testing and related issues and elaborate on areas of concern and opportunities for improved communication…

Standardized measures are often used as an index of students' reading comprehension and scores have important implications, particularly for students who perform below expectations. This study examined secondary-level students' patterns of responding and the prevalence and impact of non-attempted items on a timed, group-administered,…

In almost all high-stakes testing programs, test equating is necessary to ensure that test scores across multiple test administrations are equivalent and can be used interchangeably. Test equating becomes even more challenging in mixed-format tests, such as Advanced Placement Program® (AP®) Exams, that contain both multiple-choice and constructed…

The fuzziness of assessing second language speaking performance raises two difficulties in scoring speaking performance: "indistinction between adjacent levels" and "overlap between scales". To address these two problems, this article proposes a new approach, "confidence scoring", to deal with such fuzziness, leading to "confidence" scores between…