In many school districts, the pressure to raise test scores has created overnight celebrity status for formative assessment. Its powers to raise student achievement have been touted, however, without attending to the research on which these claims were based. Sociocultural learning theory provides theoretical grounding for understanding how…

Obtaining the intrinsic rational validity Ebel describes (TM 508 146) requires several steps that have been neglected in developing tests of ability. These include defining and describing the behaviors that demonstrate possession or lack of the ability, analyzing behaviors, translating the analyses into specifications for test items, and writing…

Schulz (2006) provides a different perspective on standard setting than that provided in Reckase (2006). He also suggests a modification to the bookmark procedure and some alternative models for errors in panelists' judgments than those provided by Reckase. This article provides a response to some of the points made by Schulz and reports some…

The validity of high-stakes assessments and accountability systems is discussed in relation to the requirements of No Child Left Behind (NCLB). The extent to which content standards and assessments are cognitively rich, the challenges in setting performance standards, and the impact of high-stakes assessments on instruction and student learning…

Whether the measurement profession should consider developing and adopting a code of professional conduct is explored after a brief review of existing references to standards of conduct and a review of other professional codes. Issues include the need for a code of ethics, its usefulness, and its enforcement. (SLD)

This essay argues that reliability should be reconceptualized in a way that reflects the importance of the theoretical expectations of the test specialist and the learning and problem solving of the test takers. It is time to characterize clearly the substantive theoretical framework supporting reliability studies and the technical evaluation of…

The goal of the Standards for Educational and Psychological Testing is to improve testing practices, but their impact on practice appears spotty. Self-regulation clearly fails in some instances. The establishment of an external agency to oversee testing practices and adherence to the Standards would face substantial hurdles, and the ambiguity of…

It is argued that consequences are a logical part of the evaluation of test use, which has been an accepted part of test validity for several decades. The examination of effects following from test use is essential in evaluating test validity and not merely the domain of policymakers and politicians. (SLD)

It is argued that consequential validity is a concept worth considering. The solution to defining "validity" is not to narrow the concept, but to allow for the differential prediction provided by tests in different circumstances. Consequences of the uses and interpretations of test scores are central to their evaluation. (SLD)

Test developers have an obligation to create the most appropriate assessment instruments possible as L. J. Kilian argues, but whether they do or not, educators still are responsible for providing test preparation that is professionally ethical and educationally defensible. (SLD)

Developers of high stakes criterion-referenced tests are urged to formulate test tasks so that efficient instruction can promote student mastery of the domain. Time-on-task has been promoted by instructional psychologists as preferable to test teaching. (DWH)

This paper seeks to identify ways of making measurement more relevant to classroom teachers, and making classroom realities more apparent to measurement specialists. Measurement elements of teacher education textbooks and courses, nontraditional assessment topics, informal classroom measurement techniques, terminological issues, and three types of…

Issues concerning the selection of judges for standard setting are discussed. Determining the consistency of judges' recommendations, or their congruity with other expert recommendations, would help in selection. Enough judges must be chosen to allow estimation of recommendations by an entire population of judges. (SLD)

This reaction paper stresses the importance of matching the qualitatively different types of educational decisions with appropriate types of tests. Reading instruction is used to illustrate that while standardized tests are helpful for student classification and program accountability, they are hazardous to instructional process decisions which…

In 1987, a survey report by J. Cannell was published criticizing state reports based on national standardized testing of elementary school students that indicated that all 50 states were above the national average. This phenomenon--known as the "Lake Wobegon Effect"--is analyzed. Technical problems with the study are considered. (TJH)