In 2018, 26 states administered a college admissions test to all public school juniors. Nearly half of those states proposed to use those scores as their academic achievement indicators for federal accountability under the Every Student Succeeds Act (ESSA); many others are planning to use those scores for other accountability purposes.…

As Paul Newton so ably demonstrates, the concept of validity is both important and problematic. Over the last several decades, a consensus definition of validity has emerged; the current edition of "Standards for Educational and Psychological Testing" notes, "Validity refers to the degree to which evidence and theory support the interpretations of…

The 1999 "Standards for Educational and Psychological Testing" defines validity as the degree to which evidence and theory support the interpretations of test scores entailed by proposed uses of tests. Although quite explicit, there are ways in which this definition lacks precision, consistency, and clarity. The history of validity has taught us…

Paul E. Newton's "Clarifying the Consensus Definition of Validity" addresses the single most important, yet stubbornly protean, value in educational and psychological assessment. "Standards for Educational and Psychological Testing" (American Educational Research Association, American Psychological Association, & National Council on Measurement in…

Developed jointly by the American Educational Research Association, American Psychological Association, and the National Council on Measurement in Education, "Standards for Educational and Psychological Testing" (Revised 2014) addresses professional and technical issues of test development and use in education, psychology, and…

Assessments labeled as formative have been offered as a means to improve student achievement. But labels can be a powerful way to miscommunicate. For an assessment use to be appropriately labeled "formative," both empirical evidence and reasoned arguments must be offered to support the claim that improvements in student achievement can be linked…

In many school districts, the pressure to raise test scores has created overnight celebrity status for formative assessment. Its powers to raise student achievement have been touted, however, without attending to the research on which these claims were based. Sociocultural learning theory provides theoretical grounding for understanding how…

The relationship between the validity theory of the past 50 years and actual validity practices was studied by comparing published test standards with the practices of measurement professionals expressed in the "Mental Measurements Yearbook" test reviews. Results show a symbiotic relationship between theory and practice on the influence…

This paper describes the Myers-Briggs Type Indicator (MBTI), developed by I. Myers and K. Briggs (1940s) to assess personality type. Based on Jungian theory, the MBTI has become a tool for identifying the 16 different patterns of action into which every person fits. The 16 personality types are based on patterns of: (1) extraversion-introversion;…

This report analyzes the validity issues that arise in the context of educational accountability systems. The report addresses validity from three interlocking perspectives. The first explores the theory of action underlying accountability provisions. It considers problems emerging from the distance between aspirations for accountability in…

The standards outlined in this book have been developed to provide criteria for the evaluation of tests, testing practices, and the effects of test use. The "Standards" provides a frame of reference to ensure that relevant issues are addressed. The first part of the book, "Test Construction, Evaluation, and Documentation,"…

Two achievement testing programs in New York City that have had and are having some consequences are described, and how the school system tries to determine and deal with these consequences is explored. Cooperation and open dialog between test publishers and the school district are making it possible to deal with the real-life problems of test…

Discusses some problems, pitfalls, and paradoxes that challenge measurement theory and practice, especially for K-12 achievement testing. Considers a number of technical issues, especially some related to reliability. Also discusses a number of practical or political issues related to validation and accountability. (SLD)

This field memo is the third in a series of updates from the Deputy Commissioner of the Office of Elementary, Middle, Secondary and Continuing Education to all teachers and administrators of public and nonpublic schools on standards and assessments in New York. This document explains how the New York State Education Department determines if test…

The National Assessment Governing Board and the National Center for Education Statistics sponsored a Joint Conference on Standard Setting for Large-Scale Assessments to provide a forum for technical and policy issues relevant to setting standards at local, state, and national levels. This executive summary conveys the essence of the conference by…