Educational tests are standardized so that all examinees are tested on the same material, under the same testing conditions, and with the same scoring protocols. This uniformity is designed to provide a level "playing field" for all examinees so that the test is "the same" for everyone. Thus, standardization is designed to…

The goal in personalized assessment is to best fit the needs of each individual test taker, given the assessment purposes. Design-in-Real-Time (DIRTy) assessment reflects the progressive evolution in testing from a single test, to an adaptive test, to an adaptive assessment "system." In this article, we lay the foundation for DIRTy…

With the recent adoption of the Common Core standards in many states, there is a need for quality information about textbook alignment to standards. While there are many existing content analysis procedures, these generally have little, if any, validity or reliability evidence. One exception is the Surveys of Enacted Curriculum (SEC), which has…

The 1990 Standards for Teacher Competence in Educational Assessment of Students (AFT, NCME, & NEA, 1990) made a documentable contribution to the field. However, the Standards have become a bit dated, most notably in two ways: (1) the Standards do not consider current conceptions of formative assessment knowledge and skills, and (2) the Standards…

Items on test score scales located at and below the Proficient cut score define the content area knowledge and skills required to achieve proficiency. Alternately, examinees who perform at the Proficient level on a test can be expected to be able to demonstrate that they have mastered most of the knowledge and skills represented by the items at…

There has been much discussion recently about why the percentage of students scoring Proficient or above varies as much as it does on state assessments across the country. However, most of these discussions center on the leniency or rigor of the cut score. Yet, the cut score is developed in a standard-setting process that depends heavily on the…

This article describes an alignment procedure, called Links for Academic Learning (LAL), for examining the degree of alignment of alternate assessments based on alternate achievement standards (AA-AAS) to grade-level content standards and instruction. Although some of the alignment criteria are similar to those used in general education…

Standards-based reform, as codified by the No Child Left Behind Act, relies on the ability of assessments to accurately reflect the learning that takes place in U.S. classrooms. However, this property of assessments--their instructional sensitivity--is rarely, if ever, investigated by test developers, states, or researchers. In this paper, the…

Following a review of educational standard-setting experiences in the United States, the procedures used by the National Assessment of Educational Progress (NAEP) to arrive at a consensus on educational objectives are described. These objectives represent the only definitions of educational quality that are widely accepted by professional and lay…

Standards-based education has now reached a stage where it is possible to evaluate its overall effectiveness. Several earlier papers in the special issue of "Educational Measurement: Issues and Practice" on "Test Scores and State Accountability" (Volume 24, Number 4) examined specific state policies and their effects on schools…

Some of the challenges that test publishers face in constructing educational assessments that meet high technical quality as prescribed in the "Standards for Educational and Psychological Testing" (AERA, APA, NCME, 1999) are examined. Federal educational initiatives are used to illustrate demands on technical quality that challenge the efforts of…

The "Standards for Educational and Psychological Testing" have evolved in the breadth and depth of coverage of issues in educational testing and measurement since their first publication in 1954. There were a number of substantive changes in the 1999 revision that addressed validity, fairness, accommodations, and compliance with the…

This commentary discusses three essential themes in performance assessment and its scoring. First, scores should mean something. Second, performance scores should permit fair and meaningful comparisons. Third, validity-reducing errors should be minimal. Increased attention to performance assessment may overcome these problems. (SLD)

The National Assessment of Educational Progress incorporates key elements of a responsible standard-setting process: (1) choice of educational objective; (2) description of current group performance ranges and trends; and (3) the identification of educational contexts differentially related to performance. NAEP results can direct public values…

Six distinguishable aspects of construct validity are discussed as they apply to performance assessment, emphasizing content, substantive, structural, generalizability, external, and consequential aspects. Taken together, these aspects provide a way to address validity questions in score interpretation and use. (SLD)