The purpose of this study was to test methods that strengthen the comparability claims about annual determinations of student proficiency in English language arts, math, and science (Grades 3-12) in the New Hampshire Performance Assessment of Competency Education (NH PACE) pilot project. First, we examined the literature in order to define…

In the United Kingdom, the majority of national assessments involve human raters. The processes by which raters determine the scores to award are central to the assessment process and affect the extent to which valid inferences can be made from assessment outcomes. Thus, understanding rater cognition has become a growing area of research in the…

The engagement of teachers as raters to score constructed response items on assessments of student learning is widely claimed to be a valuable vehicle for professional development. This paper examines the evidence behind those claims from several sources, including research and reports over the past two decades, information from a dozen state…

Using an argument-based approach to validation, this study examines the quality of teacher judgments in the context of a standards-based classroom assessment of English proficiency. Using Bachman's (2005) assessment use argument (AUA) as a framework for the investigation, this paper first articulates the claims, warrants, rebuttals, and backing…

Analyzes the three major inferences involved in interpretation of performance assessments: (1) scoring of the observed performances; (2) generalization to a domain of assessment performances like those included in the assessment; and (3) extrapolation to the large performance domain of interest. Suggests ways to improve the validity of performance…

This commentary discusses three essential themes in performance assessment and its scoring. First, scores should mean something. Second, performance scores should permit fair and meaningful comparisons. Third, validity-reducing errors should be minimal. Increased attention to performance assessment may overcome these problems. (SLD)

Conceptual, practical, and logistical issues in the development of science performance assessments (SPAs) are discussed. The conceptual framework identifies task, response format, and scoring system as components, and conceives of SPAs as tasks that attempt to recreate conditions in which scientists work. Developing SPAs is a sophisticated effort…

Generalizability of performance-based assessment scores across raters and tasks is examined, focusing on implications of generalizability analyses for specific uses and interpretations of assessment results. Although it seems probable that assessment conditions, task characteristics, and interactions with instructional experiences affect the…

An analytic framework is presented for examining the properties and objectives of assessments and scoring systems piloted in a number of state and district testing programs. The article explores the application of this framework, which considers the cognitive components of competence described in studies of expertise, to the analysis of science…

Results are presented from a study of portfolio use for large-scale assessment in the Pittsburgh (Pennsylvania) public schools that are more encouraging for the use of this form of assessment than many other studies have been. Requirements for the psychometric integrity of portfolio assessment are discussed. (SLD)

The extent to which performance assessments can have unintended and undesirable consequences is not yet clear, but preliminary evidence suggests several issues of bias and fairness. Among these is the sheer difficulty of scoring complex performance assessments, as Vermont's experiences with portfolio assessment illustrate. (SLD)