Several extensions to the Angoff method of standard setting are described that can accommodate characteristics of performance-based assessment. A study involving 12 panelists supported the effectiveness of the new approach but suggested that panelists preferred an approach that was at least partially conjunctive. (SLD)

A performance-standard setting procedure termed judgmental policy capturing (JPC) and its application are described. A study involving 12 panelists demonstrated the feasibility of the JPC method for setting performance standards for classroom teachers seeking certification from the National Board for Professional Teaching Standards. (SLD)

Cost factors associated with the development, administration, and scoring of performance assessment tasks are examined in the context of a statewide or other large-scale assessment program. Resources of money, time, and expertise are discussed. (SLD)

Portfolios are the centerpiece of Vermont's statewide assessment program in mathematics. Portfolio scores in the first two years were not reliable enough to permit the reporting of student-level results, but increasing the number of readers or the number of portfolio pieces is not operationally feasible. (SLD)

It is proposed that criteria for evaluating the quality of performance should be defined, at least tentatively, during the initial design of a performance assessment. Six characteristics of sound criteria are (1) significance; (2) fidelity; (3) generalizability; (4) developmental appropriateness; (5) accessibility; and (6) utility. (SLD)

A study with more than 1,000 students in grades 3 and 8 explored the advantages of analytical scoring in writing assessment. Cluster analysis revealed 11 patterns of student scores, reflecting different patterns of writing strength and weakness, and replicated across 5 modes of writing. (SLD)

Using responses of 123 elementary school teachers, a battery of performance-assessment tasks designed to generate responses to reading tests was evaluated from task development and scoring perspectives. More than one dozen types of errors were identified. Practical outcomes of the study and improvement of task development and scoring are…

This article provides a framework for the rest of the articles in this special issue comparing the utility of three standard-setting methods with complex performance assessments. The context of the standard setting study is described, and the methods are outlined. (SLD)

This study provides empirical evidence about the sampling variability and generalizability (reliability) of a statewide performance assessment for grade six. Results for 600 students at individual and school levels indicate that task-sampling variability was the major source of measurement error. Rater-sampling variability was negligible. (SLD)

The validity of several criticisms of objective tests is addressed, and the viability of some alternatives to objective testing is discussed. Evidence against multiple-choice tests is not as strong as has been claimed. Authentic assessments may not always be better, and research about new forms of assessment is necessary. (SLD)

Issues pertaining to the quality of performance assessments, including reliability and validity, are discussed. The relatively limited generalizability of performance across tasks is indicative of the care needed to evaluate performance assessments. Quality control is an empirical matter when measurement is intended to inform public policy. (SLD)

Alternatives to multiple-choice tests for teacher licensing examinations are described, and their advantages are cited. Concerns are expressed in the areas of cost and practicality, reliability, corruptibility, and validity. A suggestion for reducing costs using multiple-choice responses calibrated to constructed-response tasks is proposed. (SLD)

The University of California's (Los Angeles) Center for Research on Evaluation, Standards, and Student Testing survey of state assessment directors reveals that about 25 states currently study or develop performance assessments. Obstacles to statewide use of performance assessments were expressed. The new Student Assessment Exchange should…

A series of 19 questions illuminates the characteristics of effective performance assessments in 3 sections: (1) the nature of assessment; (2) properties of effective tasks; and (3) making tasks meaningful and engaging. A fourth section offers practical suggestions for the construction of performance assessments and for teacher involvement. (SLD)

The old era of educational assessment and reasons for its demise are described. The new era will be different and will present four challenges to the assessment community: (1) new assessment targets; (2) technical challenges of alternative assessments; (3) demystifying assessment and evaluation; and (4) reexamining assessment priorities. (SLD)