Attempted to improve the measurement quality of a complex performance assessment through principled assessment design using the example of the National Board for Professional Teaching Standards Early Childhood/Generalist examination. All indexes examined improved after revisions were made. Results show the importance of attention to assessment…

Attempts to place the issues associated with computer-automated scoring within the context of current validity theory and presents a taxonomy of automated scoring procedures as a framework for discussing threats to validity that may take on increased importance for specific approaches to automated scoring. (SLD)

Examined the effects of providing students with background information about the structure and scoring of mathematics performance assessments (PA). Results for 187 elementary school students who had PA orientation and 182 who did not show the effects of test wiseness training for average and above-average students, but not for below-average…

Studied methods for structuring a performance domain for a certification test in emergency nursing based on task frequency ratings from 659 emergency nurses or task similarity ratings from 21 experts. A 125-job analysis survey was used. Similarity judgment results are more easily interpreted and adequately modeled by multivariate analysis. (SLD)

Development of a multistage dominant profile method for setting standards on complex performance assessments is detailed. The method grew from experiences with a judgmental policy-capturing procedure and an extended Angoff method. The design of an early adolescence English language arts assessment illustrates the complexity of decisions panelists…

Cost factors associated with the development, administration, and scoring of performance assessment tasks are examined in the context of a statewide or other large-scale assessment program. Resources of money, time, and expertise are discussed. (SLD)

It is proposed that criteria for evaluating the quality of performance should be defined, at least tentatively, during the initial design of a performance assessment. Six characteristics of sound criteria are (1) significance; (2) fidelity; (3) generalizability; (4) developmental appropriateness; (5) accessibility; and (6) utility. (SLD)

Describes an automated scoring algorithm for a computer-based simulation examination of physicians' patient-management skills. Results with 280 medical students show that scores produced using this algorithm are highly correlated to actual clinician ratings. Scores were also effective in discriminating between case performance judged passing or…

Using responses of 123 elementary school teachers, a battery of performance-assessment tasks designed to generate responses to reading tests was evaluated from task development and scoring perspectives. More than one dozen types of errors were identified. Practical outcomes of the study and improvement of task development and scoring are…

Issues pertaining to the quality of performance assessments, including reliability and validity, are discussed. The relatively limited generalizability of performance across tasks is indicative of the care needed to evaluate performance assessments. Quality control is an empirical matter when measurement is intended to inform public policy. (SLD)

Alternatives to multiple-choice tests for teacher licensing examinations are described, and their advantages are cited. Concerns are expressed in the areas of cost and practicality, reliability, corruptibility, and validity. A suggestion for reducing costs using multiple-choice responses calibrated to constructed-response tasks is proposed. (SLD)

The University of California's (Los Angeles) Center for Research on Evaluation, Standards, and Student Testing survey of state assessment directors reveals that about 25 states currently study or develop performance assessments. Obstacles to statewide use of performance assessments were expressed. The new Student Assessment Exchange should…

A series of 19 questions illuminates the characteristics of effective performance assessments in 3 sections: (1) the nature of assessment; (2) properties of effective tasks; and (3) making tasks meaningful and engaging. A fourth section offers practical suggestions for the construction of performance assessments and for teacher involvement. (SLD)

Guidelines for developing performance assessments in science education allied with current research and reform are presented. The guidelines are applied to 3 hands-on science investigations performed by over 300 fifth and sixth graders and scored by science educators. Although difficult to develop, such assessments can be scored reliably. (SLD)