In this paper we assert that the practice of principled assessment design renders traditional standard-setting methodology redundant at best and contradictory at worst. We describe the rationale for, and methodological details of, Embedded Standard Setting (ESS; previously, Engineered Cut Scores. Lewis, 2016), an approach to establish performance…

Outlines a series of security steps that might be taken by researchers or organizations that are contemplating Web-based tests and performance assessments. Focuses on what can be done to avoid the loss, compromising, or modification of data collected by or stored through the Internet. (SLD)

Describes educational assessment in Singapore and discusses changes planned to focus assessment on more real-life situations while taking into account research on learning and critical thinking. (SLD)

This instructional module presents and illustrates rules for designing performance assessments in a step-by-step strategy through: (1) reason for the assessment; (2) type of performance evaluated; (3) exercises to elicit performance; and (4) systematic rating procedures. Guidelines for maximizing reliability, validity, and economy are presented.…

A newly developed performance standard-setting procedure, termed iterative judgmental policy capturing (JPC), is applicable to assessments composed of distinct multidimensional exercises. The procedure is described, and results are reported from the application of JPC in a study involving a panel of 20 teachers and 6 performance exercises. (SLD)

Points out areas in which aspects of assessment practice remain resistant to change: (1) evaluation of student achievement; (2) appropriate application of assessment innovation; and (3) preparation of educators in educational assessment. (SLD)

The accuracy of statistics based on performance assessments that represent percentages of students reaching standards is explored using data from a large-scale performance assessment, the Maryland School Performance Assessment Program. Results with students in grades 3, 5, and 8 support the accuracy of pooling results to produce the statistics.…

Conceptual, practical, and logistical issues in the development of science performance assessments (SPAs) are discussed. The conceptual framework identifies task, response format, and scoring system as components, and conceives of SPAs as tasks that attempt to recreate conditions in which scientists work. Developing SPAs is a sophisticated effort…

The rise of criterion-referenced measurement from 1970 to 1984 was an important period in test development. Since 1984 there has been less research published on criterion-referenced measurement, but there is ample evidence that the concept remains important, especially in the authentic measurement and performance assessment movements. (SLD)

Generalizability of performance-based assessment scores across raters and tasks is examined, focusing on implications of generalizability analyses for specific uses and interpretations of assessment results. Although it seems probable that assessment conditions, task characteristics, and interactions with instructional experiences affect the…

The application of generalizability theory to the reliability and error variance estimation for performance assessment scores is discussed. Decision makers concerned with performance assessment need to realize the restrictions that limit generalizability such as limitations that lead to reductions in the number of tasks possible, rater quality,…

Results are presented from a study of portfolio use for large-scale assessment in the Pittsburgh (Pennsylvania) public schools that are more encouraging for the use of this form of assessment than many other studies have been. Requirements for the psychometric integrity of portfolio assessment are discussed. (SLD)

A conceptual framework is presented for the development of the Quantitative Understanding: Amplifying Student Achievement and Reasoning (QUASAR) Cognitive Assessment Instrument (QCAI) that focuses on the ability of middle-school students to problem solve, reason, and communicate mathematically. The instrument will provide programatic rather than…

This instructional module from the National Council on Measurement in Education (NCME) explains how to design assessments for written reports, develop performance criteria and communicate them to students, monitor progress, and assess the outcome. Learning how to give students a role in assessment adds a dimension to their responsibilities. (SLD)

The testing practices and tests described in the preceding articles are reviewed and compared. This cross-cultural examination identifies some issues that are common to the United States and Europe: (1) the emphasis on performance tasks, (2) the accommodation of language minorities, and (3) concerns about gender differences. (SLD)