In this paper we assert that the practice of principled assessment design renders traditional standard-setting methodology redundant at best and contradictory at worst. We describe the rationale for, and methodological details of, Embedded Standard Setting (ESS; previously, Engineered Cut Scores. Lewis, 2016), an approach to establish performance…

Performance assessments, scenario-based tasks, and other groups of items carry a risk of violating the local item independence assumption made by unidimensional item response theory (IRT) models. Previous studies have identified negative impacts of ignoring such violations, most notably inflated reliability estimates. Still, the influence of this…

Provides a general framework for examining the consequences of assessment programs, especially statewide programs that intend to improve student learning by holding schools accountable. The framework is intended for use with programs using performance-based tasks but can be used with programs using traditional item formats as well. (SLD)

Conceptual, practical, and logistical issues in the development of science performance assessments (SPAs) are discussed. The conceptual framework identifies task, response format, and scoring system as components, and conceives of SPAs as tasks that attempt to recreate conditions in which scientists work. Developing SPAs is a sophisticated effort…

Gender-related differential item functioning (DIF) was examined in a context in which 3,946 middle school students received mathematics instruction focusing on problem solving. Reasons why four tasks on the performance assessment favored female students and two favored male students are discussed. (SLD)

The contributions of open-ended tasks in examining students' mathematical performance were studied with 250 U.S. and 425 Chinese sixth graders. Open-ended tasks allow for analysis of student performance that cannot be assessed solely by percent correct or incorrect, but they pose many problems, such as those of translation. (SLD)