A critical component of the standard setting process is collecting evidence to evaluate the recommended cut scores and their use for making decisions and classifying students based on test performance. Kane (1994, 2001) proposed a framework by which practitioners can identify and evaluate evidence of the results of the standard setting from (1)…

Standard setting theory has largely developed with reference to a typical situation, determining a level or levels of performance for one exam for one context. However, standard setting is now being used with international reference frameworks, where some parameters and assumptions of classical standard setting do not hold. We consider the…

[Slides] presented at the annual conference of the National Council on Measurement in Education, 2011, New Orleans.

Throughout the world, cut scores are an important aspect of a high-stakes testing program because they are a key operational component of the interpretation of test scores. One method for setting standards that is prevalent in educational testing programs--the Bookmark method--is intended to be a less cognitively complex alternative to methods…

Considerable research about standard setting has revolved around a U.S.-centric policy context. That is, over the past decade, conclusions about thought processes and the interaction of education policy and panelists' judgments have been based on assumptions of comparable policy settings. However, whether these assumptions generalize to other…

Many testing programs face the practical challenge of having limited resources to conduct comprehensive standard setting studies. Some researchers have suggested that replicating a group's recommended cut score on a full-length test may be possible by using a subset of the items. However, these studies were based on simulated data. This study…

Many educational testing programs report examinee performance at more than two levels of proficiency. Whether these assessments have the capacity to support these multiple inferences, though, is a topic that has not been widely discussed. This study proposes a method for evaluating the minimum number of measurement opportunities for reporting…

Compared simplified variations on the Angoff and Bookmark methods for setting cut scores on educational assessments with data from a grade 7 mathematics test (23 panelists in all). Although the Angoff method is more widely used, results show that the Bookmark method has some promising features. (SLD)

This paper presents a comparison of two commonly used methods, Angoff (W. Angoff, 1971) and Bookmark (D. Lewis, H. Mitzel, and D. Green, 1996), for setting cut scores on selected response tests. This comparison is presented through an application to a grade 7 mathematics assessment in a suburban Midwestern school district. Training and operational…

Many school districts are developing assessments that incorporate both selected response and constructed response formats. Scores on these assessments can be used for a variety of purposes ranging from subject remediation to promotion decisions. These policy decisions are informed by recommendations for Minimum Passing Scores (MPSs) from standard…