Responds to a negative evaluation of the National Assessment of Educational Progress (NAEP) by the National Academy of Sciences (NAS) and asserts that a review of the evidence for the NAEP performance standards indicates that there is support for the current approach to NAEP standard setting. Considers the scholarship of the NAS evaluation…

Notes that the methods used to set standards for National Assessment of Education Progress (NAEP) tests suggest recommendations for state-level policymakers. Explains the national assessment, basic assumptions in setting performance standards on NAEP, selection of judges, standard-setting methodology for NAEP, and measurement issues in setting…

Specific steps for applying the Angoff method are described. In the Angoff method, judges are asked to estimate the probabilities of minimally competent candidates' answering multiple choice test items correctly. Initial information must be obtained for designing the standard-setting process, beginning with the purpose of the examination and any…

Several extensions to the Angoff method of standard setting are described that can accommodate characteristics of performance-based assessment. A study involving 12 panelists supported the effectiveness of the new approach but suggested that panelists preferred an approach that was at least partially conjunctive. (SLD)

To address the issues associated with testing standards, sets of context-setting variables and technical matters associated with standard-setting are presented to assist groups or committees desiring to set standards in a systematic way. (Author/CM)

Applied the analytical judgment standard setting method to 90 papers from the 1996 Grade 8 National Assessment of Educational Progress science assessment. Compared sorting versus direct classification, long and short versions of the classification scale, and effects of discussion on cutscores. Results from 17 Georgia teachers and 8 Michigan…

This paper reports on a standard-setting method designed for complex performance assessments with multiple performance categories. The method studied, the Analytical Judgment Method, involves panelists' making analytical classification decisions for each of the test's components individually. It also allows for discussion and reconsideration of…

The number of performance-based assessments is increasing rapidly, but to date there is no established procedure for setting standards on these assessments. This paper describes several extensions to the Angoff procedure to accommodate the characteristics of a performance-based assessment and presents the results of research in applying this…

Reviews a number of promising methods for setting performance standards and discusses their strengths and weaknesses. Outlines some areas for future research that address the role of feedback to panelists and validation efforts for performance standards among other topics. (SLD)

A brief history of developments in the assessment of the reliability of credentialing examinations is presented, and some new results are outlined that highlight the interactions among scoring, standard setting, and the reliability and validity of pass-fail decisions. Decision consistency is an important concept in evaluating credentialing…

Considerable evidence suggests that policy-makers, educators, the media, and the public do not understand national and state test results. The problems appear to be two-fold: the scales on which scores are reported seem confusing, and the report forms themselves are often too complex for the intended audiences. This paper addresses two topics. The…

Performance assessments in education and credentialing are becoming popular. At the same time, there do not exist any well established and validated methods for setting standards on performance assessments. This paper describes several of the new standard-setting methods that are emerging for use with performance assessments and considers their…

Introduces the articles of this theme issue focusing on performance assessment methodology. Papers address: (1) merging item formats; (2) scoring models; (3) equating and linking; (4) generalizability theory; (5) standard setting methods; and (6) validity issues and methods. (SLD)

A critical, but often neglected, component of any large-scale assessment program is the reporting of test results. In the past decade, a body of evidence has been compiled that raises concerns over the ways in which these results are reported to and understood by their intended audiences. In this study, current approaches for reporting…

The National Assessment of Educational Progress (NAEP), a congressionally mandated national assessment program, can provide valuable data to educational policy makers in Massachusetts and other New England states concerning the status of their educational reform initiatives and performance standards. The three purposes of this paper are: (1) to…