Reviews a number of promising methods for setting performance standards and discusses their strengths and weaknesses. Outlines some areas for future research that address the role of feedback to panelists and validation efforts for performance standards among other topics. (SLD)

Introduces the articles of this theme issue focusing on performance assessment methodology. Papers address: (1) merging item formats; (2) scoring models; (3) equating and linking; (4) generalizability theory; (5) standard setting methods; and (6) validity issues and methods. (SLD)

Several extensions to the Angoff method of standard setting are described that can accommodate characteristics of performance-based assessment. A study involving 12 panelists supported the effectiveness of the new approach but suggested that panelists preferred an approach that was at least partially conjunctive. (SLD)

The rise of criterion-referenced measurement from 1970 to 1984 was an important period in test development. Since 1984 there has been less research published on criterion-referenced measurement, but there is ample evidence that the concept remains important, especially in the authentic measurement and performance assessment movements. (SLD)

Applied the analytical judgment standard setting method to 90 papers from the 1996 Grade 8 National Assessment of Educational Progress science assessment. Compared sorting versus direct classification, long and short versions of the classification scale, and effects of discussion on cutscores. Results from 17 Georgia teachers and 8 Michigan…

This paper reports on a standard-setting method designed for complex performance assessments with multiple performance categories. The method studied, the Analytical Judgment Method, involves panelists' making analytical classification decisions for each of the test's components individually. It also allows for discussion and reconsideration of…

The number of performance-based assessments is increasing rapidly, but to date there is no established procedure for setting standards on these assessments. This paper describes several extensions to the Angoff procedure to accommodate the characteristics of a performance-based assessment and presents the results of research in applying this…

Authentic measurement has become an important topic recently in educational testing. Advocates of authentic measurement feel that objective tests, multiple-choice tests in particular, cannot meet the demands required of today's tests and should be replaced by tests that can be closely matched to instruction and can assess higher-order cognitive…

Increasing social awareness of the need for accurate and fair testing has combined with technological developments to produce dramatic advances in psychometric models, methods, and procedures. Three of the most important new directions within the measurement field concern the following areas: (1) item response theory; (2) item bias or differential…

A brief history of developments in the assessment of the reliability of credentialing examinations is presented, and some new results are outlined that highlight the interactions among scoring, standard setting, and the reliability and validity of pass-fail decisions. Decision consistency is an important concept in evaluating credentialing…

Considerable evidence suggests that policy-makers, educators, the media, and the public do not understand national and state test results. The problems appear to be two-fold: the scales on which scores are reported seem confusing, and the report forms themselves are often too complex for the intended audiences. This paper addresses two topics. The…

The validity of several criticisms of objective tests is addressed, and the viability of some alternatives to objective testing is discussed. Evidence against multiple-choice tests is not as strong as has been claimed. Authentic assessments may not always be better, and research about new forms of assessment is necessary. (SLD)

Performance assessments in education and credentialing are becoming popular. At the same time, there do not exist any well established and validated methods for setting standards on performance assessments. This paper describes several of the new standard-setting methods that are emerging for use with performance assessments and considers their…

A review of educational measurement journals indicates that the decade of the 1970s and up until about 1984 was the time when substantial numbers of studies dealt with criterion-referenced measurement, and the era in which criterion-referenced measurement advances were made. Although there appears to be less interest in publications today, a…