This paper provides a historical review of the changes and improvements made in estimating numerical cutpoints for the National Assessment of Educational Progress (NAEP). While reviewing the various methodologies used for collecting judgment data, the paper discusses: (1) the incorporation of Item Response Theory for setting standards; (2)…

The primary objective of this research was to examine the effect of scoring items known to be multidimensional using a unidimensional model. Although several simulation studies have examined this, few studies have been conducted using data obtained from actual test administrations. In this study, open-ended mathematics items from a mandated state…

The effects of using recommended data collection procedures on median recommended test standards, variability of recommended test standards, and reliability of recommended standards for 7 subtests of the National Teacher Examinations Communications Skills and General Knowledge Tests were explored, using 236 evaluators (75 public school teachers…

This Louisiana System for Teaching and Learning Assessment and Review (STAR) illustrates conceptual, methodological, and social/political issues emerging when the decision-making model used to make teacher certification decisions does not fit the assumptions of the measurement system used to collect the data. (SLD)

This article provides a framework for the rest of the articles in this special issue comparing the utility of three standard-setting methods with complex performance assessments. The context of the standard setting study is described, and the methods are outlined. (SLD)

Methods for setting cutoff scores based on criterion performance, normative comparison, and absolute judgment were compared for scores on a multiple-choice psychology examination for 121 undergraduates and 251 undergraduates as a comparison group. All methods fell within the standard error of measurement. Implications of differences for decision…

A rigorous placement exam introduced into a university Developmental Mathematics program increased course success rates, retention, and enrollment rates in math courses, but reduced completion rates in Developmental Math and standard math courses required for a degree. Results suggest an inverse relationship between course success rates and total…

Considerable evidence suggests that policy-makers, educators, the media, and the public do not understand national and state test results. The problems appear to be two-fold: the scales on which scores are reported seem confusing, and the report forms themselves are often too complex for the intended audiences. This paper addresses two topics. The…

Studies of the Angoff method of standard setting suggest that judges agree in their estimates of the relative difficulties of test questions for minimally competent examinees and that each judge's estimates correlate well with the observed item difficulties for examinees whose total test scores are near the judge's personal standard (G. E.…

Praxis III is an assessment procedure that provides information for making instructional and licensing decisions about beginning teachers. The Praxis III Assessor's job is to interview the beginning teacher, observe the teacher in the classroom, score the teacher's performance on 19 criteria, and summarize the evidence for each score. The Assessor…

This study examined the contribution of five categories of variables to the standard-setting recommendations of a representative sample of 241 judges recommending standards for the General Knowledge and Communication Skills Tests of the National Teacher Examinations (NTE) for the state of North Carolina. It was based on the assumption that…

The purpose of this guide is to provide state and local Job Training Partnership Act (JTPA) program administrators with an approach to developing meaningful and relevant negotiated performance standards that will serve as a management and evaluation tool. The guide is organized in four chapters. Chapter I provides background information on the…

The Massachusetts Policy on Basic Skills Improvement was formulated to help students master basic skills prior to high school graduation. The aim of this guide is to assist local districts in complying with the policy's requirements for the setting of minimal standards according to levels of achievement students are expected to reach on stated…

Compares two approaches for establishing student achievement standards and expectations for province-wide assessment in Saskatchewan (Canada). The first approach based standards on judges' estimates regarding the percentage of students who should be able to attain desired levels of performance. The second approach required judges to reach a…

The process of assigning grades to students is analyzed, and a specific approach to grading is recommended that distinguishes between minimal and developmental objectives. Criterion-referenced and norm-referenced concepts are used in the approach, which is best suited for secondary school or college. (SLD)