Field trials were conducted to test rating methods and the impact of feedback about consequences for the 1998 achievement levels-setting (ALS) process for the National Assessment of Educational Progress (NAEP). The field trials provided the opportunity to try out different methods similar to those used successfully by others, as well as to try out…

The impact of overall test length and difficulty on the expert judgments of item performance by the Nedelsky method were studied. Five university-level instructors predicting the performance of minimally competent candidates on a mathematics examination were fairly consistent in their assessments regardless of length or difficulty of the test.…

A discussion of the National Board of Medical Examiners' (NBME) pass-fail grading for Part I and Part II examinations gives background on procedures previously used for setting criterion-based grading standards, gives an overview of research on standard-setting, and outlines NBME plans for determining new standards and incorporating improvements.…

The research reported in this paper was conducted to gain information to guide the selection of standard setting procedures for use with polytomous items to set achievement levels on the National Assessment of Educational Progress (NAEP) assessments in U.S. History and geography. Standard-setting procedures were evaluated to determine the relative…

The National Assessment Governing Board (NAGB) of the National Assessment of Educational Progress (NAEP) adopted mathematics achievement standards on May 11, 1991 that will be used to report results on the 1990 assessment of mathematics. NAEP achievement levels are descriptions, based on informed judgments, of what students should know and be able…

This report presents the results of an initial, iterative performance standards-setting (SS) task of a comprehensive on-the-job statewide teacher assessment system--the System for Teaching and Learning Assessment and Review (STAR). The 1990-91 STAR assesses and makes inferences about the quality of teaching and learning on sets of assessment…

Nedelsky (1954) has suggested a procedure for determining the minimum passing score on a multiple-choice test. In this procedure expert judges estimate the probable score of a minimally competent examinee. The technique does not refer to the students' performance data. The purposes of this paper are: (1) to introduce a modification to the Nedelsky…

The assignment of grades to students is discussed, and a grading process is recommended. The process begins with the separation of minimal and developmental objectives. Minimal objectives represent course outcomes that all students are expected to achieve; developmental objectives are more complex goals toward which students strive, but which few…

The increasing interest in oral proficiency during the past decade prompted Educational Testing Service (ETS) to undertake the development of the Test of Spoken English (TSE), a standardized test of speaking proficiency of non-native speakers of English. The test has been validated for the selection of non-native teaching assistants applying to…

Technical advances of the last 15 years in measurement theory and practice are described, notably criterion-referenced testing, item response theory, and computers and testing. Several remaining problems concerning the development and validation of credentialing examinations are also considered. (Author/LMO)

The description of achievement levels is important to the process of reporting student performance on the National Assessment of Educational Progress (NAEP). A process was designed to develop achievement level descriptions (ALDs) for writing and civics. To begin, focus groups were conducted in each of the four NAEP regions for both writing and…

Steps taken by various states and individual institutions to determine a minimum level of accomplishment among students and to define the basic components of a college education are outlined. These include curriculum evaluations, standards review, and testing. (JMF)

OSCE examinations were held in May and June 2002 for all third and fourth year and some fifth year medical students at the University of Leeds. There has been an arbitrary pass mark of 65% for these examinations. However, we recognise that it is important to adopt a systematic approach towards standard setting in all examinations so held a trial…

This paper provides an analysis of the controversy surrounding the standard setting process conducted by ACT Inc. for the National Assessment Governing Board (NAGB). This process is the most thoroughly planned, carefully executed, exhaustively evaluated, completely documented, and most visible of any standard setting process of which the author is…

This collection of 18 papers (selected from a total of 57 presented at a conference of the International Association for Educational Assessment) represents efforts by examining bodies and institutes to: improve the examination system and testing techniques; develop reliable instruments; and establish standards for public examinations. The papers…