A group-process approach to standard setting was compared to an independent approach for a medical specialty certification examination. Both approaches used the Angoff (1971) standard-setting method. In the group-process method, reviewers discussed items and their ratings during the rating process; in the independent condition, reviewers provided…

This study investigated the impact of task directions on the mathematical performance of high school students from six classes. Students analyzed data regarding school dropout by answering six short-answer questions and writing a letter discussing the trends and their predictions about school dropout. Tasks were scored using two methods: (1) trait…

This paper compares various methods of smoothed equipercentile equating and linear equating in the random groups equating design. Three presmoothing methods (based on the beta binomial model, four-parameter beta binomial model and a log-linear model) are compared to postsmoothing using cubic splines, linear equating and unsmoothed equipercentile…

Standard setting is a fairly widespread activity in educational and psychological measurement, but there is no formal psychometric theory to guide the development of standard setting methodology. This paper presents a conceptual framework for such a psychometric theory and uses the conceptual framework to analyze a number of methods for setting…

To determine the extent of influence that a reader's perspective as a member of a specific audience has on the assessment of student writing, a study examined the holistic judgment and the positive and negative comments made by four different types of writing evaluators on 40 different letters, each of which was written to one of four audience…

In the Spring 2003 issue of "Harvard Educational Review," Roy Freedle stated that the SAT® is both culturally and statistically biased, and he proposed a solution to ameliorate this bias. His claims, which garnered national attention, were based on serious errors in his analysis. We begin our analyses by assessing the psychometric…

Techniques derived from item response theory are useful for estimating the reliability of test classification above and below the cutting score. Test developers can construct a test whose information is peaked in the region of the cutting score; users can select a test which provides the most information in this region. The Cut-Score…

Students in an introductory college course (n=275) responded to equivalent 20-item halves of a test under number-right and formula-scoring instructions. Formula scores of those who omitted items overaged about one point lower than their comparable (formula adjusted) scores on the test half administered under number-right instructions. In contrast,…

This research investigated the effect on estimated lower asymptotes of the instructions to Graduate Record Examination (GRE) examinees about how the test would be scored. This effect was assessed for four different verbal item types (analogies, antonyms, sentence completion, and reading comprehension) using a two-way, unweighted means analysis of…

Specific steps for applying the Angoff method are described. In the Angoff method, judges are asked to estimate the probabilities of minimally competent candidates' answering multiple choice test items correctly. Initial information must be obtained for designing the standard-setting process, beginning with the purpose of the examination and any…

Traditionally, judgmental standard setting methods have been used exclusively at the test item level. In this study, the Ebel and Angoff methods of standard setting were utilized to determine minimum competency standards on a list of 175 identified competency statements for vocational evaluators. The following research questions were addressed:…

The Comprehensive Tests of Basic Skills, Form U, is scored in two ways: number-correct and pattern. The latter makes use of the information about which particular items are answered correctly, giving more weight to the more discriminating items and making allowances for guessing. Critics have suggested that black students are penalized by pattern…

Self-Knowledge Development Theory (SKDT) by Weinstein and A. Alschuler (1985) is a structural developmental theory positing four stages in the development of self-knowledge. The Experience Recall Test-2 (ERT2) is described, which is the most recent instrument developed for assessing the SKDT. Self-knowledge is defined as the ability to describe…

In preparation for adding an essay test to the General Educational Development (GED) test, the GED Testing Service undertook a series of studies to establish (1) whether acceptable reading reliabilities were attainable in decentralized holistic scoring sessions often involving no more than a dozen papers; (2) whether essay readers in a variety of…

This publication contains some of the open-ended art exercises used by the National Assessment of Educational Progress in its 1978-79 assessment of the art ability of students ages nine through 17. The objective is to provide classroom teachers easy access to released and tested art assessment materials. The open-ended exercises required students…