The vertical scales of large-scale achievement tests created by using item response theory (IRT) models are mostly based on cluster (or correlated) educational data in which students usually are clustered in certain groups or settings (classrooms or schools). While such application directly violated assumption of independent sample of person in…

Recently, educational researchers and practitioners have turned to value-added models to evaluate teacher performance. Although value-added estimates depend on the assessment used to measure student achievement, the importance of outcome selection has received scant attention in the literature. Using data from a large, urban school district, I…

In this study, we describe a framework for monitoring rater performance over time. We present several statistical indices to identify raters whose standards drift and explain how to use those indices operationally. To illustrate the use of the framework, we analyzed rating data from the 2002 Advanced Placement English Literature and Composition…

Although the Angoff procedure is among the most widely used standard setting procedures for tests comprising multiple-choice items, research has shown that subject matter experts have considerable difficulty accurately making the required judgments in the absence of examinee performance data. Some authors have viewed the need to provide…

In this article, we introduce a person-fit statistic called the hierarchy consistency index (HCI) to help detect misfitting item response vectors for tests developed and analyzed based on a cognitive model. The HCI ranges from -1.0 to 1.0, with values close to -1.0 indicating that students respond unexpectedly or differently from the responses…

The implications of "A Nation at Risk" for the field of measurement are examined. These are a need for more frequent and varied tests; responsibility for eliminating measurement problems; and a resurgence of standardized testing at the high school level. The need for measures of teaching quality will increase. (DWH)

This paper reports two experiments whose purpose was to relate two bodies of research on anxiety: test and trait-state anxiety. It was reasoned that state anxiety measures obtained in an evaluation testing condition should be more similar to test anxiety than state anxiety measures obtained in non-evaluative situations, such as a game in Study I…

The author offers some thoughts on commentator's reactions to the substance of the measures, particularly those about measuring teacher learning and change, based on the major uses of the measures, and because this is a significant challenge facing test development as an enterprise. If teacher learning results in more integrated knowledge or…

In this article, the author echoes his co-author's and colleague's pleasure (Hill, this issue) at the thoughtfulness and far-ranging nature of the comments to their initial attempts at test validation for the mathematical knowledge for teaching (MKT) measures using the validity argument approach. Because of the large number of commentaries they…

The practice of analyzing all available project children in as large a group as possible is considered not to be justifiable when distinct subgroups of pupils are represented. Instead, the approach suggested here determines the test score gain a pupil achieves from the beginning to the end of the year, with all of the pupil gain scores of a single…

A discussion of performance contracting, defined as an agreement between a group offering instruction and a school needing the services, is presented. Four major hazards to direct measurement of specific learning are considered: poor statement of objectives; selection of the wrong tests; misinterpretation of test scores; and depersonalization of…

Seven major topics were included in the conference proceedings: (1) Improving Evaluation of Educational Outcomes at the College Level; (2) Individual versus Group Decision Making; (3) Problems and Procedures in Profile Analysis; (4) Making Test Results Meaningful; (5) The Teaching of Educational Measurement; (6) The Interview as an Evaluation…

This guide is essentially designed as a teaching aid for those who would inform planners, officials of educational ministries, school administrators, principals, and teachers about educational outcome measurements. In outline and graphic form, the guide presents topics for discussion in a seminar dealing with how to obtain information on…

THE DIRECTOR OF TESTING OF THE MODERN LANGUAGE ASSOCIATION (MLA), WITH THE ASSISTANCE OF 28 SELECTED IMPARTIAL PROFESSIONALS, PRODUCED INDIVIDUALLY AND IN TEAMS THIS CRITICAL APPRAISAL OF THE MLA FOREIGN LANGUAGE PROFICIENCY TESTS IN ORDER TO POINT OUT EXISTING DEFICIENCIES AND TO SUGGEST IMPROVEMENTS IN FUTURE TEST DEVELOPMENT. THIS HANDBOOK…