Any examination that involves moderate to high stakes implications for examinees should be psychometrically sound and legally defensible. Currently, there are two broad and competing families of test theories that are used to score examination data. The majority of instructors outside the high-stakes testing arena rely on classical test theory…

This article provides an introduction to the kind of computer software that is used to score student writing in some high stakes testing programs, and that is being promoted as a teaching and learning tool to schools. It sketches the state of play with machines for the scoring of writing, and describes how these machines work and what they do.…

The purpose of this study is to determine the extent of scale drift on a test that employs cut scores. It was essential to examine scale drift for this testing program because new forms in this testing program are often put on scale through a series of intermediate equatings (known as equating chains). This process may cause equating error to…

This handbook describes the responsibilities of district and school assessment coordinators in the administration of the Pennsylvania System of School Assessment (PSSA). This updated guidebook contains the following sections: (1) General Assessment Guidelines for All Assessments; (2) Writing Specific Guidelines; (3) Reading and Mathematics…

This report presents the results of a research and development effort for SpeechRater? Version 1.0 (v1.0), an automated scoring system for the spontaneous speech of English language learners used operationally in the Test of English as a Foreign Language™ (TOEFL®) Practice Online assessment (TPO). The report includes a summary of the validity…

National Assessment of Educational Progress writing sample essays from 1988 and 1990 (495 and 599 essays) were subjected to computerized grading and human ratings. Cross-validation suggests that computer scoring is superior to a two-judge panel, a finding encouraging for large programs of essay evaluation. (SLD)

This paper is based on guidelines developed in 1989 for training workshops for state and local educators to demonstrate the processes by which performance assessments could be created, validated, and used in statewide assessment programs. These guidelines are based on work with the National Assessment of Educational Progress and several statewide…

Cost factors associated with the development, administration, and scoring of performance assessment tasks are examined in the context of a statewide or other large-scale assessment program. Resources of money, time, and expertise are discussed. (SLD)

Writing assessment began in Alaska in the 1970s, and the Alaska Writing Assessment (AWA) that was piloted in 1997 built on previous efforts. The 1997 AWA involved more than 20,000 students in grades 5, 7, and 10 from 43 school districts, and the mandatory assessment planned for 1998 will include approximately 28,000 students. This review of the…

In the 1995-96 school year, the Memphis (Tennessee) City Schools released standards for student performance in seven content areas and began laying the foundation for a standards-based curriculum and assessment system. The steps taken to develop and implement this project are outlined as follows: (1) defining the objectives and the project scope;…

Year-to-year rater variation may result in constructed response (CR) parameter changes, making CR items inappropriate to use in anchor sets for linking or equating. This study demonstrates how rater severity affected the writing and reading scores. Rater adjustments were made to statewide results using an item response theory (IRT) methodology…

The 1992-93 school year saw the second statewide implementation of the Vermont portfolio-assessment program, and RAND continued its ongoing evaluation of the program's implementation, effects, and data quality. While the first year's study found evidence of the impact of the assessment program and low reliability of portfolio scoring, this year's…

A college faculty member who has graded Advanced Placement exam essays on U.S. government and politics, taken mostly by high school juniors and seniors, suggests that high school teachers and college faculty who assess the essays are not the best qualified persons to do so and that despite efforts to ensure consistency, the resulting scores are…

A faculty consultant to the Educational Testing Service for advanced placement (AP) test reading in U.S. government and politics responds to an article criticizing essay evaluation methods and criteria, finding in it a fundamental misunderstanding of the AP reading process and explaining why the essays are subject to less scrutiny for style,…

This report describes the North Carolina Annual Testing Programs writing task which was administered in 1985-86. Grade six students were tested on their ability to write a clarification composition; while grade 8 students were evaluated on their skills in writing a persuasive composition. The timed composition (50 minutes) was scored by two…