Best linear prediction (BLP) and penalized best linear prediction (PBLP) are techniques for combining sources of information to produce task scores, section scores, and composite test scores. The report examines issues to consider in operational implementation of BLP and PBLP in testing programs administered by ETS [Educational Testing Service].

In this research, we investigated the suitability of implementing "e-rater"® automated essay scoring in a high-stakes large-scale English language testing program. We examined the effectiveness of generic scoring and 2 variants of prompt-based scoring approaches. Effectiveness was evaluated on a number of dimensions, including agreement…

This paper provides a history of ETS's role in developing assessment instruments and psychometric procedures for measuring change in large-scale national assessments funded by the Longitudinal Studies branch of the National Center for Education Statistics. It documents the innovations developed during more than 30 years of working with…

To help teachers, administrators, and parents understand student performance in writing, a state-level report is prepared each year to analyze students writing scores and provide guidelines for the interpretation of the results. This report compares students scores on the 2001 Delaware Student Testing Program (DSTP) with scores on the 2000 DSTP…

The North Carolina Writing Assessment, which is part of the state's End-of-Grade testing program, requires students in grades 4 and 7 to write essays in response to a standardized prompt. This report contains results for the Wake County Public School System (WCPSS). Fourth grade writing assessment scores across North Carolina rose in 1999, and in…

Many testing programs include a section of optional questions in addition to mandatory parts of a test. These optional parts of a test are not often truly parallel to one another, and groups of examinees selecting each optional test section are not equivalent to one another. This paper provides a general method based on missing-data methods for…

This guide explains results and other information in the "Test Item Analysis Report," the "School Report," and the "District Report" for the Massachusetts Comprehensive Assessment System (MCAS) tests for spring 2000. It is designed to help superintendents, principals, and other administrators as they review the…

This study was designed to address two frequent criticisms of the SAT essay--that essay length is the best predictor of scores, and that there is an advantage in using more "sophisticated" examples as opposed to personal experience. The study was based on 2,820 essays from the first three administrations of the new SAT. Each essay was…

The effect of rapid feedback for a state writing assessment on subsequent writing performance was investigated. In addition, the agreement between teachers' scores for the state writing assessment and state department scores was analyzed. Eighth grade English teachers (n=8) were trained in analytic trait scoring of writing assessments. They then…

This report describes the procedures followed in scoring the October 1978 Florida Minimal Writing Production Skills Assessment and reports the results of that assessment. The assessment was conducted on a sample of Florida public school students in grades 3, 5, 8, and 11. Sections include descriptions of the rating scale and scorer's guide as well…

This study invesitigated possible reasons for the low performance on the text-based writing assessment of the Delaware Student Testing Program (DSTP) in 2000, especially for grades 3 and 5, and considered ways to improve classroom instruction. In the first part of the study, a panel of teachers reviewed the anchor papers from the assessment and…

Portfolios are the centerpiece of Vermont's statewide assessment program in mathematics. Portfolio scores in the first two years were not reliable enough to permit the reporting of student-level results, but increasing the number of readers or the number of portfolio pieces is not operationally feasible. (SLD)

This study evaluated agreement between expert system and human scores on 12 algebra word problems taken by Graduate Record Examinations (GRE) General Test examinees from a general sample of 285 and a study sample of 30. Problems were drawn from three content classes (rate x time, work, and interest) and presented in four constructed-response…

The Connecticut Mastery Test was designed to assess specific skill levels of students by measuring performance on various learning objectives that students can be expected to master. The grade 4 Connecticut Mastery Test, given for the first time in the fall of 1985, provides information which can be used to improve instruction and the basic skills…

Year-to-year rater variation may result in constructed response (CR) parameter changes, making CR items inappropriate to use in anchor sets for linking or equating. This study demonstrates how rater severity affected the writing and reading scores. Rater adjustments were made to statewide results using an item response theory (IRT) methodology…