In this research, we investigated the suitability of implementing "e-rater"® automated essay scoring in a high-stakes large-scale English language testing program. We examined the effectiveness of generic scoring and 2 variants of prompt-based scoring approaches. Effectiveness was evaluated on a number of dimensions, including agreement…

This study was designed to address two frequent criticisms of the SAT essay--that essay length is the best predictor of scores, and that there is an advantage in using more "sophisticated" examples as opposed to personal experience. The study was based on 2,820 essays from the first three administrations of the new SAT. Each essay was…

The primary purpose of this study was to investigate the efficacy of formulaic writing such as the five-paragraph theme (FPT) or essay for the purpose of earning high scores on high-stakes writing assessments. This qualitative descriptive study analyzed more than 1000 essays from Delaware Grade 8 and 10 writers, written for a statewide…

Examined a scoring anomaly that became apparent in a state-mandated writing assessment. Results for 3,660 essays by sixth graders show that using a spiral model for training raters and scoring papers results in higher mean ratings than does using a sequential model for training and scoring. Findings demonstrate the importance of making decisions…

The writing assessment program in the Cherry Creek, Colorado, schools is described. Writing assessments for students in grades three, six, eight, and 10 have been developed. A holistic approach is used by teachers in scoring these instruments. Improvement of students' essay-writing ability indicates the value of the assessment program. (DWH)

A representative group of 4,000 California high school seniors was randomly assigned one of five essay topics. Each essay was given two independent ratings based on criteria specified for each topic by an advisory group of English teachers. On the basis of the scores, a representative subset of 750 essays was randomly selected from the top…

The purpose of this study was to examine rater, domain, and gender influences on the assessed quality of student writing using weighted and unweighted scores. Twenty rates were randomly selected from a group of 87 operational raters contracted to rate essays as part of the 1993 field test of the Georgia High School Writing Test. All of the raters…

The purpose of this study was to describe the Rasch model for measurement and apply the model to examine the relationship between raters, domains of written compositions, and student writing ability. Twenty raters were randomly selected from a group of 87 operational raters contracted to rate essays as part of the 1993 field test of the Georgia…

The studies reported here focus on the effects of mode of administration of the Massachusetts Comprehensive Assessment System (MCAS) in grades 4, 8, and 10. These studies, which draw on earlier studies, also examine the mode of administration effect at different levels of keyboarding speed and for students who received special education (SPED)…

The effects of presenting a choice of writing tasks on the quality of essays produced by eleventh graders were studied with 34,200 students in Georgia. The choice condition had no substantive effect on the quality of essays, but race, gender, and the writing task variable did. (SLD)

The necessity of controlling the order in which trained essay raters for a statewide writing assessment program receive student essays was studied. The underlying theoretical question concerns possible rater bias caused by raters reading long strings of essays of homogeneous quality; this problem is usually referred to as context effect or…

This study compared the different modes of evaluation at work in Greece's secondary school graduation and university admission process. The goals of the study were (1) to provide background information regarding the contribution of in-school assessment scores and external examination scores to higher education admissions decisions, and (2) to…

The differential performance of black and white college freshmen on a direct measure of writing skills, the essay portion of the American College Testing Program's Collegiate Assessment of Academic Proficiency (CAAP), was studied. The data consisted of responses of a sample of 998 black and 3,727 white examinees to two individual prompts for…

The viability of the Partial Credit Model (PCM) of G. N. Masters (1982) for scaling and banking individual essay prompts from a statewide writing assessment program was investigated. Preliminary analyses indicated the feasibility of using the PCM, the general polychotomous form of the Rasch model, with essay data from the Maryland Writing Test…

Statistical equating of scores from direct writing assessments is performed in Maryland to hold the difficulty of passing the Maryland Writing Test (MWT) equivalent from year to year and to allow a demonstration of growth in student writing achievement over the years. Passage of the test is a high school graduation requirement. Prior to…