Measurement problems which jeopardize the reliability and validity of competency-based writing assessments are analyzed. Methods to stabilize rating criteria and readers' application of them are necessary. Most writing assessment programs use guidelines from norm-referenced test methodology. Use of this method of criteria application based on…

A summary of the writing assessment programs reviewed in this journal is presented. The problems inherent in the programs are outlined. A coordinated research program on major problems in writing assessment is proposed as being beneficial and cost-effective. (DWH)

Several studies have shown that essays receive higher marks when preceded by poor quality scripts than when preceded by good quality scripts. This study investigated the effectiveness of providing scorers with model essays to reduce the influence of context. Context effects persisted despite the scoring procedures used. (Author/EGS)

Three pilot studies--effects of writing prompt modality on writing performance; effect of topic familiarity; and effect of topic, sample and rater group membership on the stability of scoring criteria application--are used to identify the variability in student writing performance. A writing competency test aims to assess writing-specific skills…

A number of studies have found that essays are scored higher when preceded by poor quality scripts than when preceded by good quality scripts. This study investigated the effects of scoring procedures designed to reduce the influence of context. Context effects were found irrespective of the scoring procedure used. (Author/PN)

Two factors which may affect the ratings assigned to an essay test are investigated: (1) context effects; and (2) score level effects. Context effects exist in essay scoring if an essay is rated higher when preceded by poor quality essays than when preceded by high quality essays. A score level effect is defined as a change in the score (value)…

To compare the different images of writing that different assessment methods produce, a study examined two formal writing assessments--holistic and the specially developed intra-subject paired comparison method (IPC)--of pre/post university freshman composition-course writing. The samples of writing were unrehearsed, 50 minute, in-class essays.…

A balanced appraisal of holistic scoring of writing is presented via: examination of the present popularity of holistic scoring; analysis of several weaknesses associated with the holistic scoring method; and recommendations for remedying these weaknesses. Six reasons for the popularity of holistic scoring are: (1) relative lack of expense; (2)…

When the Mesa, Arizona Public Schools initiated an ambitious writing instruction program in 1978, two assessments based on student writing samples were developed. The first is based on a ninth grade proficiency test. If the student does not pass the test, high school remediation is provided. After 1987, students must pass this test in order to…

The occurrence of discrepant, or non-contiguous, scores during holistic scoring of writing proficiency is usually blamed on variables of the test reader or external factors. It is hypothesized that characteristics of the essays themselves sometimes may generate discrepancies in essay scoring. Data from 152 essays of college sophomores in teacher…

This study is an attempt at a cohesive characterization of the concept of essay reliability. As such, it takes as a basic premise that previous and current practices in reporting reliability estimates for essay tests have certain shortcomings. The study provides an analysis of these shortcomings--partly to encourage a fuller understanding of the…

A scoring rubric transition study is described. It was designed to evaluate possible drift in scoring the Maryland Writing Test from year to year (when using a modified holistic scoring method), to evaluate strategies for revising swing rubrics from narrative and explanatory writing while maintaining original scoring standards, and to establish…

Contradictions are inherent in the evaluation of placement test writing, contradictions that at once value and devalue writers and writing, readers and reading. In testing, the evidence for the essay's effectiveness rests almost entirely on the writer's choice of linguistic forms. The characteristics that distinguish evaluation in competency…

These comments provide further details which expand upon the recommendations of the subcommittee on National Assessment of Educational Progress (NAEP) Writing. Three issues are discussed. The first concerns how objectives of assessment in writing ought to be stated. The 1984 objectives are stated in general terms without reference to what might…

In 1990, a study was conducted at Rio Hondo College (Whittier, California) to determine if readers exhibited any bias in scoring test papers that were composed on a word processor as opposed to being written by hand. The study began with the formulation of tentative pilot study questions and the development of procedures to address them. Three…