Purpose: This study intended to examine the generalizability and reliability of essay ratings within the scope of the generalizability (G) theory. Specifically, the effect of raters on the generalizability and reliability of students' essay ratings was examined. Furthermore, variations of the generalizability and reliability coefficients with…

Evaluation of student written work during summative assessments is an important and critical task for instructors at all educational levels. Nevertheless, few research studies exist that provide insights into how different instructors approach this task. Chemistry faculty (FIs) and graduate student instructors (GSIs) regularly engage in the…

The current paper on writing assessment surveys the literature on the reliability and validity of essay tests. The paper aims to examine the two concepts in relationship with essay testing as well as to provide a snapshot of the current understandings of the reliability and validity of essay tests as drawn in recent research studies. Bearing in…

A central aim in any assessment program is to ensure fair and stable scoring from administration to administration. When administrations are decentralized, not only in location, but in frequency and in logistical configuration, it is imperative to construct training, certifying, and monitoring systems that provide continuity between the original…

Reviews "Written Language Assessment" (WLA), a new standardized test to evaluate children's and adolescents' written language competence by having students write essays instead of answer multiple choice questions. Finds problems with the WLA in terms of interrater reliability. (RS)

This study examined the reliability of holistic scoring for a sample of essays written during the Spring 1985 MCAT administration. Analysis of variance techniques was used to estimate the reliability of scoring and to partition score variance into that due to level differences between papers and to context-specific factors. (Author/LMO)

Examines the CUNY-WAT program and questions many aspects of it, especially the choice and phrasing of topics. (FL)

Discusses the problems of readers' different responses to essay test passages and the implications for the subsequent writing assignment. Describes a model of the writing assessment episode and discusses four generalizations about writing task interpretation. (HTH)

In preparation for adding an essay test to the General Educational Development (GED) test, the GED Testing Service undertook a series of studies to establish (1) whether acceptable reading reliabilities were attainable in decentralized holistic scoring sessions often involving no more than a dozen papers; (2) whether essay readers in a variety of…

Examines reasons for using essay tests in the direct assessment of writing ability. Reviews the steps in developing a large-scale testing program; e.g., creating a pool of topics or prompts; developing scoring procedures; training raters; field-testing the system; scoring writing samples; assessing reliability; and assessing validity. (DMM)

The accuracy issues involved in detecting student writing skill deficiencies, factors to be dealt with or controlled, and procedural and statistical methods for controlling these factors are the focus of this article. The accuracy needed for effective instructional planning requires stability in writing assessment programs. (DWH)

In this naturalistic study of essay marking, 15 university lecturers graded an examination paper and completed the Eysenck Personality Questionnaire. A significant positive correlation was found between the marks given and the grader's lie score, indicating possible effects of staff-student interactions or social desirability on biases in grading.…

Two factors which may affect the ratings assigned to an essay test are investigated: (1) context effects; and (2) score level effects. Context effects exist in essay scoring if an essay is rated higher when preceded by poor quality essays than when preceded by high quality essays. A score level effect is defined as a change in the score (value)…

The primary purpose of this study was to examine the relationship between scores on the multiple-choice General Educational Development (GED) Writing Skills test and scores on holistically graded essays. Secondary purposes included the following: (1) examining the relationship of essay scores to scores on the multiple-choice GED Reading Skills…

This study examines differences and similarities in the information provided by direct and indirect measures of writing from the Collegiate Assessment of Academic Proficiency (CAAP). The indirect measure was a 72-item multiple-choice test, while the direct measure involved responding to two essay prompts. The 40-minute multiple-choice test can be…