The hierarchical rater model (HRM) recognizes the hierarchical structure of data that arises when raters score constructed response items. In this approach, raters' scores are not viewed as being direct indicators of examinee proficiency but rather as indicators of essay quality; the (latent categorical) quality of an examinee's essay in turn…

A new method for evaluating rater accuracy within the context of performance assessments is described. It uses an extended Rasch measurement model, FACETS, which is illustrated with 373 benchmark papers from the Georgia High School Graduation Writing Test rated by 20 operational raters and an expert panel. (SLD)

Reports the development of an experimental essay test for use with young children based on the now-obsolete STEP Essay Tests. The child's essay on an assigned topic is rated in comparison with previously rated and scaled essays which serve as performance models. (Author)

The relationship between scores on objective tests of language skills and on free writing tasks was analyzed for third and eighth graders. Correlations between scores were of the same magnitude as reported for college students. Differences in relationships between free-writing performance and objective test scores are discussed. (Author/RD)

Test results of 915 high school students taking a history examination with a choice of topics show that students were generally able to pick the topic on which they could get the highest score. Implications for fair scoring when topic choice is allowed are discussed. (SLD)

Investigates the effect of initial blocks of either very good or poor essay question responses on the grades assigned to subsequent essay responses. (Author/DEP)

Raters judged essays on two occasions making it possible to address the question of whether multiple ratings, however obtained, represent the same true scores. Multiple ratings of a given rater did represent the same true scores, but ratings of different raters did not. Reliability, validity, and invalidity coefficients were computed. (Author/DWH)

Several studies have shown that essays receive higher marks when preceded by poor quality scripts than when preceded by good quality scripts. This study investigated the effectiveness of providing scorers with model essays to reduce the influence of context. Context effects persisted despite the scoring procedures used. (Author/EGS)

The effect of context on the scoring of essays was examined by arranging that the scoring of the criterion essay would be preceded either by five superior essays or by five inferior essays. The contrast in essay quality had the hypothesized effect. Other effects were not significant. (CTM)

Proposition analysis was used to equate the text base of two essays with different readability levels. Easier reading essays were given higher scores than difficult reading essays. The results appear to identify another noncontent influence on essay test scores, leaving increasingly less variance for differences in content. (Author/PN)

Study was an attempt to evaluate the West African Examinations Council efforts in terms of the extent to which its marking procedures have ensured high reader reliability for the English Language Essay examination, a test which was designed to measure writing ability. (Author)

Approximately 300 prospective graduate students each wrote two essays for the Graduate Record Examinations in 40-minute and 60-minute time periods. Analysis revealed that performance was, on average, significantly better with the 60-minute limit. There was no interaction between self-described test-taking style (fast versus slow) and time limits.…

A method of assessing rater reliability via a design of overlapping rater teams is presented. Covariances or correlations of ratings can be analyzed with LISREL models. Models in which the rater reliabilities are congeneric, tau-equivalent, or parallel can be tested. Two examples based on essay ratings are presented. (TJH)

This paper assessed the relationships among holistic writing ability, the Test of Standard Written English, and four tests of organizational ability. Findings showed a significant correlation between writing ability and the tests. It was concluded that tests assessing organizational strategies ought to be included in assessments of writing…

It is hypothesized that the readers of an essay respond to a variable in terms of its context with other variables. Sex, race, reader expectation, and quality of handwriting were crossed to study their interaction effects. Results showed complex interactions of expectations, writing, and sex within race. (Author/LMO)