This study examined the reliability of holistic scoring for a sample of essays written during the Spring 1985 MCAT administration. Analysis of variance techniques was used to estimate the reliability of scoring and to partition score variance into that due to level differences between papers and to context-specific factors. (Author/LMO)

Used generalizability theory to investigate the impact of the number of raters and the type of decision (relative versus absolute) on the reliability of writing scores. Results from 251 middle school students and 20 intermediate grade students show that reliability coefficients decline as the number of raters declines and when absolute decisions…

Ratings of student performance for two essay questions rendered by professors of English and by professors in other disciplines were compared for reliability and concurrent validity. It was concluded that the reliability and validity of the ratings of the two groups were nearly comparable. (Author/BW)

To investigate the effect on item parameter estimation of pooling the raters' ratings and to examine the information content of 2 types of writing samples, the partial credit model was fit to 2,000 writing samples from secondary students that had been holistically scored. Implications for test construction are discussed. (SLD)

Studied gender differences in answers to constructed-response mathematics items on approximately 500 papers from grades 7 and 10 from the Kansas Assessment Program. Rubric-relevant variables were highly predictive of holistic scores and accounted for some of the gender differences, especially in grade 7. (SLD)