Each fall the author assigns undergraduate teacher education students to design rubrics that they can use to evaluate auditions for reality television shows like "American Idol." The point is to draw students into a critical reflection on assessment literature and practices they have been studying. Through these reflections, the author…

This study examined the effects of marking method and rater experience on ESL (English as a Second Language) essay test scores and rater performance. Each of 31 novice and 29 experienced raters rated a sample of ESL essays both holistically and analytically. Essay scores were analysed using a multi-faceted Rasch model to compare test-takers'…

This study evaluated a "substantively driven" method for scoring NAEP writing assessments automatically. The study used variations of an existing commercial program, e-rater[R], to compare the performance of three approaches to automated essay scoring: a "brute-empirical" approach in which variables are selected and weighted solely according to…

Based on the responses of 50 ESL instructors to a composition-scoring exercise, a detailed method of scoring compositions was developed that divides the writing into basic components (structure, content, vocabulary, organization, and mechanics) and provides a scoring mechanism for each component for each of three competency levels. (MSE)

This report identifies the methods and issues in the evaluation of student writing in classrooms and large scale assessments using an analytical scoring system from the Illinois Inventory of Educational Progress (IIEP). This guide is written for staff in state offices of education conducting large-scale writing assessments, and also for personnel…

The effectiveness of the "read-comp" technique in assessing writing ability and the usefulness of a rubric and procedure devised for scoring read-comp samples and essays were evaluated. Subjects were 100 freshman students enrolled in general and remedial English classes in a 6-week summer session at Indiana University of Pennsylvania.…

Four systems for scoring student essays were used to classify eleventh grade and undergraduate students according to writing ability. The reliabilities of the raters and the validities of the systems in classifying students were emphasized. Two analytic scoring systems--which assume that quality writing is characterized by the inclusion of certain…