The research described in this paper examines the relationship between raters and the training raters receive to apply that rating. The primary concern is maintaining scoring consistency within the context of raters' perceptions. If the criteria for holistic and/or analytic scoring can be recognized as communication based, rather than topic based,…

Technical questions regarding the reliability and validity of large-scale portfolio assessment were studied which focused on: (1) whether raters can score collections of writing reliably with rubrics designed for single samples; (2) whether ratings derived from different frameworks differ in their capacities to support technically sound…