Raters must score accurately and consistently for classroom observation scores to be valid. This requires (a) a standard defining when scoring is accurate and consistent enough and (b) measuring and remediating rater performance against that standard. Current practice has focused on this second problem to the exclusion of the first. My goal here…

Background: To establish credible, defensible and acceptable passing scores for written tests is a challenge for health profession educators. Angoff procedures are often used to establish pass/fail decisions for written and performance tests. In an Angoff procedure judges' expertise and professional skills are assumed to influence their ratings of…

Examined a procedure for identifying and resolving discrepancies in ratings, focusing on the third rater adjudication procedure used in scoring the Test of Spoken English. Results for 1,446 adult examinees demonstrate that implementing a discrepancy resolution procedure is not sufficient in itself for quality control monitoring. (SLD)