Studied the use of descriptive graphic rating scales by 11 raters to evaluate students' work, exploring different design features. Used a Rasch-model based rating scale analysis to determine that all the continuous scales could be considered to have at least five points, and that defined midpoints did not result in higher student separation…

The impact of item response theory (IRT) on the measurement practitioner is discussed, with a review of potential benefits. The complexity of IRT theory and procedures and the lack of robustness of IRT procedures to violation of assumptions must be recognized for the measurement practitioner to realize its advantages. (SLD)

Development of a multistage dominant profile method for setting standards on complex performance assessments is detailed. The method grew from experiences with a judgmental policy-capturing procedure and an extended Angoff method. The design of an early adolescence English language arts assessment illustrates the complexity of decisions panelists…

Describes an automated scoring algorithm for a computer-based simulation examination of physicians' patient-management skills. Results with 280 medical students show that scores produced using this algorithm are highly correlated to actual clinician ratings. Scores were also effective in discriminating between case performance judged passing or…

The nature and quality of chapter-end tests accompanying social studies and science textbooks used in elementary school and middle school grades were studied through reviews by 3 judges of 91 tests. Identified shortcomings lead to the recommendation that these tests not be used intact in classroom assessment. (SLD)