Studies on automatic scoring systems in writing assessments have also evaluated the relationship between human and machine scores for the reliability of automated essay scoring systems. This study investigated the magnitudes of indices for inter-rater agreement and discrepancy, especially regarding human and machine scoring, in writing assessment.…

These proceedings include four papers on the status of evaluation Michael Scriven ("Evaluation: The State of the Science") examines the general picture of evaluation, identifies a number of trends in the field, and presents several new evaluation concepts. Included are discussions of "responsibility evaluation,""value…

Causes of evaluator bias are: overemphasizing concrete, salient, and retrievable information; reporting only evidence which confirms hypothesis; focusing on stable personality factors, rather than on situation and environment; developing positive perceptions of a program as both an evaluator and a highly involved participant; statistical naivete;…