A multiple baseline design across four teachers was used to investigate the effects of an online Direct Behavior Rating (DBR) training module on rater accuracy of academic engagement, respect, and disruptive behavior. Rater accuracy was examined by comparing Direct Behavior Rating-Single Item Scales (DBR-SIS) completed by classroom teachers to systematic direct observation. Teacher acceptability of DBR-SIS was also investigated. Teachers rated the use of DBR-SIS as a formative evaluation tool favorably. Rater accuracy of DBR-SIS was generally higher after teachers completed the online training module. However, a visual analysis of the data revealed no clear impact of the online training.