Classroom observation is a common approach to teacher evaluation. Yet, concerns about differences in rater judgment are widespread. Despite this concern, few researchers have examined the practical impact of such differences in rater judgments on teachers' judged effectiveness. This study fills that gap. Using data from a large-scale teacher…

Identifying and describing children's play behaviours is an important component of evaluating child development. The Behaviour Mapping Schedule is a direct observational tool which aims to describe and quantify children's play behaviours but is yet to undergo reliability testing. This study aimed to determine the intra- and inter-rater reliability…

Growing awareness of autism spectrum disorders has increased the demand for diagnostic services in adulthood. High rates of mental health problems have been reported in young people and adults with autism spectrum disorder. However, sampling and methodological issues mean prevalence estimates and conclusions about specificity in psychiatric…

This paper presents the voice emotion recognition part of the FILTWAM framework for real-time emotion recognition in affective e-learning settings. FILTWAM (Framework for Improving Learning Through Webcams And Microphones) intends to offer timely and appropriate online feedback based upon learner's vocal intonations and facial expressions in order…

This study examined the psychometric properties of the Revised Edition of the School Observation Coding System (REDSOCS). Participants were 68 children ages 3 to 6 who completed parent-child interaction therapy for Oppositional Defiant Disorder as part of a larger efficacy trial. Interobserver reliability on REDSOCS categories was moderate to…

An asymmetric version of J. Cohen's kappa statistic is presented as an appropriate measure for the agreement between two observers classifying items into nominal categories, when one observer represents the "standard." A numerical example with three categories is provided. (SLD)

In a project on the development of performance assessments in science, researchers have developed four types of assessment: comparative, component identification, classification, and observation. They have noted that assessments belonging to the same type of task can be scored for the same performance properties. This paper reports on the…