Taking more than one measurement of the same variable also hosts the possibility of contamination from error sources, both singly and in combination as a result of interactions. Therefore, although the internal consistency of scores received from measurement tools is examined by itself, it is necessary to ensure interrater or intra-rater agreement…

Drawing parallels to classical test theory, this article clarifies the difference between rater accuracy and reliability and demonstrates how category marginal frequencies affect rater agreement and Cohen's kappa. Category assignment paradigms are developed: comparing raters to a standard (index) versus comparing two raters to one another…

Assessing global interrater agreement is difficult as most published indices are affected by the presence of mixtures of agreements and disagreements. A previously proposed method was shown to be specifically sensitive to global agreement, excluding mixtures, but also negatively biased. Here, we propose two alternatives in an attempt to find what…

This study compared reading rate to reading fluency (as measured by a rating scale). After listening to first graders read short passages, we assigned an overall fluency rating (low, average, or high) to each reading. We then used predictive discriminant analyses to determine which of five measures--accuracy, rate (objective); accuracy, phrasing,…

As an initial step toward improving the outcomes of learners with disabilities, special educators have formulated guidelines for identifying evidence-based practices. We describe the Council of Exceptional Children's new set of standards for identifying evidence-based practices in special education and how they (a) were systematically vetted by…

The purpose of this investigation was to evaluate the models for interpretation and use that serve as the foundation of an interpretation/use argument for the Social and Academic Behavior Risk Screener (SABRS). The SABRS was completed by 34 teachers with regard to 488 students in a Midwestern high school during the winter portion of the academic…

Growing awareness of autism spectrum disorders has increased the demand for diagnostic services in adulthood. High rates of mental health problems have been reported in young people and adults with autism spectrum disorder. However, sampling and methodological issues mean prevalence estimates and conclusions about specificity in psychiatric…

This paper presents the voice emotion recognition part of the FILTWAM framework for real-time emotion recognition in affective e-learning settings. FILTWAM (Framework for Improving Learning Through Webcams And Microphones) intends to offer timely and appropriate online feedback based upon learner's vocal intonations and facial expressions in order…

With over 800 million users worldwide, the global importance of Facebook as a social-networking platform is beyond doubt. This popularity, particularly among university-students, has encouraged research to explore ways in which social networking can be adapted into virtual learning environments. In particular, this study uses the think-aloud…

Students' writing can provide better insight into their thinking than can multiple-choice questions. However, resource constraints often prevent faculty from using writing assessments in large undergraduate science courses. We investigated the use of computer software to analyze student writing and to uncover student ideas about chemistry in an…

Quantitative behaviour analysis requires the classification of behaviour to produce the basic data. This can be challenging when the theoretical taxonomy does not match observational limitations, or if a theoretical taxonomy is unavailable. Binary keys allow qualitative observation to be used to modify a theoretical taxonomy to produce a practical…

Introduction: The Accreditation Council for Continuing Medical Education (ACCME) provides guidelines for continuing medical education (CME) materials to mitigate problems in the independence or validity of content in certified activities; however, the process of peer review of materials appears largely unstudied and the reproducibility of…