Behavior rating scales are frequently used to assess social-emotional behaviors of children. While broadband behavior rating scales often measure similarly named constructs, it is unclear how consistently different instruments measure those constructs. Head Start teachers completed the preschool versions of the Behavior Assessment System for…

"The Scales for Assessing Emotional Disturbance--Third Edition Rating Scale" (SAED-3 RS; Epstein et al.) is a standardized, norm-referenced measure designed to aid in the identification process by providing useful data to professionals determining eligibility of students with an emotional disturbance (ED). Three studies are reported to…

A methodologically sound systematic review is characterized by transparency, replicability, and a clear inclusion criterion. However, little attention has been paid to reporting the details of interrater reliability (IRR) when multiple coders are used to make decisions at various points in the screening and data extraction stages of a study. Prior…

Background: The Test of Masticating and Swallowing Solids (TOMASS) is an international standardized swallowing assessment tool. However, its psychometric characteristics have not been analysed in patients with dysphagia. Aims: To analyse TOMASS's (1) inter- and intra-rater reliability in a clinical population of patients with dysphagia, (2)…

For some students, standardized tests serve as a conduit to disclose sensitive issues of harm or distress that may otherwise go unreported. By detecting this writing, known as "crisis papers," testing programs have a unique opportunity to assist in mitigating the risk of harm to these students. The use of machine learning to…

This article proposes a Boolean approach to representing and analyzing interobserver agreement in dichotomous coding. Building on the notion that observations are samples of a universe of observations, it submits that coding can be viewed as a process in which observers sample pieces of evidence on constructs. It distinguishes between formal and…

Purpose: The reliability of auditory-perceptual judgments between listeners is a long-standing problem in the assessment of voice disorders. The purpose of this study was to determine whether a relatively novel experimental scaling method, called visual sort and rate (VSR), yielded stronger reliability than the more frequently used method of…

Emotional and behavioral problems (EBPs) frequently occur in young autistic children. Discrepancies between parents and other informants are common but can lead to uncertainty in formulation, diagnosis and care planning. This study aimed to explore child and informant characteristics are associated with reported child EBPs across settings.…

Assuring the quality of education is a concern of learning institutions. To do so, it is necessary to have assertive learning management, with consistent data on students' outcomes. This research provides associate deans and researchers, a roadmap with which to gather evidence to improve the quality of open-ended assessments. Based on statistical…

The Performance Assessment and Evaluation System (PAES) is used by all major universities in the state of Utah to measure the effective teaching skills of preservice candidates as they progress through their teaching preparation program. The resulting ratings are used to make high-stakes decisions relating to course completion as well as…

The educative Teacher Performance Assessment (edTPA) has been adopted by many state legislatures and teacher preparation programs (TPP). These states require teacher candidates to pass the edTPA with a state-specific passing score to be recommended for licensure. In the 19 states where passing the edTPA has not been required as a condition of…

The Classroom Strategies Assessment System (CSAS) is a multi-rater, multi-method (direct observation and rating scale methodology) assessment of teachers' use of research-based instructional and behavior management strategies. The present study investigated the association between teacher self-report and school administrator ratings using the CSAS…

Human raters' assessment of interpreting is a complex process. Previous researchers have mainly relied on verbal reports to examine this process. To advance our understanding, we conducted an empirical study, collecting raters' eye-movement and retrospection data in a computerised interpreting assessment in which three groups of raters (n = 35)…

Peer review is widely accepted as critical to legitimating scholarly publication, and yet, it runs the risk of reproducing inequities in publishing processes and products. Acknowledging at once the historical need to legitimize SoTL publications, the current danger of reproducing exclusive practices, and the aspirational goal to "practice…

The purpose of this synthesis study was to assess the reliability and validity of the Draw-A-Scientist Test (DAST) and its variations across multiple studies, aiming to understand limitations and propose modifications for future application within and beyond the science domain. Given the existence of multiple DAST versions, this study quantified…