In this secondary analysis of a previously conducted systematic review, we analyze social validity assessments in intervention research for transition-age autistic youth. Social validity is concerned with the acceptability of the intervention goals, the acceptability and feasibility of the intervention procedures, and the perceived importance of…

In this perspective, the authors critically examine "rater training" as it has been conceptualized and used in medical education. By "rater training," they mean the educational events intended to "improve" rater performance and contributions during assessment events. Historically, rater training programs have focused…

The agreement between a wearable inertial sensor (GYKO, G) and the force platform (P) was assessed by evaluating "test-retest" and "inter-rater reliability." Thirty-eight subjects were enrolled; the selected indices of balance were investigated over foot positions and (un)stable conditions. Intraclass correlation coefficient…

The current review examined the use of self-report measures in autistic individuals in the context of psychiatric assessments. It focused on inter-rater agreement, internal consistency, test-retest reliability, and criterion validity with clinical diagnoses. It also gathered information on constructs measured, the nature of the samples, and the…

Despite a tremendous increase in the use of video for conducting research in classrooms as well as preparing and evaluating teachers, there remain notable challenges to using classroom videos at scale, including time and financial costs. Recent advances in artificial intelligence could make the process of analyzing, scoring, and cataloguing videos…

For autism spectrum disorder (ASD) in adults there are several diagnostic instruments available with a need for consideration of the psychometric properties. This study aimed to conduct a first psychometric evaluation of a new diagnostic ASD instrument, the NIDA (Dutch Interview for Diagnostic assessment of ASD in adults) in 90 adult males without…

Measuring classroom behavior among young children is important to guide assessment and intervention decisions, yet there is limited literature on appropriate direct observation tools for this purpose. This article describes the psychometric properties of the Behavior Assessment System for Children, Student Observation System (BASC-3 SOS) with 135…

Practitioners often struggle to assess reflective learning in the workplace because of difficulties conceptualizing reflection and its effects in the workplace. This article addresses this problem by offering a pragmatic approach to assessment that asks practitioners to specify why they are using reflection, what they are hoping to gain from it,…

Purpose: Auditory training has been shown to reduce rater variability in perceptual voice assessment. Because rater variability is also a central issue in the auditory-perceptual assessment of dysarthria, this study sought to determine if training produces a meaningful change in rater reliability, criterion validity, and scaling magnitude of four…

Interdisciplinary thinking is critical for equipping students to apply scientific knowledge and tackle societal challenges across various disciplines, which has been recognized as a key objective of twenty-first century science education. However, research on effective interdisciplinary assessment in secondary school science education is still…

In this study, we examined the scoring and generalizability assumptions of an explicit instruction (EI) special education teacher observation protocol using many-faceted Rasch measurement (MFRM). Video observations of classroom instruction from 48 special education teachers across four states were collected. External raters (n = 20) were trained…

Single-case research design is a useful methodology for evaluating the presence of a functional relation between an intervention and the mathematical performance of students with a learning disability. However, a functional relation cannot be established with threats to internal validity of the design. External validity is impacted if researchers…

To examine reliability and validity of the new Social Motor Function Classification System for Children with Autism Spectrum Disorders (SMFCS-ASD). The SMFCS-ASD reliability was examined on 25 children (62.4 months SD 7.8) with ASD among six physical therapists. The validity study involved 1001 children (57.0 months, SD 9.9) with ASD using the…

Rater-mediated assessments require the evaluation of the accuracy and consistency of the inferences made by the raters to ensure the validity of score interpretations and uses. Modeling rater response processes allows for a better understanding of how raters map their representations of the examinee performance to their representation of the…

When rating performance assessments, raters may ascribe different scores for the same performance when rubric application does not align with the intended application of the scoring criteria. Given performance assessment score interpretation assumes raters apply rubrics as rubric developers intended, misalignment between raters' scoring processes…