The purpose of this synthesis study was to assess the reliability and validity of the Draw-A-Scientist Test (DAST) and its variations across multiple studies, aiming to understand limitations and propose modifications for future application within and beyond the science domain. Given the existence of multiple DAST versions, this study quantified…

The current review examined the use of self-report measures in autistic individuals in the context of psychiatric assessments. It focused on inter-rater agreement, internal consistency, test-retest reliability, and criterion validity with clinical diagnoses. It also gathered information on constructs measured, the nature of the samples, and the…

Visual analysis is the primary methodology used to determine treatment effects from graphed single-case design data. Previous studies have demonstrated mixed findings related to interrater agreement between both expert and novice visual analysts, which represents a critical limitation of visual analysis and supports calls for also presenting…

Procedural fidelity refers to the degree to which procedures for an assessment or intervention (i.e., independent variables) are implemented consistent with the prescribed protocols. Procedural fidelity is an important factor in demonstrating the internal validity of an experiment and clinical treatments. Previous reviews evaluating the inclusion…

Objective: The precise measurement of treatment fidelity (quantity and quality in the delivery of treatment strategies in an intervention) is essential for intervention development, evaluation, and implementation. Various informants are used in fidelity assessment (e.g., observers, practitioners [clinicians, teachers], clients), but these…

Background: Intercoder reliability is a statistic commonly reported by researchers to demonstrate the rigour of coding procedures during data analysis. Its importance is debatable in the analysis of qualitative interview data. It raises a question on whether researchers should identify the same codes and themes in a transcript or they should…

Introduction: Assessment of resonance characteristics is essential in research and clinical practice in individuals with velopharyngeal impairment. The purpose of this study was to systematically review correlations between auditory perceptual ratings and nasalance scores obtained by a nasometer in individuals with resonance disorders and to…

This meta-analytic study investigates the impact of online peer assessment on academic achievement in higher education. By synthesizing 20 effect sizes, we provide a comprehensive understanding of how online peer assessment influences student learning outcomes. The findings reveal a statistically significant positive effect (Hedges's g = 0.672),…

Human raters' assessment of interpreting is a complex process. Previous researchers have mainly relied on verbal reports to examine this process. To advance our understanding, we conducted an empirical study, collecting raters' eye-movement and retrospection data in a computerised interpreting assessment in which three groups of raters (n = 35)…

While skin color is relevant and important in the Latinx community, as it is associated with colorism, little is known about how often it is measured or the best way to measure it. This article presents results from two studies examining these key concerns in three prominent journals, where Latinx research is typically published (i.e., the…

There has recently been a surge of interest in improving the replicability of second language (L2) research. However, less attention is paid to replicability in the context of L2 meta-analyses. I argue that conducting interrater reliability (IRR) analyses is a key step toward improving the replicability of L2 meta-analyses. To that end, I first…

Systematic reviews of single-case experimental research (SCER) in special education often use the What Works Clearinghouse (WWC) Standards to assess the methodological rigor of studies within a given literature base. While significant changes were made between the two most recent versions of the WWC standards, no research to date has evaluated the…

Child assessment practices have undergone, and are continuing to undergo, significant changes. Among the most prominent changes is the movement toward measuring child well-being, in general, and emotional and behavioral strengths, in particular. The Behavioral and Emotional Rating Scale (BERS) is a strength-based instrument which is widely used in…

Missing data incidents are common in experimental studies of motor learning and development. Inadequate handling of missing data may lead to serious problems, such as addition of bias, reduction in power, and so on. Thus, this study aimed to conduct a systematic review of the past (2007) and present (2017) practices used for reporting and…

Hierarchical linear modeling (HLM) has been recommended as a meta-analytic technique for the quantitative synthesis of single-case experimental design (SCED) studies. The HLM approach is flexible and can model a variety of different SCED data complexities, such as intervention heterogeneity. A major advantage of using HLM is that participant…