Introduction: The What Works Clearinghouse (WWC) reviews rigorous research on educational interventions with a goal of identifying "what works" and making that information accessible to educators and policymakers. In rating the quality of causal research, the WWC has historically prioritized internal validity over external validity. One…

Social validity in the field of applied behaviour analysis is the measurement of the social significance of goals, the social appropriateness of procedures, and the social importance of the effects of a treatment. There is a paucity of rigorous research on social validity measurement as it relates to feeding treatment. To our knowledge, this is…

Single-case comparative studies could help identify efficient instructional procedures for individuals with disabilities. However, previous literature reported inconsistent efficiency results if multiple comparisons were conducted, indicating that within-participant replication was uncommon. In this review, we examined single-case comparative…

Many teacher education researchers have expressed concerns about the lack of rigorous impact evaluations of teacher preparation practices. I summarize these various concerns as they relate to issues of internal validity, measurement, and external validity. I then assess the prevalence of these issues by reviewing 166 impact evaluations of teacher…

Conversational skills, which are essential for effective social interactions and typically pose difficulties for individuals with autism spectrum disorder (ASD), include abilities such as initiating topics, engaging in back-and-forth dialog, and responding to conversational cues. Chatbots have been used in mental health fields, and the development…

Implementation fidelity is often thought of as a necessary condition to achieve internal validity and as having a relation to student outcomes. This relation is increasingly modeled; we reviewed reading intervention studies for students in pre-K-12 in which measures of implementation fidelity were included in final data analysis. A systematic…

Many teacher education researchers have expressed concerns with the lack of rigorous impact evaluations of teacher preparation practices. I summarize these various concerns as they relate to issues of internal validity, external validity, and measurement. I then assess the prevalence of these issues by reviewing 166 impact evaluations of teacher…

High-stakes examinations enjoy widespread use as summative assessments in higher education. We review the arguments for and against their use, across seven common themes: memory recall and knowledge retention; student motivation and learning; authenticity and real-world relevance; validity and reliability; academic misconduct and contract…

This quality review is an analysis of the methodological quality of social communication intervention research identifying which social communication interventions for this population could be considered evidence-based practices (EBPs). The purpose of the quality review was to apply the What Works Clearinghouse (WWC) guidelines to the literature…

Professional standards related to outcomes assessment call for student affairs professionals to use research to inform programming. If professionals are to rely on research to build programs that positively impact student learning outcomes, the research should be credible. We examined the quality of program effectiveness research available for…

Background: Evidence syntheses provide the basis for evidence-based decision making in healthcare. To judge the certainty of findings for the specific decision context evidence syntheses should consider context suitability (ie, generalizability, external validity, applicability or transferability). Our objective was to determine the status quo and…

Brief multiple-choice question workshops are a prevalent part of the faculty development landscape. But do they work? Studies have documented that faculty member-written multiple-choice questions (fMCQs) are frequently flawed and do not live up to quality standards. Poor fMCQs have real consequences for students beyond annoyance. Fourteen studies…

Treatment acceptability is an aspect of social validity that refers to participants' beliefs and perceptions about the intervention, such as the helpfulness of the strategies or the interventions' efficacy to improve performance. The purpose of this study was to conduct a systematic review of treatment acceptability measures administered during…

Tools for evaluating the quality and rigor of single case research designs (SCD) are often used when conducting SCD syntheses. Preferred components include evaluations of design features related to the internal validity of SCD to obtain quality and/or rigor ratings. Three tools for evaluating the quality and rigor of SCD (Council for Exceptional…

This systematic literature review was conducted to explore the social validity of augmented reality (AR), virtual reality (VR), and mixed reality (MR) as a means of providing social skill instruction to students with autism spectrum disorder (ASD). Forty-one articles met the inclusion criteria, including five studies utilizing AR and the remaining…