School leadership research literature has a large number of widely used constructs. Could fewer constructs bring more clarity? This study evaluates construct content validity, defined as the extent to which a measure's items reflect a theoretical content domain, in school leadership literature. To do so, we reviewed 29 articles that used Teaching…

Introduction: The What Works Clearinghouse (WWC) reviews rigorous research on educational interventions with a goal of identifying "what works" and making that information accessible to educators and policymakers. In rating the quality of causal research, the WWC has historically prioritized internal validity over external validity. One…

Outside of education, many academic disciplines developed and promulgated checklists to evaluate the quality of qualitative studies. This article used an embedded mixed methods approach in three stages: development of a conceptual framework reviewing 62 guidelines; a content analysis of key areas of the guidelines; and review of dissertations to…

In this article, we examine whether cross-national studies disclose enough information for independent researchers to evaluate the validity and reliability of the findings (evaluation transparency) or to perform a direct replication (replicability transparency). The first contribution is theoretical. We develop a heuristic theoretical model…

Research has increasingly focused on the intersection between gender diversity and autism. Understanding the foci, methods, and rigor of recent studies could help guide and maximize impact in this accelerating research area. We conducted a scoping review of peer-reviewed literature on this topic published between 2018 and 2023. The search…

Comparison studies conducted to determine which instructional interventions are most efficient for teaching discrete behaviors to individuals with disabilities are potentially valuable, although some threats to internal validity may be more likely in these studies. Studies included in this review typically met common internal validity standards,…

While there has been a lot of debate over the impact of online and remote learning on mental health and well-being, there has been no systematic syntheses or reviews of the research on this particular issue. In this paper, we review the research on the relationship between mental health/well-being and online or remote learning. Our review shows…

This report presents a review of the statistical practices of 30 journals representative of the second language field. A review of 150 articles showed a number of prevalent statistical violations including incomplete reporting of reliability, validity, non-significant results, effect sizes, and assumption checks as well as making inferences from…

This research study aims to evaluate the graduate theses and articles conducted on the concept of instructional leadership over the period between 2002 and 2017 in Turkey by means of methodological and statistical analysis techniques. In the study, which is configured by using the case study design as one of the qualitative research methods, the…

Student self-assessment is a central component of current conceptions of formative and classroom assessment. The research on self-assessment has focused on its efficacy in promoting both academic achievement and self-regulated learning, with little concern for issues of validity. Because reliability of testing is considered a sine qua non for the…

Bias poses a significant threat to the validity of research findings as well as practices and policies based on research. Reforms to academic publishing have been proposed and implemented in other fields to address bias in research. In this paper I review some sources of bias in research, some proposed areas of reform in academic publishing (e.g.,…

School choice programs continue to be controversial, spurring a number of researchers into evaluating them. When possible, researchers evaluate the effect of attending a school of choice using randomized designs to eliminate possible selection bias. Randomized designs are often thought of as the gold standard for research, but many circumstances…

The growing importance of genomics and bioinformatics methods and paradigms in biology has been accompanied by an explosion of new curricula and pedagogies. An important question to ask about these educational innovations is whether they are having a meaningful impact on students' knowledge, attitudes, or skills. Although assessments are…

Purpose: This policy brief, the third in the AERI-NEPC eBrief series "Understanding validity issues around the world", discusses validity issues surrounding International Large Scale Assessment (ILSA) programs. ILSA programs, such as the well-known Programme of International Student Assessment (PISA) and the Trends in International…

In experimental research, internal validity refers to what extent researchers can conclude that changes in dependent variable (i.e. outcome) are caused by manipulations in independent variable. The causal inference permits researchers to meaningfully interpret research results. This article discusses (a) internal validity threats in social and…