Despite the sensitivity of fit indices to various model and data characteristics in structural equation modeling, these fit indices are used in a rigid binary fashion as a mere rule of thumb threshold value in a search for model adequacy. Here, we address the behavior and interpretation of the popular Comparative Fit Index (CFI) by stressing that…

A class of effect size indices are discussed that evaluate the degree to which two nested confirmatory factor analysis models differ from each other in terms of fit to a set of observed variables. These descriptive effect measures can be used to quantify the impact of parameter restrictions imposed in an initially considered model and are free…

Over the last twenty years, education researchers have increasingly conducted randomised experiments with the goal of informing the decisions of educators and policymakers. Such experiments have generally employed broad, consequential, standardised outcome measures in the hope that this would allow decisionmakers to compare effectiveness of…

Students often believe that statistical significance is the only determinant of whether a quantitative result is "important." In this paper, I review traditional null hypothesis statistical testing to identify what questions inferential statistics can and cannot answer, including statistical significance, effect size and direction,…

Researchers commonly interpret effect sizes by applying benchmarks proposed by Jacob Cohen over a half century ago. However, effects that are small by Cohen's standards are large relative to the impacts of most field-based interventions. These benchmarks also fail to consider important differences in study features, program costs, and scalability.…

Active learning research emerged from the undergraduate STEM education communities of practice, some of whom identify as discipline-based education researchers (DBER). Consequently, current frameworks of active learning are largely inductive and based on emergent patterns observed in undergraduate teaching and learning. Alternatively, classic…

Education experiments frequently assign students to treatment or control conditions within schools. Longitudinal components added in these studies (e.g., students followed over time) allow researchers to assess treatment effects in average rates of change (e.g., linear or quadratic). We provide methods for a priori power analysis in three-level…

This commentary begins by summarizing the five contributions to this special issue and briefly recapping the background to the topic of student learning in higher education. Narrative and systematic reviews are compared, and the relative value of different bibliographic databases in the context of systematic reviews is assessed. The importance of…

There is currently little consensus on how special education teachers should be evaluated in a way that is effective, fair and responsive to their unique teaching responsibilities. In this paper, we explain several of the current approaches to teacher evaluation under consideration, and then provide an overview of the challenges associated with…

The statistical analysis of mediation effects has become an indispensable tool for helping scientists investigate processes thought to be causal. Yet, in spite of many recent advances in the estimation and testing of mediation effects, little attention has been given to methods for communicating effect size and the practical importance of those…

This report summarizes the complex research literature on quantitative methods for assessing how impacts of educational interventions on instructional practices and student learning differ across students, educators, and schools. It also provides technical guidance about the use and interpretation of these methods. The research topics addressed…

Quantitative meta-analysis is a very useful, yet underutilized, technique for synthesizing research findings in higher education. Meta-analytic inquiry can be more challenging in higher education than in other fields of study as a result of (a) concerns about the use of regression coefficients as a metric for comparing the magnitude of effects…

In this article, a 2-way multigroup common factor model (MG-CFM) is presented. The MG-CFM can be used to estimate interaction effects between 2 grouping variables on 1 or more hypothesized latent variables. For testing the significance of such interactions, a likelihood ratio test is presented. In a simulation study, the robustness of the…

This article focuses on a method of policy analysis that has evolved from the previous articles in this issue. The first section, "Toward a Theory of Educational Production," identifies concepts from science and achievement production to be incorporated into this policy analysis method. Building on Kuhn's (1970) discussion regarding paradigms, the…

Data from single case designs (SCDs) have traditionally been analyzed by visual inspection rather than statistical models. As a consequence, effect sizes have been of little interest. Lately, some effect-size estimators have been proposed, but most are either (i) nonparametric, and/or (ii) based on an analogy incompatible with effect sizes from…