Recent critiques of physics education research (PER) studies have revoiced the critical issues when drawing causal inferences from observational data where no intervention is present. In response to a call for a "causal reasoning primer" in PER, this paper discusses some of the fundamental issues in statistical causal inference. In…

Multilevel structural equation models (MSEM) are typically evaluated on the basis of goodness of fit indices. A problem with these indices is that they pertain to the entire model, reflecting simultaneously the degree of fit for all levels in the model. Consequently, in cases that lack model fit, it is unclear which level model is misspecified.…

Analogical reasoning is a core facet of higher cognition in humans. Creating analogies as we navigate the environment helps us learn. Analogies involve reframing novel encounters using knowledge of familiar, relationally similar contexts stored in memory. When an analogy links a novel encounter with a familiar context, it can aid in problem…

Sound educational policy recommendations require valid estimates of causal effects, but observational studies in physics education research sometimes have loosely specified causal hypotheses. The connections between the observational data and the explicit or implicit causal conclusions are sometimes misstated. The link between the causal…

Applying qualitative comparative analysis (QCA) to large Ns relaxes researchers' case-based knowledge. This is problematic because causality in QCA is inferred from a dialogue between empirical, theoretical, and case-based knowledge. The lack of case-based knowledge may be remedied by various robustness tests. However, being a case-based method,…

Basic knowledge of ideas of causal inference can help students to think beyond data, that is, to think more clearly about the data generating process. Especially for (maybe big) observational data, qualitative assumptions are important for the conclusions drawn and interpretation of the quantitative results. Concepts of causal inference can also…

A common motivation for adding control variables to statistical models is to reduce the potential for spurious findings when analyzing non-experimental data and to thereby allow for more reliable causal inferences. However, as I show here, unless "all" potential confounding factors are included in an analysis (which is unlikely to be…

Confidence intervals for effect sizes (CIES) provide readers with an estimate of the strength of a reported statistic as well as the relative precision of the point estimate. These statistics offer more information and context than null hypothesis statistic testing. Although confidence intervals have been recommended by scholars for many years,…

Advances in validity theory and alacrity in validation practice have suffered because the term "validity" has been used to refer to two incompatible concerns: (1) the degree of support for specified interpretations of test scores (i.e. intended score meaning) and (2) the degree of support for specified applications (i.e. intended test…

This manuscript describes an approach to critical qualitative data analysis that combines (1) Carspecken's critical qualitative methodological framework with (2) the conceptual resources of critical discourse analysis (CDA), as framed by Fairclough and colleagues. Carspecken's methodological theory illuminates the connection between sociopolitical…

Business analytics courses, such as marketing research, data mining, forecasting, and advanced financial modeling, have substantial predictive modeling components. The predictive modeling in these courses requires students to estimate and test many linear regressions. As a result, false positive variable selection ("type I errors") is…

This paper briefly discusses four measurement challenges of data science or "big data" in educational assessments that are enabled by technology: 1. Dealing with change over time via time-based data. 2. How a digital performance space's relationships interact with learner actions, communications and products. 3. How layers of…

This Methodological Brief introduces the reader to the regression discontinuity design (RDD), which is a method that when used correctly can yield estimates of research treatment effects that are equivalent to those obtained through randomized control trials and can therefore be used to infer causality. However, RDD does not require the random…

The area of statistics is one in which teachers may be encouraged to make important links to other curriculum areas and social issues. Statistical literacy is a key component of being numerate and living as an informed citizen. The teaching of statistics provides an opportunity to inform and educate students about social issues and moral…

Many students graduate from K-12 mathematics programs without flexible, conceptual mathematics knowledge. This article reviews psychological and educational research to propose that refining K-12 classroom instruction such that students draw connections through relational comparisons may enhance their long-term ability to transfer and engage with…