Pedagogical content knowledge (PCK) traditionally has been used in research concerning prospective and practicing classroom teachers. This essay argues that PCK is also relevant to other professions including those advancing quantitative reasoning (QR). To illustrate, the case of PCK for teaching QR is considered. Those in fields such as public…

BASIE is a framework for interpreting impact estimates from evaluations. It is an alternative to null hypothesis significance testing. This guide walks researchers through the key steps of applying BASIE, including selecting prior evidence, reporting impact estimates, interpreting impact estimates, and conducting sensitivity analyses. The guide…

Propensity and prognostic score methods are two statistical techniques used to correct for the selection bias in nonexperimental studies. Recently, the joint use of propensity and prognostic scores (i.e., two-score methods) has been proposed to improve the performance of adjustments using propensity or prognostic scores alone for bias reduction.…

Despite the importance of replication, it remains rare in the interactive learning research community. In this paper, we attempt to replicate recent quasi-experimental results suggesting that the ALEKS intelligent tutoring system is effective at improving student course outcomes in higher education (Mojarad et al., 2018). In this paper, we conduct…

The purpose of this paper is to provide guidance in choice of analytic bias reduction methods for educational studies in which the goal is to estimate a treatment effect in the presence of selection bias into treatment. In addition, issues of dimensionality, collinearity, omitted confounders, missing outcomes, and non-independence may be factors…

Nonnormality of univariate data has been extensively examined previously (Blanca et al., 2013; Micceri, 1989). However, less is known of the potential nonnormality of multivariate data although multivariate analysis is commonly used in psychological and educational research. Using univariate and multivariate skewness and kurtosis as measures of…

Randomized experiments are considered the gold standard for causal inference because they can provide unbiased estimates of treatment effects for the experimental participants. However, researchers and policymakers are often interested in using a specific experiment to inform decisions about other target populations. In education research,…

Recent methods to improve generalizations from nonrandom samples typically invoke assumptions such as the strong ignorability of sample selection, which is challenging to meet in practice. Although researchers acknowledge the difficulty in meeting this assumption, point estimates are still provided and used without considering alternative…

Recently, Stephen Gorard has outlined strong objections to the use of significance testing in social research. He has argued, first, that as the samples used in social research are almost always non-random it is not possible to use inferential statistical techniques and, second, that even if a truly random sample were achieved, the logic behind…

Analysis of variance is one of the most frequently used statistical analyses in the behavioral, educational, and social sciences, and special attention has been paid to the selection and use of an appropriate effect size measure of association in analysis of variance. This article presents the sample size procedures for precise interval estimation…

Propensity score matching (PSM) has been used to estimate causal effects of treatment, especially in studies where random assignment to treatment is difficult to obtain. The main purpose of this article is to provide some practical guidance for propensity score sample matching, including definitions, procedures, decisions on each step, and methods…

Often it is infeasible or unethical to use random assignment in educational settings to study important constructs and questions. Hence, educational research often uses observational data, such as large-scale secondary data sets and state and school district data, and quasi-experimental designs. One method of reducing selection bias in estimations…

This article reviews all (quasi)experimental studies appearing in the first 19 volumes (1997-2015) of "Language Teaching Research" (LTR). Specifically, it provides an overview of how statistical analyses were conducted in these studies and of how the analyses were reported. The overall conclusion is that there has been a tight adherence…

Evaluations of vocational education and training (VET) programs play a key role in informing training policy in Australia and elsewhere. Increasingly, such evaluations use observational data from surveys or administrative collections to assess the effectiveness of VET programs and interventions. The difficulty associated with using observational…

The goal of this research is to make sensitivity analysis accessible not only to empirical researchers but also to the various stakeholders for whom educational evaluations are conducted. To do this it derives anchors for the omitted variable (OV)-program participation association intrinsically, using the Love plot to present a wide range of…