Methods to assess the significance of mediated effects in education and the social sciences are well studied and fall into two categories: single sample methods and computer-intensive methods. A popular single sample method to detect the significance of the mediated effect is the test of joint significance, and a popular computer-intensive method…

Although widely used, student evaluations of teaching do not address several factors that should be considered in evaluating teaching performance such as new course preparations, teaching larger classes, and inconvenient class times. Consequently, the incentive exists to avoid certain teaching assignments to achieve high SET scores while…

Multisite trials, which are being used with increasing frequency in education and evaluation research, provide an exciting opportunity for learning about how the effects of interventions or programs are distributed across sites. In particular, these studies can produce rigorous estimates of a cross-site mean effect of program assignment…

The use of statistics in second language acquisition (SLA) research has increased over the past 30-40 years and continues to increase in both complexity and sophistication (Gass, 2009; Loewen & Gass, 2009). The increased use of statistical procedures has drawn attention to the current state of statistical literacy among second language (L2)…

Cluster randomized trials involving participants nested within intact treatment and control groups are commonly performed in various educational, psychological, and biomedical studies. However, recruiting and retaining intact groups present various practical, financial, and logistical challenges to evaluators and often, cluster randomized trials…

Recently, MDS has been utilized to identify and evaluate cognitive ability latent profiles in a population. However, dimension coordinates do not carry any statistical properties. To cope with statistical incompetence of MDS, we investigated the common aspects of various studies utilizing bootstrapping, and provided an R function for its…

One of the highest ambitions in educational technology is the move towards personalized learning. To this end, computerized adaptive learning (CAL) systems are developed. A popular method to track the development of student ability and item difficulty, in CAL systems, is the Elo Rating System (ERS). The ERS allows for dynamic model parameters by…

Since the seminal introduction of the propensity score (PS) by Rosenbaum and Rubin, PS-based methods have been widely used for drawing causal inferences in the behavioral and social sciences. However, the PS approach depends on the ignorability assumption: there are no unobserved confounders once observed covariates are taken into account. For…

Analysis of variance (ANOVA) is a test of "mean" differences, but the reference to "variances" in the name is often overlooked. Classroom activities are presented to illustrate how ANOVA works with emphasis on how to think critically about inferential reasoning.

Purpose: This paper aims to introduce matching in propensity score analysis (PSA) as an alternative statistical approach for researchers looking to make causal inferences using intact groups. Design/methodology/approach: An illustrative example demonstrated the varying results of analysis of variance, analysis of covariance and PSA on a heuristic…

Single case studies can provide vital contributions to theory-testing in social science studies. Particularly, by applying the process-tracing method, case studies can test theoretical frameworks through a rigorous research design that ensures substantial empirical leverage. While most scholarly contributions on process-tracing focus on either…

This thesis examined middle school students' current understanding of variability using a constructed response item assessment question. Variability is an essential concept in the teaching and learning of statistics. However, many students have difficulty with the concept of variability especially when constructing boxplots. Using a framework…

Applied researchers are increasingly interested in whether and how treatment effects vary in randomized evaluations, especially variation not explained by observed covariates. We propose a model-free approach for testing for the presence of such unexplained variation. To use this randomization-based approach, we must address the fact that the…

We describe how students can be encouraged to actively review course contents on inferential statistics by creating application-oriented exercises and sample solutions on the basis of concrete and realistic research articles and their data. For evaluation purposes, we use students' reactions to the activity and investigate its effects on the final…

To identify the ways teachers and educational systems can improve learning, researchers need to make causal inferences. Analyses of existing datasets play an important role in detecting causal patterns, but conducting experiments also plays an indispensable role in this research. In this article, we advocate for experiments to be embedded in real…