Most existing solutions to the current replication crisis in science address only the factors stemming from specific poor research practices. We introduce a novel mechanism that leverages the experts' predictive abilities to analyze the root causes of replication failures. It is backed by the principle that the most accurate predictor is the most…

The purpose of this article is to illustrate the dynamic process involved in developing and utilizing a theoretical model in a mixed methods study. Specifically, I illustrate how the theoretical model can serve as the starting point in framing the study, as a lens for guiding the data collection and analysis, and as the end point in explaining the…

The product Bayes factor (PBF) synthesizes evidence for an informative hypothesis across heterogeneous replication studies. It can be used when fixed- or random effects meta-analysis fall short. For example, when effect sizes are incomparable and cannot be pooled, or when studies diverge significantly in the populations, study designs, and…

Purpose: This article provides a tutorial introduction to ordinal pattern analysis, a statistical analysis method designed to quantify the extent to which hypotheses of relative change across experimental conditions match observed data at the level of individuals. This method may be a useful addition to familiar parametric statistical methods…

In light of the replication crisis in psychology, null-hypothesis significance testing (NHST) and "p"-values have been heavily criticized and various alternatives have been proposed, ranging from slight modifications of the current paradigm to banning "p"-values from journals. Since the physics education research community…

Purpose: The purpose of this work is to illustrate the processes involved in managing teams in order to assist designers and developers to build software that support teamwork. A deeper investigation into the role of team analytics is discussed in this article. Design/methodology/approach: Many researchers over the past several decades studied the…

Research studies in Educational Data Mining (EDM) often involve several variables related to student learning activities. As such, it may be necessary to run multiple statistical tests simultaneously, thereby leading to the problem of multiple comparisons. The Benjamini-Hochberg (BH) procedure is commonly used in EDM research to address this…

Data reasoning is an essential component of scientific reasoning, as a component of evidence evaluation. In this paper, we outline a model of scientific data reasoning that describes how data sensemaking underlies data reasoning. Data sensemaking, a relatively automatic process rooted in perceptual mechanisms that summarize large quantities of…

Explanatory item response models (EIRMs) have been applied to investigate the effects of person covariates, item covariates, and their interactions in the fields of reading education and psycholinguistics. In practice, it is often assumed that the relationships between the covariates and the logit transformation of item response probability are…

Explanatory item response models (EIRMs) have been applied to investigate the effects of person covariates, item covariates, and their interactions in the fields of reading education and psycholinguistics. In practice, it is often assumed that the relationships between the covariates and the logit transformation of item response probability are…

Evidence-based assessment (EBA) requires that investigators employ scientific theories and research findings to guide decisions about what domains to measure, how and when to measure them, and how to make decisions and interpret results. To implement EBA, investigators need high-quality assessment tools along with evidence-based processes. We…

This case is designed to be used in business analytics courses; particularly those that emphasize predictive analytics. Students are given background information on money laundering and data from People's United Bank, a regional bank in the northeast United States. The students must develop their hypothesis, analyze the data, develop and optimize…

Background: Scientific thinking is an essential learning goal of science education and it can be fostered by inquiry learning. One important prerequisite for scientific thinking is procedural understanding. Procedural understanding is the knowledge about specific steps in scientific inquiry (e.g. formulating hypotheses, measuring dependent and…

There has been an ongoing debate of which physical labs or virtual labs are better. To resolve this issue, a remote lab provides an online lab that can do real experiments to obtain real data from a distant physical lab. Instead of relying on a remote lab, this article suggests that students collect experimental data locally with low-cost data…

This article introduces ethnographic upscaling, an innovative procedure to explore and test hypotheses drawn from in-depth ethnographic findings in spatially continuous cases. The approach combines the strength of localized ethnographic descriptions with questionnaire-based regional surveys to study the distribution of ethnographic findings across…