Inter-rater reliability (IRR), which is a prerequisite of high-quality ratings and assessments, may be affected by contextual variables, such as the rater's or ratee's gender, major, or experience. Identification of such heterogeneity sources in IRR is important for the implementation of policies with the potential to decrease measurement error…

It was aimed to reveal the informal statistical inferences of primary school students on the concept of variability. A phenomenographic research approach was adopted. A task was adapted and used as a data collection tool. Clinical interviews were conducted with 20 primary school students from each grade level. Data were analyzed within the context…

Machine learning (ML) methods for causal inference have gained popularity due to their flexibility to predict the outcome model and the propensity score. In this article, we provide a within-group approach for ML-based causal inference methods in order to robustly estimate average treatment effects in multilevel studies when there is cluster-level…

One part of COVID-19's staggering impact on education has been to suspend or fundamentally alter ongoing education research projects. This article addresses how to analyze the simple but fundamental example of a multi-cohort study in which student assessment data for the final cohort are missing because schools were closed, learning was virtual,…

An online authentic learning environment (OnALE) is proposed in this study to facilitate students' learning of inferential statistics in a real-life context. The efficacy of the OnALE, in comparison to the conventional approach relative to the students' performance, was explored. Respondents from the experimental group were purposively selected to…

The Programme for International Student Assessment (PISA) introduced creative thinking as an innovative domain in 2022. This paper examines the unique methodological issues in international assessments and the implications of measuring creative thinking within PISA's framework, including stratified sampling, rotated form designs, and a distinct…

There exists a variety of interval estimators for the overall treatment effect in a random-effects meta-analysis. A recent literature review summarizing existing methods suggested that in most situations, the Hartung-Knapp/Sidik-Jonkman (HKSJ) method was preferable. However, a quantitative comparison of those methods in a common simulation study…

Conventional estimators for indirect effects using a difference in coefficients and product of coefficients produce the same results for continuous outcomes. However, for binary outcomes, the difference in coefficient estimator systematically underestimates the indirect effects because of a scaling problem. One solution is to standardize…

If cues from different sensory modalities share the same cause, their information can be integrated to improve perceptual precision. While it is well established that adults exploit sensory redundancy by integrating cues in a Bayes optimal fashion, whether children under 8 years of age combine sensory information in a similar fashion is still…

Since the fall semester of 2016, first-time international student enrollment (ISE[subscript ft]) has declined at U.S. colleges and universities. This trend disrupts a steady upwards trajectory of ISE[subscript ft] rates. Previous research has demonstrated that various political, social, and macroeconomic factors influence the number of…

As a foundational approach in inferential statistics, hypothesis testing (HT) is considered as one of the most challenging topics for teaching and learning. A promising approach is through the consideration of students' learning modalities, as demonstrated in vast applications; however, contentions that surround the use of learning modality in…

Background: Over the past decade, research on the development and assessment of tools to improve the generalizability of experimental findings has grown extensively (Tipton & Olsen, 2018). However, many experimental studies in education are based on small samples, which may include 30-70 schools while inference populations to which…

When working with multimodal Bayesian posterior distributions, Markov chain Monte Carlo (MCMC) algorithms have difficulty moving between modes, and default variational or mode-based approximate inferences will understate posterior uncertainty. And, even if the most important modes can be found, it is difficult to evaluate their relative weights in…

Sir Ronald Fisher's venerable experiment "The Lady Tasting Tea" is revisited from a Bayesian perspective. We demonstrate how a similar tasting experiment, conducted in a classroom setting, can familiarize students with several key concepts of Bayesian inference, such as the prior distribution, the posterior distribution, the Bayes…

Empirical studies often demonstrate multiple causal mechanisms potentially involving simultaneous or causally related mediators. However, researchers often use simple mediation models to understand the processes because they do not or cannot measure other theoretically relevant mediators. In such cases, another potentially relevant but unobserved…