Many methods are used to measure interrater reliability for studies where each target receives ratings by a different set of judges. The purpose of this study is to explore the use of hierarchical modeling for estimating interrater reliability using the intraclass correlation coefficient. This study provides a description of how the ICC can be…

One of the most serious forms of cheating occurs when examinees have item preknowledge and prior access to secure test material before taking an exam for the purpose of obtaining an inflated test score. Examinees that cheat and have prior knowledge of test content before testing may have an unfair advantage over examinees that do not cheat. Item…

Background: Grade retention is an educational aspect that concerns teachers, families, and experts. It implies an economic cost for families, as well as a personal cost for the student, who is forced to study one more year. The objective of the study was to evaluate the effect of course repetition on math, science and reading competencies, and…

Growing scholarship on the pedagogical applications of statistical modeling is currently taking place to create adaptations of this practice to introduce novices to statistics. These are intended to promote novices' reasoning, and are typically void of formal mathematical procedures and calculations. In this article, we define the potential…

Although item response theory (IRT) models such as the bifactor, two-tier, and between-item-dimensionality IRT models have been devised to confirm complex dimensional structures in educational and psychological data, they can be challenging to use in practice. The reason is that these models are multidimensional IRT (MIRT) models and thus are…

The aim of the study is to examine the mediating role of empathy in the relationship between rational and experiential thinking styles and was designed in a correlational design. The sample of the study consists of 593 university students who were determined by simple random sampling method. Emotional Intelligence Questionnaire--Short Form,…

Difficulties in learning (and thus teaching) statistical inference are well reported in the literature. We argue the problem emanates not only from the way in which statistical inference is taught but also from what exactly is taught as statistical inference. What makes statistical inference difficult to understand is that it contains two logics…

This article presents two alternative methods to null hypothesis significance testing (NHST) for improving inferences from underpowered research designs. Post hoc design analysis (PHDA) assesses whether an NHST analysis generating null findings might otherwise have had sufficient power to detect effects of plausible magnitudes. Bayesian analysis…

Periodic changes in occupational classifications make it difficult to obtain consistent measures of social class over time, potentially jeopardizing research on class-based trends. The severity of this problem depends, in part, on the measurement strategies used to address those changes. The authors propose that when a sample has been coded partly…

The synthetic control method (SCM) is a popular approach for estimating the impact of a treatment on a single unit in panel data settings. The "synthetic control" is a weighted average of control units that balances the treated unit's pre-treatment outcomes and other covariates as closely as possible. A critical feature of the original…

As a contribution to the discussion on the assessment of informal inferential reasoning (IIR) and the transition from this to formal inferential reasoning (FIR), we present a review of research on how these two types of inferential reasoning have been conceptualized and assessed. Based on our review, we discuss the need to redefine the conceptions…

Courses: Research methods for undergraduates or graduates. Objectives: The aims of this activity are: (1) to clarify the basics of experimental design; (2) to illustrate the concept of levels of measurement; (3) to demonstrate in-person/hands-on data collection procedures; (4) to understand and practice the steps in null hypothesis testing; and…

Library and information science (LIS) research is becoming more quantitative. However, statistics is not extensively taught within LIS research methods courses, and statistics courses are uncommon within LIS programs. Previous research on statistics in LIS revealed that researchers have mainly relied on descriptive statistics in publications. This…

Randomized control trials are considered the pinnacle for causal inference. In many cases, however, randomization of participants in social work research studies is not feasible or ethical. This paper introduces the co-twin control design study as an alternative quasi-experimental design to provide evidence of causal mechanisms when randomization…

Randomized A/B tests within online learning platforms enable us to draw unbiased causal estimators. However, precise estimates of treatment effects can be challenging due to minimal participation, resulting in underpowered A/B tests. Recent advancements indicate that leveraging auxiliary information from detailed logs and employing design-based…