The quality of exams drives test-taking behavior of examinees and is a proxy for the quality of teaching. As most university exams have strict time limits, and speededness is an important measure of the cognitive state of examinees, this might be used to assess the connection between exams' quality and examinees' performance. The practice of…

The Bayesian information criterion (BIC) can be useful for model selection within multilevel modeling studies. However, the formula for BIC requires a value for N, which is unclear in multilevel models, since N is observed in at least two levels. The present study uses simulated data to evaluate the rate of false positives and power when using a…

There have been increasing calls for applied researchers to see and utilize effect sizes as the primary outcomes of their research. However, this sometimes places a methodological burden on researchers whose primary interests are substantive. Motivated by a desire to help applied researchers better report effect sizes and their confidence…

This paper drills deeper into the documented effects of the Cognitive Tutor Algebra I and ASSISTments intelligent tutoring systems by estimating their effects on specific problems. We start by describing a multilevel Rasch-type model that facilitates testing for differences in the effects between problems and precise problem-specific effect…

Research about the development of the graphomotor side of writing is very scarce. The goal of this study was to gain a better understanding of what constitutes graphic complexity of written material by determining the impact of graphic characteristics on handwriting production. In this end, the pen stroke of cursive handwriting was precisely…

Recent studies show cluster randomized trials may be well powered to detect mediation or indirect effects in multilevel settings. However, literature has rarely provided guidance on designing cluster-randomized trials aim to assess indirect effects. In this study, we developed closed-form expression to estimate the variance of and the statistical…

Multiple-baseline studies provide meta-analysts the opportunity to compute effect sizes based on either within-series comparisons of treatment phase to baseline phase observations, or time specific between-series comparisons of observations from those that have started treatment to observations of those that are still in baseline. The advantage of…

We compared students' performance on a paper-based test (PBT) and three computer-based tests (CBTs). The three computer-based tests used different test navigation and answer selection features, allowing us to examine how these features affect student performance. The study sample consisted of 9,698 fourth through twelfth grade students from across…

Bayesian Knowledge Tracing (BKT) models were in active use in the Intelligent Tutoring Systems (ITS) field for over 20 years. They have been intensively studied, and a number of useful extensions to them were proposed and experimentally tested. Among the most widely researched extensions to BKT models are various types of individualization.…

Mediation in multi-level data can be examined using conflated multilevel modeling (CMM), unconflated multilevel modeling (UMM), or multilevel structural equation modeling (MSEM). A Monte Carlo study was performed to compare the three methods on bias, type I error, and power in a 1-1-1 model with random slopes. The three methods showed no…

Growth modeling has been of interest in many assessment programs, including both highstakes and low-states tests. Growth could be modeled using different approaches. This study models growth with an Item Response Theory (IRT) based approach that utilizes item response data. It investigates the impact of complex student clustering structure where…

Problem-based learning (PBL) is an instructional design approach for promoting student learning, in context-rich settings. GlobalEd 2 (GE2) is PBL intervention that combines face-to-face and online environments into a 12-week simulation of international negotiations of science advisors on global water resource issues. The GE2 environment is…

This original research examined the intersection of school leadership and teacher to parent communication. The analysis used hierarchical linear modeling (HLM) to determine how principal actions and school level factors were related to teacher actions and the focus of communication. Secondary educators in four district school boards were surveyed…

Research into open education has identified a "high number of participants" and "unpredictable mixed abilities" as factors responsible for the relatively weak presence of language Massive Open Online Courses (MOOCs). This contribution presents a model for open online language courses that aims to bridge this gap. The tangible…

Intelligent tutoring systems yield data with many properties that render it potentially ideal to examine using multi-level models (MLM). Repeated observations with dependencies may be optimally examined using MLM because it can account for deviations from normality. This paper examines the applicability of MLM to data from the intelligent tutoring…