The identification of differential item functioning (DIF) is often performed by means of statistical approaches that consider the raw scores as proxies for the ability trait level. One of the most popular approaches, the Mantel-Haenszel (MH) method, belongs to this category. However, replacing the ability level by the simple raw score is a source…

Field experiments with nested structures are becoming increasingly common, especially designs that assign randomly entire clusters such as schools to a treatment and a control group. In such large-scale cluster randomized studies the challenge is to obtain sufficient power of the test of the treatment effect. The objective is to maximize power…

Growth curve models with different types of distributions of random effects and of intraindividual measurement errors for robust analysis are compared. After demonstrating the influence of distribution specification on parameter estimation, 3 methods for diagnosing the distributions for both random effects and intraindividual measurement errors…

The present study explored the long-term impact of admission policies at two aspiring research institutions in Texas. Six years of longitudinal institutional data were analyzed for all full-time first time in college undergraduate students at both universities. Descriptive and inferential statistics were used to identify relationships and…

This paper explores the effect of digital differentiation on student performance using a randomized experiment. The experiment is conducted in a second year biology class among 115 prevocational students in the Netherlands. Differentiation allowed students in the treatment group to work at three different levels. The results show that there is a…

Little is known about the importance of instructional quality in American higher education because few recent studies have had access to direct measures of student learning that are comparable across sections of the same course. Using data from two developmental algebra courses at a large community college, I found that student learning varies…

Public school critics often point to rising expenditures and relatively flat test scores to justify their school reform agendas. The claims are flawed because their analyses fail to account for the difference in data types between dollars (ratio) and test scores (interval). A cost-benefit analysis using dollars as a common metric for both costs…

Outdoor adventure education (OAE) is widely recognised for its ability to elicit personal and social development for its participants. However, quantitative evidence on which this recognition is based is frequently questioned, and is virtually absent in Scotland. To provide some of the first statistically determined evidence from Scotland that OAE…

The root mean square error of approximation (RMSEA) is a popular fit index in structural equation modeling (SEM). Typically, RMSEA is computed using the normal theory maximum likelihood (ML) fit function. Under nonnormality, the uncorrected sample estimate of the ML RMSEA tends to be inflated. Two robust corrections to the sample ML RMSEA have…

Goodness-of-fit testing in factor analysis is based on the assumption that the test statistic is asymptotically chi-square, but this property may not hold in small samples even when the factors and errors are normally distributed in the population. Robust methods such as Browne's (1984) asymptotically distribution-free method and Satorra Bentler's…

It is difficult to assess the extent to which course evaluations reflect how much students truly learn from a course because valid measures of learning are rarely available. This paper makes use of a unique setting in which students take a common, high-stakes post-test which is centrally graded and serves as the basis for capturing actual student…

Science education trends promote student engagement in authentic knowledge in practice to tackle personally consequential problems. This study explored how partnering scientists and students on a social media platform supported students' development of disciplinary practice knowledge through practice-based learning with experts during two pilot…

This paper investigates the effectiveness (measured using assignment and examination performance) of an assessment design incorporating formative feedback through summative tutorial-based assessments to improve student performance, in a second-year Finance course at an Australian university. Data was collected for students who were enrolled in an…

Schools and neighborhoods are thought to be two of the most important contextual influences on student academic outcomes. Drawing on a unique data set that permits simultaneous estimation of neighborhood and school contributions to student test score gains, we analyze the distributions of these contributions to consider the relative importance of…

Although many parametric statistical tests are considered to be robust, as recently shown in Methodologist's Corner, it still pays to be circumspect about the assumptions underlying statistical tests. In this paper I show that robustness mainly refers to "[alpha]", the type-I error. If the underlying distribution of data is ignored there…