The distribution of the standardized mean difference is well understood. However, in many situations, researchers need to estimate an effect size to represent the relationship between a continuous outcome and a dichotomous grouping variable, adjusting for the effect of a covariate (or a set of covariates). Typically, this adjustment takes place…

Experiments in psychology or education often use logistic regression models (LRMs) when analyzing binary outcomes. However, a challenge with LRMs is that results are generally difficult to understand. We present alternatives to LRMs in the analysis of experiments and discuss the linear probability model, the log-binomial model, and the modified…

Many generalization studies in education are typically based on a sample of 30-70 schools while the inference population is at least twenty times larger. This small sample to population size ratio limits the precision of design-based estimators of the population average treatment effect. Prior work has shown the potential of small area estimation…

Using data from multiple informants has long been considered best practice in education. However, multiple informants often disagree on similar constructs, complicating decision-making. Polynomial regression and response-surface analysis (PRA) is often used to test the congruence effect between multiple informants on an outcome. However, PRA…

This study explores the performance of classical methods for detecting publication bias--namely, Egger's regression test, Funnel Plot test, Begg's Rank Correlation and Trim and Fill method--in meta-analysis of studies that report multiple effects. Publication bias, outcome reporting bias, and a combination of these were generated. Egger's…

The standard regression technique for modeling binary response variables in education research is logistic regression. The odds ratios from these models are used to quantify and communicate variable effects. These effects are sometimes pooled together as in a meta-analysis. We argue that this process is problematic as odds ratios calculated from…

Classification measures and entropy variants can be used as indicators of model fit for logistic regression. These measures rely on a cut-point, "c," to determine predicted group membership. While recommendations exist for determining the location of the cut-point, these methods are primarily anecdotal. The current study used Monte Carlo…

Researchers are often reluctant to rely on classification rates because a model with favorable classification rates but poor separation may not replicate well. In comparison, entropy captures information about borderline cases unlikely to generalize to the population. In logistic regression, the correctness of predicted group membership is known,…

Multilevel Rasch models are increasingly used to estimate the relationships between test scores and student and school factors. Response data were generated to follow one-, two-, and three-parameter logistic (1PL, 2PL, 3PL) models, but the Rasch model was used to estimate the latent regression parameters. When the response functions followed 2PL…

Propensity score (PS) adjustments have become popular methods used to improve estimates of treatment effects in quasi-experiments. Although researchers continue to develop PS methods, other procedures can also be effective in reducing selection bias. One of these uses clustering to create balanced groups. However, the success of this new method…

Randomized controlled trials are not always feasible in educational research, so researchers must use alternative methods to study treatment effects. Propensity score matching is one such method for observational studies that has shown considerable growth in popularity since it was first introduced in the early 1980s. This paper outlines the…

Some IRT models can be equivalently modeled in alternative frameworks such as logistic regression. Logistic regression can also model time-to-event data, which concerns the probability of an event occurring over time. Using the relation between time-to-event models and logistic regression and the relation between logistic regression and IRT, this…

The purpose of this article is to develop a statistical model that best explains variability in the number of school days suspended. Number of school days suspended is a count variable that may be zero-inflated and overdispersed relative to a Poisson model. Four models were examined: Poisson, negative binomial, Poisson hurdle, and negative…

Studies analyzing clustered data sets using both multilevel models (MLMs) and ordinary least squares (OLS) regression have generally concluded that resulting point estimates, but not the standard errors, are comparable with each other. However, the accuracy of the estimates of OLS models is important to consider, as several alternative techniques…

An important concern when planning research studies is to obtain maximum precision of an estimate of a treatment effect given a budget constraint. When research designs have a "multilevel" or "hierarchical" structure changes in sample size at different levels of the design will impact precision differently. Furthermore, there…