Standard meta-analysis methods are vulnerable to bias from incomplete reporting of results (both publication and outcome reporting bias) and poor study quality. Several alternative methods have been proposed as being less vulnerable to such biases. To evaluate these claims independently we simulated study results under a broad range of conditions…

Doubly robust (DR) estimators that combine regression adjustments and inverse probability weighting (IPW) are widely used in causal inference with observational data because they are claimed to be consistent when either the outcome or the treatment selection model is correctly specified (Scharfstein et al., 1999). This property of "double…

The standard regression technique for modeling binary response variables in education research is logistic regression. The odds ratios from these models are used to quantify and communicate variable effects. These effects are sometimes pooled together as in a meta-analysis. We argue that this process is problematic as odds ratios calculated from…

There are many well-known data sets that can be used to illustrate Simpson's Paradox. The Stand Your Ground data presented here shows Simpson's Paradox. In these data, race plays the key role--and not in the way that some students expect.

Organizational and applied sciences have long struggled with improving causal inference in quasi-experiments. We introduce organizational researchers to propensity scoring, a statistical technique that has become popular in other applied sciences as a means for improving internal validity. Propensity scoring statistically models how individuals in…

Because of the Flynn effect, IQ scores rise as a test norm ages but drop on the introduction of a newly revised test norm. The purpose of the current study was to determine the impact of the Flynn effect on learning disability (LD) diagnoses, the most prevalent special education diagnosis in the United States. Using a longitudinal sample of 875…

This methodological note illustrates how a commonly used calculation of the Delta-p statistic is inappropriate for categorical independent variables, and this note provides users of logistic regression with a revised calculation of the Delta-p statistic that is more meaningful when studying the differences in the predicted probability of an…

Asserts that traditional approaches to multivariate analysis, including standard linear regression techniques, ignore the special character of count data. Explicates three suitable alternatives to standard regression techniques, a simple Poisson regression, a modified Poisson regression, and a negative binomial model. (MJP)

This monograph covers the theory, application, and interpretation of both descriptive and inferential statistical techniques in institutional research. Each chapter opens with a hypothetical case study, which is used to illustrate the application of one or more statistical procedures to typical research questions. Chapter 2 covers the comparison…