In studies of educational achievement, students' self-reported number of books in the family home is a frequently used proxy for social, cultural, and economic background. Absent hard evidence about what this variable captures or how well, its use has been motivated by strong associations with student outcomes. I show that these associations rest…

Selection bias is a persistent, and often hidden, problem in educational research. It is the primary obstacle standing in between increasingly available large education datasets and the ability to make valid causal inferences to inform policymaking, research, and practice (Stuart, 2010). This article provides an accessible discussion on the…

Educational researchers commonly use the rule of thumb of "design effect smaller than 2" as the justification of not accounting for the multilevel or clustered structure in their data. The rule, however, has not yet been systematically studied in previous research. In the present study, we generated data from three different models…

Nearly all quantitative analyses in higher education draw from incomplete datasets-a common problem with no universal solution. In the first part of this paper, we explain why missing data matter and outline the advantages and disadvantages of six common methods for handling missing data. Next, we analyze real-world data from 5,905 students across…

Organizational and applied sciences have long struggled with improving causal inference in quasi-experiments. We introduce organizational researchers to propensity scoring, a statistical technique that has become popular in other applied sciences as a means for improving internal validity. Propensity scoring statistically models how individuals in…

This methodological note illustrates how a commonly used calculation of the Delta-p statistic is inappropriate for categorical independent variables, and this note provides users of logistic regression with a revised calculation of the Delta-p statistic that is more meaningful when studying the differences in the predicted probability of an…

United States Department of Agriculture defines food insecure as answering affirmatively to three or more food insecurity questions describing a household's ability to acquire enough food. Households indicating low levels of food insecurity (one or two affirmative responses) are considered food secure. This paper compares the characteristics of…

The increased use of multiple regression analysis in research warrants closer examination of the coefficients produced in these analyses, especially ones which are often ignored, such as structure coefficients. Structure coefficients are bivariate correlation coefficients between a predictor variable and the synthetic variable. When predictor…

Ridge regression offers advantages over ordinary least squares estimation when a validity shrinkage criterion is considered. Comparisons of cross-validated multiple correlations indicate that ridge estimation is superior when the predictors are multicollinear, the number of predictors is large relative to sample size, and the population multiple…

It is generally believed that the correction formula will yield exact correlational values only when the regression of z on x is both linear and homoscedastic. The formula is shown to hold for nonlinear heteroscedastic relationships. A simple sufficient condition for formula validity and estimation predictions is demonstrated in a numerical…

Researchers sometimes use stepwise methods to eliminate variables from analyses when the variables do not appreciably improve the ability to explain or predict inferences about the importance of various predictor variables. It is argued that stepwise methods are usually not appropriate for these purposes for three reasons. First, some researchers…

This paper examines some of the problems that arise when conducting multivariate analyses with incomplete data. The literature on the effectiveness of several missing data procedures (MDP) is summarized. The most widely used MDPs are: (1) listwise deletion; (2) pairwise deletion; (3) variable mean; (4) correlational methods. No MDP should be used…

L. A. Marascuilo and R. C. Serlin (1988) note that stepwise regression is a method used frequently in social science research. C. Huberty (1989) characterizes such applications as being "common". In support of this latter statement, a review of dissertations by B. Thompson (1988) demonstrated that dissertation students frequently use…

There are times in survey research when missing values need to be estimated. The robustness of four variations of regression and substitution by mean methods was examined using a 3x3x4 factorial design. The regression variations included in the study were: (1) regression using a single best predictor; (2) two best predictors; (3) all available…

The serious problems associated with the use of stepwise methods are well documented. Various authors have leveled scathing criticisms against the use of stepwise techniques, yet it is not uncommon to find these methods continually employed in educational and psychological research. The three main problems with stepwise techniques are: (1)…