In studies of educational achievement, students' self-reported number of books in the family home is a frequently used proxy for social, cultural, and economic background. Absent hard evidence about what this variable captures or how well, its use has been motivated by strong associations with student outcomes. I show that these associations rest…

Selection bias is a persistent, and often hidden, problem in educational research. It is the primary obstacle standing in between increasingly available large education datasets and the ability to make valid causal inferences to inform policymaking, research, and practice (Stuart, 2010). This article provides an accessible discussion on the…

The purpose of this article is to clarify the discrepancy in the use of "queer" as a sexual identity classification in education survey research. This study extends the work completed by Dugan and Yurman (2011), who empirically demonstrated problems with treating LGB students as a homogenous population through collapsing all respondents…

In experimental research, internal validity refers to what extent researchers can conclude that changes in dependent variable (i.e. outcome) are caused by manipulations in independent variable. The causal inference permits researchers to meaningfully interpret research results. This article discusses (a) internal validity threats in social and…

Educational researchers commonly use the rule of thumb of "design effect smaller than 2" as the justification of not accounting for the multilevel or clustered structure in their data. The rule, however, has not yet been systematically studied in previous research. In the present study, we generated data from three different models…

Nearly all quantitative analyses in higher education draw from incomplete datasets-a common problem with no universal solution. In the first part of this paper, we explain why missing data matter and outline the advantages and disadvantages of six common methods for handling missing data. Next, we analyze real-world data from 5,905 students across…

In this chapter, the authors discuss the issue of research design in conducting inquiry on college impact and demonstrate the importance of longitudinal pretest-posttest designs in maximizing the internal validity of findings. They begin by discussing the strengths and weaknesses of different types of research design in the college impact…

This methodological note illustrates how a commonly used calculation of the Delta-p statistic is inappropriate for categorical independent variables, and this note provides users of logistic regression with a revised calculation of the Delta-p statistic that is more meaningful when studying the differences in the predicted probability of an…

Education research rarely lends itself to large scale experimental research and true randomization, leaving the researcher to quasi-experimental designs. The problem with quasi-experimental research is that underlying factors may impact group selection and lead to potentially biased results. One way to minimize the impact of non-randomization is…

The call for practice on the basis of evidence is slowly impacting the link between research and practice. Accompanying the call for evidence-based practice, concerns have been raised about constructing research questions that address the reality of applied settings, as well as arguments for including additional measures to ensure the internal and…

One goal of basic research in education is to identify the variables of effective instruction. As this pursuit has been conceived, however, its theoretical problems make it unlikely that the effort will provide a clear picture of instructional variables, their interactions, or the kind of teacher training that is implied by instructional variables.

Purpose: The use of e-learning is largely predicated upon the assumption that it can facilitate improvements in student learning and therefore can be more effective than conventional techniques. This assumption has been supported by some in the literature but has been questioned by a continuing body of contrary or indifferent evidence. The purpose…

Per definition, CSCL research deals with the data of individuals nested in groups, and the influence of a specific learning setting on the collaborative process of learning. Most well-established statistical methods are not able to analyze such nested data adequately. This article describes the problems which arise when standard methods are…

Despite the widespread belief in a positive influence of research on education, the empirical evidence is lacking (Hattie and Marsh 1996). Several authors have questioned the appropriateness of the operationalisation of both aspects of the relation between teaching and research. This article takes a closer look at the research questions in…

The authors express their concerns about the research methodology and assumptions in Coker, Medley, and Soar's work on competency-based teacher education. (WD)