Concerns about the conduct of research are pervasive in many fields, including education. In this preregistered study, we replicated and extended previous studies from other fields by asking education researchers about 10 questionable research practices and five open research practices. We asked them to estimate the prevalence of the practices in…

A substantial number of randomized trials of educational interventions that have been conducted over the past two decades have produced null results, with either no impact or an unreliable estimate of impact on student achievement or other outcomes of interest. The investment of time and money spent implementing such trials warrants more useful…

In this response, we first show that Simpson's proposed analysis answers a different and less interesting question than ours. We then justify the choice of prior for our Bayes factors calculations, but we also demonstrate that the substantive conclusions of our article are not substantially affected by varying this choice.

As a result of the evidence-based decision-making movement, the number of randomized trials evaluating educational programs and curricula has increased dramatically over the past 20 years. Policy makers and practitioners are encouraged to use the results of these trials to inform their decision making in schools and school districts. At the same…

A recent paper uses Bayes factors to argue a large minority of rigorous, large-scale education RCTs are "uninformative." The definition of "uninformative" depends on the authors' hypothesis choices for calculating Bayes factors. These arguably overadjust for effect size inflation and involve a fixed prior distribution,…

The advent of online platforms such as Amazon's Mechanical Turk (MTurk) has expanded considerably researchers' options for collecting research data. Many researchers, however, express understandable skepticism of the viability of using platforms such as MTurk. In this article, we provide a background on the use of MTurk as a mechanism for…

In the current article, we consider the influential position of the Programme for International Student Assessment (PISA) and discuss several methodological areas that demonstrate the need for caution when using and interpreting PISA results. We motivate our argument by briefly describing the program's increased influence in educational policy…

Design-based research (DBR) evolved near the beginning of the 21st century and was heralded as a practical research methodology that could effectively bridge the chasm between research and practice in formal education. In this article, the authors review the characteristics of DBR and analyze the five most cited DBR articles from each year of this…

Discusses the role of research methodology as disciplined inquiry in educational research. Uses examples to address issues of generalizability of results, precision, and choice of research methods. (MK)

Three major propositions discussed in this article include: (1) There are two major obstacles to research influencing educational practice in the long term. (2) These are the pragmatic-practical notion that research should pay off and that it should be relevant to contemporary social and educational problems. (Author/AM)

Argues that defining "progress" is fundamental to understanding how effective educational research is in improving practice. Discusses the problem of partial specification and its consequences for defining the research-practice relationship. Presents three different, but complementary, approaches to defining "progress":…

Criticizes the previous article on positivism in educational thought by Denis Phillips. Takes issue with Phillips' assumption that, at the base of theoretical disputes and inquiry, there exists a final and absolute truth. (GC)

Ascribes difficulties associated with the use of ethnography in educational research to faulty or partial transmission of traits from one sociocultural system to another. Maintains that the appropriate use of ethnography demands that the whole trait complex be borrowed. Describes educational studies that used ethnographic methods. (Author/MJL)

Identifies and discusses three myths that pervade experimental counseling and psychotherapy: (1) that experimental subjects receive treatments appropriate to their clinical problems; (2) that the treatments are deployed as purported; and (3) that control groups customarily used by researchers allow them to determine the existence of a treatment…

Suggests that the results of research and the practice of teaching are related in many more ways than as a dialogue between a psychologist and a teacher. (Author)