ERIC Number: EJ958843
Record Type: Journal
Publication Date: 2012-Mar
Pages: 8
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1082-989X
EISSN: N/A
Available Date: N/A
Publication Bias in Psychological Science: Comment on Ferguson and Brannick (2012)
Rothstein, Hannah R.; Bushman, Brad J.
Psychological Methods, v17 n1 p129-136 Mar 2012
It is well documented that studies reporting statistically significant results are more likely to be published than are studies reporting nonsignificant results--a phenomenon called "publication bias". Publication bias in meta-analytic reviews should be identified and reduced when possible. Ferguson and Brannick (2012) argued that the inclusion of unpublished articles is ineffective and possibly counterproductive as a means of reducing publication bias in meta-analyses. We show how idiosyncratic choices on the part of Ferguson and Brannick led to an erroneous conclusion. We demonstrate that their key finding--that publication bias was more likely when unpublished studies were included--may be an artifact of the way they assessed publication bias. We also point out how the lack of transparency about key choices and the absence of information about critical features of Ferguson and Brannick's sample and procedures might have obscured readers' ability to assess the validity of their claims. Furthermore, we demonstrate that many of the claims they made are without empirical support, even though they could have tested these claims empirically, and that these claims may be misleading. With their claim that addressing publication bias introduces subjectivity and bias into meta-analysis, they ignored a large body of evidence showing that including unpublished studies that meet the inclusion criteria of a meta-analysis decreases (rather than increases) publication bias. Rather than exclude unpublished studies, we recommend that meta-analysts code study characteristics related to methodological quality (e.g., experimental vs. nonexperimental design) and test whether these factors influence the meta-analytic results. (Contains 1 footnote.)
Descriptors: Research Reports, Academic Discourse, Context Effect, Bias, Identification, Periodicals, Writing for Publication, Selection, Meta Analysis, Evidence, Coding, Research Methodology
American Psychological Association. Journals Department, 750 First Street NE, Washington, DC 20002-4242. Tel: 800-374-2721; Tel: 202-336-5510; Fax: 202-336-5502; e-mail: order@apa.org; Web site: http://www.apa.org/publications
Publication Type: Journal Articles; Opinion Papers; Reports - Descriptive
Education Level: Higher Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A