ERIC Number: EJ1316119
Record Type: Journal
Publication Date: 2021-Nov
Pages: 20
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1759-2879
EISSN: N/A
Available Date: N/A
Detecting Publication Selection Bias through Excess Statistical Significance
Stanley, T. D.; Doucouliagos, Hristos; Ioannidis, John P. A.; Carter, Evan C.
Research Synthesis Methods, v12 n6 p776-795 Nov 2021
We introduce and evaluate three tests for publication selection bias based on excess statistical significance (ESS). The proposed tests incorporate heterogeneity explicitly in the formulas for expected and ESS. We calculate the expected proportion of statistically significant findings in the absence of selective reporting or publication bias based on each study's SE and meta-analysis estimates of the mean and variance of the true-effect distribution. A simple proportion of statistical significance test (PSST) compares the expected to the observed proportion of statistically significant findings. Alternatively, we propose a direct test of excess statistical significance (TESS). We also combine these two tests of excess statistical significance (TESSPSST). Simulations show that these ESS tests often outperform the conventional Egger test for publication selection bias and the three-parameter selection model (3PSM).
Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A