Forecasting the publication and citation outcomes of COVID-19 preprints
Abstract
Many publications on COVID-19 were released on preprint servers such as medRxiv and bioRxiv. It is unknown how reliable these preprints are, and which ones will eventually be published in scientific journals. In this study, we use crowdsourced human forecasts to predict publication outcomes and future citation counts for a sample of 400 preprints with high Altmetric score. Most of these preprints were published within 1 year of upload on a preprint server (70%), with a considerable fraction (45%) appearing in a high-impact journal with a journal impact factor of at least 10. On average, the preprints received 162 citations within the first year. We found that forecasters can predict if preprints will be published after 1 year and if the publishing journal has high impact. Forecasts are also informative with respect to Google Scholar citations within 1 year of upload on a preprint server. For both types of assessment, we found statistically significant positive correlations between forecasts and observed outcomes. While the forecasts can help to provide a preliminary assessment of preprints at a faster pace than traditional peer-review, it remains to be investigated if such an assessment is suited to identify methodological problems in preprints.
Document Details
- Document Type
- Pub Defense Publication
- Publication Date
- Sep 01, 2022
- Source ID
- 10.1098/rsos.220440
Entities
People
- Anna Dreber
- Brandon Goldfedder
- Charles Twardy
- Felix Holzmeister
- Juntao Wang
- Louisa Tran
- Magnus Johannesson
- Michael Bishop
- Michael Gordon
- Thomas Pfeiffer
- Yang Liu
- Yiling Chen
Organizations
- George Mason University
- Harvard University
- Jacobs Engineering Group
- Massey University
- Stockholm School of Economics
- University of California, Santa Cruz
- University of Innsbruck