Estimating the deep replicability of scientific findings using human and artificial intelligence
Abstract
After years of urgent concern about the failure of scientific papers to replicate, an accurate, scalable method for identifying findings at risk has yet to arrive. We present a method that combines machine intelligence and human acumen for estimating a study’s likelihood of replication. Our model—trained and tested on hundreds of manually replicated studies and out-of-sample datasets —is comparable to the best current methods, yet reduces the strain on researchers’ resources. In practice, our model can complement prediction market and survey replication methods, prioritize studies for expensive manual replication tests, and furnish independent feedback to researchers prior to submitting a study for review.
Document Details
- Document Type
- Pub Defense Publication
- Publication Date
- May 04, 2020
- Source ID
- 10.1073/pnas.1909046117
Entities
People
- Brian Uzzi
- Wu Youyou
- Yang Yang
Organizations
- Air Force Office of Scientific Research
- Army Research Office
- Northwestern University