The Age of Instant Science
First, what exactly is a preprint? Think of it as a research paper’s first draft, made public before it has undergone formal peer review. Scientists post them on online servers like bioRxiv or medRxiv to share findings quickly, claim priority on a discovery,
and get early feedback from their peers. The practice exploded during the COVID-19 pandemic, when the need for rapid information was critical. This speed is a huge benefit for accelerating science, but it comes with a significant trade-off: the lack of the traditional quality control that peer review provides. This means the research is unvetted, and the conclusions are preliminary.
What 70,000 Papers Told Us
The headline-making study that sparked this discussion involved a massive analysis of nearly 73,000 biomedical manuscripts. Using a large language model, researchers compared the initial preprint version of a paper to its final, peer-reviewed published version. The findings were broadly positive. Around 90% of papers saw their main conclusions either remain the same (39.9%) or undergo only minor revisions (50%). Only about 10% of studies required major changes to their core findings after being scrutinised by other experts. Furthermore, the analysis found that papers appearing first as preprints were retracted at about half the rate of those that didn't. On the surface, this suggests that the preprint ecosystem is remarkably stable and reliable.
The Dangerous Fallacy of Averages
Herein lies the risk. While it’s encouraging that on average preprints are reliable, this average masks a crucial reality: the outliers can be incredibly dangerous. A 10% rate of major changes is not insignificant, especially when a study’s findings have public health implications. Some preprints contain brilliant, field-changing work, while others are flawed, contain honest mistakes, or in rare cases, are outright bad science. One study on COVID-19 preprints found that 17% had major changes in their conclusions upon publication, compared to just 7% for non-COVID studies. Treating every preprint as trustworthy because the average one is, is like assuming every mushroom in the forest is safe to eat because most are not poisonous. This assumption can have serious consequences.
When Bad Science Goes Viral
The potential for harm is not theoretical. Early in the pandemic, a preprint was published that misleadingly suggested similarities between the novel coronavirus and HIV, hinting at human engineering. Though it was quickly withdrawn, it spread rapidly online, fuelling misinformation campaigns. Another risk is 'spin', where results are framed to seem more significant than they are—a practice that peer review often, but not always, tempers. When news outlets and the public pick up on these preliminary, unvetted findings without understanding their provisional nature, it can lead to confusion, false hope, and a general erosion of trust in the scientific process. Studies have shown that a vast majority of the public doesn't understand what a preprint is, even when a definition is provided.
How to Be a Smarter Science Reader
Navigating this new landscape requires a more critical approach from all of us. When you see a news report about a new scientific breakthrough, ask yourself a few questions. First, is it based on a preprint or a peer-reviewed study? News articles should state this clearly, but often don’t. Look for mentions of servers like 'bioRxiv' or phrases like 'has not yet been peer-reviewed'. Second, see what other experts are saying. Is there a healthy scientific debate around the finding, or is it being presented as a fact? Finally, remember that science is a process, not a series of one-off pronouncements. Preprints are a valuable part of that process, representing the start of a conversation, not the final word. The final word comes only after rigorous review and replication.
