First, What Is a Preprint?
Think of a preprint as a sneak peek of a scientific paper. It's a full research manuscript made available to the public on servers like bioRxiv or medRxiv before it has undergone peer review. This process, where independent experts in the same field scrutinize
the work, is a cornerstone of formal academic publishing designed to ensure quality and validity. Scientists post preprints to share their findings quickly, get early feedback from a wide range of peers, and stake a claim to their discoveries. This practice has been common in fields like physics for decades, but its adoption in biomedical science has exploded, especially during urgent health crises like the COVID-19 pandemic. This speed is a double-edged sword: it accelerates science but also releases unvetted information into the wild, where it can be misinterpreted.
What the Giant 70,000-Preprint Study Found
The headline-making study offers a sweeping look at how research changes between its initial preprint stage and its final, peer-reviewed publication. Researchers at the University of Southern California analyzed 72,644 pairs of biomedical preprints and their final published versions. The key finding is reassuring: for the vast majority of studies, the core conclusions hold up surprisingly well. The analysis found that in about 40% of cases, the main claim remained the same, and in another 50%, it was only revised in a minor way. Only about 10% of papers had their conclusions substantially rewritten after peer review. When claims did change, they were twice as likely to become more cautious and tentative rather than more confident. The rate of major changes also decreased over time, dropping from 17% in 2019 to less than 6% in 2024. This suggests that while peer review is crucial for refinement, most preprints provide a solid initial look at the research.
The Risks of Reading Science Raw
While the large analysis is encouraging, the 10% of studies that do change significantly—and other studies noting major conclusion changes in nearly one in five COVID-19 preprints—highlight the need for caution. During the pandemic, the world saw the real-world consequences of misinterpreting preliminary findings. Preprints on potential treatments or vaccine side effects went viral, were amplified by media and public figures, and sometimes influenced behaviour before the claims could be validated or debunked by the scientific community. This is the primary danger: treating a preprint, which is a dispatch from the research frontier, as a final, verified fact. Peer review often catches errors in methodology, pushes authors to tone down overstated claims, and adds important context—all of which is missing when a reader engages only with the initial preprint.
How to Be a Smarter Science Reader
Navigating the world of preprints doesn't require a PhD, just a healthy dose of critical thinking. First, always check if you are reading a preprint. Reputable servers clearly label them as such, often with a disclaimer that the work has not been peer-reviewed. Second, treat its findings as preliminary. They are a starting point for a scientific conversation, not the final word. Ask yourself: who are the authors, and what is their track record? What are other experts in the field saying about the study? Often, commentary on social media from other scientists can provide valuable context. Finally, resist the temptation of a single, sensational study. Science builds knowledge incrementally. A finding that radically deviates from the existing body of evidence is more likely to be revised or proven incorrect than one that aligns with previous research.
