The Age of the Preprint
The COVID-19 pandemic didn't just change our lives; it changed how science is communicated. Suddenly, 'preprints'—complete scientific manuscripts made public before formal peer review—became front-page news. This was a double-edged sword. On one hand,
it allowed for the rapid sharing of life-saving research at an unprecedented speed. On the other, it raised concerns about the spread of unvetted, potentially inaccurate information. Before the pandemic, preprints were mostly used in fields like physics. But their explosion in biomedical science, on servers like bioRxiv and medRxiv, has created a new reality for anyone who consumes health information: from a student writing a paper, to a journalist on a deadline, or a researcher building on new findings. The central question has been: how reliable is this early information?
A Landmark Study Provides Clarity
To answer this, researchers have been working to quantify just how much a scientific paper changes between its initial preprint version and its final, peer-reviewed publication. One of the most revealing analyses, published in PLOS Biology, manually compared hundreds of life science preprints to their eventually published counterparts. The study focused on papers from the critical early months of the pandemic, providing a unique window into science under pressure. The researchers meticulously tracked changes to the abstract—the paper's summary—as well as figures, tables, and authorship, creating one of the clearest pictures yet of the evolution of scientific findings. This wasn't just about counting edits; it was about understanding whether the core conclusions of the research held up after scrutiny.
What Actually Changes from Preprint to Publication?
The findings were nuanced and illuminating. The good news is that for the vast majority of papers, the fundamental conclusions don't change. The study found that the main conclusions of about 93% of non-COVID-19 life science preprints remained stable upon publication. However, the picture was different for the more rapidly produced COVID-19 research. For these papers, 17.2% saw major changes in their conclusions after peer review, compared to just 7.2% for non-COVID papers. These 'major changes' could involve strengthening or softening a conclusion, or in rare cases, even contradicting the initial finding. Most alterations were more minor, such as adding new data or making textual changes. This suggests that while preprints are often directionally correct, a significant minority undergo important revisions, especially when research is being conducted during a crisis.
A New Playbook for Journalists and Students
These findings have profound implications for how non-specialists should approach preprints. For journalists and students, the key is to treat preprints with 'critical assessment' rather than taking them at face value. The study reinforces the need to always highlight that the findings are preliminary and have not been peer-reviewed—a step that is crucial given that studies show only a small fraction of the public can even define what a preprint is. Experts advise focusing on the big-picture findings rather than specific statistics, which might change. It’s also wise to seek input from unaffiliated scientists who can provide context and point out potential flaws. The goal isn't to stop reporting on preprints, especially during public health emergencies, but to do so with a clear understanding of their provisional nature, thereby balancing speed with responsibility.
Rethinking Research Communication
For researchers themselves, this data provides confidence in using preprints to disseminate work early, while also serving as a reminder of the value of the peer-review process. Peer review, for all its limitations, often makes papers stronger by catching errors, increasing sample sizes, or making the authors' language more conservative and precise. The new landscape doesn’t position preprints as a replacement for traditional journals, but as a complementary part of the scientific ecosystem. The most effective way to read any scientific paper—preprint or published—is to engage with it actively. This means starting with the data in the figures and tables before reading the authors' interpretation in the discussion. By forming your own conclusions from the evidence first, you can more effectively judge whether the authors' claims are truly supported. This skill is more vital than ever in an era of accelerated science.
