#sciencepublishing

With apologies to Madonna

I made it through academia Somehow I made it through Didn't know how lost I was Until I found you

I was beat Incomplete I'd been had, I was sad and blue But you made me feel Yeah, you made me feel Shiny and new

Hoo, Like a physicist (Posting) preprints for the very first time Like a physicist When your e-print Is next to mine

Gonna give you all my data, bioRxiv My fear is fading fast Been saving it all for you 'Cause open science will last

You're so fine And you're free Make me strong, yeah you make me bold Oh your archive thawed out Yeah, your archive thawed out What was scared and cold

Like a physicist, hey (Posting) preprints for the very first time Like a physicist When your e-print Is next to mine

Whoa Whoa, ah Whoa

A few answers to @hormiga post about why he’s not bothering with preprints.

1.     I don’t mind waiting a few months or a couple years for people to see my paper. If there’s anybody who I want to find out about my stuff right away, then I’ll send it to them. I don’t think I’m making the world suffer by waiting a little longer than they’re already waiting.

Answer: You publish to communicate your science so, hopefully, others will be influenced and can build on it to do more great stuff. You may not mind making the world wait longer but that only means you’ll unnecessarily delay influencing and impacting others.

2.     I don’t expect that I’ll get more or better peer review by using preprints than using other approaches.

A: You’ll never find out I guess. That’s like saying there is no point in presenting a poster or giving a talk at a conference because you’re not going to get better feedback than through formal pre-publication peer-review. In reality, you never know where the good suggestions will come from, so the wider you disseminate your work, the more chance that you’ll get useful feedback.

3.     I want my papers to have a single (and better) version available to the public. Let’s say I have manuscript that I’m ready to submit to a journal. And so I upload the preprint and submit it at the same time. Then, the peer reviews from the journal come back with editorial remarks. And I then make some changes that improve the paper. Then that one will get published. So now, the world has access to this paper that’s not as good? Why would I want this? Can I avoid this by not posting a preprint? Yay!

A: Indeed, it takes guts to post a preprint. We want to see your own version, not the editor or reviewer-modified one. That first unaltered submission is a true reflection of your quality as a scientist. If you call yourself an expert on something then you better get the preprint right. Am I going to hire a biochemist whose preprint experiment lacks obvious controls? Nope.

4.     I’d prefer a paper to splash than trickle out. If I have a paper that’s interesting and exciting, and then I want to be able to share this once the legit peer-reviewed version comes out.

A: Why not have the splash earlier? And what kind of a splash will you have once you get scooped?

5.     I might have fucked up in a paper, and maybe I don’t want to outsource the detection of this fuckup to the entire world? And I might not want to expose my student coauthors to this possibility?

A: Again, it takes guts to post a preprint. And we want to know when you fuck up. Better, we want to know how you react when you find out that you fucked up. Currently, we lack incentives for self-correction. Your students would only become better scientists if they learn to admit that they were wrong and proceed to fix the record.

6.     I think preprints exacerbate the malady of careerism that makes it difficult for junior scientists to treat science like a normal job. To get ahead in academia, you gotta do MORE! FASTER! BIGGER! and it feels like the epitome of this is the need to magically publish your papers before they get published. I think preprint advocates might think that they are a prescription to fix the rat race, but I think it just puts the rats in a more convoluted maze. Po-tay-to, po-tah-to.

A: Srsly? One of the most frustrating aspects of academia is publishing. It’s stressful, takes ages, and there is a lingering feeling that it’s unfair and gamed. You write up your work and proudly submit it to some journal only to have an editor or reviewer tell you that it’s not broad impact, not appropriate for this journal, you need to do more work etc. In my experience, this is turning off many youngsters from science. Colleagues tell me that posting preprints relieves a lot of that pressure. It’s a liberating feeling to finish writing up a paper and immediately share it with everyone. You should try it.

7.     If preprints do become the norm, then there’ll be a ton of crappy preprints with unsubstantiated results and highly flawed experimental designs that wouldn’t get past the editorial process of a legit journal. But I’m sure every single one of you reads every scientific paper from start to finish and comprehensively decides whether the methods were sound and the analyses were robust before citing it.

A: There is already tons of crappy peer-reviewed papers. Deeply flawed papers get published in all sorts of journals (#arseniclife anyone?), and continue to get cited even after they’re debunked or retracted.

8.     If I were to create a list of 100 things that we should be changing about how we do and publish science, preprints wouldn’t be on my list. So until it becomes the norm, I’ll fuss over things that matter more to me.

A: The way we publish science is costly both in terms of the scientists time commitment to editing and reviewing, and the outrageous page charges that end up fattening the profit margins of scientific publishers. Preprints are free.

Find out more about preprints at #ASAPbio.

Image from PrePubmed (June 2017 stats).

·      The Old schooler

o   "I grew up learning how to play the publishing game and now you want to change the rules?”

o   “Preprints don’t fit in the games I play with Editors and reviewers.”

·      The Metrics Maniac

o   “My publishing decisions are driven by Journal Impact Factors.”

o   “Preprints don’t have an impact factor.”

o   “I only need papers to get a job/promotion. Preprints don’t count.”

·      The Paranoiac

o   "I want to get this paper in a CNS journal, and I want to stack as many chips on my side as I can. Preprints won’t help my chances.”

o   “I’m convinced preprints would hurt my chances to get in a CNS journal.”

·      The Insecure

o   “I’m never that sure about the first version of my papers and I don’t want the world to see it.”

o   “Reviewers always improve my papers. I only want people to read the better version.”

o   “What if my paper is flawed?”

·      The Libertarian

o   “I don’t like preprints. I don’t want anyone to tell me what to do.”

·      The Unenlightened

o   "I don’t know what preprints are.”

o   “I heard of them but I’m not really sure what it’s all about.”

·      The Cynic

o   “It’s another fad and it will blow away.”

Preprints—open versions of scientific papers that typically precede formal publication in scientific journals—have emerged as an important source of scientific information. How do preprints differ from standard scientific publications? The primary difference is that classic scientific papers normally go through the process of pre-publication peer-review. This is thought to give the publication a stamp of approval—experts have rigorously checked and acknowledged the quality of the science and the conclusions drawn from the work. Other scientists can now read the work with a degree of confidence knowing that it has been carefully vetted by their peers.

That’s at least the theory. In practice, the quality of pre-publication peer-review is highly variable and the system can be gamed. It is common that deeply flawed papers get published in all types of journals, from minor outlets to the famous glam-mags. Although the reasons are complex, peer-review is known to be slow, inconsistent, biased and subject to abuse. In addition, the review process generally lacks transparency leading to frustration among authors and readers alike. As a consequence, scientists often approach the peer-reviewed literature with a great degree of skepticism. With the emergence of internet commentary and social media, post-publication evaluation has become routine. Post-publication peer-review is now an important factor in the scientific debate. Scientists are openly commenting and criticizing published papers using a variety of online platforms. It is now fairly common that erroneous research papers published in peer-reviewed journals get corrected or retracted following online criticism. The evaluation of the scientific literature doesn’t end with publication in peer-reviewed journals. The life of the paper extends beyond its acceptance date.

What’s flawed in the standard publication system is its reliance on pre-publication peer-review to filter publications and weigh on the rather subjective decision of what gets published where. This has created corrosive incentives for scientists. The focus is not on producing high-quality reproducible science that stands the test of time but to convince, and may be even fool, reviewers and editors. Authors have developed several stratagems to address the somewhat arbitrary nature of pre-publication peer-review. One example is serial submissions. Authors keep submitting a manuscript until they stumble on sympathetic reviewers and editors. This process can take years. Typically, the paper’s conclusions don’t change that much over the multiple versions. Some authors keep submitting their work even when reviewers highlight important flaws. Ultimately, some journal, possibly a reputable one, ends up publishing the manuscript. However, once the paper is out, the scientific community is largely unaware of its protracted history. The combined effort of the anonymous reviewers—a costly affair—remains buried in the journal’s archive and out of reach from the scientific community. All the pro bono time that the referees have devoted to the review process ends up serving little to advance science. I wish an economist would calculate the overall cost of this sterile exercise. [It turned out somebody precisely did that. Thanks @jessicapolka​ for the link.]

Pre-publication peer-review can of course improve the quality of the science but the fact is bad science eventually filters through thus defeating the purpose of the entire exercise. Flawed and irreproducible papers frequently get published in all types of journals. The authors of such papers often hide behind pre-publication peer-review to defend their work. “But it’s in [insert name of journal here]!”, they would claim. Ironically, the same authors tend to be dismissive of post-publication peer-review as if arguments raised through that process somehow carry less weight than the closed-door deliberations of an editor and a couple of referees. Indeed, post-publication review suffers from the apathy of authors and editors–a hesitance to fix the record. A truly shocking stance given that snubbing criticism goes counter to our fundamental ethos as scientists and would ultimately erode society’s trust in the scientific community. Calls to class the failure to address errors as research misconduct aim at creating incentives for self-correction. The responsibility to fix the record, even after formal publication in peer-reviewed journals, rests primarily with the authors.

The true stamp of approval in science is reproducibility. Can the work be repeated? Are other scientists building on it to advance their own research? If the study stands the test of time then it would have gone through a much more rigorous and stringent vetting than pre-publication peer-review by two or three referees. This is where preprints come in play. Do we need to devote so much time and money to the tedious and expensive process of pre-publication peer-review when there are painless and free platforms to disseminate scientific knowledge and accelerate the capacity of the scientific community to build on the published work? Is the significant material and human cost of pre-publication peer-review justified? Can the peer-review process be more transparent and focused solely on improving the science? The current scientific publishing format is based on a pre-internet era when the only way to get published was to have the article printed on paper—an expensive process that required a certain degree of filtering. Those days are long gone. We have yet to fully transition scientific publishing into the digital age. We don’t need to sift as stringently and we certainly don’t need to outsource the process of flagging the most exciting science to a handful of for-profit publishing houses.

What we need is to disseminate new research findings as early as possible and then, and only then, curate the literature. I foresee a future where all scientists would first publish their work as preprints. Journals would have to reinvent themselves to become forums for systematic analysis and discussion of this preprint literature. What we need is not pre-publication peer-review but post-preprint peer-review. Peer-review of the scientific literature needs to be transparent and dynamic. It should become more than just a mechanism to reach that dreaded accept/reject decision. The peer-review itself becomes a much more recognizable unit of scholarship. Review articles and commentaries will become more analytical and influential. Important science will get widely discussed and peer-reviewed by experts in readily accessible and searchable web platforms.

Journals need to reinvent scientific publishing. This is where the future lies—generalized use of preprints with journals serving as forums for curation and discussion of this literature. To achieve this, the first step would be for funding agencies to fully embrace their open science manifesto and mandate preprints for their grantees. Journals would only evaluate papers that have already been released to the public domain as preprints, and therefore totally ban pre-publication peer-review. All evaluations and reviews, whether positive or negative, are made public and linked to the preprint. These reviews should be posted as soon as they are received. This live peer-review system needn’t be limited to editors and commissioned reviewers but can include author led reviewing and crowdsourced community feedback. As the reviews accumulate, the journal editors may engage in the discussion, elect to tag a particular paper and promote a given version of the paper to a formal article. The editor’s role is more akin to curator than gatekeeper. The process doesn’t need to be static. As the community further comments on the article and follow-up studies are published, the journal editors may decide to revise the tags and annotations of the paper to reflect new knowledge and scientific advances.

The fundamental feature of this radical scientific publishing model is that every single evaluation and review of every submitted article is published through a fully transparent open process. This would add badly needed accountability and increase scrutiny of the published literature. It would also save time and money by eliminating redundant evaluations and reducing reviewer load. Science and society can only benefit.

P.S. To clarify the proposal, I’m posting a roadmap to deliver this vision: Journals 2.0: a roadmap to reinvent scientific publishing. Zenodo. http://doi.org/10.5281/zenodo.1466784. [added on October 18, 2018] 

Posted by Sophien @kamounlab

I summarized some thoughts on peer-review following exchanges with colleagues prompted by this tweet. I hope this clarifies my views. A more extensive article on this topic will follow soon. 

These are my quick thoughts in no particular order:

• I have no problems with peer-review per se. But I have a problem with pre-publication peer-review to decide what gets published where. That system is messed up. Bigly.

• Do we need to peer-review every single article that's ever been written? Probably not. In fact, just the sheer volume of what gets submitted to journals is such that the current review system is becoming impractical. Every Editor will tell you: too many papers, not enough qualified reviewers. See Figure 1 of this article by Plume & van Weijen.

• The peer-reviewed literature creates an illusion of rigor. This truly is an illusion.

• Post-preprint peer-review works and the community will engage ONLY when it matters. Here is an example of community post-preprint peer-review in the comments section of a biorxiv paper. Note that the paper was ultimately published in a peer-reviewed journal. 

• Note the disclaimer on biorxiv explaining what a preprint is.

• Unrelated to peer-review but a critical point nonetheless is the broken economic model of scholarly publishing. Example: it costs $5200 to publish in Nature Communications. That’s one third of PhD studentship in China.  It’s obscene. Publishing in biorxiv is free.

Check these great plant-microbe interactions e-prints/preprints published just these last two weeks on bioRxiv:

Brachi et al. Plant genes influence microbial hubs that shape beneficial leaf communities. (August 26, 2017)

Shahid et al. MicroRNAs from the parasitic plant Cuscuta campestris target host messenger RNAs. (August 25, 2017)

Miller et al. De novo assembly and phasing of dikaryotic genomes from two isolates of Puccinia coronata f. sp. avenae, the causal agent of oat crown rust. (August 25, 2017)

Rosas-Diaz et al. A plant receptor-like kinase promotes cell-to-cell spread of RNAi and is targeted by a virus. (August 24, 2017)

Gladieux et al. Coexistence of multiple endemic and pandemic lineages of the rice blast pathogen. (August 24, 2017)

Bailey et al. Dominant integration locus drives continuous diversification of plant immune receptors with exogenous domain fusions. (August 19, 2017)

Schweizer et al. Positively selected effector genes and their contribution to virulence in the smut fungus Sporisorium reilianum. (August 16, 2017)

Grandaubert et al. The genomic rate of adaptation in the fungal wheat pathogen Zymoseptoria tritici. (August 15, 2017)

To celebrate, here is a song for you: Like a Physicist (with apologies to Madonna).

Follow the latest bioRxiv plant-microbe interactions papers on the biorxiv Twitter feeds for Plant Biology, Microbiology and Genomics. You’ll also find some of the latest bioRxiv papers on the Scoop.it pages Plants and Microbes and  Plant Pathogenomics.