One of the things I hate about the business of science more than anything else is the paper fight. I don’t have any problems with tough peer review. (I have seen a number of cases where peer review has been incredibly helpful – finding errors and giving new insights that improve papers.) My problem is that in our current system, papers fall down from fancy journals, and that can take years. More importantly, the journey down from fancy (“high impact”) to less fancy journals is rarely due to problems in the science itself, but is more due to editor’s opinions about the importance / impact / snazziness of the results.
We’ve all been there – you take a paper to a fancy journal. The peer reviewers find some minor issues, but in the confidential-to-the-editor section, they say “this isn’t important enough for your fancy journal to fill your limited space with”. Maybe you fight with the editors. Maybe you appeal. But then, after a year of arguing, it is clear that they are not going to take it, and you have to try the whole process again. So, it takes another year at a slightly less-fancy journal. And it takes years to publish the paper, even though the science has not actually changed, and the original paper was, honestly, just fine, thank you very much.
The scientific community has been complaining about this for years. I was asked once to be on a focus group for a fancy GlamourMag (which will remain nameless) – they offered us lunch at SFN if we would answer some questions for them. It turned out what they wanted to ask was “how much will you pay for open access?” The 12 of us all said that open access was a red herring and unimportant, because the scientific community could easily find any published paper, either through institutional access, through our websites, or by emailing us directly, and, moreover, every paper funded by NIH (we were all NIH-funded researchers) would have to be publicly available on pubmedcentral after twelve months. We said that we didn’t care about open access. What we cared about was the fact that it took years to fall down the journal hierarchy. They grumbled at us and asked again “how much will you pay for open access?” One of my colleagues said “nothing”. It did not make the GlamourMag personnel happy.
What the journal publishers seems to have come to as a solution is to make it easier to fall down the impact-tree within their family of journals through “consortia” that send reviews on with the paper. THIS IS NOT A SOLUTION TO THIS PROBLEM! It still takes months (or years) to fight with each journal, usually over importance not scientific validity, and, in my experience, one often ends up getting re-reviewed and starting the process over in the next round down, even when the reviews get passed down. (Because it’s not usually the reviews that are the reason for the rejection.)
Some people have suggested that preprints [e.g. BioRxiv, put it out unreviewed, but then go ahead and do the journal fight] solves this problem. While this solves the priority problem, it does so by losing all the good things that journals actually do. (Next post is on my problems with BioRXiv.) So I’m not happy with that solution.
Other people have argued that one should have a single, flat journal [e.g. PLoS ONE, which has no journal fight, because it explicitly does not take “importance / impact / snazziness” into account]. In practice, because we do still use journals for cachet (q.v. oldocmurphy/author-pays), these big journals have become places to put minor work that failed at the fancier journals. And I will note that PLoS has a family of journals (from flagship journals like PLoS Biology to workhorse journals like PLoS Computational Biology – both excellent journals), belying the idea of a single, flat journal.
My favorite solution was the one put forward by Frontiers, but, to my knowledge, this seems to have fallen by the wayside and is no longer being used and the Frontiers family of journals are being treated as more standard journals (like Frontiers in Decision Making, or Frontiers in Integrative Neuroscience). It’s too bad because it was a good idea. That idea was that papers should move up, not down. In this system, all papers go through peer review for basic scientific integrity. They get published in a “Tier I” journal (think Frontiers in Some Type of Specialized Neuroscience). This means they are peer reviewed, full publications (avoiding the preprint problem). Then, using online metrics (number of downloads, number of citations, number of likes, number of comments, etc), a subset of authors are invited to write a second, follow-up article for a “Tier II” journal (say Frontiers in Neuroscience) that would appeal to a broader audience. The second paper is more of a review, providing a more general view on the results. The second paper is also fully peer-reviewed and edited, making it a second citable publication. (In the early days of Frontiers, I had several papers go through this two-stage transition. It worked really well.) There was supposed to be a “Tier III” stage, which would be across-fields, say Frontiers in the BioSciences or even Frontiers in Science, but I don’t think that ever appeared. The beauty of this solution was that papers appeared immediately (no priority problem!), they were peer reviewed (no preprint problem!), but that there were opportunities for results to gain cachet (Glamour, here we come!). Importantly, the decision on what made it into the fancier journals was a community decision, not solely an editor’s. But it seems to have fallen by the wayside. The Frontiers family of journals is now more like every other journal. They’re fine journals, but we seem to have lost this innovative solution to the GlamourMag problem.
Whatever happened to it? Why didn’t it catch on? Can we implement it somehow?