Scientists complain a lot about peer review. It’s a safe bet that most of us think that our papers are generally not improved in the process, but in the usual self-congratlulatory way, most of us probably think that we’re in the minority of good referees who actually make useful suggestions, or catch egregious errors. We can’t be right about both, not most of us anyway.

Nature, arguably the most prestigious scientific journal across all fields, unquestionably a bastion of the establishment, has decided its time to re-examine the peer review paradigm, prompted largely by the ongoing revolution in electronic publishing — which has caused many of us to wonder whether science’s version of “mainstream media” is needed any longer.

First, it’s hosting a peer review trial and debate. They’ve invited editors of other journals, experts in electronic publishing, ethicists, and working scientists to comment on the purpose of peer review, the way it is practiced today and how it might be modified in what seems to be the coming era of open journals and fully electronic access. There’s even a contribution from Chris Anderson, editor of Wired, who first enunciated the idea of the Long Tail, that in the age of the internet, there’s as much of interest in the tail of the distribution, millions of works of art or science, even if each of them only reaches a tiny audience. Here, he talks about collaborative review in the style of Wikipedia, Slashdot, or Digg, marshalling the so-called wisdom of crowds. (Anderson will be in London this week.)

It’s harder in the world of science, of course, where specialist knowledge is necessary, but Nature is willing to experiment. Parallel to their standard external peer review, they’re letting authors opt into a public peer review trial. The paper is posted on the web, open for blog-style (moderated, and not anonymous) comments by “the scientific community”, which the editors will consider when deciding on the fate of the paper. Of course, the supposedly anonymous referees are free to comment themselves — and forego their anonymity. In astrophysics, most of us already post on the ArXiv at the same time that we submit to a journal, but a sanctioned way of soliciting comments from the community could transform the lecture into a conversation (not that many of us will find the time to comment, I suspect).

Of course, Nature is only one journal, and it doesn’t publish much cosmology, but some colleagues and I are trying to create an “Open Journal of Astrophysics and Cosmology” which would overlay these sorts of ideas on the extant (and public) ArXiv.