Arxiv.org Peer Review and the Humanities Journal
Traditionally, the humanities journal has relied on Peer Review much like many academic disciplines. I believe firmly that Arxiv.org or something very much like it will replace the humanities journal in the near future, much like it has in the sciences. One of the main objections to this direction in the humanities is “what about peer review?” This is a major sticking point. The other is “will it count on the RAE/whatever management insanity we have to deal with at the moment?” I cannot answer the latter but I will give a vision of a solution for the former.
Peer Review is anonymous. This allows for people to be straight-forward in their assessment of a piece of writing. Errors both large and small can be brought to the author’s attention and fixed. How would one go about implementing a system to deal with this? First, Peer Review is usually done gratis so people could sign up to be in a pool of Peer Reviewers. On an Arxiv.org type site, this might show up in your “profile”. When an author submits a work, it is marked as “not peer reviewed” but it will show up in searches, etc. The author then submits the article for Peer Review. The system then randomly picks two Peer Reviewers to review the work. They then read and vote on the work. If they give “two thumbs up”, the article will be marked. If they give it “two thumbs up with changes”, the changes are given to the author who then makes (or not) the changes and it is resubmitted to the reviewers who give the final “up or down vote”. In the case of a tie in the voting, another Peer Reviewer is randomly selected and they look over the case for and against, the tie-breaker then does his job.
As you can see, this can all be automated. All you need is people willing to do the Peer Review thing. Also, who nominates people for Peer Review? That is a political question really but I would suggest that you use people who have credentials (PhD or above) from accredited universities. Yes, there will be some cranks but this will be evened out by the possibility of the third tie-break reviewers.
In the large, a user could watch the “un-peer reviewed” articles stream to look for new ideas and but only reference those which are peer reviewed or something like that. This would make scholarship so much easier to do rather than having to deal with publishers (who in the end only rent back to us work that was given mostly for free, which is a monumentally insane way of doing things) and Jstor.
Celticist, Computer Scientist, Nerd, sometimes a poet…
My boss at PhilPapers wrote a paper on a similar (but more elaborated) ideas in the context of philosophy: http://philpapers.org/rec/BOUPPA (also have a look at the comments where I link to some more papers from FirstMonday).
There’s a missing element. To make this take off, there has to be a selfish reason for the peer reviewers to volunteer their effort. “Reputation” can work for this, but what’s even better is if the reviewers gain some direct selfish reward. F.ex. if they are interested in the content inside the system, and you can find a way in which doing reviews somehow makes the content a little more valuable for themselves, they’ll reap an immediate (if small) reward for every bit of work they put into the system. Ideally the value is such that it gets multiplied by network effects. If you can come up with such a motivation, and get enough users into the system quickly, then it will pretty much sell itself.
Yes, if I could think of something that would be of value to them.
Currently, Peer Review is “just done” and universities consider it “a thing that you do” and do not give academics any incentive to do it.