Traditionally, the humanities journal has relied on Peer Review much like many academic disciplines. I believe firmly that Arxiv.org or something very much like it will replace the humanities journal in the near future, much like it has in the sciences. One of the main objections to this direction in the humanities is “what about peer review?” This is a major sticking point. The other is “will it count on the RAE/whatever management insanity we have to deal with at the moment?” I cannot answer the latter but I will give a vision of a solution for the former.
Peer Review is anonymous. This allows for people to be straight-forward in their assessment of a piece of writing. Errors both large and small can be brought to the author’s attention and fixed. How would one go about implementing a system to deal with this? First, Peer Review is usually done gratis so people could sign up to be in a pool of Peer Reviewers. On an Arxiv.org type site, this might show up in your “profile”. When an author submits a work, it is marked as “not peer reviewed” but it will show up in searches, etc. The author then submits the article for Peer Review. The system then randomly picks two Peer Reviewers to review the work. They then read and vote on the work. If they give “two thumbs up”, the article will be marked. If they give it “two thumbs up with changes”, the changes are given to the author who then makes (or not) the changes and it is resubmitted to the reviewers who give the final “up or down vote”. In the case of a tie in the voting, another Peer Reviewer is randomly selected and they look over the case for and against, the tie-breaker then does his job.
As you can see, this can all be automated. All you need is people willing to do the Peer Review thing. Also, who nominates people for Peer Review? That is a political question really but I would suggest that you use people who have credentials (PhD or above) from accredited universities. Yes, there will be some cranks but this will be evened out by the possibility of the third tie-break reviewers.
In the large, a user could watch the “un-peer reviewed” articles stream to look for new ideas and but only reference those which are peer reviewed or something like that. This would make scholarship so much easier to do rather than having to deal with publishers (who in the end only rent back to us work that was given mostly for free, which is a monumentally insane way of doing things) and Jstor.