Current academic practice
Our approach to peer reviewing is informed by Whitworth and Friedman’s [WaF2009a] criticism of current academic publishing as a form of competitive economics in which “scarcity reflects demand, so high journal rejection rates become quality indicators”. This self-reinforcing system where journals that reject more attract more results in a situation where “avoiding faults becomes more important than new ideas. Wrongly accepting a paper with a fault gives reputation consequences, while wrongly rejecting a useful paper leaves no evidence”.
Whitworth and Friedman [WaF2009b] propose an alternative evaluation system:
1. higher rating discrimination: a many-point scale, not just accept-reject
2. more submissions to be rated: rate all
3. more people to rate: community involvement
4. different ways of rating: formal review vs. informal use ratings.
Our process is also informed by Toni Prug’s [Prug2010] paper on community reviewing: submission proposals will be made on our open email list. Prospective authors will then be told by our peer review community whether their proposal is appropriate for the journal and if any additional key elements are missing.
JOURNAL OF PEER PRODUCTION review process
Prospective authors and reviewers are invited to carefully read this section.
1. PAPER PROPOSAL
1.1. Non-special issues
Authors use the contact form to send a proposal to the editorial team who will assess the suitability of the proposal for the journal.
1.2. Special issues
Authors submit proposals directly to the special issue editors.
2. SUBMISSION AND REVIEW
Once authors have completed a full submission, they send it to the editor who assigns it to three anonymous reviewers. The reviewers assess the paper following our suggested review categories (see Appendix A). Once the reviewers have provided any necessary recommendations for improvement, these reviews are sent to the author for possible revision and the author decides whether to follow these recommendations.
3.REVISION AND SIGNALS
The revised paper is sent back to reviewers for final evaluation following signaling categories (see Appendix B). The editor forwards the signals to the author, who then decides whether to publish the paper or not.
The paper is published alongside the signals.
Signals will remain anonymous to ensure frank and fearless signaling.
IMPORTANT – PLEASE NOTE:
- All reviews will be released alongside published papers. Reviewers may opt whether to remain anonymous or not.
- All initial draft submissions will be released alongside published papers unless the author provides compelling reasons (such as privacy of author or subjects) why this should not be the case.
Since draft submissions will be published, authors are encouraged to use a respectful tone when dealing with other researchers’ work in their drafts.
Outcomes and benefits
It is up to the author to decide whether they are happy with publishing a paper with the given signals. Letting authors decide whether to publish enables the journal to release a wider variety of submissions whilst protecting its scientific reputation.
Signaled papers will be published alongside reviewers’ initial reports and recommendations. This will help to promote review quality, as well as ensure that reviewers receive public appreciation for their work.
To offer a counterpoint to expert reviews, anyone who registers an identity with the site will be able to rate articles, and comment on articles. Statistics regarding article views and ratings, and reviewer activity will be available on request for authors and reviewers, or sent directly by the journal to the required institutions.
Appendix A: Suggested categories for peer review
1. Is the subject matter relevant?
2. Is the treatment of the subject matter intellectually interesting? Are there citations or bodies of literature you think are essential to which the author has not referred?
3. Are there any noticeable problems with the author’s means of validating assumptions or making judgments?
4. Is the article well written?
5. Are there portions of the article that you recommend be shortened, excised or expanded?
Appendix B: Suggested categories for signaling
Article proposes a critique of a policy or practice with specific action proposals or suggestions: yes / no
Article follows conventions of academic research article — e.g. position in literature, cited sources, and claimed contribution: yes / no
Article is based on developments that have not yet occurred: yes / no
Article is based on formal logic or mathematical technique: yes / no
Standard of English expression in article is excellent: yes / no
Scope of debate
Article addresses an issue which is widely known and debated: yes / no
Most related sources are mentioned in article [this is an invitation to careful selection rather than a demonstration of prowess in citation collection -- i.e. apt and representative choices made in source citations]: yes / no
Ideas are well organised in article: yes / no
The argument presented in article is new: yes / no
The article has been significantly changed as a result of the review process: yes / no
Reviewers indicate their appreciation of the article in the form of a 50 word statement.
[Prug2010] Prug T (2010) “Open-process Academic Publishing”
[WaF2009a] Whitworth B and R Friedman (2009a) “Reinventing academic publishing online. Part I: Rigor, relevance and practice”, First Monday, Volume 14, Number 8 – 3 August 2009.
[WaF2009b] Whitworth B and R Friedman (2009b) “Reinventing academic publishing online. Part II: A socio-technical vision”, First Monday, Volume 14, Number 9 – 7 September 2009.