The Journal of Peer Production - New perspectives on the implications of peer production for social change New perspectives on the implications of peer production for social change
Journal of Open Source Software – Publish Your Software: Introducing the Journal of Open Source Software image

Article and rationale:

Daniel S. Katz, Kyle E. Niemeyer and Arfon M. Smith (2018) Publish Your Software: Introducing the Journal of Open Source Software (2018) Computing in Science & Engineering, May/June, 84-88.

Daniel S. Katz: JOSS offers developers a venue to publish their complete research software wrapped in relatively short high-level articles, thus enabling citation credit for their work. Submissions should be easy for authors – we state, ‘if your software is already well documented, then paper preparation should take no more than an hour’. This article was specifically written to attract potential JOSS authors (i.e., research software developers) and to build up interest within the overall open source community.

Download PDF: OAB Journal of Open Source Software


The Journal of Open Source Software (JOSS,[1]

Daniel S. Katz, Tania Allard, Christopher R. Madan, Kevin Moerman, Karthik Ram, Arfon Smith

The primary goal of the Journal of Open Source Software (JOSS) is to give researchers who develop, contribute to, and maintain open source software a means to get citable credit for their work within today’s research ecosystem. In short, JOSS is a developer-friendly journal for publishing research software packages. It’s also an online academic journal (ISSN 2475-9066) with a formal peer review process that is designed to improve the quality of the software submitted by making sure it meets minimum standards and includes standard identifiers for content on digital networks (Digital Object Identifiers, or DOIs) for all accepted papers.

In a perfect world, we would want to see credit given directly to software and those who develop it, but in today’s research world, citations to papers and other products are the standard means for tracking the impact and use of research work. (Certainly there are problems with citations; this is simply a practical choice.) We recognize that for most researchers, papers, not software, are the currency of academic research, and citations are required for a good career. In academic and government research, citations of papers are one of the two main factors that count in hiring and promotion decisions, along with getting research funding. We’ve created JOSS so that software developers can get the same credit as traditional authors, by making it easy for developers to turn their software into citable papers.

After you’ve done the hard work of writing great software, it shouldn’t take weeks or months to write a paper about your work. In fact, as we say on our About page, if your software is already well documented, then paper preparation should take no more than an hour.

This includes a simple submission workflow and extensive documentation to help you prepare your submission, which creates an issue in our GitHub repository. The paper itself is a short markdown file that contains a summary describing the high-level functionality and purpose of the software for a diverse, non-specialist audience; a clear statement of need that illustrates the purpose of the software; a list of key references, including a link to the software archive; and mentions (if applicable) of any ongoing research projects using the software or recent scholarly publications enabled by it.

After this, it’s our turn. An editor-in-chief, 3 associate editors-in-chief, and  16 topic editors make up our editorial board. One of the editors-in-chief does a brief scan to make sure the paper looks suitable for JOSS (is it about a software package likely to be used and cited in the research ecosystem?), then an editor takes responsibility for the rest of the review process. The editor finds typically two reviewers who agree to review the paper, creates a new GitHub issue for the review itself, and closes the pre-review issue.

A bot (named Whedon) monitors the JOSS issues, and acting on keywords from editors, performs many of the core mechanical parts of the process, such as assigning reviewers and editors, creating review issues, and compiling PDFs upon acceptance.

The review issue contains a checklist of 18 items for the reviewer(s) to work through, including agreeing with JOSS’s policies on conflicts of interest and code of conduct; general checks on source code availability, use of an OSI-approved license, matching the software version in the paper and the repository and confirming that the submitter is an author of the software; checking on functionality, including installation and performance claims, if appropriate; checking on documentation, which should include a statement of need for the software, installation instructions, example usage, and documentation of functionality, tests, and community guidelines; and finally, checking on the software paper itself, including ensuring that the list of authors is reasonable, the statement of need is in the paper, and that sufficient and well-structured references are present.

Because this is done via a GitHub issue, when a reviewer finds a problem, they can comment in the review issue or open a new issue for the specific problem in the repository of the submission being reviewed. The article submitter can then interact with the reviewer, asking questions, making changes, etc. until the reviewer is satisfied, as moderated by the assigned JOSS editor. The goal of JOSS is not to reject a percentage of papers (although some rejections do happen when authors decide to withdraw rather than revise the submission based on the reviewer’s feedback), but to improve submissions until they are acceptable. This usually takes a few weeks, but if everyone is very responsive, it has been known to be completed in a day. However, in some cases, particularly with larger projects, the review process can take much longer, especially when documentation is unclear or features need to be added.

Once the reviewer(s) and editor are satisfied with the paper, the submitting author deposits a copy of the source code repository in an archival data repository such as Zenodo or figshare. A permanent link to this software archive is then added to the paper, which is converted from Markdown to PDF, and article metadata is deposited in Crossref. The review issue is closed, and the JOSS website is updated with the final version of the JOSS paper (PDF), including its DOI issued by Crossref. Just like the papers being reviewed, the final articles and metadata are versioned in a public repository as well as the JOSS website.

Now the authors have a paper with a DOI for the software and can add this information to their repository, in either the README or a CITATION file, telling users how to appropriately cite the software in a way that fits naturally into today’s research ecosystem. When users cite the software, the authors will be able to see this and get credit for the software, just as they would had they written a research article.

In its first 2 1/2 years (May 11, 2016 – January 26, 2019), JOSS has accepted 468 papers. You can read more about JOSS and its status in an open access paper in PeerJ Computer Science.


[1] This article is lightly adapted from Daniel S. Katz, Karthik Ram, Arfon Smith, Christopher Madan, “Who Gets the Credit?”, 21 Dec. 2017.