Open Peer Review in Scientific Publishing: A Web Mining Study of PeerJ Authors and Reviewers

Purpose: To understand how authors and reviewers are accepting and embracing Open Peer Review (OPR), one of the newest innovations in the open science movement.

Design: This research collected and analyzed data from the Open Access journal PeerJ over its first three years (2013-2016). Web data were scraped, cleaned, and structured using several Web tools and programs. The structured data were imported into a relational database. Data analyses were conducted using analytical tools as well as programs developed by the researchers.

Findings: PeerJ, which supports optional OPR, has a broad international representation of authors and referees. Approximately 73.89% of articles provide full review histories. Of the articles with published review histories, 17.61% had identities of all reviewers and 52.57% had at least one signed reviewer. In total, 43.23% of all reviews were signed. The observed proportions of signed reviews have been relatively stable over the period since the journal’s inception.

Limitations: This research is constrained by the availability of the peer review history data. Some peer reviews were not available when the authors opted out of publishing their review histories. The anonymity of reviewers made it impossible to give an accurate count of reviewers who contributed to the review process.

Implications: These findings shed light on the current characteristics of OPR. Given the policy that authors are encouraged to make their articles’ review history public and referees are encouraged to sign their review reports, the three years of PeerJ review data demonstrate that there is still some reluctance by authors to make their reviews public and by reviewers to identify themselves.

Originality/Value: This is the first study to closely examine PeerJ as an example of an OPR model journal. As open science moves further towards open research, OPR is a final and critical component. Research in this area must identify the best policies and paths towards a transparent and open peer review process for scientific communication.