Free scoping read. A written assessment of your manuscript in two business days, at no cost.

Start one
Contents

Academy · Research Publication · Peer Review Models

Peer Review Models in Scholarly Publishing: A Review of the Evidence

Single-blind, double-blind and open peer review are not interchangeable labels. Evidence on peer review should be interpreted in the context of the study’s discipline, dataset and review process.

Pubrica AcademyWritten by a scholarly publishing analyst Reviewed bya peer review process specialist Published10 October 2026 Last checked10 October 2026 Reading time8 minutes Sources10 peer-reviewed

Journals conduct peer review under different models: single-blind, in which reviewers know the authors’ identities but not the reverse; double-blind, in which neither party knows the identity of the other; and open review, in which reviewer identities, reports or both are published alongside the article. These are not minor procedural variations. Separate, well-controlled studies, including a randomised controlled trial conducted within a major medical journal, have measured differences in reviewer behaviour and outcomes between them, providing a more precise account of the effects of changing the model than intuition alone would suggest. Evidence on peer review should be interpreted in the context of the study’s discipline, dataset and review process.

Learning objectives

  • What a large-scale study of conference submissions found about bias differences between single-blind and double-blind review
  • What happened when the BMJ randomly assigned manuscripts to open versus blinded review in a controlled trial
  • Why making reviewer reports public changes reviewer behavior, according to a 2019 study across five journals

Common misconceptions

  • “Double-blind review and single-blind review produce roughly the same outcomes, so the distinction barely matters.” A 2017 study analysing reviewer bidding behaviour at a major computer science conference found that reviewers under single-blind review were significantly more likely than reviewers under double-blind review to bid to review papers from famous authors and well-known institutions, a measurable difference in behaviour directly associated with the model used.

Large-scale evidence on single-blind and double-blind review

Andrew Tomkins, Min Zhang and William Heavlin conducted a large experiment at the Web Search and Data Mining (WSDM) conference, randomly assigning submitted papers to review under either single-blind or double-blind conditions and measuring reviewer behaviour under each1. They found that single-blind reviewers were significantly more likely than double-blind reviewers to choose to review papers written by famous authors, papers from top institutions and papers from top companies, and were more likely to recommend acceptance for those same categories of papers independently of the papers’ actual content1. This constitutes a different and more direct form of evidence than studies that examine only whether blinding changes a reviewer’s final numerical score; it shows that blinding affects which papers reviewers choose to engage with in the first place, before any substantive judgement about quality is made. A related line of earlier research examined the effect of double-blind review on author demographics specifically: a 2008 analysis found that journals adopting double-blind review saw an increase in the representation of female first authors among published papers, a pattern the authors linked to reduced identity-based bias in the review process2. Both studies share a methodological strength: they measured an outcome (whom reviewers chose to review or recommend, and who was ultimately published) rather than relying on reviewers’ self-reports of whether their judgements were influenced by author identity, which constitutes substantially stronger evidence than a survey of reviewers’ stated impressions of their own objectivity. It also implies that the documented bias is not necessarily a matter of reviewers consciously favouring famous names; a more charitable and equally consistent explanation is that reputation acts as an unconscious heuristic for expected quality, which double-blind conditions interrupt by removing the information needed to apply it, irrespective of whether an individual reviewer would endorse that heuristic if asked directly. This is also why structural changes to the review process, rather than appeals to the conscious intentions of individual reviewers, have been the more consistently effective intervention documented in this body of research.

A randomized trial of open peer review at the BMJ

Separately from the question of single-blind versus double-blind review, the BMJ conducted a randomised controlled trial specifically testing open peer review, in which reviewers’ identities were revealed to authors, against standard anonymous review, measuring the quality of the resulting reviews and the reviewers’ recommendations3. A companion randomised trial, conducted largely by the same research group, compared review quality and reviewers’ recommendations directly between open and blinded conditions and found that revealing reviewer identity did not meaningfully reduce review quality, addressing a common concern that reviewers would be more reluctant to provide critical feedback if their names were attached to it4. Fiona Godlee, writing on the rationale for the eventual move of the BMJ to full open review, argued that the change concerned making reviewers visible, accountable and properly credited for their work, presenting transparency as a value in its own right rather than solely as a means of detecting reviewer misconduct5. A systematic review of the open peer review literature by Tony Ross-Hellauer catalogued the many distinct practices grouped under the single label “open peer review” (open identities, open reports, open participation and open interaction), finding sufficient variation across journals that the term alone does not specify a single, well-defined model6.

Effects of publishing reviewer reports

A 2019 study by Giangiacomo Bravo and colleagues examined specifically the effect on reviewer behaviour of making the review reports themselves, and not only reviewer names, public, analysing data from five journals that had implemented this practice7. The study found that publishing review reports did not reduce reviewers’ willingness to accept review invitations and had no detectable negative effect on review quality, contrary to a common assumption that reviewers would decline more often or write more superficial, less critical reports if they knew that their reports would become a permanent, citable public record7. This finding is broadly consistent with a separate international survey of researcher attitudes towards peer review, which found that although many researchers express reservations about full transparency, actual behaviour under transparent systems, measured directly rather than through stated preference, has tended to be more favourable than those reservations would predict8.

Divergence between editorial gatekeeping and subsequent scholarly impact

None of the review models discussed here is free of documented risk of bias, which is itself an important finding. A broad review of bias in peer review by Lee, Sugimoto, Zhang and Cronin catalogued evidence of bias associated with author prestige, institutional affiliation, nationality and gender across review models generally, concluding that no single model has been shown to eliminate bias entirely, only to shift its specific sources and visibility9. A separate, large analysis of acceptance patterns by Kyle Siler, Kirby Lee and Lisa Bero examined how papers that subsequently became highly cited had, in a substantial number of cases, first been rejected by one or more journals, sometimes repeatedly, before eventually being published and recognised10. Their analysis found that the judgements of reviewers and editors at the time of submission correlated only moderately with the subsequent citation and recognition of the same papers by the field, a divergence the authors attribute partly to the inherent difficulty of judging the eventual importance of a paper from a single reading at submission, and partly to the same prestige- and novelty-related biases documented elsewhere in the peer review literature, irrespective of the review model (single-blind, double-blind or open) in use at a given journal at the time10. This finding cautions against treating any single rejection, or the gatekeeping decision of any single model, as a definitive judgement of the underlying value of a paper.

Implications for the selection of journals and review models

For an author selecting a target journal, or a reviewer considering how to approach an assignment, the practical implication of this body of work is that the review model in use (single-blind, double-blind or open) measurably alters specific behaviours and risks; verifying which model a target journal uses, rather than assuming that “peer-reviewed” denotes one uniform process, is therefore a reasonable and evidence-based step. A rejection under any model is not, by itself, a reliable judgement of the ultimate scholarly value of a paper, which should inform the weight an author places on a single negative decision relative to seeking a second, independent assessment at a different venue. None of the studies reviewed here suggests that bad faith on the part of reviewers or editors is the typical explanation for these documented differences in outcome; the more consistent picture across the WSDM bidding study, the open-review trials of the BMJ and the citation-impact analysis is of ordinary, well-intentioned reviewers operating under the specific incentive structure and information constraints that each review model creates. This is precisely why changing the model itself, rather than asking individual reviewers to make greater efforts to be unbiased, has been the more empirically supported means of changing outcomes.

Key points

Key points

Single-blind, double-blind and open peer review are not interchangeable in practice. Controlled studies have measured real differences in reviewer bidding behavior, bias related to author prestige and reviewer willingness; the specific model used by a journal should therefore be verified rather than assumed.

Practical application

Practical application

  1. The specific review model used by a target journal should be checked before submission. The designation ‘peer-reviewed’ should not be assumed to imply a single standard process; single-blind, double-blind and open review have each been shown to produce measurably different reviewer behavior.
  2. Where a choice is available, double-blind review may be considered when prestige bias is a plausible concern. The evidence on reviewer bidding and recommendation behavior specifically ties single-blind review to increased favoritism toward well-known authors and institutions.
  3. Reviewers should not assume that transparency will require more lenient assessment. Studies of published review reports found no detectable decline in review quality or willingness to review under more transparent conditions, a finding relevant to reviewers’ assessment of open models.

Frequently asked questions

Identities are guessed in a meaningful proportion of cases, but the WSDM study measured behavioural differences (which papers bidding reviewers chose to review and how they rated famous relative to unknown authors) that remained statistically significant even with imperfect blinding, indicating that the model still alters outcomes when anonymity is incomplete.

No. A systematic review of open peer review practices found substantial variation in what is made open (reviewer identity, the review report itself, reader participation), so the specific practices of a given journal should be verified rather than inferred from the general label.

The available evidence, specifically a study of five journals that published review reports, found no reduction in reviewers’ willingness to accept review invitations under that more transparent condition.

The research reviewed here does not establish a single best model; it shows that each model alters specific, measurable behaviours and patterns of bias in different ways, so the more relevant question for a given journal or field is which specific risks are most important to address.

Yes. The documented increase in female first authorship following the adoption of double-blind review by a journal is one concrete, measured example of a change in review model being associated with a shift in the demographics of published authors.

References

  1. Tomkins A, Zhang M, Heavlin WD. Reviewer bias in single- versus double-blind peer review. Proceedings of the National Academy of Sciences. 2017;114(48):12708-12713. read 10 October 2026. View on PNAS
  2. Budden AE, Tregenza T, Aarssen LW, Koricheva J, Leimu R, Lortie CJ. Double-blind review favours increased representation of female authors. Trends in Ecology & Evolution. 2008;23(1):4-6. read 10 October 2026. View on Trends in Ecology & Evolution
  3. van Rooyen S, Godlee F, Evans S, Smith R, Black N. Effect of open peer review on quality of reviews and on reviewers' recommendations: a randomised trial. BMJ. 1999;318(7175):23-27. read 10 October 2026. View on BMJ
  4. Walsh E, Rooney M, Appleby L, Wilkinson G. Open peer review: a randomised controlled trial. British Journal of Psychiatry. 2000;176(1):47-51. read 10 October 2026. View on British Journal of Psychiatry
  5. Godlee F. Making reviewers visible: openness, accountability, and credit. JAMA. 2002;287(21):2762-2765. read 10 October 2026. View on JAMA
  6. Ross-Hellauer T. What is open peer review? A systematic review. F1000Research. 2017;6:588. read 10 October 2026. View on F1000Research
  7. Bravo G, Grimaldo F, López-Iñesta E, Mehmani B, Squazzoni F. The effect of publishing peer review reports on referee behavior in five scholarly journals. Nature Communications. 2019;10:322. read 10 October 2026. View on Nature Communications
  8. Mulligan A, Hall L, Raphael E. Peer review in a changing world: An international study measuring the attitudes of researchers. Journal of the American Society for Information Science and Technology. 2013;64(1):132-161. read 10 October 2026. View on JASIST
  9. Lee CJ, Sugimoto CR, Zhang G, Cronin B. Bias in peer review. Journal of the American Society for Information Science and Technology. 2013;64(1):2-17. read 10 October 2026. View on JASIST
  10. Siler K, Lee K, Bero L. Measuring the effectiveness of scientific gatekeeping. Proceedings of the National Academy of Sciences. 2015;112(2):360-365. read 10 October 2026. View on PNAS

Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.

How to cite this article

Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.

Pubrica Academy. (2026, October 10). Peer Review Models in Scholarly Publishing: A Review of the Evidence. Pubrica. https://pubrica.com/academy/concepts-definitions/peer-review-of-manuscripts-a-important-aspect-of-scholarly-publishing/

This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.

Review Model Awareness Checklist

For use before submitting to, or reviewing for, a journal whose review model has not yet been verified.

[ ] Confirmed whether the target journal uses single-blind, double-blind, or open review
[ ] Checked what 'open' specifically means at that journal if applicable (identities, reports, both)
[ ] If given a choice of venue and prestige bias is a concern, considered a double-blind option
[ ] As a reviewer, based recommendations on manuscript content rather than author reputation
[ ] Noted that no model has been shown to eliminate bias entirely, only to shift its sources

Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.

How Pubrica can help

Pubrica’s publication advisory team verifies a target journal’s specific review model and practices before submission, so that authors understand the type of scrutiny, and the associated risk, to which their manuscript will be subject.

Find something else

Search the Pubrica Academy

Access is free and requires no account. Enter a topic, or open the full library to filter by category, sub-topic or resource type.

Identifying the peer review model used by a target journal

A scoping assessment is free and takes two business days. It provides a written assessment of what the work requires and which specialist would undertake it.

Get a free quote

Regulations

Check your project against this rule

  1. 1Your work
  2. 2Your details
  3. 3Send
What do you need?

Material you send is held under signed confidentiality, with access limited to the assigned team and encrypted transfer. It is not used to train anything and is not passed to third-party tools. Ask for the agreement before you send anything you would rather cover first.