Contents
On this page
Academy · Research Data, Analytics & AI · Impact Factor: The Case Against Misuse
The Tyranny of the Journal Impact Factor: Citation Skew and the Case for Responsible Assessment
A 1997 paper in the BMJ showed that between 25% and 50% of a journal’s articles account for essentially all of its citations, and a 2012 international declaration signed by thousands of researchers and institutions explicitly calls for the impact factor to be excluded from hiring, funding and promotion decisions. This article examines the statistical basis of that position and its practical implications.
The journal impact factor was designed by Eugene Garfield as a tool to help librarians decide which journals to subscribe to, providing a means of comparing the average citation rates of journals’ recent output. It was never intended to evaluate individual researchers, individual papers or grant applicants, yet this has become its predominant use. The term ‘tyranny’ refers not to the metric itself but to this mismatch: an average constructed to compare the subscription value of journals is used as a proxy for the quality of the researchers and papers it aggregates, a use that its statistical structure cannot support.
Learning objectives
- Why the citations of a journal are usually dominated by a small proportion of its articles, which undermines any average-based metric
- What the 2012 San Francisco Declaration on Research Assessment formally recommends, and why thousands of organisations have signed it
- How the use of the impact factor for journal-level comparison differs from its use to judge an individual paper or researcher
Common misconceptions
- “The impact factor of a journal indicates the likely quality of any given paper published in it.” Per-Olav Seglen’s 1997 analysis in the BMJ found that between 25% and 50% of a journal’s cited articles typically account for roughly 90% of the citations the journal receives, meaning that the average reported as the impact factor is driven by a small minority of the journal’s papers rather than by a typical one.
The uneven distribution of citations within a journal
The 1997 paper by Per-Olav Seglen in the BMJ, “Why the impact factor of journals should not be used for evaluating research,” remains one of the most direct statistical critiques of the metric. Seglen showed that citation counts across a journal’s articles are not distributed around the journal’s average but are heavily skewed, with a minority of articles accounting for the great majority of citations: typically, 25% to 50% of a journal’s cited papers account for approximately 90% of the citations that contribute to its impact factor1. Because of this skew, a journal with a high impact factor can, and routinely does, publish individual papers that are subsequently rarely cited, while a journal with a lower impact factor can publish a paper that becomes highly cited; the journal-level average cannot reliably predict either outcome for a single article1. Seglen also noted that the impact factor varies substantially by field, owing to differences in average reference-list length and publication rates across disciplines, which renders cross-field comparisons based on the raw figure statistically unsound irrespective of the skew1.
The 2012 declaration against misuse of the impact factor
These structural problems, compounded by years of documented misuse in hiring and funding decisions, led a group of editors and publishers meeting at the 2012 annual meeting of the American Society for Cell Biology to draft the San Francisco Declaration on Research Assessment, now known as DORA2. The central recommendation of DORA is explicit: funding agencies, institutions and publishers should not use journal-based metrics such as the impact factor as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion or funding decisions2. According to the organisation’s own public records, DORA has been signed by tens of thousands of individuals and thousands of research institutions, publishers and funders worldwide, making it one of the most widely endorsed documents on research-assessment reform in the sciences2. The 2007 editorial by Rossner, Van Epps and Hill in the Journal of Cell Biology, published before DORA but frequently cited in the same debate, independently documented how the mechanics of impact-factor calculation, including which articles are classified as “citable” for the denominator, can shift a journal’s published figure by several points without any real change in how often its research is cited3.
Appropriate uses of the impact factor
This does not mean that the impact factor is meaningless, only that it is being used to answer a question it was not designed to answer. The 2012 review by Vanclay in Scientometrics argues that the impact factor retains some legitimate use as one input into a librarian’s subscription decisions, or as a very approximate aggregate signal when journals of similar type and field are compared, provided that it is read alongside other journal-level indicators rather than in isolation4. The 2014 commentary by Casadevall and Fang in mBio describes the practical consequence of ignoring this distinction as “impact factor mania,” a self-reinforcing cycle in which authors select journals by impact factor, institutions reward publication in those journals irrespective of an individual paper’s citation performance, and journals in turn manage their published figure through editorial policy rather than through the quality of what they publish5. The remedy recommended across this literature is consistent: the impact factor should be used, if at all, only to compare journals of similar scope at the journal level, and individual papers and researchers should be evaluated on the content and citation record of their own work.
Alternative metrics and their limitations
The response to the weaknesses of the impact factor has not been to abandon citation-based evaluation altogether but to move towards article-level and more statistically robust journal-level alternatives, each with its own documented limitations. CiteScore, the journal metric of Scopus, extends the citation window to four years rather than two and counts a broader range of document types, which reduces some of the single-paper volatility that Seglen described but does not eliminate the underlying skew, since citations within a four-year window can still be dominated by a small number of papers4. The SCImago Journal Rank and the Eigenfactor weight citations by the prestige of the citing journal rather than treating every citation equally, a refinement intended to make the metric more resistant to manipulation through citation stacking among lower-tier journals; however, Vanclay’s review notes that this introduces a circularity of its own, since a journal’s weighted prestige depends partly on the prestige rankings of other journals, which are themselves derived from the same kind of citation data4. At the article level, Larivière and colleagues and other bibliometricians have urged publishers to report the full distribution of citations across a journal’s articles rather than a single average, so that readers can see directly how skewed that distribution is for a given journal rather than inferring it from Seglen’s general finding; several major publishers, including some DORA signatories, now publish these distributions voluntarily2. None of these alternatives resolves the core problem identified by DORA, namely that any journal-level number, however calculated, describes the journal and not the specific paper or researcher under evaluation; for this reason, the central recommendation of DORA is a change in the evidence used, not an improved version of the same kind of metric.
Documented management of impact factors by journals
Beyond the passive statistical artefacts described by Seglen and the Journal of Cell Biology editorial, the mBio commentary by Casadevall and Fang documents more deliberate editorial practices intended to raise a journal’s published impact factor rather than to improve the research it publishes. These include coercive citation, in which an editor asks an author to add citations to the journal’s own recent papers as a condition of acceptance, a practice that directly inflates the numerator of the journal’s future impact factor; the publication of a disproportionate number of review articles, which tend to attract more citations than original research articles of equivalent quality, raising a journal’s overall citation average without a corresponding change in its original research output; and the reclassification of content such as editorials or correspondence as non-citable items, which, through the mechanism the Journal of Cell Biology editorial documented in Current Biology, reduces the denominator and raises the ratio independently of any change in how often the journal’s substantive articles are cited5. These practices are not universal, and most journals do not engage in them; nevertheless, their documented existence forms part of the case against treating a published impact factor as an unmanipulated signal of quality. The figure is calculated from inputs that a journal’s own editorial choices can influence, which is a different kind of problem from the statistical skew Seglen identified, and one that compounds rather than offsets it.
Alternative questions for funding and hiring panels
The practical recommendations of DORA for institutions go beyond discontinuing the use of the impact factor and specify what evaluators should consider instead: a clear and specific account of the applicant’s own contribution to each cited work, direct examination of a small number of the applicant’s most significant papers on their own merits, and consideration of a broader set of outputs and impacts, including data, software, mentorship and policy influence, rather than restricting “impact” to journal-level citation counts alone2. This approach to evaluation is more labour-intensive than reading the journal names listed in a curriculum vitae, and the apparent convenience of the impact factor, a single number that seems to summarise quality without requiring a panel member to read the paper, is part of what has sustained its use long after its statistical limitations were documented. The commentary by Casadevall and Fang makes this incentive problem explicit: impact factor mania persists not because evaluators are unaware of the metric’s flaws, which have been published and discussed for decades, but because reading and judging individual papers on their substantive merits requires time and expertise for which a single summary number appears to substitute, even though DORA and the broader methodological literature are clear that it cannot do so reliably5.
Key points
Key points
The impact factor of a journal is a skewed average dominated by a minority of its articles rather than a prediction about any single paper, and the 2012 DORA declaration, now signed by tens of thousands of researchers and institutions, formally recommends against its use to judge individual papers, researchers or funding applications.
Practical application
Practical application
- A specific paper should be evaluated by its own citation record rather than by the impact factor of its journal. The citation count of the individual article should be consulted directly rather than its quality inferred from the venue of publication.
- Members of hiring, promotion and grant panels should compare institutional policy with the recommendations of DORA. Many funders and universities have formally signed DORA; panels should confirm whether their evaluation criteria nevertheless default to the impact factor.
- The impact factor should be used, if at all, only to compare journals of similar scope. It should be treated as one weak signal among several (scope fit, indexing and rigour of peer review) rather than as a standalone decision criterion.
Frequently asked questions
Eugene Garfield designed it as a tool to help librarians compare the average citation rates of journals when making subscription decisions, not as a means of evaluating individual papers, researchers or grant applicants.
Because citations within a journal are heavily skewed: Seglen’s 1997 analysis found that 25% to 50% of a journal’s articles typically account for approximately 90% of its citations, so the average conceals far more than it reveals about any single paper.
The central recommendation of DORA is that funders, institutions and publishers should stop using journal-based metrics such as the impact factor as a proxy for the quality of individual articles or researchers in hiring, promotion and funding decisions.
Vanclay’s 2012 review argues that it retains limited legitimate use for comparing journals of similar type and field, as one input among several, but not as a standalone measure of the quality of an individual paper or researcher.
Yes. The analyses by Seglen and by Rossner, Van Epps and Hill document cases in which denominator reclassification or citation concentration shifted the published figure without a corresponding change in the performance of the journal’s typical paper.
References
- Seglen PO. Why the impact factor of journals should not be used for evaluating research. BMJ. 1997;314(7079):498-502. read 10 October 2026. View on BMJ
- San Francisco Declaration on Research Assessment (DORA). The Declaration. read 10 October 2026. View on DORA
- Rossner M, Van Epps H, Hill E. Show me the data. Journal of Cell Biology. 2007;179(6):1091-1092. read 10 October 2026. View on Journal of Cell Biology
- Vanclay JK. Impact factor: outdated artefact or stepping-stone to journal certification? Scientometrics. 2012;92(2):211-238. read 10 October 2026. View on Scientometrics
- Casadevall A, Fang FC. Causes for the persistence of impact factor mania. mBio. 2014;5(2):e00064-14. read 10 October 2026. View on mBio
- Seglen PO. Why the impact factor of journals should not be used for evaluating research. BMJ. 1997;314(7079):498-502. read 10 October 2026. View on BMJ
- San Francisco Declaration on Research Assessment (DORA). The Declaration. read 10 October 2026. View on DORA
- PLOS Medicine Editors. The impact factor game. PLoS Med. 2006;3(6):e291. read 10 October 2026. View on PLOS Medicine
- Rossner M, Van Epps H, Hill E. Show me the data. Journal of Cell Biology. 2007;179(6):1091-1092. read 10 October 2026. View on Journal of Cell Biology
- Casadevall A, Fang FC. Causes for the persistence of impact factor mania. mBio. 2014;5(2):e00064-14. read 10 October 2026. View on mBio
Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.
How to cite this article
Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.
Pubrica Academy. (2026, October 10). The Tyranny of the Journal Impact Factor: Citation Skew and the Case for Responsible Assessment. Pubrica. https://pubrica.com/academy/study-guide/publication/tyranny-journal-impact-factor/
Pubrica Academy. “The Tyranny of the Journal Impact Factor: Citation Skew and the Case for Responsible Assessment.” Pubrica, 10 Oct. 2026, pubrica.com/academy/study-guide/publication/tyranny-journal-impact-factor/.
Pubrica Academy (2026) The Tyranny of the Journal Impact Factor: Citation Skew and the Case for Responsible Assessment. Pubrica. Available at: https://pubrica.com/academy/study-guide/publication/tyranny-journal-impact-factor/ (Accessed: 15 October 2026).
Pubrica Academy. “The Tyranny of the Journal Impact Factor: Citation Skew and the Case for Responsible Assessment.” Pubrica. October 10, 2026. https://pubrica.com/academy/study-guide/publication/tyranny-journal-impact-factor/
Pubrica Academy, ‘The Tyranny of the Journal Impact Factor: Citation Skew and the Case for Responsible Assessment’ (Pubrica, 10 October 2026) <https://pubrica.com/academy/study-guide/publication/tyranny-journal-impact-factor/> accessed 15 October 2026.
Pubrica Academy. The Tyranny of the Journal Impact Factor: Citation Skew and the Case for Responsible Assessment. [Internet]. Pubrica; 2026 Oct 10 [cited 2026 Oct 15]. Available from: https://pubrica.com/academy/study-guide/publication/tyranny-journal-impact-factor/
Pubrica Academy, “The Tyranny of the Journal Impact Factor: Citation Skew and the Case for Responsible Assessment,” Pubrica, Oct. 10, 2026. [Online]. Available: https://pubrica.com/academy/study-guide/publication/tyranny-journal-impact-factor/
@misc{/la_tyranny_journal_impact_factor_2026,
author = {{Pubrica Academy}},
title = {The Tyranny of the Journal Impact Factor: Citation Skew and the Case for Responsible Assessment},
howpublished = {Pubrica},
year = {2026},
month = {oct},
url = {https://pubrica.com/academy/study-guide/publication/tyranny-journal-impact-factor/},
note = {Accessed: <span data-cite-acc="iso">2026-10-15</span>}
}This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.
Impact-Factor Misuse Self-Audit
This checklist may be used to determine whether an evaluation practice relies inappropriately on the impact factor.
[ ] Checked whether your hiring/promotion/funding criteria name impact factor directly as a decision factor [ ] Compared those criteria against DORA's published recommendations [ ] For any specific paper under review, pulled its own citation count rather than inferring from its journal [ ] If comparing journals, limited the comparison to journals of similar scope and field [ ] Checked whether your institution or funder has formally signed DORA
Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.
How Pubrica can help
Pubrica’s research data and bibliometric advisory service assists institutions and researchers in developing evaluation criteria consistent with the recommendations of DORA, replacing journal-level proxies with article-level and researcher-level evidence.