Contents
On this page
- Effects of a single paper or reclassification on the impact factor
- Uneven distribution of citations within a journal
- Citation stacking and self-citation as sources of volatility
- Quantified effects of a single paper across journals
- Suppression of impact factors by indexers
- Alternative metrics and sources of disagreement
- Reform initiatives beyond a single metric
- Interpreting changes in the impact factor
Academy · Research Data, Analytics & AI · Impact Factor Variation
Temporal Variation of the Journal Impact Factor: A Submission and Citation-Based Analysis
A single highly cited paper, a reclassified set of front-matter articles or one conference issue can change a journal’s impact factor by several points within a single citation cycle. This indicates that year-to-year change in the metric often reflects counting mechanics rather than a genuine change in quality.
The Journal Impact Factor (JIF) is calculated as the number of citations a journal receives in a given year to items it published in the two preceding years, divided by the number of those items counted as “citable.” Although the ratio appears precise, both its numerator and its denominator are more sensitive than a single decimal figure suggests. Because the window spans only two years and the denominator for many journals ranges from a few hundred to a few thousand articles, a small number of unusual events (a single outlier paper, a reclassification of article types, or a concentrated burst of citations from a related special issue) can shift the published figure by an amount unrelated to the journal's average quality in that year.
Learning objectives
- A documented case in which a journal’s impact factor rose sharply because its citable-item denominator fell, not because citations increased
- How citations concentrated in a small proportion of a journal’s papers can dominate its impact factor
- A case in which citation stacking inflated one journal’s impact factor by approximately seven times, depending on the citation window used
Common misconceptions
- “A journal’s impact factor is a stable and precise year-on-year measure of its quality.” Editors at the Journal of Cell Biology publicly documented how the reclassification of articles changed citable-item counts sufficiently to alter journal rankings, and a separate bibliometric audit found that a journal’s impact factor fell from 8.91 to 1.27 when the same citations were measured over a different time window.
Effects of a single paper or reclassification on the impact factor
In 2007, the editors of the Journal of Cell Biology published an editorial, "Show me the data," after purchasing the underlying citation data Thomson Scientific used to calculate impact factors for their own journal and several competitors1. They documented that Current Biology's impact factor rose from 7.00 in 2002 to 11.91 in 2003, although the journal published more articles overall, because the number of items classified as “citable” for the denominator fell from 1,032 to 634 over the same period1. The rise in the published ratio therefore reflected a change in how articles were classified for the denominator rather than a change in how often the journal's research was cited. The same editorial reported that, when the editors purchased and recalculated citation totals for their own journals from Thomson Scientific's data, the counts differed from the figures published on the Journal Citation Reports website by as much as 19% for some journals, a discrepancy large enough to alter relative rankings1.
Uneven distribution of citations within a journal
The same 2007 editorial reported that, in Nature's own analysis of its 2005 impact factor, 89% of the citations counted toward the figure came from only 25% of the papers Nature had published in the relevant window1. A mean-based metric calculated over such a distribution is dominated by a small number of unusually well-cited papers and does not reflect the typical paper a journal publishes. The editors also cited the original human genome sequencing paper in Nature, which had accumulated 5,904 citations by November 2007, as an illustration of how a single landmark paper can anchor a journal's citation average for years after its own two-year citation window has formally closed, through the field's enduring practice of citing it1.
Citation stacking and self-citation as sources of volatility
A separate bibliometric analysis by mathematicians Douglas Arnold and Kristine Fowler examined how citation patterns inflated the impact factor of the International Journal of Nonlinear Science and Numerical Simulation (IJNSNS)2. They found that in 2008, 71.5% of all citations IJNSNS received fell within the narrow two-year window used to calculate its impact factor, compared with only 16% for a comparison journal (CPAM) and 8% for another (SIREV)2. A single issue of a conference proceedings journal linked to IJNSNS's editor-in-chief supplied 294 citations, more than 20% of the total counted toward the impact factor, and a special issue of a different journal supplied a further 2062. The authors estimated that citations connected to the journal's own editors accounted for more than 70% of the citations counted, whereas conventional self-citation by authors of their own prior work accounted for a comparatively small 7%2. When the authors recalculated the impact factor using a 2000-2005 citation window instead of the standard two-year window, IJNSNS's figure fell from 8.91 to 1.27, whereas the comparison journals remained approximately stable or rose2.
Quantified effects of a single paper across journals
The Current Biology and IJNSNS cases are not isolated instances; a 2019 analysis by Michael Antonoyiannakis quantified the same mechanism across the entire 2017 Journal Citation Reports dataset, covering 11,639 journals and approximately 3.09 million papers published in 2015-20163. The study found that a journal's single most-cited paper raised its impact factor by more than 0.5 points at 381 journals and by more than 25% at 818 journals, and that for 1,292 journals, the top three cited papers combined inflated the impact factor by more than half3. The most extreme documented case was CA: A Cancer Journal for Clinicians, where a single 2016 paper cited 3,790 times in 2017 raised the journal's impact factor by 68.3 points, from approximately 171.8 to 240.13. For the Journal of Statistical Software, one paper cited 2,708 times accounted for 73% of the journal's citations that year; without it, the impact factor would have fallen from 21.6 to 5.83. The study also found that the mechanism affects small journals most strongly: a hypothetical paper cited 100 times would raise the impact factor of a 50-article journal by 1.902 points but that of a 50,000-article journal by only 0.0019 points, which explains why a journal with very few citable items in the two-year window is structurally the most volatile3.
Suppression of impact factors by indexers
Citation manipulation has become sufficiently prevalent that Clarivate, the company that publishes the Journal Citation Reports, now routinely withholds impact factors from journals it flags for “anomalous citation behavior,” a category it defines to include excessive journal self-citation and citation stacking among a small number of journals8. In its 2024 Journal Citation Reports, Clarivate suppressed the impact factor of 17 journals on these grounds, up from four the year before8. Documented examples from that round include the Ukrainian Journal for Physical Optics, where the editor confirmed that 46% of its 2023 citations came from a single other journal, Optik, and Information Sciences, which Clarivate suppressed for one year after flagging anomalous citation patterns in twelve of its 2023 articles8. The mechanism underlying a citation cartel, a group of journals or authors that cite one another at a disproportionately high rate relative to the rest of the field, was formally described in bibliometric network terms by Fister, Fister, and Perc, who note that the practice originated as an inter-journal arrangement among editors before extending to editor-author and author-author patterns4. A separate large-scale analysis by Chorus and Waltman, comparing journal self-citation to recent versus older papers across the sciences, found that the proportion of journals showing an unusually high self-citation ratio remained approximately stable from the 1980s through the early 2000s before rising sharply across nearly every field thereafter5.
Alternative metrics and sources of disagreement
Because the Journal Impact Factor's two-year window and its reliance solely on Web of Science citation data make it unusually sensitive to the volatility documented above, several alternative metrics are designed to correct for different aspects of the problem, and each is calculated differently. Elsevier's CiteScore, drawn from the Scopus database rather than Web of Science, measures citations to a journal's output against the number of documents it published, providing a broader denominator and a different, larger pool of indexed journals than the Journal Citation Reports (JCR) covers6. SCImago Journal Rank (SJR), also based on Scopus data, additionally weights each citation according to the prestige of the citing journal, so that a citation from a highly cited source counts for more than one from an obscure outlet; this directly addresses the type of citation-stacking problem documented in the IJNSNS case10. Larivière and Sugimoto's historical review of the Impact Factor notes that, because citation counts within any journal are heavily right-skewed, typically only around one-third of a journal's own articles reach or exceed its published impact factor value; the “average” that the metric reports therefore does not describe a typical paper even in a journal without anomalies, independently of any manipulation9.
Reform initiatives beyond a single metric
The cumulative weight of these documented problems (denominator sensitivity, concentrated citation, self-citation and cartel inflation, and single-paper volatility) motivated a formal reform movement rather than only individual critiques. The San Francisco Declaration on Research Assessment (DORA), coordinated in 2013 by the American Society for Cell Biology and documented by Cagan in Disease Models & Mechanisms, sets out 18 recommendations directed at funders, institutions, publishers and researchers, the central recommendation being to cease using the journal-level impact factor as a proxy for the quality of an individual article or researcher in hiring, promotion, or funding decisions7. More than 150 scientists and 75 scientific organizations signed the declaration at its launch, and it has since been cited as a reference point by journals and funders revising their own evaluation policies7. DORA does not contend that citation data are meaningless; rather, it holds that a single two-year average, given how readily it shifts, is the wrong level at which to apply such data to the work of an individual.
Interpreting changes in the impact factor
Both cases support the same practical conclusion: a single year's impact factor, considered in isolation, cannot distinguish a genuine shift in a journal's standing from a denominator reclassification, a landmark paper, or concentrated editor-linked citation12. Responsible interpretation of the metric therefore requires examining more than one year of data, checking whether a journal's citation counts agree across sources, and treating a sudden increase as a prompt for investigation rather than as self-evident proof of improved quality.
Key points
Key points
A single year’s change in impact factor should be treated as a potential measurement artefact requiring investigation, rather than as evidence of quality, until the possibility of an outlier paper, a denominator reclassification or citation stacking has been examined.
Practical application
Practical application
- At least three years of impact factor data should be obtained for any journal under evaluation. Single-year fluctuations are common; a multi-year trend is considerably more informative than one annual figure considered in isolation.
- The citation window should be examined for outlier papers. The two most recent citable-item years of the journal should be searched for any single paper with a disproportionate number of citations relative to the rest of the issue.
- The figure should be cross-checked against a second metric or source. The published Journal Impact Factor should be compared with CiteScore or the journal’s own reported citation data to determine whether the pattern is consistent across methods.
Frequently asked questions
Their citable-item denominators are smaller, so the same absolute change in citations or in the count of articles classified as citable moves the ratio proportionally further, which is the mechanism documented in the Journal of Cell Biology editorial's Current Biology example.
Yes. Both the Journal of Cell Biology editorial and the IJNSNS analysis document real cases in which a rising or inflated impact factor reflected denominator changes or concentrated editor-linked citation rather than a broader improvement in the journal's typical paper.
Not necessarily. Clarivate states explicitly that suppression reflects anomalous citation behavior and does not presume motive or accuse a journal of wrongdoing; some 2024 cases involved a small number of flagged articles or a narrow subfield with few other journals to cite, rather than deliberate manipulation.
They draw on different citation databases (Scopus versus Web of Science) with different journal coverage, use different citation windows, and in SJR's case weight citations by the citing journal's own prestige rather than counting them equally, so disagreement reflects different measurement choices rather than an error in any one figure.
References
- Rossner M, Van Epps H, Hill E. Show me the data. Journal of Cell Biology. 2007;179(6):1091-1092. read 10 October 2026. View on Journal of Cell Biology
- Arnold DN, Fowler KK. Nefarious Numbers. arXiv:1010.0278 [math.HO]. 2011. read 10 October 2026. View on arXiv
- Antonoyiannakis M. How a Single Paper Affects the Impact Factor: Implications for Scholarly Publishing. Proceedings of the 17th International Conference of the International Society for Scientometrics and Informetrics (ISSI 2019), Vol. II:2306-2313. read 10 October 2026. View on arXiv
- Fister I Jr, Fister I, Perc M. Toward the Discovery of Citation Cartels in Citation Networks. Frontiers in Physics. 2016;4:49. read 10 October 2026. View on Frontiers in Physics
- Chorus C, Waltman L. A Large-Scale Analysis of Impact Factor Biased Journal Self-Citations. PLOS ONE. 2016;11(8):e0161021. read 10 October 2026. View on PLOS ONE
- Elsevier. What is CiteScore? Scopus support documentation. read 10 October 2026. View on Elsevier Support
- Cagan R. The San Francisco Declaration on Research Assessment. Disease Models & Mechanisms. 2013;6(4):869-870. read 10 October 2026. View on Disease Models & Mechanisms
- Oransky I, Marcus A. Seventeen journals lose impact factors for suspected citation manipulation. Retraction Watch. 2024. read 10 October 2026. View on Retraction Watch
- Larivière V, Sugimoto CR. The Journal Impact Factor: A brief history, critique, and discussion of adverse effects. In: Springer Handbook of Science and Technology Indicators. Springer; 2018. read 10 October 2026. View on arXiv
- SCImago Journal & Country Rank. SCImago Journal Rank (SJR) indicator. read 10 October 2026. View source
Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.
How to cite this article
Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.
Pubrica Academy. (2026, October 10). Temporal Variation of the Journal Impact Factor: A Submission and Citation-Based Analysis. Pubrica. https://pubrica.com/academy/concepts-definitions/submission-and-citation-based-study-on-temporal-variation-of-journal-impact-factor/
Pubrica Academy. “Temporal Variation of the Journal Impact Factor: A Submission and Citation-Based Analysis.” Pubrica, 10 Oct. 2026, pubrica.com/academy/concepts-definitions/submission-and-citation-based-study-on-temporal-variation-of-journal-impact-factor/.
Pubrica Academy (2026) Temporal Variation of the Journal Impact Factor: A Submission and Citation-Based Analysis. Pubrica. Available at: https://pubrica.com/academy/concepts-definitions/submission-and-citation-based-study-on-temporal-variation-of-journal-impact-factor/ (Accessed: 15 October 2026).
Pubrica Academy. “Temporal Variation of the Journal Impact Factor: A Submission and Citation-Based Analysis.” Pubrica. October 10, 2026. https://pubrica.com/academy/concepts-definitions/submission-and-citation-based-study-on-temporal-variation-of-journal-impact-factor/
Pubrica Academy, ‘Temporal Variation of the Journal Impact Factor: A Submission and Citation-Based Analysis’ (Pubrica, 10 October 2026) <https://pubrica.com/academy/concepts-definitions/submission-and-citation-based-study-on-temporal-variation-of-journal-impact-factor/> accessed 15 October 2026.
Pubrica Academy. Temporal Variation of the Journal Impact Factor: A Submission and Citation-Based Analysis. [Internet]. Pubrica; 2026 Oct 10 [cited 2026 Oct 15]. Available from: https://pubrica.com/academy/concepts-definitions/submission-and-citation-based-study-on-temporal-variation-of-journal-impact-factor/
Pubrica Academy, “Temporal Variation of the Journal Impact Factor: A Submission and Citation-Based Analysis,” Pubrica, Oct. 10, 2026. [Online]. Available: https://pubrica.com/academy/concepts-definitions/submission-and-citation-based-study-on-temporal-variation-of-journal-impact-factor/
@misc{/la_journal_impact_factor_temporal_variation_2026,
author = {{Pubrica Academy}},
title = {Temporal Variation of the Journal Impact Factor: A Submission and Citation-Based Analysis},
howpublished = {Pubrica},
year = {2026},
month = {oct},
url = {https://pubrica.com/academy/concepts-definitions/submission-and-citation-based-study-on-temporal-variation-of-journal-impact-factor/},
note = {Accessed: <span data-cite-acc="iso">2026-10-15</span>}
}This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.
Impact-Factor Sanity Check
This checklist should be completed before a single year's impact factor is cited as evidence of a journal's standing.
[ ] Pulled at least 3 years of the journal's impact factor, not just the latest figure [ ] Checked whether the citable-item count (denominator) changed between years [ ] Searched for any single paper responsible for an unusually large share of citations [ ] Compared the figure against an independent source (CiteScore, SCImago, or the publisher's own reported citations) [ ] Noted whether a special issue, conference proceedings, or editor-linked publication contributed heavily to recent citations
Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.
How Pubrica can help
Pubrica’s data and bibliometric review service examines a shortlist of candidate journals across multiple years and citation sources, and identifies any single-year figure that appears to be driven by an outlier rather than a genuine trend.