Free scoping read. A written assessment of your manuscript in two business days, at no cost.

Start one
Contents

Academy · Research Data, Analytics & AI · Altmetrics & Guideline Tracking

Measuring the Real-World Impact of Research with Altmetrics and Clinical Guideline Citations

Citation counts take years to accumulate and do not indicate whether a study changed practice. Altmetrics, and the more recent practice of tracking citations within clinical guidelines, are designed to address that gap more rapidly and in closer relation to real-world use. This article examines what each measures and how they should be reported.

Pubrica AcademyWritten by a research impact analyst Reviewed bya bibliometrics reviewer Published10 October 2026 Last checked10 October 2026 Reading time8 minutes Sources10 peer-reviewed

Altmetrics are quantitative indicators of attention a piece of research receives outside the traditional citation record: mentions in news coverage, policy documents, social media, reference-manager libraries, blogs, and, increasingly, clinical practice guidelines. They were proposed in 2010 as a faster, broader complement to citation counts, not a replacement for them, because citations to a paper can take two to five years to accumulate while online attention and guideline uptake can appear within weeks of publication.

Learning objectives

  • What altmetrics measure that citation counts cannot, and why the time lag between the two matters
  • How tracking the appearance of a paper in clinical practice guidelines differs from tracking social media mentions
  • What the peer-reviewed evidence indicates about whether altmetric attention predicts later citation impact

Common misconceptions

  • “A high altmetric score indicates that a study is scientifically important.” Multiple empirical studies have found only a weak-to-moderate correlation between altmetric attention and subsequent citation counts, and the relationship varies substantially by discipline and by altmetric source; a high score therefore reflects reach and timing more reliably than it reflects eventual scholarly importance.

The rationale for altmetrics alongside citations

Citation counts are the oldest and most trusted measure of the influence of a piece of research on subsequent scholarship, but they are inherently slow: a typical paper requires two to three years before its citation count becomes a meaningful signal, and in some fields the delay is longer1. Jason Priem and colleagues proposed the term “altmetrics” in a 2010 manifesto to describe a family of faster, web-based indicators (downloads, bookmarks, mentions in blogs and social media, and references in policy and news documents) that could reveal how a paper was being used and discussed almost immediately after publication1. The manifesto stated explicitly that altmetrics were intended to supplement citation-based measures, providing a picture of attention and reach that citations alone cannot provide, rather than to serve as a direct substitute for them1. Later commentary described their appeal clearly: altmetrics can show a funder or an author that a paper reached clinicians, journalists or policymakers long before a citation count would reflect any of that activity6.

The distinct significance of citation in clinical guidelines

Most altmetric sources (tweets, news stories and blog posts) measure attention rather than adoption. A mention in a clinical practice guideline is a different kind of signal, because guideline bodies such as the World Health Organization (WHO), national bodies such as the National Institute for Health and Care Excellence (NICE) in the United Kingdom, and specialty societies cite evidence only after a formal, documented appraisal process78. The published NICE manual for guideline development describes a structured sequence of evidence identification, quality appraisal and committee deliberation before any study is cited in a recommendation, and specialty bodies that operate similar processes typically require a stated level of evidence before citing a source79. The presence of a paper in a published guideline therefore indicates that it has passed a vetted, multi-step evaluation directed at clinical usefulness, which is a materially stronger claim than appearance in a tweet or a general news article, even though both are captured within the broad scope of altmetrics tracking. This distinction is most important in specialties in which guideline bodies publish frequent, versioned updates, such as oncology and cardiology, since a paper that is cited, removed and subsequently re-cited across successive guideline versions provides a more complete account of the durability of its influence than a single mention, a trajectory that platforms tracking guideline citations over successive versions, rather than only a single appearance, are increasingly designed to reveal.

Evidence on the relationship between altmetrics and citations

Because altmetrics are often proposed as an early proxy for eventual scholarly impact, several research teams have tested directly whether the two are associated. Mike Thelwall and colleagues analysed over 200,000 articles across twelve social-web sources and found some correlation between a small number of sources and later citations, but concluded that most altmetric sources (including the one then called Mendeley readership) showed only weak relationships with citation counts, and that the strength of the relationship varied considerably by discipline2. Rodrigo Costas, Zohreh Zahedi and Paul Wouters extended this comparison across a broader set of altmetric indicators and multiple fields and reported that correlations between altmetric counts and citations were generally low, supporting the view that the two measure substantially different things rather than two aspects of the same underlying quantity3. Lutz Bornmann’s review of the broader altmetrics literature similarly concluded that altmetrics can indicate broader societal attention but should not be interpreted as a leading indicator of scientific quality or eventual citation impact4. Nicholas Trueger and colleagues, writing about the proprietary Altmetric Attention Score used by many publishers, described it as a useful measure of dissemination, that is, how far and how quickly a paper travelled, while cautioning that dissemination and scientific merit are not the same property10.

Susceptibility of attention metrics to manipulation and implications for trust

Because altmetric counts are derived largely from open, low-barrier platforms, such as social media shares, downloads and bookmarking tools, they are inherently more exposed to deliberate manipulation than a citation count, which requires another author to have written and published a formal, reviewed reference to the work. Bornmann’s review identifies coordinated social media promotion, self-sharing campaigns and bulk automated downloads as documented or plausible means by which a paper’s attention score can be inflated without any corresponding change in its scientific merit or even its actual readership4. The comparative work of Costas, Zahedi and Wouters makes a related point from a different perspective: because different altmetric sources are weighted and aggregated differently by different providers, and because the underlying platforms change their algorithms and visibility rules over time, a paper’s attention score is not a fixed, stable property in the way that a DOI or a publication date is; it can change retrospectively as platforms are added, removed or reweighted by the altmetric provider3. This instability compounds the concern about manipulation: a single altmetric number, examined without information about its sources and their known limitations, is a weak basis for any high-stakes decision, such as hiring, promotion or funding, which are precisely the decisions that altmetrics are sometimes informally used to support even though the evidence base does not yet justify that degree of reliance. Institutions that have begun using altmetrics in formal evaluation have generally been advised, including by the signatories of the Declaration on Research Assessment, to treat any single attention-based number as supplementary context rather than a primary criterion, for this reason.

Responsible use of attention data in impact reporting

The practical implication for authors and institutions is that altmetric and guideline-tracking data should be reported for what they demonstrate, namely reach, timing and, in the case of guidelines, a documented appraisal of clinical relevance, rather than presented as a proxy for citation-level scholarly impact. Catalina Barnes’s review of altmetrics as a research-impact tool for libraries and institutions recommended using these indicators as one strand of evidence in impact reporting, alongside citation counts and qualitative impact narratives, rather than as a standalone score5. The analysis by Zhichao Fang and Rodrigo Costas of how altmetric data accumulate over time also found that the speed and volume of attention a paper receives vary considerably by platform and subject area, which means that comparisons of raw altmetric scores across papers from different fields or publication dates can be misleading unless these differences are normalised10. A defensible impact report, whether for a grant renewal, a promotion dossier or an institutional communications summary, states the specific source of each number (a named guideline and its publishing body, a named database of social media mentions, a citation count from a named index), the date on which it was retrieved, and an explicit note of what each figure does not show. Grant reports, promotion dossiers and institutional impact summaries are strongest when they distinguish wide discussion of a paper from a demonstrated change in practice, and use guideline citation, where it exists, as the evidence for the latter claim rather than an unexplained composite score. This discipline is also helpful when a paper’s altmetric profile appears unusually strong or weak relative to its citation trajectory, since a divergence between the two is itself informative: a paper with high attention but few citations may be more recent, more accessible to a lay audience, or the subject of a single widely circulated event, rather than being over- or under-valued by the scholarly community, whereas a highly cited paper with minimal public attention is more likely to be explained by a subject outside the topics that attract general media or social interest than by any flaw in the work.

Key points

Key points

Altmetrics measure the speed and reach of attention rather than scientific quality, and the published evidence shows only a weak correlation with eventual citation counts. The appearance of a paper in a vetted clinical guideline is the one altmetric-adjacent signal supported by a documented evidence-appraisal process and should be treated differently from a social media mention.

Practical application

Practical application

  1. Attention metrics should be separated from claims of impact in reporting. Reports should state explicitly whether a figure reflects online mentions, citations or documented adoption in guidelines, rather than combining these into a single unexplained score.
  2. Authors should establish which guideline bodies cite evidence from their field, and on what basis. The published methods manual of a body such as WHO, NICE or the relevant specialty society should be reviewed to establish its evidence-appraisal threshold before guideline impact is claimed.
  3. Altmetric comparisons should be normalised by field and publication date. Because attention accumulates at very different rates across disciplines and platforms, the altmetric profile of a paper should be compared only with those of papers in the same field and a similar time window.

Frequently asked questions

No. The altmetrics manifesto itself proposed them as a faster complement to citations, not a replacement, and empirical studies have consistently found only a weak correlation between the two; high online attention therefore does not reliably predict eventual citation-based impact.

Guideline bodies such as NICE and WHO publish methods manuals describing a structured evidence-appraisal process that a study must pass before it is cited in a recommendation; this constitutes a documented filter for clinical relevance to which ordinary social media or news mentions are not subject.

Not without adjustment. Research on the accumulation of altmetric data has found substantial differences in the volume and speed of attention by discipline and platform, so comparisons of raw scores across fields can be misleading.

No. A paper can be highly cited and scientifically important while having little online attention, particularly in fields with less public or social media engagement, so a low score mainly indicates limited public-facing reach rather than limited scientific value.

It is a proprietary metric produced by the company Altmetric, which aggregates mentions from a defined set of sources and weights them by source type and estimated reach; the exact weighting formula is published by the company but is not an open academic standard.

References

  1. Priem J, Taraborelli D, Groth P, Neylon C. Altmetrics: A manifesto. 2010. read 10 October 2026. View the Altmetrics Manifesto
  2. Thelwall M, Haustein S, Larivière V, Sugimoto CR. Do Altmetrics Work? Twitter and Ten Other Social Web Services. PLOS ONE. 2013;8(5):e64841. read 10 October 2026. View on PLOS ONE
  3. Costas R, Zahedi Z, Wouters P. Do 'altmetrics' correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. Journal of the Association for Information Science and Technology. 2015;66(10):2003-2019. read 10 October 2026. View on JASIST
  4. Bornmann L. Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics. Journal of Informetrics. 2014;8(4):895-903. read 10 October 2026. View on Journal of Informetrics
  5. Barnes C. The use of altmetrics as a tool for measuring research impact. Australian Academic & Research Libraries. 2015;46(2):121-134. read 10 October 2026. View on Australian Academic & Research Libraries
  6. Trueger NS, Thoma B, Hsu CH, Sullivan D, Peters L, Lin M. The Altmetric Score: A New Measure for Article-Level Dissemination and Impact. Annals of Emergency Medicine. 2015;66(5):549-553. read 10 October 2026. View on Annals of Emergency Medicine
  7. National Institute for Health and Care Excellence (NICE). Developing NICE guidelines: the manual. read 10 October 2026. View NICE's guideline manual
  8. World Health Organization. WHO handbook for guideline development, 2nd edition. 2014. read 10 October 2026. View on WHO
  9. American Society of Clinical Oncology. ASCO Guidelines Methodology Manual. read 10 October 2026. View ASCO's methodology manual
  10. Fang Z, Costas R. Studying the accumulation velocity of altmetric data tracked by Altmetric.com. Scientometrics. 2020;123:1077-1101. read 10 October 2026. View on Scientometrics

Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.

How to cite this article

Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.

Pubrica Academy. (2026, October 10). Measuring the Real-World Impact of Research with Altmetrics and Clinical Guideline Citations. Pubrica. https://pubrica.com/academy/study-guide/research-writing/altmetric-clinical-guidelines-impact/

This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.

Research Attention Reporting Template

This checklist may be used to separate claims of attention, citation and guideline adoption when the impact of a paper is reported.

[ ] Reported altmetric/attention data separately from citation counts, clearly labeled
[ ] Checked whether the paper is cited in any formally published clinical guideline, and by which body
[ ] If citing guideline adoption, named the specific guideline, its publishing body, and the appraisal process it follows
[ ] Normalized any cross-paper altmetric comparison by field and publication date
[ ] Avoided presenting a single combined 'impact score' without explaining what it does and does not measure

Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.

How Pubrica can help

Pubrica’s research impact reporting service assists authors and institutions in compiling citation, altmetric and guideline-adoption evidence into a single, clearly labelled impact narrative for grant reports, dossiers and publication strategy.

Find something else

Search the Pubrica Academy

Access is free and requires no account. Enter a topic, or open the full library to filter by category, sub-topic or resource type.

Impact reports that distinguish reach from quality

A scoping assessment is free and takes two business days. It provides a written assessment of what the work requires and which specialist would undertake it.

Get a free quote

Regulations

Check your project against this rule

  1. 1Your work
  2. 2Your details
  3. 3Send
What do you need?

Material you send is held under signed confidentiality, with access limited to the assigned team and encrypted transfer. It is not used to train anything and is not passed to third-party tools. Ask for the agreement before you send anything you would rather cover first.