Free scoping read. A written assessment of your manuscript in two business days, at no cost.

Start one
Contents

Academy · Research Writing · Mass Gathering Evidence Levels

Levels of Evidence-Based Medicine in Mass-Gathering Public Health and Emergency Medicine: A Literature Review

A review of a decade of mass-gathering health literature found that more than one third of the evidence base consists of expert opinion rather than observed data, a structural limitation that researchers should plan for rather than apologise for.

Pubrica AcademyWritten by a disaster and emergency health research writer Reviewed bya public health epidemiologist Published10 October 2026 Last checked10 October 2026 Reading time6 minutes Sources10 peer-reviewed

Mass gatherings, such as pilgrimages, festivals and sporting finals, place substantial demand on emergency health services, yet they are difficult to study with the designs that journals prefer. A 2013 review by Arbon, Cusack and Verdonk, published in the Australasian Journal of Paramedicine, sought to establish the strength of the evidence underpinning mass-gathering health planning. The question was significant less as a methodological matter than as a planning problem: event medical directors and health authorities were, and often still are, making resourcing decisions on the basis of whatever evidence existed, regardless of its grade.

Learning objectives

  • The distribution of evidence in mass-gathering health research, based on a 10-year sample of published literature
  • Why World Health Organisation categories and Joanna Briggs Institute effectiveness classifications are used together to grade this field
  • The meaning of ‘mid-level’ evidence where randomised designs are rarely feasible, and why it represents a realistic target rather than a shortcoming

Common misconceptions

  • “Mass-gathering medicine research is dominated by randomised controlled trials, like most of clinical medicine.” The literature reviewed from 2001–2011 found that only 2.18% of studies were quasi-experimental and none were randomised controlled trials (RCTs); the evidence base consists overwhelmingly of observational and cohort studies, with little scope for randomisation at a live event.
  • “Expert opinion in this field is a minor, marginal category.” Arbon, Cusack and Verdonk found expert opinion or consensus accounted for 36.68% of the literature sampled — almost as large a share as observational studies (38.86%) — meaning more than a third of what the field relies on is not derived from directly observed event data.

Scope of the review

Arbon P, Cusack L and Verdonk N published “Mass gathering public health and emergency medicine literature review: levels of evidence” in the Australasian Journal of Paramedicine, volume 10, issue 1, 20131. The team searched peer-reviewed mass-gathering public health and emergency medicine articles published between 2001 and 2011, deliberately excluding news reports and grey literature so that the sample reflected the citable, peer-reviewed evidence base on which clinicians and planners would draw1.

Each included study was sorted against World Health Organisation evidence categories and then rated using Joanna Briggs Institute (JBI) effectiveness classifications — a combination chosen because neither framework on its own was designed for the particular constraints of event-based field research1.

The distribution of evidence identified

The resulting breakdown of study designs was: observational studies 38.86% (n = 89), expert opinion or consensus 36.68% (n = 84), cohort studies 20.09% (n = 46), case-control studies 2.18% (n = 5), and quasi-experimental studies 2.18% (n = 5)1. No randomised controlled trials appeared in the sample at all.

Two features of this distribution are notable. First, observational and cohort designs together make up roughly 59% of the field — a usable, if modest, evidence tier. Second, expert opinion is not a small residual category; at 36.68% it is nearly as large as the observational share, which means that a sizeable proportion of operational guidance in this field derives from consensus and experience rather than measured outcomes.

Structural barriers to high-level evidence

The authors’ own conclusion is direct: “High-level evidence studies may not be possible in the mass gathering context”1. A mass gathering is a one-off, uncontrolled, safety-critical environment — researchers cannot randomise crowd density, withhold medical resources to create a control arm, or re-run the same festival to test an intervention. This is a structural constraint rather than an indication of weak science in the field.

Given this limit, the practical recommendation of the review is to deliberately expand the mid-level evidence tier — well-designed observational and cohort studies — so that event planning decisions rest on something closer to measured data and rely less heavily on experience and expert opinion by default1.

Grading the evidence with WHO, JBI and GRADE frameworks

Arbon and colleagues graded their sample against WHO evidence categories and JBI effectiveness levels together, but the two frameworks serve different purposes. The JBI Levels of Evidence, published by the JBI Levels of Evidence and Grades of Recommendation Working Party in October 2013, rank study designs for effectiveness questions into five tiers from strongest to weakest: Level 1 covers systematic reviews of randomised and pseudo-randomised trials; Level 2 covers quasi-experimental designs, including pre-test/post-test and historically controlled studies; Level 3 covers observational analytic designs such as cohort and case-control studies; Level 4 covers observational descriptive designs such as cross-sectional studies and case series; and Level 5 covers expert opinion and bench research4. Mapped onto this scale, the 38.86% observational share of the mass-gathering literature sits mostly at Level 3, its 20.09% cohort share also at Level 3, and its 36.68% expert-opinion share at Level 5 — which means that more than a third of the citable output of the field occupies the weakest tier the JBI scale defines.

The GRADE framework, first set out by the GRADE Working Group in the BMJ in 2004, addresses the opposite end of the process: not the design of an individual study, but the overall confidence a reader should place in a specific finding once study design, consistency, directness and precision have all been weighed2. Randomised trials start at high confidence and observational studies start low, but GRADE allows an observational finding to be rated up again if the association is strong, shows a dose-response gradient, or persists despite plausible confounding working against it2. For a field with no RCTs, this has practical significance: a well-conducted mass-gathering cohort study showing a strong, consistent effect is not automatically “low-quality evidence” under GRADE solely because it lacks randomisation — but the reviewer must make and justify that upgrade explicitly rather than assume it.

The evolving definition of the field

The composition of the evidence base partly reflects the fact that “mass gathering” itself has no single agreed definition. Yezli and Alotaibi, writing in the Saudi Medical Journal in 2016, note that earlier definitions used a crowd-size threshold of more than 1,000 people, while much of the published literature in practice describes events with more than 25,000 attendees; the World Health Organization's own definition instead focuses on whether an event could strain the host community's public health resources, regardless of headcount3. The authors argue that “mass gatherings health” is a more accurate term than “mass gathering medicine,” since the discipline covers infectious and non-communicable disease, trauma, heat illness, drug and alcohol harms and deliberate attacks — considerably more than clinical care alone3. They trace the field's formal emergence to the 2010 Global Forum on Mass Gatherings Medicine held in Jeddah and its launch as a recognised discipline at the 2014 World Health Assembly3 — a field that had only recently been formally established when Arbon et al.’s review was published, which partly explains why so much of its literature still consists of expert consensus rather than measured outcomes.

Evidence on the impact of mass gatherings on health services

A separate systematic review by Ranse, Hutton, Keene and colleagues, published in Prehospital and Disaster Medicine in 2017, looked specifically at how mass gatherings affect ambulance and hospital services rather than at in-event care alone6. It identified 24 eligible studies, all of them retrospective case studies. Twenty-three described in-event first-responder services, and all reported that ambulance services transported some patients to hospital — but only nine studies tracked the outcomes of those patients once they reached the emergency department, and none measured the broader impact on ambulance or hospital capacity6. The review concluded that research on the downstream effect of mass gatherings on external health services is “minimal,” and it recommended linking event-level patient data to hospital and ambulance records so that future planning could be based on measured service impact rather than on in-event anecdote alone6. This gap falls within the same evidence ceiling that Arbon et al. describe: even the existing studies tend to stop measuring at the point at which an ambulance leaves the event site.

Infectious disease is the one sub-area with a denser, more internationally coordinated literature, largely because of recurring events such as the Hajj. Abubakar, Gautret, Brunette and colleagues, writing in The Lancet Infectious Diseases in 2012, reviewed global perspectives on preventing infectious disease spread at mass gatherings and argued for stronger international surveillance coordination, since gatherings that draw travellers from many countries create conditions for cross-border transmission that a single host nation cannot monitor alone7. Even here the underlying evidence is overwhelmingly descriptive — the same ceiling Arbon et al. identified, documented in more depth only because a small number of very large, recurring events have drawn sustained attention.

A parallel problem: predicting patient presentation rates

A planning-focused strand of the literature addresses a narrower, related problem: predicting how many patients an event will generate so that organisers can plan medical staffing in advance. Van Remoortel, Scheers, De Buck, Haenen and Vandekerckhove’s 2020 systematic review in PLOS ONE examined prediction-modelling studies for medical usage rates at mass gatherings and found the same evidence-quality pattern as Arbon et al., in relation to a more applied question8. Existing models vary widely in which event characteristics they include — crowd size, weather, event type, alcohol availability — and in how they were validated, which the review treats as a barrier to generalising any one model from one event to another. The practical consequence is that even a well-supported prediction model should be treated as decision support, not a precise forecast.

Risk-assessment tools developed for an evidence-poor field

Given that the underlying evidence base cannot readily be strengthened to RCT level, much of the practical effort in the field has been directed towards structured risk-assessment tools that make the best use of the mid-level evidence that does exist. The Jeddah Tool, described by Yezli and Khan in the Saudi Medical Journal in 2020, is a health risk assessment framework developed specifically for recurrent mass gatherings such as the Hajj, intended to standardise how organisers identify hazards and estimate risk level across repeated iterations of the same event rather than beginning anew each time5. The World Health Organization’s Generic All-Hazards Risk Assessment Tool for Mass Gathering Events, published in 2023, generalises the same approach for any national government or event organiser, helping them to identify hazards, assess overall risk level, and select proportionate precautions for a specific event9. Neither tool generates evidence where none exists; both are explicitly structured decision frameworks designed to make consistent, auditable use of the observational and expert-consensus evidence available to the field.

Underpinning several of these tools is a strand of foundational theoretical work by Lund, Turris and colleagues, who proposed population-level models for mass gatherings in Prehospital and Disaster Medicine10. Their approach treats a mass gathering as a bounded population with its own demographic and behavioural characteristics — crowd density, mobility, alcohol and substance use patterns, weather exposure — that can be modelled systematically rather than treated as a one-off event that cannot be analysed. This theoretical framework is what allows prediction models and risk-assessment tools to generalise lessons across different gatherings, even though no individual gathering can be randomised or replicated.

Implications for authors in the field

Taken together, these sources indicate a consistent, field-wide research agenda rather than a problem specific to Arbon et al.’s 2013 sample: Yezli and Alotaibi call for the inclusion of the discipline in formal training programs3, Ranse et al. call for linking in-event data to hospital records6, and Van Remoortel et al. call for better-validated prediction models8 — none of these expects randomisation to become possible, and all aim to strengthen the mid-level observational tier. For a literature review or thesis chapter in this field, the practical implication is that sources should be graded accurately rather than against a hierarchy built around RCTs that the field cannot supply. Citing an expert-consensus paper is legitimate provided it is labelled as such. Where a specific sub-question already has a denser literature — infectious disease at recurring events, or patient-load prediction — citing the systematic reviews of that sub-literature provides a stronger basis than relying on a single decade-old sample alone, and citing a structured tool such as the Jeddah Tool or the WHO risk-assessment framework is more defensible for a planning claim than citing raw observational data directly.

Key points

Key points

In mass-gathering health research, observational and cohort studies constitute the strongest realistically attainable tier of evidence, and more than one third of the field consists of expert opinion; both facts should be stated explicitly in any review that cites this literature.

Practical application

Practical application

  1. Every source should be graded against JBI effectiveness levels before it is cited. A published paper should not be assumed to carry the weight of a clinical trial; each source should be classified as observational, cohort, case-control or expert opinion so that the review’s evidence base is transparent.
  2. Search windows comparable to 2001–2011 should be used deliberately, rather than recent years alone. The evidence base in this field accumulates slowly; a narrow search window of two to three years will miss most of the citable cohort and observational studies.
  3. The evidence ceiling should be stated at the outset. The methods or discussion section should acknowledge that RCT-level evidence is largely unobtainable at mass gatherings, so that readers do not mistake an observational finding for a controlled one.

Frequently asked questions

A live event is a single, uncontrolled, safety-critical occurrence — attendees cannot be randomised to different levels of medical cover without creating real risk, and the same crowd and conditions cannot be replicated for a second arm. The 2013 Arbon, Cusack and Verdonk review found no RCTs in its 2001–2011 sample for this reason.

Yes, provided it is labelled as such. At 36.68% of the sampled literature, expert consensus is too large a share of the field to exclude, but a rigorous review should distinguish it clearly from the 38.86% observational and 20.09% cohort tiers so that readers can weigh claims accordingly.

References

  1. Arbon P, Cusack L, Verdonk N. Mass gathering public health and emergency medicine literature review: levels of evidence. Australasian Journal of Paramedicine. 2013;10(1). read 10 October 2026. View source
  2. GRADE Working Group. Grading quality of evidence and strength of recommendations. BMJ. 2004;328(7454):1490-1494. read 10 October 2026. View source
  3. Yezli S, Alotaibi BM. Mass gatherings and mass gatherings health. Saudi Medical Journal. 2016;37(7):729-730. read 10 October 2026. View on PubMed Central
  4. Joanna Briggs Institute Levels of Evidence and Grades of Recommendation Working Party. JBI Levels of Evidence. Joanna Briggs Institute; 2013. read 10 October 2026. View source
  5. Yezli S, Khan AA. The Jeddah tool: A health risk assessment framework for mass gatherings. Saudi Medical Journal. 2020;41(2):121-122. read 10 October 2026. View source
  6. Ranse J, Hutton A, Keene T, Lenson S, Luther M, Bost N, Johnston ANB, Crilly J, Cannon M, Jones N, Hayes C, Burke B. Health service impact from mass gatherings: A systematic literature review. Prehospital and Disaster Medicine. 2017;32(1):71-77. read 10 October 2026. View source
  7. Abubakar I, Gautret P, Brunette GW, Blumberg L, Johnson D, Poumerol G, Memish ZA, Barbeschi M, Khan AS. Global perspectives for prevention of infectious diseases associated with mass gatherings. The Lancet Infectious Diseases. 2012;12(1):66-74. read 10 October 2026. View source
  8. Van Remoortel H, Scheers H, De Buck E, Haenen W, Vandekerckhove P. Prediction modelling studies for medical usage rates in mass gatherings: A systematic review. PLOS ONE. 2020;15(6):e0234977. read 10 October 2026. View source
  9. World Health Organization. The generic all-hazards risk assessment tool for mass gathering events. WHO; 2023. read 10 October 2026. View source
  10. Lund A, Turris SA, Bowles R, Steenkamp M, Hutton A, Ranse J, Arbon P. Mass-gathering health research foundational theory: Part 1: population models for mass gatherings. Prehospital and Disaster Medicine. read 10 October 2026. View source

Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.

How to cite this article

Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.

Pubrica Academy. (2026, October 10). Levels of Evidence-Based Medicine in Mass-Gathering Public Health and Emergency Medicine: A Literature Review. Pubrica. https://pubrica.com/academy/literature-review/levels-of-evidence-based-medicine-in-mass-gathering-public-health-and-emergency-medicine-literature-review/

This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.

Evidence-grading checklist for mass-gathering literature reviews

This checklist should be applied before any source is cited in a mass-gathering or event-medicine review.

[ ] Classified the source against a WHO evidence category
[ ] Rated the source using JBI effectiveness levels
[ ] Applied GRADE's directness/consistency/precision checks, not just study design, when judging overall confidence
[ ] Labelled expert-opinion/consensus sources explicitly as such
[ ] Searched a multi-year window, not just the last 2-3 years
[ ] Checked whether a denser sub-literature (infectious disease, prediction modelling) already covers the specific question
[ ] Stated the field's evidence ceiling in the discussion section

Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.

How Pubrica can help

Pubrica’s medical writers grade and classify mass-gathering and disaster-medicine literature against WHO and JBI frameworks so that a review’s evidence base is defensible at peer review.

Find something else

Search the Pubrica Academy

Access is free and requires no account. Enter a topic, or open the full library to filter by category, sub-topic or resource type.

Defensible evidence grading for mass-gathering and disaster medicine reviews

A scoping assessment is free and takes two business days. It provides a written assessment of what the work requires and which specialist would undertake it.

Get a free quote

Regulations

Check your project against this rule

  1. 1Your work
  2. 2Your details
  3. 3Send
What do you need?

Material you send is held under signed confidentiality, with access limited to the assigned team and encrypted transfer. It is not used to train anything and is not passed to third-party tools. Ask for the agreement before you send anything you would rather cover first.