Home/Insights/Methodology reference/Quasi-experimental methodology
Quasi-experimental methodology
An intervention without random assignment. Causal inference is possible, but only on assumptions that have to be stated and defended.
When it is the right choice
Quasi-experimental designs are used where randomisation is impossible or unethical: policy changes, service reconfigurations, public health interventions applied to whole populations, natural experiments.
The exposure is still deliberate — something was done — but who received it was determined by policy, geography, timing or eligibility rather than by chance.
The main designs, and what each assumes
Interrupted time series. Repeated measures before and after an intervention, with level and slope change estimated. Assumes the pre-intervention trend would have continued unchanged. Strong when there are enough pre-intervention points to establish that trend — a single before-and-after pair is not an interrupted time series.
Difference-in-differences. Compares change in an exposed group with change in an unexposed one. Assumes parallel trends: both groups would have moved together absent the intervention. The assumption is testable in the pre-period and should be shown, not asserted.
Regression discontinuity. Exploits a threshold — an age, a score, an eligibility cut-off. Those just either side are comparable, so the design is strong near the threshold and says nothing about those far from it.
Non-equivalent control group. The weakest form; relies on measured covariates being sufficient.
Threats to address explicitly
History — something else changed at the same time. Maturation — the outcome was already moving. Regression to the mean, which is severe when the intervention was triggered by an extreme value. Selection, because assignment was not random.
Each of these produces an apparent effect where none exists, and each should be named and argued against in the paper rather than left to the reader.
In practice
Fix, flag or query.
The distinction that keeps an edit defensible: what an editor may change silently, what must be shown to the author, and what only the author can answer.
- Show, do not assert: the parallel-trends assumption, using pre-period data.
- Beware: regression to the mean when the intervention followed an extreme value.
- Report to: TREND, for non-randomised evaluations of behavioural interventions.
Reporting standards
- TREND Statement, for non-randomised evaluations
- ROBINS-I, for risk of bias in non-randomised intervention studies
Reporting guidelines named here are hosted by the EQUATOR Network. Writing to the checklist for your design before submission is the cheapest available improvement to a methods section.
Related