Pre-Experimental Research Design: Overview and Types
A pre-experimental design is not a smaller experiment. It is a design that lacks the one feature an experiment depends on: a credible comparison. The three named types come from Campbell and Stanley, who presented the first as “a minimum reference point” and the second explicitly as a “bad example”. Knowing what each leaves uncontrolled is the point of studying them.
- The three designs in their original notation, and what X and O each stand for
- The named rival explanations each design cannot rule out, quoted from the source
- When a pre-experimental design is a defensible choice, and how to report one honestly
- “A pre-experimental design gives weaker evidence of cause, but evidence all the same.” For the one-shot case study there is no evidence of cause at all, because there is nothing to compare. Campbell and Stanley state the principle in one line: “securing scientific evidence involves making at least one comparison”.1 A weak comparison can be discounted; an absent one cannot be weighed.
What makes a design pre-experimental
The term comes from Campbell and Stanley’s 1963 monograph, which groups three designs under the heading “THREE PRE-EXPERIMENTAL DESIGNS”.1 What unites them is an absence rather than a presence. None involves random allocation, and none secures a comparison group whose equivalence can be defended.
That single absence governs everything else. A true experiment uses random assignment, which “when properly implemented, eliminates selection bias, balancing both known and unknown prognostic factors”.2 Nothing in a pre-experimental design performs that work, and nothing in the analysis can supply it afterwards, because the factors that need balancing include those nobody measured.
The designs are therefore studied less as options to choose than as a catalogue of what goes wrong without a comparison. Campbell and Stanley are explicit about their purpose in presenting them: the one-shot case study “is introduced here as a minimum reference point”, and the one-group pretest-posttest design is “introduced here as a ‘bad example’ to illustrate several of the confounded extraneous variables”.1
Reading the notation
The shorthand used throughout this literature is theirs, and it is worth reading precisely because so much is carried by position. “An X will represent the exposure of a group to an experimental variable or event”, while “O will refer to some process of observation or measurement”.1 Symbols are read left to right in time order, and a row is one group.
| Design | Notation | What it says |
|---|---|---|
| One-shot case study | X O | One group, treated, then measured once1 |
| One-group pretest-posttest | O1 X O2 | One group, measured before and after1 |
| Static-group comparison also published as the post-test-only design with non-equivalent groups | X O above a dashed rule, O below it | Two groups, each measured once, neither formed at random1 |
The three designs
The one-shot case study
A single group receives the intervention and is measured once afterwards. Campbell and Stanley observe that “much research in education today conforms to a design in which a single group is studied only once”.1
The objection is not that the evidence is weak. It is that the structure produces no evidence about cause at all, because “securing scientific evidence involves making at least one comparison”.1 A single post-intervention measurement can be compared with nothing: not with the same group beforehand, not with anyone untreated. Whatever the number is, it cannot be contrasted, so nothing can be attributed to the intervention.
The one-group pretest-posttest design
The same group is measured before and after, which at least supplies a comparison. The comparison is with the group’s own earlier state, and that is where the trouble begins: everything else that happened between the two measurements is also a candidate explanation.
Campbell and Stanley work through five such candidates in turn, and section 4 below sets out each one.1 This is the design they single out as the instructive bad example, precisely because the pretest makes it look rigorous while leaving every one of those rivals in play.
The static-group comparison
Two groups, one of which received the intervention, each measured once afterwards. The same design is commonly published under the later label post-test-only design with non-equivalent groups, which describes the same structure in modern terminology. In the authors’ words, this is “a design in which a group which has experienced X is compared with one which has not, for the purpose of establishing the effect of X”.1
It supplies a concurrent comparison, which the other two lack. What it does not supply is any reason to believe the groups were alike to begin with. “In marked contrast with the ‘true’ experiment of Design 6, below, there are in these Design 3 instances no formal means of certifying that the groups would have been equivalent had it not been for the X.”1 Without a baseline measurement, a difference at the end cannot be separated from a difference at the start.
The threats each one leaves open
Campbell and Stanley enumerate the rival explanations that a design must handle to support a causal claim. Their definitions are compact, and each one names a specific way a before-and-after difference can arise without the intervention doing anything.
| Threat | Definition, verbatim | What it looks like in practice |
|---|---|---|
| History | “The specific events occurring between the first and second measurement in addition to the experimental variable”1 | A policy change, a news story or a seasonal shift that moved the outcome on its own |
| Maturation | “Processes within the respondents operating as a function of the passage of time per se”1 | Participants grew older, more practised, more tired or simply recovered |
| Testing | “The effects of taking a test upon the scores of a second testing”1 | The pretest itself taught, sensitised or rehearsed the participants |
| Instrumentation | “Changes in the calibration of a measuring instrument or changes in the observers or scorers used”1 | A different rater, a revised form, or an observer who got better at scoring |
| Statistical regression | “Operating where groups have been selected on the basis of their extreme scores”1 | A group recruited because it scored worst will tend to score better next time regardless |
| Selection | “Biases resulting in differential selection of respondents for the comparison groups”1 | The people who took up the intervention differed from those who did not |
| Experimental mortality | “Differential loss of respondents from the comparison groups”1 | Those who did worst dropped out, leaving a flattering remainder |
For the one-group pretest-posttest design the authors name five of these as uncontrolled rival hypotheses, taking them in order: history, maturation, the effect of the pretest itself, instrumentation and statistical regression.1 For the static-group comparison they name the differential recruitment of the groups and experimental mortality.1
Interventions are often aimed at whoever is doing worst: the lowest-scoring class, the sickest patients, the worst-performing site. Selecting on an extreme score is exactly the condition under which statistical regression operates.1 Such a group tends to score closer to average on a second measurement whether or not anything was done to it, so a one-group pretest-posttest study of a remedial programme is close to guaranteed to show an improvement.
How large are these effects in practice?
Naming a threat is not the same as showing it matters. For three of the five that affect the one-group design, the size of the effect has been measured directly, and the figures are large enough to account for a study’s entire result.
Regression to the mean
Barnett, van der Pols and Dobson describe the phenomenon in one line: regression to the mean “is a statistical phenomenon that can make natural variation in repeated data look like real change”.7 The mechanism is simply that “unusually large or small measurements tend to be followed by measurements that are closer to the mean”.7
Two conditions make it worse, and both are ordinary in applied research. The effect “becomes more noticeable with increasing measurement error”, and when “follow-up measurements are only examined on a sub-sample selected using a baseline value”.7 A service that enrols the patients scoring worst, using an imperfect instrument, has met both conditions before the intervention starts.
Linden, writing on health care initiatives, is blunter about the consequence. Regression to the mean “often leads to an inaccurate conclusion that the intervention resulted in a treatment effect”, the single-group before-and-after approach is “problematic from a statistical standpoint”, and “failure to address RTM may result in wasteful pursuit of ineffective interventions”.8
Testing and practice effects
Bartels and colleagues define a practice effect as an “increase in a subject’s test score from one administration to the next”, and their study measured it in healthy adults receiving no intervention of any kind.9 They report “clinically relevant practice effects during high-frequency testing until month 3 (Cohen’s d 0.36–1.19)”.9
That upper figure is a large effect by any convention, produced by repeated testing alone in people who received no treatment. A one-group pretest-posttest study has no way to separate an effect of that size from the intervention it is evaluating. The authors draw the conclusion themselves: “treatment effects might be overestimated, particularly in the absence of adequate control groups”.9
The direction is not always upward, which makes the threat harder to reason about than it looks. Roediger and Karpicke showed that testing changes later performance in ways that depend on the interval: at five minutes, restudying beat testing, “81% vs. 75%”, but after a week “the tested group recalled 56% of the material, whereas the restudy group recalled only 42%”.10 A pretest can therefore depress or inflate a post-test depending on how long the gap is.
Maturation and natural history
The clearest illustration comes from a condition that resolves on its own. Coste and colleagues followed an inception cohort of patients presenting to primary care with acute low back pain and found that “90% of patients recovered within the first two weeks”, with a median episode duration of seven days.11
Set that beside a one-group study of any treatment for acute back pain measured at two weeks. Such a study would report roughly nine in ten participants improved, and every one of those recoveries was available without the intervention. The design cannot distinguish the two.
At the level of a service rather than a person, the same problem appears as secular trend. Ramsay and colleagues set out the before-and-after logic — “some measure of compliance or outcome is taken before the intervention and then the same measure is taken after the intervention has occurred”, after which “any changes are then inferred to have occurred because of the intervention” — and then name the flaw: “the outcome may be increasing or decreasing with time”.12 If referrals were already falling, a before-and-after study will credit the intervention with a fall that was happening anyway.12
Reactivity: being studied changes behaviour
Campbell and Stanley flagged this as a distinct problem, noting that “the reactive effect can be expected whenever the testing process is in itself a stimulus to change rather than a passive record of behavior”.1 It is usually discussed under the name of the Hawthorne effect.
Here the honest summary is more qualified than the folklore. A systematic review and meta-analysis in primary care computed “the overall OR of the HE in primary care” as “1.41 (95% CI: [1.13;1.75]; I²=97%)”, which is a real effect — but the authors add that “the significance of the HE disappears in well-designed studies”.13 The heterogeneity is extreme, and the effect is partly a property of weak designs rather than a fixed quantity to subtract. For a pre-experimental study the practical implication stands either way: participants know they are being studied, and nothing in the design accounts for it.
Consider a single-group study of a back-pain intervention, measured before and after with a two-week gap, in patients recruited because they scored worst. Natural history alone can deliver recovery in around 90% of them.11 Regression to the mean operates on top of that, because selection was on an extreme score.7 Repeated testing can add up to a large effect by itself.9 None of this requires the intervention to do anything. That is why the design cannot support a causal claim, and why the objection is structural rather than a matter of sample size.
What happens when nobody is treated at all
The sharpest test of a single-group design is to ask what the same measurements would have shown with no intervention. Trials that include a genuine no-treatment arm answer that question directly, and the answer is not nothing.
Krogsbøll, Hróbjartsson and Gøtzsche pooled 37 such trials covering eight clinical conditions and 2,900 patients. Across all of them, “there was a statistically significant change from baseline in all three arms” — including the arm that received no treatment, where the standardised mean difference for change from baseline was “−0.24 (95% confidence interval −0.36 to −0.12)”.32 Setting that against the active arms, “the relative contributions of spontaneous improvement and of placebo to that of the active interventions were 24% and 20%, respectively”.32
Spontaneous improvement and placebo together account for close to half of what an active arm shows, on those pooled figures.32 A controlled trial separates the three. A one-group pretest-posttest study measures their sum and reports it as the intervention.
The contribution varies sharply by condition. The same analysis found spontaneous improvement most pronounced in “nausea (45%), smoking (40%), depression (35%), phobia (34%) and acute pain (25%)”.32 Those are precisely the areas where single-group evaluations are common.
Natural history, condition by condition
Three bodies of evidence put numbers on how much recovery is available without any intervention at all.
| Condition | What was observed | Important qualifier |
|---|---|---|
| Major depression | “23% of prevalent cases of untreated depression will remit within 3 months”, “32% within 6 months” and “53% within 12 months”33 | Genuinely untreated cases, modelled from primary-care samples33 |
| Respiratory infections in children | “In 90% of children, earache was resolved by seven to eight days”, and “acute cough by 25 days, common cold by 15 days”34 | The review notes these estimates “also apply to children who are given (usually symptomatic) drugs”, so this is usual care rather than no care34 |
| Acute low back pain | Pain “decreased rapidly (by between 12% and 84% of initial levels, pooled mean 58%) within one month”35 | Data were extracted “for the control group, defined as the group receiving the least active intervention”35 |
The qualifiers in the third column matter, and are given here rather than smoothed over. Only the depression figures describe genuinely untreated people. The other two describe least-active or usual-care arms, which is a weaker comparison — but still one that a single-group study does not have at all.
And placebo is smaller than the folklore suggests
It is tempting to attribute the remainder to a powerful placebo response. The evidence does not support that either. Hróbjartsson and Gøtzsche compared placebo with no treatment across clinical trials and concluded: “we found little evidence in general that placebos had powerful clinical effects”.36 Placebo “did not have a significant effect on binary outcomes”, with possible small benefits confined to “studies with continuous subjective outcomes”.36
That makes the problem for a single-group design worse, not better. If the improvement is not mostly placebo, much of it is the condition changing on its own — which is maturation, the second threat on the list, and the one a pretest cannot touch.1
Bland and Altman give the clearest statement of the mechanism in a clinical setting. Taking patients selected for high blood pressure, they note that “even if subjects are not treated the mean blood pressure will go down, owing to regression towards the mean”, and that “this should not be interpreted as showing the effect of the treatment”.37 The task, as they put it, is to “separate any genuine reductions due to treatment from the effect of regression towards the mean”37 — and a design with one group has nothing to separate them with.
The analysis cannot rescue the design
A final hope is that the right statistical handling will fix things. It will not, and two of the most common choices make matters worse.
Vickers and Altman show that “analysing change does not control for baseline imbalance because of regression to the mean”, because “baseline values are negatively correlated with change” — patients with low scores at baseline “generally improve more than those with high scores”.38 The consequence is directional: the treatment effect will be “overestimated by looking at change scores (because of regression to the mean)”.38 Their recommended approach, analysis of covariance, requires a comparison group that a single-group design does not have.38
Percentage change from baseline, a common choice in service evaluations, is weaker still. It “has the lowest statistical power”, is “highly sensitive to changes in variance”, and — “perhaps counterintuitively” — “does not correct for imbalance between groups at baseline”. The conclusion is unambiguous: “percentage change from baseline should therefore not be used in statistical analysis”.39
One further caution applies whatever the analysis. Vickers and Altman note that “it is incorrect to choose whichever analysis gives a more significant finding”, and that “the method of analysis should be specified in the trial protocol”.38
Internal and external validity
The threats above are all threats to internal validity, and it is worth being precise about what that term covers, because the two kinds of validity pull against each other.
Campbell and Stanley define the distinction sharply. “Internal validity is the basic minimum without which any experiment is uninterpretable”, while “external validity asks the question of generalizability”.1 They add the complication that matters for design choices: “both types of criteria are obviously important, even though they are frequently at odds in that features increasing one may jeopardize the other”.1
That trade-off is why a pre-experimental study is not rescued by being realistic. Running in an ordinary clinic with unselected patients may give it good external validity, but external validity is generalisability of a finding — and there is no secure finding to generalise until internal validity is established first.
What happens when these designs are checked against stronger ones
Everything above is an argument from structure. There is also a direct empirical test: take questions answered by an uncontrolled before-after study, answer them again with a controlled design, and see whether the answers agree.
They often do not, and the disagreement has a direction. Reviewing the evidence on non-randomised intervention studies, Deeks and colleagues record that “it was noted that non-randomised studies overestimated more often than they underestimated treatment effects”.23 They add two warnings that matter for anyone hoping to repair the problem in the analysis: such studies “may still give seriously misleading results when treated and control groups appear similar in key prognostic factors”, and “standard methods of case-mix adjustment do not guarantee removal of bias”.23
The same review reports a comparison of randomised trials with historically controlled trials across six interventions: “across all six interventions 20% of RCTs showed benefit compared with 79% of the HCTs”.23 That figure should be read with the caveat its authors attach to it — “as the conclusions were based on statistical significance and not size of effect, this may be confounded by sample size”23 — but the gap between a fifth and four-fifths is not a rounding error.
Goodacre gives a case where the uncontrolled answer was not just too large but pointed the wrong way: “uncontrolled before-after data suggested that intravenous thrombolysis could reduce mortality in cardiac arrest”, whereas subsequent randomised data showed “this treatment increased intracranial haemorrhage” and did not reduce mortality.22 A treatment that looked life-saving in a before-and-after comparison turned out to cause harm. That is the practical cost of a design that cannot control for “contemporaneous changes in case mix, referral patterns or other elements of care”.22
Goodacre also notes the long-standing nature of the problem — “the use of historical controls in clinical trials has long been recognised to overestimate the benefit of new treatments” — and a further limitation that is easy to miss: “most before-after studies do not continue long enough to determine whether the intervention and its apparent effect are sustainable”.22
The three designs, drawn
Set side by side in Campbell and Stanley’s own notation, the three designs differ in exactly two ways: whether anything is measured before the intervention, and whether anyone who did not receive it is measured at all.
X marks the exposure, O marks an observation. Time runs left to right. The dashed rule in design 3 marks groups that were not formed at random.
What to look for. Campbell and Stanley define the notation plainly: “an X will represent the exposure of a group to an experimental variable or event”, and “O will refer to some process of observation or measurement”.1 Design 1 has nothing to compare against. Design 2 compares a group with its own earlier self. Design 3 compares two groups that were never shown to be alike. Each of those is a different way of being short of a control.
Schematic representation of the three designs as Campbell and Stanley set them out, not data.1
Taken from the explanatory text rather than from a summary table. A marked cell is a rival explanation the design cannot rule out.
What to look for. For design 2 Campbell and Stanley name five uncontrolled rival explanations in turn: history, maturation, “the effect of testing, the effect of the pretest itself”, instrumentation, and statistical regression.1 For design 3 they name the differential selection of the groups and experimental mortality.1 Design 1 is not better off for having fewer marks against it — it has nothing to compare, so there is no inference for a threat to spoil.
Schematic. Compiled from the authors’ explanatory text for each design, not reproduced from their summary table.1
Why the one-group design persists
Bierer and colleagues put the limit in one sentence: “all one can say when using a one-group pretest-posttest design is that a change has occurred, but not that an intervention caused it”.14 They attribute the problem to “threats to internal validity (i.e., history, maturation, testing, instrumentation, etc.) which precludes researchers from claiming causality”, and note that the design “will not conclusively prove causation despite its frequent use”.14
The criticism is old and well known, which raises an obvious question. Knapp put it in a title: “Why Is the One-Group Pretest–Posttest Design Still Used?”, opening with the observation that the design “has been widely criticized”.3
The practical answer is visible in the published literature, where the design remains common in education and health-professions research. Two recent studies describe themselves in exactly these terms: one “utilized a single-group pretest-posttest design” to evaluate a mixed-reality teaching session with 39 senior nursing students,4 and another used “a semi-experimental, single-group pre-test/post-test design” to measure 133 nursing students before and after a management course.5
Neither is dishonest, and in both cases a randomised comparison would have been difficult: a cohort takes a course together, and withholding teaching from half of it to form a control group raises its own problems. The design persists because it fits circumstances that occur constantly. That is a reason to report its limits carefully, not a reason to treat its results as causal.
The designs in the published literature
The studies below were located by searching for papers that name one of these designs themselves. Each is cited so the design statement can be checked against the paper. The final column is the most instructive part of the table.
| Design, in the authors’ words | Field and sample | Does the paper acknowledge the weakness? |
|---|---|---|
| “A pre-experimental single-group pretest-posttest design”16 | Interprofessional neonatal resuscitation training; 18 nurses and 9 doctors, South Korea16 | Partly. It notes “exogenous variables possibly influenced the results” but never names the missing control group16 |
| “Utilized a single-group pretest-posttest design”4 | Mixed-reality simulation; 39 senior nursing students, South Korea4 | Fully, and in detail — see below4 |
| “A one-group, pretest–posttest design was employed”17 | Mindfulness course; 93 second-year nursing students, South Korea17 | Yes. “The lack of a control group prevents ruling out alternative explanations for the observed changes”17 |
| “This study was based on a one-group pretest-posttest design”18 | Online platform and team-based learning; 68 nursing students, China18 | Yes. “Parallel control groups were not set up in this study”18 |
| A “single-arm pre–post experimental study”19 | Vaccine promotion programme; 50 residents, Hong Kong19 | Yes. “Since no control group for comparison, it might lead to uncertainty in assessing the intervention effects”19 |
| “This research used a static-group comparison design”20 | Worked examples in calculus teaching; 64 first-year undergraduates, Indonesia20 | No limitations section at all20 |
| “Pre-experimental with a one-shot case study”21 | Direct instruction and learning motivation; 11 school students, Indonesia21 | No limitations section at all21 |
A pattern runs down that last column. The four one-group pretest-posttest studies all say something about the missing comparison, and three name it precisely. The static-group and one-shot studies say nothing. The weaker the design, the less likely the paper is to discuss what the design cannot support — which is the opposite of what a reader needs.
One of these papers is worth reading as a model. Its authors write that “the program was applied to a small group of students from a single university”, that they “evaluated the effect without a control group”, and that “the inherent constraints of this one-group pretest-post-test design” “limit our capacity to conclusively attribute the observed changes solely to the program”. They then name the specific rivals: “external factors such as concurrent educational activities or testing effects”.4 That is the standard to write to. It does not weaken the paper; it is what makes the rest of it credible.
A note on the one-shot case study
Searching for studies that call themselves one-shot case studies produces an unexpected result. The label survives in education research, particularly in journals that still teach Campbell and Stanley’s original typology, but it has effectively disappeared from nursing, medicine and public health. What persists in the health literature is the same structure under a different description: a single-arm survey administered after an intervention, with no design label attached at all.
The naming matters less than the structure. A post-intervention satisfaction survey with no baseline and no comparison group is a one-shot case study whatever the methods section calls it, and it carries the same inability to support a causal claim.1
How common is this in practice?
Direct counts are scarce, but one systematic scoping review gives a usable indication. King and colleagues reviewed 68 research education programmes for nurses and allied health professionals, described across 69 papers. They report that “more than half of the program evaluations (n = 38) used pre- and post-intervention surveys”, while “twelve evaluation studies included a control group”.15
The authors do not use the word pre-experimental, and it should not be put into their mouths. What the two figures establish is narrower but still telling: in that literature, before-and-after measurement is the common approach, and a comparison group is the exception rather than the rule.15
Reporting and appraising a pre-experimental study
If one of these designs is the right choice, the reporting has to carry the weight the design cannot. Two established tools set the expectations.
What TREND asks for
The TREND statement was written for exactly this situation: non-randomised evaluations of behavioural and public health interventions. Its authors give the reason plainly — “if nonrandomized designs are to be systematically used in building evidence-based public health practices, it will be necessary to improve the reporting quality of these types of studies”.27
Several of its checklist items name exactly what a pre-experimental write-up tends to omit: “information on how units were allocated to interventions”; the “method used to assign units to study conditions”; “how sample size was determined”; “data on study group equivalence at baseline and statistical methods used to control for baseline differences”; and an interpretation that takes into account “sources of potential bias, imprecision of measures” and “other limitations or weaknesses of the study”.27 The checklist also warns against vague comparison descriptions: “‘usual care’ is not a helpful description of a comparison condition”.27
What risk-of-bias tools can and cannot assess
ROBINS-I is the standard tool for non-randomised studies of interventions, organised around seven bias domains: “bias due to confounding”, “bias in selection of participants into the study”, “bias in classification of interventions”, “bias due to deviations from intended interventions”, “bias due to missing data”, “bias in measurement of outcomes”, and “bias in selection of the reported result”.28
ROBINS-I describes itself as a tool for estimates “of the comparative effectiveness (harm or benefit) of interventions from studies that did not use randomisation to allocate units … to comparison groups”.28 The scope assumes comparison groups exist. A one-shot case study or a one-group pretest-posttest study produces no estimate of comparative effectiveness, so there is nothing for the instrument to appraise. This is worth knowing before a study is designed: a design that cannot be risk-of-bias assessed is a design that will struggle to be used by anyone else.
The assumption a before-and-after design makes silently
Every design smuggles in an assumption. For designs with a control group, it is that the untreated units show what would have happened. Nianogo and colleagues name the equivalent for designs without one: “since single-group designs do not include untreated units, a similar assumption for single-group designs, termed the absence of ‘history bias’, is therefore made”.29
Stated that way, the assumption becomes checkable rather than invisible. Such bias “can occur when the effects of time-varying confounding from unexpected events or co-interventions (i.e. time shock) mix with the effects of the intervention of interest”.29 A methods section that names the assumption, and reports what else was happening during the study period, is doing the only thing available to it.
Grimshaw and colleagues put the design-choice consequence in a single sentence: “wherever possible, designs allowing head-to-head comparisons, which incorporate baseline measures of performance, should be used”.31
Where these designs are legitimately useful
Three uses survive the criticism, provided the claim is matched to the design.
- Establishing feasibility. Whether an intervention can be delivered at all, whether participants will accept it, and whether the outcome can be measured are questions about practicality, not cause. A single group answers them.
- Generating hypotheses. A pattern seen once is a reason to design a controlled study, and Campbell and Stanley’s framing of design 1 as “a minimum reference point” is an invitation to build upward from it.1
- Describing what happened. A service evaluation can report that outcomes changed after a change was made. The reporting problem arises only when “after” is written up as “because”.
If the study is a pilot, the guidance is explicit that effect estimation is not its job. Thabane and colleagues warn that “it can be dangerous to use pilot studies to estimate treatment effects, as such estimates may be unrealistic/biased because of the limited sample sizes”, and that “emphasis should not be placed on statistical significance when pilot studies are not powered to detect minimal clinically important differences”.24 The CONSORT extension for pilot and feasibility trials is firmer still: “formal hypothesis testing for effectiveness (or efficacy) is not recommended”, because “the aim of a pilot trial is not to assess effectiveness (or efficacy) and it will usually be underpowered to do this”.25
What to report instead is specified: “recruitment rates, consent rates, completion rates, variance estimates”,24 and rates of “acceptance, recruitment, retention, or uptake”.25 Where an effect estimate is given at all, it “would be reported as estimates with 95% confidence intervals without P values”.25
The vocabulary is worth getting right, because the two words are not synonyms. “A feasibility study asks whether something can be done, should we proceed with it, and if so, how”, while “a pilot study asks the same questions but also has a specific design feature: in a pilot study a future study, or part of a future study, is conducted on a smaller scale”.26 It follows that “all pilot studies are feasibility studies but not all feasibility studies are pilot studies”.26 A non-randomised pilot is a recognised category in its own right.26
Lancaster and colleagues add the warning that gives the whole category its reputation: pilot studies “play an important role in health research, but they can be misused, mistreated and misrepresented”.30 A well-conducted one, with “a clear list of aims and objectives within a formal framework”, is publishable on its own terms.30
Bierer and colleagues make the constructive version of the same point for education research. The one-group pretest-posttest design “is not useful for assessing the causal effect of an intervention (due to threats to internal validity)”, but “it can be appropriately used for other purposes”, such as “using pretest data to identify baseline learner characteristics as a needs assessment”.14
Each of those uses treats the pre-experimental study as a stage rather than a conclusion. The usual progression is to establish that an intervention can be delivered and measured, then to repeat the question with a design that secures a comparison. The next step up is a quasi-experimental design, which still lacks random allocation but obtains a defensible comparison from a control site, a pre-intervention trend or an eligibility threshold. Above that sits a true experimental design, where random allocation balances the factors nobody measured.2 Planning that progression at the outset is cheaper than discovering at write-up that the design cannot carry the claim.
Cochrane’s Effective Practice and Organisation of Care group sets minimum design standards for the studies it will include, and states that “inclusion of uncontrolled before-after studies or cross sectional studies is strongly discouraged”.6 A study built on one of these designs is unlikely to enter a systematic review of effects. If the work is intended to influence practice, that constraint is worth knowing before the data are collected rather than after.
Advantages and limitations
| Advantages | Limitations |
|---|---|
| Feasible when randomisation is impossible for ethical, practical or organisational reasons | No mechanism balances known and unknown differences, and none can be added later2 |
| Quick and inexpensive, needing one or two rounds of measurement and no comparison arm to recruit | The one-group design leaves history, maturation, testing, instrumentation and regression all in play1 |
| Useful for pilot work, feasibility testing and generating hypotheses for a controlled study | The static-group design cannot show the groups were comparable to begin with1 |
| Often the only option for a one-off change that has already happened | Results are unlikely to be eligible for a systematic review of effects6 |
| Instructive: the designs make the function of a control group visible by removing it | Samples are usually small and non-random, so findings generalise poorly |
Common mistakes
| Mistake | Stage | What it costs | What to do instead |
|---|---|---|---|
| Writing “after” as “because” | Write-up | A causal claim the design cannot carry at any sample size | Report the change, then name the rival explanations the design leaves open1 |
| Recruiting the lowest scorers and reporting the improvement | Sampling | Statistical regression produces the result on its own1 | Add a comparison group, or interpret against a predicted regression effect |
| Treating a pretest as a substitute for a control group | Design | The design still leaves five named threats unaddressed1 | Recognise that the pretest controls only the group’s starting level, not elapsed time |
| Comparing two intact groups with no baseline | Design | No way to certify the groups “would have been equivalent”1 | Measure both groups before the intervention if there is any opportunity to do so |
| Changing the rater or the instrument between measurements | Measurement | Instrumentation becomes a rival explanation for the whole effect1 | Keep the instrument and the observers fixed, and say so in the methods |
| Ignoring who dropped out | Analysis | Differential loss can manufacture an improvement1 | Report attrition and compare those who left with those who stayed |
| Calling the study quasi-experimental | Write-up | Misclassification; a quasi-experiment has a defensible comparison, these do not | Name the design precisely, using the published terminology1 |
| Planning a pre-experimental study for evidence that must inform practice | Planning | The result is unlikely to be eligible for synthesis6 | Check the inclusion criteria of the reviews you hope to enter, before collecting data |
Take-home message
Pre-experimental designs are defined by what they lack. The one-shot case study makes no comparison, so it cannot support a causal claim at all. The one-group pretest-posttest design compares a group with its own past and leaves history, maturation, testing, instrumentation and regression unaddressed. The static-group comparison has a concurrent group but no way to show the two were alike. Each remains a reasonable choice for feasibility work, pilots and description, and each is best treated as a stage before a design that secures a comparison. None of them licenses the word “because”.
How to apply this to your work
- Name the design precisely, using the published terminology rather than calling it quasi-experimental.1
- Ask what your comparison actually is. The same group earlier, another group, or nothing at all — each supports a different claim.
- Add a baseline wherever one is possible. It is the cheapest improvement available to a static-group comparison.1
- Check whether you selected on an extreme score, and if so expect regression toward the mean.1
- Hold the instrument and the observers constant across measurements, and record that you did.1
- Report attrition, and compare those who left with those who remained.1
- List the rival explanations in your limitations by name, rather than noting a generic lack of a control group.
- Match the claim to the design: feasible, acceptable, associated with a change — not caused by it.
Common questions
What is a pre-experimental research design?
A design that lacks both random allocation and a defensible comparison group. Campbell and Stanley group three of them under the heading “THREE PRE-EXPERIMENTAL DESIGNS”: the one-shot case study, the one-group pretest-posttest design and the static-group comparison.1 They are studied mainly to show what a control group does, by removing it.
What are the three types of pre-experimental design?
The one-shot case study, written X O: one group treated and measured once. The one-group pretest-posttest design, written O1 X O2: one group measured before and after. The static-group comparison: a treated group and an untreated group, each measured once, with no baseline for either.1
Can a pre-experimental design show cause and effect?
No. The one-shot case study makes no comparison at all, and “securing scientific evidence involves making at least one comparison”.1 The other two make comparisons that leave named rival explanations in play. They can show that something changed; they cannot show what caused it.
Is the static-group comparison the same as a post-test-only design with non-equivalent groups?
Yes. They are two names for one structure: a treated group and an untreated group, each measured once, with no baseline for either and no random allocation. Campbell and Stanley call it the static-group comparison;1 the later label describes the same design in modern terminology. Either name is acceptable provided the methods section states what was actually done.
What is the difference between pre-experimental and quasi-experimental?
Both lack random allocation, but a quasi-experimental design still secures a comparison that can be defended — a control site, a pre-intervention trend, an eligibility threshold. A pre-experimental design does not. That is why Cochrane EPOC accepts some non-randomised designs while stating that “inclusion of uncontrolled before-after studies or cross sectional studies is strongly discouraged”.6
Why does the one-group pretest-posttest design keep appearing?
Because the circumstances that favour it are common: a single cohort, a course or service everyone receives, and no ethical way to withhold it. Knapp addressed the question directly, noting that the design “has been widely criticized” and still appears in clinical nursing research.3 Recent published examples describe themselves in exactly these terms.45
What does X O mean in research design notation?
It is Campbell and Stanley’s shorthand. “An X will represent the exposure of a group to an experimental variable or event”, and “O will refer to some process of observation or measurement”.1 Symbols read left to right in time order, and each row is one group.
Is a pre-experimental study worth publishing?
It can be, if the claim matches the design. Feasibility, acceptability and descriptive findings are publishable and useful. The difficulty arises when a causal claim is attached, or when the work was intended to inform practice but cannot enter a systematic review of effects.6
Declared interest: Pubrica provides research design and manuscript support commercially. Nothing on this page should be read as suggesting that paid support can make a pre-experimental design support a causal claim; no design choice or service can supply a comparison that was never collected.
References and sources
- Campbell DT, Stanley JC. Experimental and Quasi-Experimental Designs for Research. Boston: Houghton Mifflin; 1963. Notation, p. 5; three pre-experimental designs, pp. 6–13. read 20 October 2026.View the full text
- Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. Journal of Clinical Epidemiology. 2010;63:e1–e37. doi:10.1016/j.jclinepi.2010.03.004 read 20 October 2026 from an institutional repository copy.View the full text
- Knapp TR. Why is the one-group pretest–posttest design still used? Clinical Nursing Research. 2016;25(5):467–472. doi:10.1177/1054773816666280 bibliographic record and the opening of the abstract read 20 October 2026; the full text is behind a paywall and is cited here only for the question it poses.View the record on RePEc
- Moon SH, Jeong H, Choi MJ. Integrating mixed reality preparation into acute coronary syndrome simulation for nursing students: a single-group pretest-posttest study. BMC Nursing. 2024;23:468. doi:10.1186/s12912-024-02110-9 read 20 October 2026.View on BMC Nursing
- Çingöl N, Karakaş M. Effect of the management in nursing course on students’ time management and career planning attitudes: a single-group pre-test post-test study. Nurse Education Today. 2023;125:105797. doi:10.1016/j.nedt.2023.105797 read 20 October 2026 from the author institution’s repository.View in the institutional repository
- Cochrane Effective Practice and Organisation of Care (EPOC). What study designs should be included in an EPOC review and what should they be called? EPOC Resources for review authors; 2017. read 20 October 2026.View the EPOC guidance
- Barnett AG, van der Pols JC, Dobson AJ. Regression to the mean: what it is and how to deal with it. International Journal of Epidemiology. 2005;34(1):215–220. first page and abstract read 20 October 2026; the remainder is behind a paywall, and the quotations here are drawn from the part that was read.View the article
- Linden A. Assessing regression to the mean effects in health care initiatives. BMC Medical Research Methodology. 2013;13:119. doi:10.1186/1471-2288-13-119 read 20 October 2026.View the full text
- Bartels C, Wegrzyn M, Wiedl A, Ackermann V, Ehrenreich H. Practice effects in healthy adults: a longitudinal study on frequent repetitive cognitive testing. BMC Neuroscience. 2010;11:118. doi:10.1186/1471-2202-11-118 read 20 October 2026.View on BMC Neuroscience
- Roediger HL III, Karpicke JD. Test-enhanced learning: taking memory tests improves long-term retention. Psychological Science. 2006;17(3):249–255. read 20 October 2026 from a university-hosted copy; no DOI was printed on the copy reached.View the article PDF
- Coste J, Delecoeuillerie G, Cohen de Lara A, LeParc JM, Paolaggi JB. Clinical course and prognostic factors in acute low back pain: an inception cohort study in primary care practice. BMJ. 1994;308(6928):577–580. doi:10.1136/bmj.308.6928.577 read 20 October 2026.View on the BMJ
- Ramsay CR, Matowe L, Grilli R, Grimshaw JM, Thomas RE. Interrupted time series designs in health technology assessment: lessons from two systematic reviews of behavior change strategies. International Journal of Technology Assessment in Health Care. 2003;19(4):613–623. read 20 October 2026 from a university-hosted copy; no DOI was printed on the copy reached.View the article PDF
- Berkhout C, Berbra O, Favre J, Collins C, Calafiore M, Peremans L, et al. Defining and evaluating the Hawthorne effect in primary care, a systematic review and meta-analysis. Frontiers in Medicine. 2022;9:1033486. doi:10.3389/fmed.2022.1033486 read 20 October 2026.View on Frontiers in Medicine
- Bierer SB, Beck Dallaghan G, Borges NJ, Brondfield S, Fung CC, Huggett KN, et al. Moving beyond simplistic research design in health professions education: what a one-group pretest-posttest design will not prove. MedEdPORTAL. 2025;21:11527. doi:10.15766/mep_2374-8265.11527 read 20 October 2026.View on MedEdPORTAL
- King O, West E, Lee S, Glenister K, Quilliam C, Wong Shee A, et al. Research education and training for nurses and allied health professionals: a systematic scoping review. BMC Medical Education. 2022;22:385. doi:10.1186/s12909-022-03406-7 read 20 October 2026.View on BMC Medical Education
- Chae S, Shon S. Effectiveness of simulation-based interprofessional education on teamwork and communication skills in neonatal resuscitation. BMC Medical Education. 2024;24:602. doi:10.1186/s12909-024-05581-1 read 20 October 2026.View on BMC Medical Education
- Lee M. The effect of a mindfulness-based class on mindfulness, achievement emotions, emotion regulation, and empathy in nursing students. Nursing Reports. 2025;15(11):374. doi:10.3390/nursrep15110374 read 20 October 2026.View on Nursing Reports
- Hu L, Li S, Zhou L. Effect of Tronclass combined with team-based learning on nursing students’ self-directed learning and academic performance: a pretest-posttest study. BMC Medical Education. 2024;24:752. doi:10.1186/s12909-024-05741-3 read 20 October 2026.View on BMC Medical Education
- Xie YJ, et al. Use of the PRECEDE-PROCEED model in piloting vaccine promotion and infection self-protection: intervention development and effectiveness examination. Vaccines. 2024;12(9):979. doi:10.3390/vaccines12090979 read 20 October 2026.View on Vaccines
- Santosa CAHF, Suryadi D, Prabawanto S, Syamsuri. The role of worked-example in enhancing students’ self-explanation and cognitive efficiency in calculus instruction. Jurnal Riset Pendidikan Matematika. 2018;5(2):168–180. doi:10.21831/jrpm.v0i0.19602 abstract and article record read 20 October 2026; cited as an instance of a study naming this design, not as evidence of its findings.View the article record
- Hidayati N, Utami IS, Wiyarsi A, Ikhsan J. Analysis of students’ learning motivation on the implementation of direct instruction learning model. Jurnal Penelitian Pendidikan IPA. 2022;8(5):2417–2422. doi:10.29303/jppipa.v8i5.2090 read 20 October 2026; cited solely as an instance of a study labelling its design a one-shot case study. It has 11 participants and no limitations section, so no finding of it is relied on here.View the article
- Goodacre S. Uncontrolled before-after studies: discouraged by Cochrane and the EMJ. Emergency Medicine Journal. 2015;32(7):507–508. doi:10.1136/emermed-2015-204761 read 20 October 2026 from an institutional repository copy.View the full text
- Deeks JJ, Dinnes J, D’Amico R, Sowden AJ, Sakarovitch C, Song F, et al. Evaluating non-randomised intervention studies. Health Technology Assessment. 2003;7(27). read 20 October 2026. The comparison of randomised with historically controlled trials is this monograph’s account of an earlier study, and is cited accordingly.View the monograph
- Thabane L, Ma J, Chu R, Cheng J, Ismaila A, Rios LP, et al. A tutorial on pilot studies: the what, why and how. BMC Medical Research Methodology. 2010;10:1. doi:10.1186/1471-2288-10-1 read 20 October 2026.View the full text
- Eldridge SM, Chan CL, Campbell MJ, Bond CM, Hopewell S, Thabane L, Lancaster GA. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. BMJ. 2016;355:i5239. doi:10.1136/bmj.i5239 read 20 October 2026 from an institutional repository copy of the version of record.View the full text
- Eldridge SM, Lancaster GA, Campbell MJ, Thabane L, Hopewell S, Coleman CL, Bond CM. Defining feasibility and pilot studies in preparation for randomised controlled trials: development of a conceptual framework. PLoS ONE. 2016;11(3):e0150205. doi:10.1371/journal.pone.0150205 read 20 October 2026.View on PLOS ONE
- Des Jarlais DC, Lyles C, Crepaz N, and the TREND Group. Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: the TREND statement. American Journal of Public Health. 2004;94(3):361–366. doi:10.2105/AJPH.94.3.361 read 20 October 2026.View the statement and checklist
- Sterne JAC, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355:i4919. doi:10.1136/bmj.i4919 read 20 October 2026 from a university-hosted copy.View the full text
- Nianogo RA, Benmarhnia T, O’Neill S. A comparison of quasi-experimental methods with data before and after an intervention. International Journal of Epidemiology. 2023;52(5):1522–1533. doi:10.1093/ije/dyad032 abstract and opening of the full text read 20 October 2026; later sections did not render, and nothing from them is relied on here.View on the International Journal of Epidemiology
- Lancaster GA, Dodd S, Williamson PR. Design and analysis of pilot studies: recommendations for good practice. Journal of Evaluation in Clinical Practice. 2004;10(2):307–312. abstract read 20 October 2026 from the author institution’s repository; the full text was not available there, and the quotations here come from the abstract.View the record
- Grimshaw J, Campbell M, Eccles M, Steen N. Experimental and quasi-experimental designs for evaluating guideline implementation strategies. Family Practice. 2000;17(Suppl 1):S11–S16. doi:10.1093/fampra/17.suppl_1.S11 abstract read 20 October 2026; the full text was not available, and the quotation here comes from the abstract.View the record
- Krogsbøll LT, Hróbjartsson A, Gøtzsche PC. Spontaneous improvement in randomised clinical trials: meta-analysis of three-armed trials comparing no treatment, placebo and active intervention. BMC Medical Research Methodology. 2009;9:1. doi:10.1186/1471-2288-9-1 abstract read 20 October 2026 from the author institution’s research portal.View the record
- Whiteford HA, Harris MG, McKeon G, Baxter A, Pennell C, Barendregt JJ, Wang J. Estimating remission from untreated major depression: a systematic review and meta-analysis. Psychological Medicine. 2013;43(8):1569–1585. abstract read 20 October 2026; no DOI was printed on the page reached.View on Cambridge Core
- Thompson M, Vodicka TA, Blair PS, Buckley DI, Heneghan C, Hay AD, on behalf of the TARGET Programme Team. Duration of symptoms of respiratory tract infections in children: systematic review. BMJ. 2013;347:f7027. doi:10.1136/bmj.f7027 read 20 October 2026.View the full text
- Pengel LHM, Herbert RD, Maher CG, Refshauge KM. Acute low back pain: systematic review of its prognosis. BMJ. 2003;327(7410):323. doi:10.1136/bmj.327.7410.323 read 20 October 2026.View the full text
- Hróbjartsson A, Gøtzsche PC. Is the placebo powerless? An analysis of clinical trials comparing placebo with no treatment. New England Journal of Medicine. 2001;344(21):1594–1602. read 20 October 2026 from a university-hosted copy.View the full text
- Bland JM, Altman DG. Statistics notes: some examples of regression towards the mean. BMJ. 1994;309(6957):780. doi:10.1136/bmj.309.6957.780 read 20 October 2026.View on the BMJ
- Vickers AJ, Altman DG. Statistics notes: analysing controlled trials with baseline and follow up measurements. BMJ. 2001;323(7321):1123–1124. doi:10.1136/bmj.323.7321.1123 read 20 October 2026.View on the BMJ
- Vickers AJ. The use of percentage change from baseline as an outcome in a controlled trial is statistically inefficient: a simulation study. BMC Medical Research Methodology. 2001;1:6. read 20 October 2026.View the full text
How to cite this article
Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.
This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.
Work through these before you write up.
[ ] Design named precisely, not called quasi-experimental [ ] Stated in notation (X O, O1 X O2, or the static-group layout) [ ] Identified what the comparison actually is, if any [ ] Baseline measured wherever it was possible [ ] Checked whether the group was selected on an extreme score [ ] Regression toward the mean considered if it was [ ] Same instrument used at both measurements [ ] Same observers or scorers at both measurements [ ] Attrition reported, leavers compared with stayers [ ] Events during the study period recorded (history) [ ] Elapsed time between measurements stated (maturation) [ ] Pretest effects considered (testing) [ ] Rival explanations named individually in the limitations [ ] Claim matched to the design: changed, not caused
We help you establish whether a stronger comparison is available before data collection begins, name the design correctly in the methods, and write limitations that address each rival explanation by name rather than in general terms.