Contents
Academy · Research Writing · Pre-registration vs post-hoc analysis
Pre-Registration and Post-Hoc Analysis: The Evidentiary Value of Confirmatory and Exploratory Findings
A result that confirms a pre-registered hypothesis and a result that emerges from exploration of the same dataset do not constitute equally strong evidence, even when the statistics appear identical.
Pre-registration means publicly recording a study's hypotheses, outcomes and analysis plan before data collection or analysis begins. Post-hoc analysis is any analysis not specified in advance, conducted after the data has been seen. The same statistical test can be confirmatory evidence in one case and hypothesis-generating in the other, depending entirely on which it was.
Learning objectives
- Why pre-registration specifically addresses the problem of flexible analysis choices made after seeing the data.
- The meaning of researcher degrees of freedom and the reasons they inflate false-positive rates.
- Why post-hoc analyses still have legitimate value when properly labelled.
Common misconceptions
- “Running a different statistical test after the first did not reach significance is a minor, harmless adjustment.” Nosek and colleagues describe how the many small, seemingly reasonable analytic choices available after seeing data — which covariates to include, which subgroup to examine, which test to run, when to stop collecting data — collectively inflate the true false-positive rate well above the nominal 5% if not pre-specified, a phenomenon sometimes termed “researcher degrees of freedom”, or p-hacking when exploited to reach significance.
- “Pre-registration prevents a study from exploring its data at all.” Nosek and colleagues are explicit that pre-registration does not prohibit additional, unplanned analyses — it requires that such analyses be clearly labelled as exploratory and distinguished from the pre-registered confirmatory analyses, preserving the value of open-ended exploration while being honest about its lower evidentiary weight.
Elements fixed by pre-registration
Nosek, Ebersole, DeHaven and Mellor describe pre-registration as publicly time-stamping a study's hypotheses, design, outcome measures and planned analysis before the data is collected or examined, creating a verifiable record that distinguishes analyses planned in advance from those devised afterward1. This record enables a reader to verify later that a reported confirmatory result was genuinely predicted in advance, rather than selected from among many possible analyses after the fact.
Researcher degrees of freedom and their significance
Nosek and colleagues explain that, without pre-registration, a researcher has considerable flexibility in analytic choices made after seeing the data — which variables to control for, which subgroups to examine, which of several plausible outcome definitions to report, when to stop collecting additional data — and that each such choice, even if individually defensible, adds an opportunity for a false-positive result to emerge by chance, collectively inflating the real false-positive rate well beyond the nominal threshold used for any single test1.
Legitimate value of appropriately labelled post-hoc analysis
Post-hoc exploration remains a legitimate and valuable part of research — it is how new hypotheses are generated for future, pre-registered confirmatory testing. Nosek and colleagues' point is not that exploration should stop, but that it must be clearly and honestly distinguished from confirmatory, pre-registered analysis in how results are reported, so readers can correctly calibrate how much evidentiary weight each finding deserves1.
Key points
Key points
Pre-registration creates a verifiable, time-stamped record of a study’s planned hypotheses and analysis, which is what allows a confirmatory result to be trusted; post-hoc analyses remain valuable for generating new hypotheses, but should be explicitly labelled as exploratory rather than presented with the same confidence as a pre-registered finding.
Practical application
Practical application
- Hypotheses, outcomes and the full analysis plan should be registered before data collection or analysis begins. A public registry (such as a trial registry, the Open Science Framework [OSF] or a journal-based registered report) should be used to time-stamp the plan.
- Every reported analysis should be labelled as either pre-registered/confirmatory or post-hoc/exploratory. The distinction should be made explicit in the methods and results rather than left to the reader to infer.
- A post-hoc finding should be treated as a hypothesis for a future pre-registered study rather than as a standalone conclusion. Unplanned significant findings should be framed as motivation for further confirmatory research, not as established results in their own right.
Frequently asked questions
No — it specifically addresses analytic flexibility and selective reporting of analyses, but does not by itself address other sources of bias such as selection bias or confounding, which require their own design safeguards.
Yes, but any deviation should be disclosed and explained, with the original pre-registered plan still available for comparison, so readers can judge the deviation's impact on the result's interpretation.
No — it is increasingly used and expected across observational studies, psychology, and other empirical research fields precisely because the researcher-degrees-of-freedom problem is not specific to trials.
References
- Nosek BA, Ebersole CR, DeHaven AC, Mellor DT. The preregistration revolution. Proceedings of the National Academy of Sciences. 2018;115(11):2600-2606. read 10 October 2026. View on PubMed Central
Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.
How to cite this article
Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.
Pubrica Academy. (2026, October 10). Pre-Registration and Post-Hoc Analysis: The Evidentiary Value of Confirmatory and Exploratory Findings. Pubrica. https://pubrica.com/academy/concepts-definitions/preregistration-vs-post-hoc-analysis/
Pubrica Academy. “Pre-Registration and Post-Hoc Analysis: The Evidentiary Value of Confirmatory and Exploratory Findings.” Pubrica, 10 Oct. 2026, pubrica.com/academy/concepts-definitions/preregistration-vs-post-hoc-analysis/.
Pubrica Academy (2026) Pre-Registration and Post-Hoc Analysis: The Evidentiary Value of Confirmatory and Exploratory Findings. Pubrica. Available at: https://pubrica.com/academy/concepts-definitions/preregistration-vs-post-hoc-analysis/ (Accessed: 15 October 2026).
Pubrica Academy. “Pre-Registration and Post-Hoc Analysis: The Evidentiary Value of Confirmatory and Exploratory Findings.” Pubrica. October 10, 2026. https://pubrica.com/academy/concepts-definitions/preregistration-vs-post-hoc-analysis/
Pubrica Academy, ‘Pre-Registration and Post-Hoc Analysis: The Evidentiary Value of Confirmatory and Exploratory Findings’ (Pubrica, 10 October 2026) <https://pubrica.com/academy/concepts-definitions/preregistration-vs-post-hoc-analysis/> accessed 15 October 2026.
Pubrica Academy. Pre-Registration and Post-Hoc Analysis: The Evidentiary Value of Confirmatory and Exploratory Findings. [Internet]. Pubrica; 2026 Oct 10 [cited 2026 Oct 15]. Available from: https://pubrica.com/academy/concepts-definitions/preregistration-vs-post-hoc-analysis/
Pubrica Academy, “Pre-Registration and Post-Hoc Analysis: The Evidentiary Value of Confirmatory and Exploratory Findings,” Pubrica, Oct. 10, 2026. [Online]. Available: https://pubrica.com/academy/concepts-definitions/preregistration-vs-post-hoc-analysis/
@misc{preregistration_vs_post_hoc_analysis_2026,
author = {{Pubrica Academy}},
title = {Pre-Registration and Post-Hoc Analysis: The Evidentiary Value of Confirmatory and Exploratory Findings},
howpublished = {Pubrica},
year = {2026},
month = {oct},
url = {https://pubrica.com/academy/concepts-definitions/preregistration-vs-post-hoc-analysis/},
note = {Accessed: <span data-cite-acc="iso">2026-10-15</span>}
}This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.
Pre-registration check
Use when planning a study's registration and analysis plan.
[ ] Hypotheses, outcomes and analysis plan registered publicly before data collection/analysis [ ] Every reported analysis labelled pre-registered/confirmatory or post-hoc/exploratory [ ] Any deviation from the registered plan disclosed and explained [ ] Post-hoc findings framed as hypothesis-generating, not standalone conclusions
Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.
How Pubrica can help
Pubrica assists in drafting a complete, registrable analysis plan before data collection and verifies that the results section distinguishes confirmatory from exploratory findings transparently.