Home/Insights/Methodology reference/The scientific approach in research
The scientific approach in research
Hypotheses are tested by trying to refute them, which is why a study designed only to confirm an expectation tells you very little.
Inference, and its direction
Deductive work starts from a theory, derives a testable prediction, and tests it. Inductive work starts from observation and builds toward explanation. Most real programmes alternate, but a single paper should be clear about which it is doing.
A prediction is scientifically useful in proportion to what would refute it. A hypothesis compatible with every possible result is not being tested.
Why confirmatory and exploratory must be separated
A confirmatory analysis tests a hypothesis stated before the data were seen. An exploratory analysis generates hypotheses from data already in hand. Both are legitimate; presenting the second as the first is not.
The practical consequence is multiplicity. Testing many outcomes and reporting the significant ones produces findings that do not replicate. Preregistration, and a stated primary outcome, exist to make the distinction verifiable rather than a matter of the authors’ word.
What a p-value does and does not say
A p-value is the probability of data at least as extreme as those observed, assuming the null hypothesis is true. It is not the probability that the hypothesis is false, and it is not a measure of effect size.
Statistical significance and clinical or practical importance are different questions. A large study can make a trivial difference significant; a small one can leave an important difference undetected. Reporting effect sizes with confidence intervals addresses both.
In practice
Fix, flag or query.
The distinction that keeps an edit defensible: what an editor may change silently, what must be shown to the author, and what only the author can answer.
- Never: present a post-hoc finding as a pre-specified hypothesis.
- Always report: effect size and interval, not significance alone.
- Preregister: the primary outcome and the analysis plan, where the design allows.
Reporting standards
- ICMJE Recommendations, on trial registration and pre-specified outcomes
- American Statistical Association, statement on statistical significance and p-values
Reporting guidelines named here are hosted by the EQUATOR Network. Writing to the checklist for your design before submission is the cheapest available improvement to a methods section.
Related