Home/Insights/Evidence synthesis/Choosing the data analysis for a research paper
Choosing the data analysis for a research paper
The analysis follows from the design, the outcome type and the structure of the data — not from what is familiar.
The three questions that determine the test
What kind of outcome? Continuous, binary, count, time-to-event, ordinal. Each has its own family of methods, and treating an ordinal scale as continuous is a decision requiring justification.
What is the structure? Are observations independent, or clustered within clinicians, sites or participants, or repeated over time? Non-independence changes the method, not just the standard errors.
What is the question? Difference between groups, association, prediction, agreement, or diagnostic accuracy. Prediction and explanation require different model-building strategies, and the same regression output serves them differently.
Common correct pairings
Two independent groups, continuous outcome — t-test or its non-parametric equivalent; regression if adjustment is needed. Binary outcome — chi-squared or logistic regression. Time to event — Kaplan–Meier with log-rank, and Cox regression for adjustment, after checking proportional hazards. Repeated measures — mixed-effects models or GEE. Agreement — Bland–Altman with limits of agreement, and kappa for categorical agreement, not correlation.
Correlation is not agreement. Two methods can correlate almost perfectly and disagree systematically, which is why Bland–Altman exists.
The errors reviewers look for
Clustered or repeated data analysed as independent. Multiple comparisons without adjustment or acknowledgement. Baseline imbalance "tested" in a randomised trial, where any imbalance is by definition chance. Change scores analysed without accounting for baseline. Dichotomising a continuous variable, which discards information and inflates type I error at the chosen cut-point. Reporting p-values without effect sizes or intervals.
Reporting
Name the software and version, state the significance level and whether tests were one- or two-sided, state how missing data were handled, and report assumption checks. SAMPL gives the statistical reporting items; CONSORT, STROBE and the other design guidelines each contain their own statistical requirements.
In practice
Fix, flag or query.
The distinction that keeps an edit defensible: what an editor may change silently, what must be shown to the author, and what only the author can answer.
- Match the method to outcome type and data structure, not to habit.
- Do not dichotomise a continuous variable without a stated reason.
- Report effect sizes with intervals, and name the software version.
Sources
- SAMPL guidelines, statistical analyses and methods in the published literature
- EQUATOR Network, design-specific statistical reporting items
Sources are given so each claim can be checked at its origin. Where a tool’s current maintenance status could not be confirmed, the page says to check it rather than asserting it.
Related