Free scoping read. A written assessment of your manuscript in two business days, at no cost.

Start one
Contents

Academy · Statistical analysis · Biostatistics in practice

Biostatistics in Practice: Correct Interpretation of Commonly Misstated Concepts

A p-value is not the probability that a result arose by chance. An earlier version of this page stated that misinterpretation, and this revision is largely concerned with correcting it.

Pubrica AcademyWritten by a research writing specialist Reviewed bya research methodology reviewer Published15 October 2026 Last checked15 October 2026 Reading time12 minutes Sources11 primary

Biostatistics is the application of statistical reasoning to biological and medical questions: determining what to measure and in how many participants, summarising the findings, and distinguishing what the data support from what they do not. This article examines the concepts most frequently misstated, beginning with the one that an earlier version of this page misstated.

The function of biostatistics in a study

Statistics enters a study at three points: when the question is turned into something measurable and a sample size is chosen, when the data are summarised, and when an inference is drawn from the sample to the population. The third is where the reasoning is hardest and where most published errors live1.

Ali and Bhaskar's overview covers the ground from descriptive measures through to the choice between parametric and non-parametric tests, ending, as they put it, with “Finally, there is a summary of parametric and non-parametric tests used for data analysis.”9 Guetterman’s primer sets out the same foundations for clinical research10.

A sourcing note. Several of the journal articles cited on this page could not be opened in full from our environment, because PubMed Central and a number of publisher sites are closed to automated retrieval. Where that is so, the citation is verified against the publisher’s own index record and nothing is quoted from the body of the paper. Each reference below names the host that was actually read.

The correct definition of the p-value

A p-value is frequently defined as the “probability that the observed data occurred by chance”. That is the standard misinterpretation, and the reasons are set out precisely below.

Reasons the chance-based definition is incorrect

That definition inverts the conditional probability. A p-value is calculated assuming the null hypothesis is true: it is the probability of data at least as extreme as those observed, under that assumption. It is neither the probability that the null hypothesis is true nor the probability that the result arose by chance, since both would require a probability attached to the hypothesis, which a p-value does not supply. The distinction is substantive: the incorrect reading converts p = 0.04 into “a 4% chance this is a fluke, so a 96% chance the effect is real”, a claim that the arithmetic does not support.

Greenland and colleagues wrote their guide specifically to catalogue these misreadings1, and the American Statistical Association’s statement on statistical significance makes the same correction in its own principles2. Andrade’s paper on the subject takes the argument further, to the point that “research data contain much more meaning than is summarized in a P value and its statistical significance”4 — which is the practical lesson. The p-value is one summary of one comparison, and Pernet’s tutorial puts the limit plainly: “The p-value is not an indication of the strength or magnitude of an effect.”5

Two consequences for how you write up a study. A result with p = 0.049 and one with p = 0.051 are not different findings, and describing one as significant and the other as not is a presentational choice rather than a scientific one. And a non-significant result is not evidence of no effect: as Altman and Bland put it, “This term wrongly implies that the study has shown that there is no difference,” “whereas usually all that has been shown is an absence of evidence of a difference.” “These are quite different statements.”3 Our page on the null hypothesis works through that distinction in detail.

A caution regarding one frequently consulted source

A caution is warranted regarding one otherwise reputable source, which is hosted by the National Center for Biotechnology Information and is therefore readily trusted. The StatPearls chapter on hypothesis testing states that the p-value is “the probability that the observed effect within the study would have occurred by chance”. That is the same error that this revision removes. Its confidence-interval entry likewise carries no warning about the standard misinterpretation. Hosting by a reputable institution does not in itself confer authority.

Descriptive and inferential statistics

Descriptive statistics summarise the data in front of you. Hazra and Gogtay’s module on the basics starts from the observation that “Data are thus the values of a variable.”7, and covers the measures of central tendency and dispersion that describe how those values are distributed7.

Inferential statistics do something different in kind: they use the sample to make a statement about the population it was drawn from. Every estimate, every interval and every hypothesis test is an inferential claim, and each one carries assumptions about how the sample relates to the population — which is why a biased sample cannot be rescued by a sophisticated analysis.

The boundary matters in writing. “The mean age in our cohort was 62” is descriptive and needs no defence. “Patients with this condition are around 62” is inferential, and needs a sample that supports it.

Confidence intervals

A confidence interval is the more informative companion to a p-value, and Hazra and Gogtay describe it as “The confidence interval (CI) is an increasingly important measure of precision.”7

Its correct interpretation is a statement about the procedure rather than about the individual interval: across repeated studies, “X% of times the CI obtained will contain the true parameter value”5 The misreading to avoid is treating the stated percentage as the probability that the true value lies within the one interval you have, which attaches a probability to the parameter — something a frequentist interval does not do5.

Why prefer it to a p-value in a results section: the interval carries the magnitude and the precision together. A confidence interval from 0.98 to 1.02 and one from 0.40 to 2.50 may both include the null, but they say entirely different things about what the study established. Reporting the interval lets a reader tell them apart; reporting only “not significant” does not1.

Type I and Type II error

The two ways an inference can go wrong, in Pernet’s wording: a Type I error is “rejecting H0 when there is no effect, the type I error”5, and a Type II error is “not rejecting H0 when there is an effect, the type II error.”5

They are controlled by different things. The Type I error rate is the significance level you adopt, and it is a choice. The Type II error rate is one minus the power, and power is mostly a function of sample size and effect size — which is why it has to be addressed at the design stage. A study that is too small does not produce a cautious answer; it produces an uninformative one, and no analysis performed afterwards can recover the information that was never collected8.

Citation note on this source

This tutorial exists in five versions on an open-review platform, and its title changed after version 3: from “a short tutorial” to “a guide to commonly misunderstood concepts and recommendations for good practice”. Its open peer review is also mixed rather than settled. It is cited here for the definitions of Type I and Type II error and for its warning about confidence intervals, which are uncontroversial; the correction of the p-value definition itself rests on Greenland and colleagues and on the American Statistical Association’s own statement.

Parametric and non-parametric tests

The distinction is about assumptions, not about difficulty. A parametric test rests on assumptions about the distribution the data came from — principally “the assumption of normality, which specifies that the means of the sample group are normally distributed.”6, together with an assumption of equal variance across the groups being compared6.

A non-parametric test makes fewer such assumptions, because “it only uses + or − signs or the rank of data sizes instead of the original values of the data.”6 That is what buys the robustness, and it is also what costs the power: when the parametric assumptions do hold, the parametric test extracts more from the same data.

So the choice is a question about the data, answered before the test is run, and not a matter of trying both and reporting the one that reached significance. That last practice inflates the Type I error rate, and it is not recoverable by disclosure.

Variable types and their role in test selection

The type of variable you have constrains almost everything downstream: how it is summarised, which test applies, and what a result can mean. Kaliyadan and Kulkarni cover the variable types alongside descriptive statistics and sample size, which is the right grouping, because the three decisions are connected8.

The practical order is: establish what kind of variable each outcome is, decide how it will be summarised, and only then choose the test. Reversing that — picking a familiar test and treating the data as whatever it requires — is how continuous outcomes get split into arbitrary categories and ordinal scales get averaged.

The appropriate stage for statistical involvement

At the design stage, and the reason is structural rather than a matter of courtesy. Once data exist, the sample size, the allocation, the outcome definitions and the comparisons available are all fixed.

Zapf and colleagues wrote on this precisely because expectations of the role are often misplaced: “the aim of this article is to outline the tasks and responsibilities of biostatisticians in clinical trials”11 and to distinguish that role from epidemiology, medical informatics and bioinformatics, which are different disciplines11.

Zapf and colleagues (2020) are sometimes cited in support of statistical methodology. The paper is, however, a commentary on the professional role of the biostatistician: what the role involves and how it differs from epidemiology, medical informatics and bioinformatics. Zapf has published separately on inter-rater reliability, which is the probable source of the confusion. On this page the reference supports a single claim: that statistical expertise belongs on the research team from the outset.

One frequently cited reference is not used here. An introductory statistics note attributed to Ranganathan and Gogtay in the Indian Journal of Critical Care Medicine could not be retrieved from any host accessible to the authors of this page, and its authors, volume, pages and identifier are therefore unverified. A reference that has not been consulted should not be cited, and a two-page supplementary note would not in any case have supported a substantive claim.

Recommended working practices

Practical application

  1. The p-value should be defined conditionally or not defined at all. It is the probability of data at least as extreme as those observed given the null hypothesis, and any phrasing that omits that condition is incorrect.
  2. Effect estimates should be reported with confidence intervals rather than with p-values alone. The interval conveys magnitude and precision; the p-value conveys neither.
  3. A non-significant result should not be described as showing no difference. Such a result is consistent with no effect, with a small effect, and with a study too small to detect a real effect.
  4. The analysis, including the test and its assumptions, should be decided before the data are examined. Selecting a test according to the result it yields inflates the error rate.
  5. Absolute numbers should be given alongside every percentage and ratio. A proportion reported without its denominator cannot be assessed.
  6. Descriptive and inferential claims should be kept distinct in the text. Summarising the sample and describing the population are different statements.
  7. Power should be addressed at the design stage. An underpowered study produces an uninformative result rather than a cautious one.
  8. Statistical input should be obtained before data collection rather than afterwards, by which time the consequential design decisions have already been made.

Key points

A p-value is the probability of data at least as extreme as those observed if the null hypothesis were true; it is not the probability that the result arose by chance. Confidence intervals should be reported because they show magnitude and precision, and a non-significant result is not evidence of no effect. The Type I error rate is the chosen significance level, whereas Type II error relates to power and therefore to sample size. Parametric tests gain power through their assumptions, and the choice of test is made before analysis. Statistical input belongs at the design stage, when decisions remain open.

The probability of observing data at least as extreme as those observed if the null hypothesis were true. This conditional clause is essential to the definition. It is not the probability that the null hypothesis is true, nor the probability that the observed result arose by chance; both would require a probability attached to the hypothesis itself, which a p-value does not provide.

No. This is the most common misinterpretation in applied statistics. The p-value is computed on the assumption that chance alone is operating; it cannot also be the probability that that assumption holds. Greenland and colleagues list this among the misinterpretations their guide exists to correct, and the American Statistical Association’s statement makes the same point.

If the study were repeated many times, the stated percentage of the intervals produced would contain the true parameter value. It is a statement about the procedure, not about any individual interval. The common misinterpretation, that there is a 95% probability that the true value lies within a particular interval, attaches a probability to the parameter, which frequentist statistics does not do.

A Type I error is the rejection of the null hypothesis when there is no effect (a false positive). A Type II error is the failure to reject it when there is an effect (a false negative). The first is controlled by the chosen significance level; the second by the power of the study, which is largely determined by sample size.

Descriptive statistics summarise the sample obtained: counts, means, medians, measures of spread and the plots that display a distribution. Inferential statistics use that sample to draw conclusions about the population from which it was drawn, and estimates, intervals and hypothesis tests belong to this domain. Confusing the two leads to a summary of 40 patients being presented as a claim about a disease.

When its assumptions hold, principally normality and equal variance. A non-parametric test makes fewer assumptions about the distribution, using the signs or ranks of the data rather than the values themselves, and is the appropriate choice when the parametric assumptions do not hold. The trade-off is some loss of statistical power when those assumptions do hold.

At the design stage, before any data are collected. Once data exist, the sample size, the allocation, the outcome definitions and the analysis plan are all fixed, and a statistician can only describe the data available rather than help to ensure that the data required are obtained. Zapf and colleagues set out the role of the biostatistician in clinical trials precisely because these expectations are often misplaced.

References

  1. Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, Altman DG. Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. European Journal of Epidemiology. 2016;31:337–350. doi:10.1007/s10654-016-0149-3 read 15 October 2026. View on European Journal of EpidemiologyThe reference work on how p-values are misread. The definitions and the two misinterpretations quoted on this page are taken from it verbatim.
  2. Wasserstein RL, Lazar NA. The ASA’s Statement on p-Values: Context, Process, and Purpose. The American Statistician. 2016;70(2):129–133. doi:10.1080/00031305.2016.1154108 read 15 October 2026. View on The American StatisticianThe American Statistical Association’s formal statement on p-values. Its numbered principles are quoted on this page as published.
  3. Altman DG, Bland JM. Statistics notes: Absence of evidence is not evidence of absence. BMJ. 1995;311(7003):485. doi:10.1136/bmj.311.7003.485 read 15 October 2026. View on The BMJOne page, and the clearest statement of what a non-significant result does and does not license you to say.
  4. Andrade C. The P value and statistical significance: Misunderstandings, explanations, challenges, and alternatives. Indian Journal of Psychological Medicine. 2019;41(3):210–215. doi:10.4103/IJPSYM.IJPSYM_193_19 read 15 October 2026. View on DOAJ recordOn the misinterpretation of p-values and significance, the arguments for and against the 5% threshold, and the alternatives. Verified from the index record; the publisher's full text was not reachable.
  5. Pernet C. Null hypothesis significance testing: a short tutorial. F1000Research. 2015;4:621. (Current version v5, 2017.) read 15 October 2026. View on F1000ResearchA tutorial written for researchers rather than statisticians. Note the article has five versions; v5 is current, and an earlier Pubrica page cited v3.
  6. Nahm FS. Nonparametric statistical tests for the continuous data: the basic concept and the practical use. Korean Journal of Anesthesiology. 2016;69(1):8–14. doi:10.4097/kjae.2016.69.1.8 read 15 October 2026. View on a university course mirrorThe source for what actually distinguishes a non-parametric test and for the assumptions a parametric test rests on. Read from a University of Vermont course PDF because the publisher's site was not reachable; the citation is to the journal.
  7. Hazra A, Gogtay N. Biostatistics series module 1: Basics of biostatistics. Indian Journal of Dermatology. 2016;61(1):10–20. doi:10.4103/0019-5154.173988 read 15 October 2026. View on DOAJ recordModule 1 of a series, covering data types, central tendency, dispersion and confidence intervals. It is a descriptive-statistics module and establishes nothing about hypothesis testing, so no p-value claim is attached to it.
  8. Kaliyadan F, Kulkarni V. Types of variables, descriptive statistics, and sample size. Indian Dermatology Online Journal. 2019;10(1):82–86. doi:10.4103/idoj.IDOJ_468_18 read 15 October 2026. View on DOAJ recordCovers variable types, basic descriptive statistics and sample-size estimation. The authors call it a snippet, and it is cited only for those three things. Volume, issue and page range come from the citation rather than the index record, which omits them.
  9. Ali Z, Bhaskar SB. Basic statistical tools in research and data analysis. Indian Journal of Anaesthesia. 2016;60(9):662–669. doi:10.4103/0019-5049.190623 read 15 October 2026. View on DOAJ recordAn overview of descriptive and inferential methods, ending in a summary of parametric and non-parametric tests. Verified from the Directory of Open Access Journals record; the journal's own full text was not reachable from here, so nothing from the body of the paper is quoted.
  10. Guetterman TC. Basics of statistics for primary care research. Family Medicine and Community Health. 2019;7(2):e000067. doi:10.1136/fmch-2018-000067 read 15 October 2026. View on BMJ (issue contents)A statistics primer for primary care research. Author, title, journal, volume, issue, article number and DOI were confirmed against the publisher's own issue contents page; the full text was not reachable, so no wording is quoted from it.
  11. Zapf A, Rauch G, Kieser M. Why do you need a biostatistician? BMC Medical Research Methodology. 2020;20(1):23. doi:10.1186/s12874-020-0916-4 read 15 October 2026. View on University Medical Center Hamburg-Eppendorf research portalA commentary on the tasks and responsibilities of biostatisticians in clinical trials, and on distinguishing them from epidemiologists and bioinformaticians. It is cited here for that and only that.

Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.

Related: the null hypothesis and what a p-value does not say · statistical significance versus effect size.

How Pubrica can help

Pubrica’s biostatisticians work on study design and analysis: sample size and power calculations before data collection, statistical analysis plans, the analysis itself, and the methods and results text that reports it to journal standards. Responses to statistical comments raised by reviewers on a submitted manuscript can be undertaken as a separate, defined piece of work.

Get a free quote

Regulations

Check your project against this rule

  1. 1Your work
  2. 2Your details
  3. 3Send
What do you need?

Material you send is held under signed confidentiality, with access limited to the assigned team and encrypted transfer. It is not used to train anything and is not passed to third-party tools. Ask for the agreement before you send anything you would rather cover first.