What Is a P-Value?

A p-value is the probability of obtaining results at least as extreme as those observed, if the null hypothesis and all other model assumptions were true. It measures how compatible the data are with a specified model, not the probability that a hypothesis is true.

What it is not

• It is not the probability that the null hypothesis is true.
• It is not the probability that the result occurred "by chance".
• It does not measure the size or importance of an effect.
• A p-value above 0.05 does not prove there is no effect; it means the data are not clearly incompatible with no effect.

Using a threshold

A conventional threshold of 0.05 is arbitrary. The American Statistical Association has warned against basing conclusions only on whether a p-value crosses a threshold, and recommended reporting effect sizes, intervals and context. Report exact p-values (for example p = 0.03) rather than only "p < 0.05".

Factors that influence it

• Effect size: larger effects tend to give smaller p-values.
• Sample size: very large samples can give tiny p-values for trivial effects; small samples may give large p-values for important effects.
• Variability and the chosen test.
• Multiple testing, which increases the chance of at least one small p-value by chance.

Illustrative example

Invented for teaching. Two trials test a drug. Trial A (20,000 participants) finds a 1 mm Hg reduction in blood pressure with p < 0.001. Trial B (40 participants) finds a 10 mm Hg reduction with p = 0.08. A is highly "significant" but clinically trivial; B is imprecise but may be important. The effect size and confidence interval tell the real story.

How to report

State the estimate, its 95% confidence interval, the exact p-value and the test used.

Common pitfalls

• Declaring "no difference" when p > 0.05.
• P-hacking: running many analyses and reporting the one with p < 0.05.

Key takeaways

• A p-value measures compatibility of data with a model, not truth of a hypothesis.
• Always report effect size and confidence interval.

This page is general educational information about research methods. It is not medical, legal or regulatory advice; follow your institution's ethics board and your target journal's instructions.