Given a specified null model and a test statistic computed from data, the p-value is the probability, under the null model, of observing a test statistic at least as extreme as the observed value.

Principle

The p-value measures data extremeness relative to a null model; it is a tail probability computed under assumptions that define that model and the statistic's sampling distribution.

Demonstration

In a t-test comparing a sample mean to a null mean, compute the t-statistic from data, then the p-value is the probability under the null Studentized distribution to exceed the observed t in magnitude.

Misapplication

Interpreting the p-value as the probability that the null hypothesis is true, or as the probability that the observed effect is practically important, or using an arbitrary threshold without considering power and multiple testing.

Consequence

Correct use yields a calibrated measure of incompatibility between data and the null model that can guide decisions about rejecting the null, subject to control of error rates across repetitions.

Reversal

A large p-value indicates the data are not unlikely under the null; this does not confirm the null but fails to provide evidence against it.

Boundary

Depends on the exact null model, the choice of test statistic, and assumptions (independence, distributional form); not defined without a fully specified null and test procedure.

Semantic Tension

Often contrasted with measures of effect magnitude or Bayesian posterior probabilities; p-values address sampling extremeness under a model, not direct probability of hypotheses or effect sizes.

Synthesis

A p-value is the tail probability, computed under a specified null model and test statistic, that quantifies how extreme the observed data are relative to that model.