A nonnegative scalar that quantifies the amount of statistical dependence between two random variables by measuring how much knowing one reduces uncertainty about the other.

Principle

Express shared information as the reduction in uncertainty about one variable given knowledge of the other without presupposing linear relationships.

Demonstration

Two binary variables that are identical have maximal mutual information equal to the variable's information content; if they are independent, mutual information is zero.

Misapplication

Interpreting a large mutual information value as evidence of causal influence confuses dependence with directionality and can be misleading without temporal or intervention data.

Consequence

Serves as a model-agnostic criterion for feature selection, variable clustering, and detection of nonlinear associations between variables.

Reversal

Independence is the inverse situation: zero mutual information implies no statistical dependence under the chosen probability model.

Boundary

Requires a joint distribution; for continuous variables practical estimation demands density or discretization methods and suffers from sample-size limitations.

Semantic Tension

Versus correlation coefficients: mutual information captures arbitrary dependence including nonlinear relationships, while correlation measures only linear association strength.

Synthesis

A scalar measure of dependence that quantifies how much one variable reduces uncertainty about another, applicable to discrete or continuous random variables with appropriate estimation.