A one-parameter family of entropy measures for a discrete probability distribution p, defined for order α≥0, α≠1 by S_α = (1/(1−α)) log(sum_i p_i^α), which interpolates between measures of support size and concentration as α varies.

Principle

The parameter α reweights probabilities: lower α emphasizes the number of nonzero outcomes, higher α accentuates the most probable events; the α→1 limit recovers the standard information entropy.

Demonstration

For a biased coin with probabilities p and 1−p, the Rényi entropy S_α(p) decreases monotonically with α; S_0 equals the logarithm of support size (log 2 for a fair coin), while S_∞ equals −log(max(p,1−p)).

Misapplication

Applying the discrete formula directly to non-normalized measures or interpreting Rényi entropy as a linear functional under mixing, which it is not for α≠1.

Consequence

Rényi entropies provide tunable diversity indices and are useful for comparing concentration across distributions, for characterizing multifractal spectra, and for defining generalized entropic quantities in information tasks.

Reversal

Taking α→0 emphasizes counting of outcomes (Hartley-type entropy), whereas α→∞ focuses exclusively on the most likely event (min-entropy).

Boundary

Defined for probability measures (discrete or via density generalizations) and parameter α values for which the sum converges; not a distance or divergence without additional construction and sensitive to support and normalization.

Semantic Tension

Often contrasted with Tsallis entropies: both are one-parameter generalizations but differ by functional form (logarithmic vs power-law combination) and consequent additivity properties.

Synthesis

Rényi entropy is the parameterized logarithmic measure that, by changing α, continuously interpolates between counting effective support, typical-event uncertainty, and worst-case concentration, providing a flexible spectrum of information measures.