##  [Exponential Family](/exponential-family-0) 

 Definition

A class of probability distributions whose densities (with respect to a dominating measure) can be written in canonical form p(x|η) = h(x) exp(η·T(x) − A(η)), where η is the natural parameter, T(x) the sufficient statistic, and A the log-partition function.

 

 

 

 

 

 





## Principle

Principle

Characterized by finite-dimensional sufficient statistics and convex log-partition functions; closed under sampling (sums of sufficient statistics) and admitting conjugate priors in Bayesian analysis when the form is canonical.

 

 

 

 

 





## Demonstration

Demonstration

Examples: Bernoulli in canonical form p(x|η)=exp(η x − A(η)) with T(x)=x; Gaussian with known variance, Poisson, multinomial; sample sums yield minimal sufficient statistics whose dimension does not grow with sample size.

 

 

 

 

## Misapplication

Misapplication

Forcing a distribution into exponential form when no dominating measure or sufficient-statistic structure exists, or assuming all parametric models are exponential families; confusing canonical parameterization with arbitrary reparameterizations.

 

 

 

 

 





## Consequence

Consequence

Leads to low-dimensional sufficient statistics, convexity properties for maximum likelihood estimation (concavity of log-likelihood in η), tractable moment calculations via derivatives of A, and convenient Bayesian conjugacy.

 

 

 

 

## Reversal

Reversal

Non‑exponential families (e.g., Cauchy, many heavy-tailed models, or mixtures of exponentials) lack fixed‑dimensional sufficient statistics and do not enjoy the same convexity or conjugacy properties.

 

 

 

 

 





## Boundary

Boundary

Requires a dominating measure and an exponential representation; when parameter constraints or nonlinear parameterizations occur one gets curved exponential families or models outside this class; mixtures of exponentials usually lie outside.

 

 

 

 

 





## Semantic Tension

Semantic Tension

Tension with mixture models: mixtures of members of an exponential family typically are not exponential families and lose finite-dimensional sufficiency, producing richer but less tractable statistical structure.

 

 

 

 

 





## Synthesis

Synthesis

An exponential family is a parametrized set of distributions expressible as an exponential of parameter times sufficient statistic minus log-partition, yielding finite-dimensional sufficiency, convex inference, and conjugacy structure.