##  [Expectation–Maximization Algorithm](/expectation-maximization-algorithm-0) 

 Definition

An iterative method for finding maximum-likelihood parameter estimates in statistical models with latent (hidden) variables by alternating between an expectation step that computes expected sufficient statistics given current parameters and a maximization step that updates parameters to maximize expected likelihood, repeated until convergence.

 

 

 

 

 

 





## Principle

Principle

Decompose an intractable likelihood into a complete-data expectation and maximize this surrogate: E-step computes the posterior expectation of hidden data conditioned on observed data and current parameters; M-step optimizes parameters treating those expectations as if they were observed.

 

 

 

 

 





## Demonstration

Demonstration

Fitting a Gaussian mixture model: E-step computes responsibilities (probability each component produced each data point) using current means and variances; M-step updates component means, variances and mixing weights by weighted averages using those responsibilities; iterate until likelihood stabilizes.

 

 

 

 

## Misapplication

Misapplication

Using EM without checking for multimodal likelihoods, interpreting a local maximum as global optimum, failing to regularize when components collapse, or applying EM with incorrect model assumptions (e.g., wrong conditional independence structure).

 

 

 

 

 





## Consequence

Consequence

When appropriate, EM yields parameter estimates that monotonically increase the observed-data likelihood and often converges to a useful local maximum; convergence can be slow and sensitive to initialization, and standard EM offers no guarantee of finding the global maximum in multimodal landscapes.

 

 

 

 

## Reversal

Reversal

Replacing iterative expectation–maximization with direct optimization of the marginal likelihood (when tractable) or with Monte Carlo methods (e.g., MCMC) converts the approach to global sampling or direct gradient-based maximization instead of alternating surrogate steps.

 

 

 

 

 





## Boundary

Boundary

Applies to latent-variable models where conditional expectations are computable or approximable; excludes models where the E-step is intractable without approximation, or where likelihoods are undefined; requires careful handling of degenerate solutions (e.g., zero-variance components).

 

 

 

 

 





## Semantic Tension

Semantic Tension

Often confused with general EM-like heuristics (e.g., coordinate ascent or variational inference); EM is specifically the iterative E-step/M-step scheme that maximizes expected complete-data log-likelihood, whereas variational methods optimize a lower bound with different convergence properties.

 

 

 

 

 





## Synthesis

Synthesis

Expectation–maximization alternates computing expectations of missing data given current parameters and maximizing parameters given those expectations: a principled surrogate-optimization framework that increases observed likelihood each step but converges to local optima and depends on tractable expectation computation.