A Markov chain Monte Carlo algorithm that generates samples from a multivariate target distribution by iteratively sampling each variable (or block) from its conditional distribution given the current values of the other variables.

Principle

Construct a Markov kernel whose invariant distribution is the joint target by cycling through conditional updates; detailed balance holds for each full-conditional update and the chain converges under standard irreducibility and aperiodicity conditions.

Demonstration

To sample from a bivariate Gaussian with known conditional normals, alternate drawing x conditional on y and y conditional on x; the sequence of pairs converges to the joint Gaussian distribution.

Misapplication

Applying Gibbs updates when full conditional distributions are intractable, assuming rapid mixing without diagnostics, or treating successive Gibbs draws as independent samples.

Consequence

When correctly implemented and converged, Gibbs sampling produces dependent draws from the desired joint distribution that can be used for posterior expectation estimation, uncertainty quantification, and predictive simulation.

Reversal

The reversal is a global proposal scheme (e.g., independent sampling or Metropolis–Hastings) that proposes joint moves instead of coordinate-wise conditional draws, trading simplicity of conditionals for potentially faster mixing on some problems.

Boundary

Requires the ability to sample exactly (or efficiently) from each full conditional; performance deteriorates with strong dependence across coordinates and for multimodal targets unless blocked updates or alternative kernels are used.

Semantic Tension

Competes with Metropolis–Hastings: Gibbs is simple and exact for tractable conditionals, whereas Metropolis–Hastings handles arbitrary targets with explicit proposals but requires acceptance mechanisms and tuning.

Synthesis

An MCMC kernel that attains the target distribution by sequentially replacing coordinates with draws from their conditionals, offering simplicity when conditionals are available and requiring care for convergence and correlation.