A stochastic process defined so that every finite collection of its random variables has a joint multivariate normal distribution, fully specified by a mean function and a positive-definite covariance (kernel) function.

Principle

Complete specification by first- and second-order statistics: the mean function sets central tendency and the covariance function encodes similarity and smoothness, determining all finite-dimensional marginals.

Demonstration

Use a zero-mean Gaussian process with a squared-exponential covariance as a prior over functions in regression; conditioning on observed input–output pairs yields predictive means and variances at new inputs.

Misapplication

Assuming Gaussianity of residuals or stationarity of the kernel without verification; or using a poorly chosen kernel that cannot represent the function class, leading to systematic bias.

Consequence

Provides nonparametric, probabilistic function estimation with closed-form posterior predictive distributions (mean and variance) under Gaussian likelihoods and linear operations.

Reversal

A deterministic parametric model (e.g., fixed-basis linear model) gives point predictions without the intrinsically quantified function uncertainty that a Gaussian process supplies.

Boundary

Requires a covariance kernel that is positive definite and, for practical inference, often assumes Gaussian or conjugate noise; does not encompass non-Gaussian processes with arbitrary finite-dimensional laws.

Semantic Tension

Tension exists between Gaussian processes as flexible Bayesian nonparametric priors and machine-learning models like deep neural networks that provide high-capacity but typically non-probabilistic function approximations.

Synthesis

A Gaussian process is an infinite-dimensional Gaussian law over functions, characterized entirely by a mean and covariance kernel, used to produce probabilistic predictions and quantify function uncertainty.