A supervised learning principle that selects a predictive model from a hypothesis class by minimizing the average loss computed over a finite training sample.

Principle

Replace the unknown population risk by its empirical sample average and choose the hypothesis that attains the smallest empirical loss; regularization or capacity control is typically added to manage generalization.

Demonstration

Fitting a linear predictor by minimizing mean squared error on observed input–output pairs is an instance: the chosen coefficients minimize the sample average squared residuals over the dataset.

Misapplication

Blindly minimizing empirical loss without any restriction on model complexity or validation, leading to overfitting where the chosen model fits noise in the sample and performs poorly on new data.

Consequence

When hypothesis class complexity is controlled and sample size is sufficient, empirical minimizers converge to low population risk and yield predictors that generalize; otherwise, guarantees break down.

Reversal

Instead of minimizing empirical loss, one may use methods that integrate model uncertainty or complexity penalties explicitly (for example by selecting based on held-out risk or incorporating explicit complexity regularizers).

Boundary

Applies to problems with a defined loss function and labeled training data; it does not by itself specify how to choose the hypothesis class, how to regularize, nor how to evaluate out-of-sample performance.

Semantic Tension

Closely related to but distinct from inference methods that optimize posterior objectives: empirical risk focuses on sample-average loss minimization, while other frameworks emphasize probabilistic modeling and uncertainty quantification.

Synthesis

Empirical risk minimization is the foundational algorithmic rule of supervised learning: pick the model that minimizes average training loss, subject to additional controls to ensure that empirical performance reflects true predictive ability.