A linear dimensionality-reduction technique that identifies orthogonal directions (principal components) in data which successively capture maximal variance; typically computed via diagonalization of the empirical covariance matrix or via singular value decomposition of the data matrix.

Principle

Find an orthogonal basis that maximizes projected variance in descending order so that a low-dimensional projection retains as much of the data's second-moment structure as possible under linear constraints.

Demonstration

Standardize a multivariate dataset, compute its sample covariance matrix, diagonalize it to obtain principal axes, and project the data onto the first k axes to reduce dimensionality while preserving predominant variance.

Misapplication

Using the method on data with strong nonlinear manifolds without preprocessing or failing to center variables, producing misleading components that do not capture intrinsic structure.

Consequence

Produces uncorrelated linear features ordered by explained variance, simplifies visualization, noise reduction and some downstream tasks while exposing directions of highest variability.

Reversal

Nonlinear dimensionality techniques (e.g., manifold learning) that are designed to capture nonlinear structure and relationships that linear principal components cannot represent.

Boundary

A linear method relying on second-moment statistics; it does not model latent generative factors explicitly and is sensitive to scaling and outliers unless data are preprocessed.

Semantic Tension

Often contrasted with factor analysis: PCA focuses on variance maximization and orthogonal projections, while factor analysis posits a latent-variable generative model and models residual variance differently.

Synthesis

PCA is a linear projection technique that derives an orthogonal coordinate system by diagonalizing the empirical covariance (or using SVD) so low-dimensional projections retain maximal variance under orthogonality constraints.