##  [Principal Component Analysis (PCA)](/principal-component-analysis-pca-0) 

 Definition

A linear dimensionality-reduction method that finds orthogonal directions (principal components) in data space which successively maximize variance; computed via eigen-decomposition of the empirical covariance or singular value decomposition of the data matrix.

 

 

 

 

 

 





## Principle

Principle

Project data onto an orthonormal basis that diagonalizes the empirical second-moment structure; retaining the top-k components yields the best k-dimensional linear approximation in mean-squared error among orthogonal projections.

 

 

 

 

 





## Demonstration

Demonstration

Given centered samples in R^d, compute the covariance matrix, find its top eigenvectors, and project the data onto the span of those eigenvectors to compress and de-noise while preserving maximal linear variance.

 

 

 

 

## Misapplication

Misapplication

Interpreting principal components as causal factors, applying PCA to strongly non-linear manifolds without preprocessing, or using PCA on uncentered data which shifts principal directions toward the mean offset.

 

 

 

 

 





## Consequence

Consequence

Produces decorrelated coordinates ordered by explained variance, enabling compression, visualization, and noise reduction when dominant directions capture signal structure.

 

 

 

 

## Reversal

Reversal

Projecting onto low-variance components yields directions that minimize variance capture and often emphasize noise; probabilistic factor models invert PCA's deterministic variance-maximization by introducing latent-variable noise models.

 

 

 

 

 





## Boundary

Boundary

Assumes linear relations and second-moment sufficiency; not optimal for non-linear structure-preserving reductions (manifold learning) or for heavy-tailed data without robust covariance estimation.

 

 

 

 

 





## Semantic Tension

Semantic Tension

Versus factor analysis or independent component analysis: PCA is an orthogonal variance-maximizer without an explicit generative noise model, whereas competing methods impose latent-variable or independence assumptions.

 

 

 

 

 





## Synthesis

Synthesis

An orthogonal projection method that diagonalizes covariance to extract ordered linear modes of maximal variance, yielding an optimal least-squares low-dimensional linear summary when data are mean-centered.