##  [Principal Component Analysis](/principal-component-analysis-0) 

 Definition

A linear dimensionality-reduction technique that identifies orthogonal directions (principal components) in data which successively capture maximal variance; typically computed via diagonalization of the empirical covariance matrix or via singular value decomposition of the data matrix.

 

 

 

 

 

 





## Principle

Principle

Find an orthogonal basis that maximizes projected variance in descending order so that a low-dimensional projection retains as much of the data's second-moment structure as possible under linear constraints.

 

 

 

 

 





## Demonstration

Demonstration

Standardize a multivariate dataset, compute its sample covariance matrix, diagonalize it to obtain principal axes, and project the data onto the first k axes to reduce dimensionality while preserving predominant variance.

 

 

 

 

## Misapplication

Misapplication

Using the method on data with strong nonlinear manifolds without preprocessing or failing to center variables, producing misleading components that do not capture intrinsic structure.

 

 

 

 

 





## Consequence

Consequence

Produces uncorrelated linear features ordered by explained variance, simplifies visualization, noise reduction and some downstream tasks while exposing directions of highest variability.

 

 

 

 

## Reversal

Reversal

Nonlinear dimensionality techniques (e.g., manifold learning) that are designed to capture nonlinear structure and relationships that linear principal components cannot represent.

 

 

 

 

 





## Boundary

Boundary

A linear method relying on second-moment statistics; it does not model latent generative factors explicitly and is sensitive to scaling and outliers unless data are preprocessed.

 

 

 

 

 





## Semantic Tension

Semantic Tension

Often contrasted with factor analysis: PCA focuses on variance maximization and orthogonal projections, while factor analysis posits a latent-variable generative model and models residual variance differently.

 

 

 

 

 





## Synthesis

Synthesis

PCA is a linear projection technique that derives an orthogonal coordinate system by diagonalizing the empirical covariance (or using SVD) so low-dimensional projections retain maximal variance under orthogonality constraints.