##  [Covariance Matrix](/covariance-matrix-0) 

 Definition

A square matrix whose (i,j) entry is the covariance between the i-th and j-th components of a random vector; it encodes pairwise linear dispersion and scaling among components.

 

 

 

 

 

 





## Principle

Principle

Collect second-moment information: compute expectations of pairwise products minus product of expectations; the matrix is symmetric and positive semi-definite, reflecting variance and linear dependence structure.

 

 

 

 

 





## Demonstration

Demonstration

Given n independent d‑dimensional observations x₁,…,x_n, the sample covariance matrix S = (1/(n-1)) ∑ (x_i − 9x)(x_i − 9x)^T yields entry S_{jk} equal to the sample covariance between coordinates j and k.

 

 

 

 

## Misapplication

Misapplication

Interpreting zero covariance as full independence (zero covariance only implies lack of linear correlation) or applying the sample covariance from strongly nonstationary sequences without removing trends or seasonal components.

 

 

 

 

 





## Consequence

Consequence

Correct use provides directions of large and small variability, informs linear estimators and error propagation, defines Mahalanobis-style distances, and parameterizes multivariate Gaussian dispersion.

 

 

 

 

## Reversal

Reversal

The precision matrix (matrix inverse when it exists) emphasizes conditional relationships and sparsity patterns rather than marginal covariances.

 

 

 

 

 





## Boundary

Boundary

Only summarizes second-order (pairwise linear) relations and requires finite second moments; it does not capture nonlinear dependence or higher moments and may be ill-conditioned for small sample sizes relative to dimension.

 

 

 

 

 





## Semantic Tension

Semantic Tension

Correlation matrix is a normalized form of the covariance matrix; confusion arises when one compares scale-sensitive covariances with scale-free correlations.

 

 

 

 

 





## Synthesis

Synthesis

The covariance matrix is the symmetric positive semi-definite matrix of pairwise covariances that compactly represents the linear dispersion and scale relationships of a multivariate distribution.