A matrix whose entries are pairwise inner products of a collection of vectors: G_{ij} = ⟨v_i, v_j⟩. In kernel methods the Gram matrix is the matrix of kernel evaluations k(x_i,x_j).

Principle

Encodes pairwise geometric relationships via the inner product; mathematically it is symmetric and positive semidefinite, with rank equal to the dimension of the span of the vectors.

Demonstration

Given vectors v_1,...,v_m in R^n, form G ∈ R^{m×m} with G_{ij} = v_i^T v_j. In machine learning, for data points x_i and kernel k, the empirical kernel matrix K_{ij}=k(x_i,x_j) is a Gram matrix in the feature space.

Misapplication

Treating a Gram matrix as a covariance matrix without centering, inverting it without regularization when it is singular, or ignoring numerical ill-conditioning in near-collinear data.

Consequence

Positive semidefiniteness allows Cholesky/eigendecompositions and kernel PCA; the rank reveals redundancy and determines whether linear dependencies exist among vectors or features.

Reversal

A distance matrix records pairwise distances rather than inner products; distances can be converted to Gram matrices only after choosing an origin/centering transformation.

Boundary

Requires an inner product or reproducing kernel; for infinite-dimensional feature spaces the Gram matrix may be defined implicitly but can be ill-conditioned or infinite-rank, requiring regularization.

Semantic Tension

Often conflated with covariance: both are symmetric PSD matrices, but covariance is an expectation of centered outer products while a Gram matrix is raw inner products and depends on centering and scaling.

Synthesis

The Gram matrix is the symmetric PSD matrix of pairwise inner products that encodes linear geometry of a set of vectors or kernel-evaluated data and determines rank and orthogonality structure.