A vector algebra perspective on data dimensionality reduction

Abstract

The paper studies methods of applying vector algebra to optimize machine learning tasks. Using the Human Activity Recognition dataset, a mathematically rigorous primary data analysis is conducted: the Gram matrix and correlation structure are examined, positive semi-definiteness of the correlation matrix is proved, and multicollinearity is quantified via the condition number 𝑘(XᵀX) ≈ 9.56 ∙ 10⁴ and median VIF = 22.4. Principal Component Analysis is examined through the lens of singular value decomposition; the orthogonality of principal components is proved. Projection onto k = 20 components increases classifier accuracy from 62.8 % to 65.5 %.