Fighting against data scarcity: a Cold Diffusion Framework for Sparse Covariance Matrix Estimation

Abstract

Accurately estimating covariance matrices from limited samples is a fundamental challenge in high-dimensional statistics. Traditional methods attempt to mitigate the effects of data scarcity through fixed statistical priors, such as regularization or shrinkage, which may not capture the true underlying data structure. This thesis introduces a new paradigm that reframes covariance estimation as a generative denoising problem. We conceptualize the noisy Sample Covariance Matrix (Sigma) as the result of a non-Gaussian statistical degradation process and develop a framework that learns to reverse this process.

Our primary contribution is ColdCov, a novel deep generative model based on the Cold Diffusion framework. ColdCov is uniquely suited to this problem for two reasons. First, it is designed to reverse arbitrary non-Gaussian degradations, aligning perfectly with the statistical properties of the degradation process. Second, its output architecture enforces the symmetric positive semi-definite (SPSD) property by construction, guaranteeing that every output is a mathematically valid covariance matrix.

Through comprehensive experiments, we show that ColdCov provides strong advantages in covariance reconstruction, particularly in low-sample regimes, achieving lower Frobenius reconstruction error and competitive precision-matrix quality relative to established baselines. However, the results also reveal a task-dependent trade-off: sparsity-enforcing methods such as Graphical Lasso and Ledoit-Wolf remain stronger for explicit graph-structure recovery, while ColdCov produces richer denoised covariance representations. The framework's practical utility is further validated on the ABIDE neuroimaging dataset, where precision matrix features extracted by ColdCov show best observed mean performance on downstream discriminative classification, with modest gains over baselines in a Control vs.\ Autism subject-level classification task.

We conclude that modeling covariance estimation as a generative denoising task is a promising complement to traditional regularization: it is particularly effective for covariance reconstruction and downstream discriminative feature extraction, while sparse regularization remains preferable when exact graph topology recovery is the primary objective.