We propose a differentially private algorithm that approximates the second moment matrix of any subsamplable dataset, and show its applicability to heavy-tailed distributions and input with outliers.