We propose CovMatch, a scalable dataset distillation method for image-text contrastive learning that aligns cross-covariance and feature distributions between real and synthetic data.