We estimate global class covariances at the server with a provably unbiased estimator requiring only local class means from clients, achieving performance competitive or superior to algorithms sharing second-order statistics

The paper proposes Core Space Merging, a method to efficiently merge LoRA-adapted models by aligning them in a shared low-rank subspace, achieving higher accuracy and major speedups over prior merging techniques.

We ensure backward compatibility through multiple transformations and a relaxed orthogonality constraint for distribution-specific adaptation.