When majority rules, minority loses: bias amplification of gradient descent

Despite growing empirical evidence of bias amplification in machine learning, its theoretical foundations remain poorly understood. We develop a formal framework for majority-minority learning tasks, showing how standard training can favor majority groups and produce stereotypical predictors that neglect minority-specific features.

Assuming population and variance imbalance, our analysis reveals three key findings:

1. the close proximity between "full-data" and stereotypical predictors
2. the dominance of a region where training the entire model tends to merely learn the majority traits
3. a lower bound on the additional training required.

Our results are illustrated through experiments in deep learning for tabular and image classification tasks.