We use scaling law derivation to compare open language-vision foundation models (CLIP, MaMMUT) and datasets (DataComp-1.4B, Re-LAION-1.4B, DFN-1.4B), identifying models and datasets that promise stronger scalability in the pre-training.

ChemPile is a large and diverse collection of chemical data for the study and development of chemical foundation models

We introduce EmoNet-Face, a suite of diverse, AI-generated facial emotion datasets with a novel 40-category taxonomy, to improve AI's ability to understand human emotions more accurately and empathetically.

This paper presents a holistic and approximate normalization approach that accelerates GPT training by up to 40% while eliminating the need for weight decay and learning rate warm-up.