We analyze formally and empirically shortcuts arising in concept-based models

Reasoning models can learn rules from simple examples and be able to solve complex ones using the rules. We identify a broad class of everyday reasoning rules that current models cannot learn and build large datasets requiring such rule learning.