In sign-diverse Hebbian/anti-Hebbian or E-I networks, inherent non-gradient “curl” terms arise, and can, depending on network architecture, destabilize gradient-descent solutions or paradoxically accelerate learning beyond pure gradient flow.

We derive well-known learning rules from an objective that casts learning rules as policies for navigating uncertain loss landscapes