This work identifies gradient conflicts as a key challenge in training PINNs and shows that quasi second-order methods—especially SOAP—effectively resolve them, leading to 2-10x accuracy gains on 10 PDE benchmarks.