We prove that the interaction of parameter symmetry and equivariance constraints can create critical points and minima in the loss landscape.