We show that multiplicative (bi-linear) hidden state transitions are a natural choice for representing state tracking behavior in linear recurrent networks.