In this paper, we systematically evaluate RL and control-based methods on a suite of navigation tasks, using offline datasets of varying quality.