We propose RPEX, an Offline-to-Online method that improves the performance of offline pretrained RL policies under a wide range of data corruptions.