Seeing through Uncertainty: Robust Task-Oriented Optimization in Visual Navigation

Yiyuan Pan, Yunzhe XU, Zhe Liu, Hesheng Wang

Visual Navigation Deep Reinforcement Learning Task-based Model Uncertainty Calibration

⋅ NeurIPS ⋅ Project Page ⋅Poster ⋅OpenReview

Abstract

Visual navigation is a fundamental problem in embodied AI, yet practical deployments demand long-horizon planning capabilities to address multi-objective tasks. A major bottleneck is data scarcity: policies learned from limited data often overfit and fail to generalize OOD. Existing neural network-based agents typically increase architectural complexity that paradoxically become counterproductive in the small-sample regime.

This paper introduce NeuRO, a integrated learning-to-optimize framework that tightly couples perception networks with downstream task-level robust optimization.

Specifically, NeuRO addresses core difficulties in this integration:

it transforms noisy visual predictions under data scarcity into convex uncertainty sets using Partially Input Convex Neural Networks (PICNNs) with conformal calibration, which directly parameterize the optimization constraints;
and it reformulates planning under partial observability as a robust optimization problem, enabling uncertainty-aware policies that transfer across environments.

Extensive experiments on both unordered and sequential multi-object navigation tasks demonstrate that NeuRO establishes SoTA performance, particularly in generalization to unseen environments. Our work thus presents a significant advancement for developing robust, generalizable autonomous agents.