Heng Ji

The paper proposes a principled reward design framework for training LLMs on tool use via reinforcement learning, leading to significant gains over SFT and baseline models in generalization and performance.

Introducing PARTONOMY and PLUM, a new benchmark and segmenting LMM that enable fine-grained, part-level visual reasoning by addressing architectural flaws in existing LMMs and setting a new standard for grounded multimodal understanding.

Variational supervised contrastive learning maximizes a posterior-weighted ELBO, replacing pairwise comparisons with class-level interactions for SOTA performance on image classification tasks.

Fire360 is a benchmark of 360° firefighting videos for evaluating vision-language models under real-world degradation, introducing five tasks including Transformed Object Retrieval (TOR) for fire-damaged object matching.