We propose two techniques to improve the data efficiency of LLM RL fine-tuning: difficulty-targeted online data selection and rollout replay.

We introduce a novel offline RL algorithm that leverages shortcut models to scale both training and inference.

A systematic multimodal RL framework that improves the policy exploration and advantage estimation.

We develop a variance-aware gap-dependent regret bound with better $H$ dependence for tabular MDPs.

We introduce a method that allows adversarial optimization to be used in general-sum settings to train more robust and diverse policies.

Scalable, simple, and practical algorithm for model-based RL with regret bounds across several RL settings and experiments on state-based, visual control and hardware tasks.

We propose DCoLT, a method to enhance diffusion language models by treating each reverse diffusion step as a latent "thinking" using reinforcement learning. Achieves promising results on several math and code metrics with SEDD and LLaDA.

We propose AdaReasoner, a reinforcement learning–based framework that adaptively tunes LLM reasoning settings in few-shot scenarios, achieving gains in multiple experimental settings.

DIET makes LLMs more token-efficient by using problem difficulty to dynamically guide compression during reinforcement learning, boosting reasoning performance and enabling superior inference scaling under fixed budgets.

SATURN enables scalable, verifiable, and curriculum-controlled reinforcement learning that enhances the reasoning capability of LLMs, and generalizes to math and programming tasks.

This paper introduces Completion Pruning Policy Optimization (CPPO) to accelerate the training of reasoning models based on Group Relative Policy Optimization (GRPO).

We design an RL environment for automated mathematical theory formation and present an evolutionary synthesis methodology for interestingness measures to guide exploration in it

We propose a unified conformal prediction framework for infinite-horizon policy evaluation that seamlessly accommodates both on-policy and off-policy scenarios.

OMiSO (Online MicroStimulation Optimization): a brain stimulation framework to drive neural population activity toward specified states by adaptively choosing stimulation parameters based on brain state information

RL to train LLMs how to generate data and update themselves to adapt to new knowledge/tasks.

A method for constructing an optimal behavior basis for the Option Keyboard, enabling zero-shot identification of optimal solutions for any linear-reward task.

We provide a computational efficient algorithm to achieve $O(H)$ deployment cost with polynomial sample complexity.

This paper proposes a novel deep reinforcement learning algorithm robust to unobserved confounding bias in observed data over complex and high-dimensional domains.

This paper presents SUFT, a causal upper-bound loss optimization strategy for DRL designed to enhance sample efficiency and reduce computational demands.

We adopt reinforcement learning to train LLMs to generate quality code with rewards derived from program analysis.

RL fine-tuning in LLMs updates a small subnetwork containing 20–30% of parameters leaving rest of the parameters unchanged.

MyoChallenge 2024 at NeurIPS introduced a benchmark for coordinated control of biological and prosthetic limbs through two tasks, manipulation and locomotion, using physiologically realistic models.

We find that online multi-task RL with high-capacity value models leads to SOTA sample efficiency and performance

BREEZE is a new zero-shot reinforcement learning framework that improves the stability, expressivity, and performance of pre-trained generalist policies, achieving state-of-the-art results on benchmark tasks.

We compare the capabilities of modern neural-networks-based world models with a simple, search-based alternative for dynamics prediction.

Training a new reasoning paradigm of LLMs explicitly contains meta-thinking in a multi-agent and multi-turn setting with RL

We compare RL methods DPO & GRPO for image generation, showing their strengths & how rewards affect generalization. Explores scaling for better CoT-based synthesis.

We improve training of spiking neural networks for energy-efficient robotic control by analyzing surrogate gradient slopes and introducing a privileged policy-guided method, achieving a 2.1× performance boost and strong real-world results.

We propose Interactive Retrosynthesis Planning, a novel framework that learns to construct retrosynthetic routes by interacting with tree MDPs and optimising a worst-path objective by self-imitation learning.

FastSVERL provides a practical and scalable approach for principled, rigourous interpretability in reinforcement learning.

Staggered resets fix harmful nonstationarity in massively parallel RL's short synchronous rollouts by varying environment start times, improving state coverage and boosting sample efficiency, performance, and scalability.

We present Time-R1, a reinforcement learning-based post-training framework that achieves state-of-the-art performance across large vision language models for temporal video grounding.

A novel intrinsic motivation method based on world-model memory mismatch enables embodied agents to exhibit robust autonomous behaviors that closely match whole-brain neural data from zebrafish.

we propose a scalable pre-training method named POLicy DiscriminAtive LeaRning (POLAR), which trains a reward model (RM) to discern identical policies and discriminate different ones.

Large Reasoning Model learns when to think via Decoupled GRPO.

We present SynLogic: a data synthesis framework and logical reasoning dataset encompassing 35 tasks.

Established a generalized BSM of state similarity between MDPs, backed by rigorously proved properties, which is applied to derive improved theoretical guarantees for policy transfer, state aggregation, and sampling-based estimation in MDPs.

We imrove sample complexity of single time scale actor critic to $O(\epsilon^{-3})$ from $O(\epsilon^{-4})$ for obtaining $\epsilon$-close global optimal policy.

We construct generalized Lyapunov certificates by augmenting value functions with residual terms to certify the stability of control policies from optimal control or reinforcement learning.

We introduce an RL framework that unify the training of answer generation and verification in a single model.

We reveal that reasoning-oriented RL fine-tuning significantly increases the prevalence of hallucinations and propose a FSPO approach via reinforcement learning.

We show evidence that LLM scaling theory, connecting Zipf's law to scaling laws, applies to AlphaZero scaling.

We propose UniteFormer, a unified neural solver that supports node-only, edge-only, and hybrid input types through a single model trained via joint edge-node modalities.

We propose Periodic Skill Discovery (PSD), an unsupervised framework that discovers diverse periodic behaviors.

We propose a flow matching method for reinforcement learning with shifted-dynamics data.

An end-to-end RL-based OARSMT construction method.

We introduce a quantization-free way to train autoregressive transformers for continuous action decision making, improving on discretized action methods.

We refine SVG generation using online reinforcement learning with image reconstruction, semantic, and code-level rewards, boosting accuracy, efficiency, and interpretability.

DCcluster-Opt introduces a high-fidelity benchmark for sustainable workload scheduling in geo-distributed data centers, integrating real-world data and physics-based models to evaluate multi-objective optimization strategies via a Gym environment.

We propose a memory-enhanced improvement search framework based on deep reinforcement learning to solve flexible job-shop scheduling problems.

Measure cost per state, not per trajectory: Stagger (State-wise-DAgger) —and our hybrid Warm‑Stagger—beats Behavior Cloning, giving the first formal proof that state‑wise interactive IL outperforms Behavior Cloning.

We conduct an empirical study to evaluate the generalization benefits of reinforcement learning fine-tuning versus supervised fine-tuning for vision-language-action models and provide some findings and analyses.

We propose a four-stage training task for LLMs—Fast Thinking, Verification, Slow Thinking, and Summarization—separating the training of intuition, verification, reasoning, and integration.

MEMENTO improves neural routing solvers by using memory to adapt decisions at inference time, outperforming fine-tuning and search methods while pushing SOTA on 11 of 12 tasks.

This paper answers the question: "under what conditions will model-free reinforcement learning give rise to thinking as a strategy for reward maximization?"

We propose RF-Agent, an automated RL reward function design framework via language agent tree search.

We propose ADRPO, a method that dynamically adjusts divergence regularization strength based on advantage estimates, enabling more effective fine-tuning of generative models by automatically balancing exploration and exploitation at the sample level.

We propose Q-chunking, a simple, effective offline-to-online RL method that uses action chunking to improve value propagation and exploration via temporally coherent actions.

Reframing black-box LLM jailbreaking as a variational inference problem to efficiently generate diverse and fluent adversarial prompts that outperform traditional genetic methods.

We provide a robust method of directly optimizing the pass at k with reinforcement learning, with theory and real world experiments.

We introduce CQN-AS, a value-based RL algorithm that learns a critic network that outputs Q-values over a sequence of actions.

This paper proposed a LLM-based plug-in, which is compatible with various RL algorithms, that enhances the efficiency of policy exploration in RL training.

Improving both regret approximation and level generation for unsupervised environment design, allowing for scaling to larger environments.

We construct a Progress Reward Model with convergence guarantee for Reinforcement Learning via Large Language Models.

Using search strategies at inference-time can provide massive performance boost on numerous complex reinforcement learning tasks, within only a couple seconds of execution time.

We propose a simple robot learning recipe leveraging privileged information to train active perception policies on real robots.

Efficient Reasoning VLM

we propose Seg-VAR, a novel framework that rethinks segmentation as a conditional autoregressive mask generation problem

Training LLMs to combine reasoning with external knowledge retrieval via RL without any supervised data on reasoning steps.

We propose BREAD, a novel and effective variant of GRPO that bridges supervised learning and reinforcement learning by employing branch rollouts from expert traces.

We propose Ensemble++, a scalable framework that achieves the low regret of Thompson Sampling using a tiny, computationally-efficient ensemble, making it practical for large-scale models.

We propose an offline goal-conditioned RL algorithm that achieves state-of-the-art performance on complex, long-horizon tasks without needing hierarchical policies or generative subgoal models.

We provide approximation bounds for an approach that solves a Partially Observable Reinforcement Learning (PORL) problem by approximating the corresponding POMDP model into a finite-state Markov Decision Process (MDP)

We present a general framework for personalized math exercise recommendation that integrates semantic knowledge tracing and model-based reinforcement learning.

We prove that transformers can achieve nearly optimal dynamic regret bounds in non-stationary environments.

A robust RL framework for quantitative trading with introducing novel elliptic uncertainty sets to model the market impact.

This paper introduces CoPDT, a method of using one unified and adaptable DT model for multi-task (multi-budget or multi-constraint) offline safe RL.

We introduce a Bayesian value framework and a Bellman–Jensen Gap analysis to rigorously quantify and exploit imperfect multi‐step transition predictions, and present BOLA, the first provable sample‐efficient algorithm for RL with predictions.

We propose Router-R1, an RL-based framework that interleaves multi-round reasoning with dynamic LLM selection, supports zero-shot integration of new models, and optimizes performance-cost trade-offs

We learn offline meta-policies from natural language supervision with contrastive language-decision pre-training, aligning text embeddings to comprehend environment dynamics.

We propose a novel system identification framework for sim-2-real transfer in reinforcement learning that combines Diffusion Evolution with Adversarial Learning (DEAL) to iteratively infer physical parameters with limited real-world data.

We provide a scalable bandit architecture for prompt tuning of decision transformers for increased downstream performance.

We propose Point-RFT, a multimodal framework using visually grounded Chain-of-Thought reasoning with two-stage finetuning, which exhibits superior generalization capability and potentials in complex real-world scenarios.

We develop policy gradient algorithms with global optimality and convergence guarantees for reinforcement learning with PID control policy parameterization.

We introduce a new class of Reinforcement Learned Teachers trained to provide effective reasoning traces for downstream distillation, yielding more effective data for distillation and cold-starting than orders of magnitude larger reasoning LMs.

While most RL methods use shallow MLPs (~2–5 layers), we show that scaling up to 1000-layers for contrastive RL (CRL) can significantly boost performance, ranging from doubling performance to 50x on a diverse suite of robotic tasks.

ESCORT: A particle-based framework for representing complex belief distributions in POMDPs

We propose Hephaestus, a generative self-reinforcing framework for solving the QoSD problem on large networks with nonlinear edge-weight functions, outperforming prior combinatorial and ML-based methods

Scaling RL-based reasoning to unverifiable data

AnytimeReasoner optimizes LLM reasoning under variable token budgets by introducing verifiable dense rewards and a variance reduction method (BRPO), enabling more efficient RL for both final and anytime reasoning performance.

We present a unified neural solver for multi-task VRPs using mixture of specialized experts based on the state-decomposable MDP.

We provide a formal definition of plasticity under minimal assumptions, and highlight a fundamental connection to empowerment

To train an adaptive video tokenizer, we introduce probabilistic taildrop to inject visual complexity prior to the tokenizer and incorporate GRPO for post-training, which further boosts efficiency in a task-aware adaptive manner.

A reinforcement-learning post-training framework teaches LLM assistants to reason about contextual integrity, slashing inappropriate information disclosure while helping users complete their tasks.

Reinforcement learning for systems described by differential equations: a model-free approach based on the HJB equation

In the context of off-policy RL, we give a theoretical analysis of the role of an additive reward correction in improving performance, accompanied by experiments on bandits and LLM posttraining.

We use VLMs to generate high-level hand-object plans for dexterous manipulation.

We propose ShorterBetter, a reinforcement learning method that trains reasoning models to generate concise yet accurate Chain-of-Thought traces by rewarding the shortest correct response among sampled outputs.

We identify the connection between the output of the encoder and the ensuing dense layers as the main underlying factor limiting scaling capabilities in deep RL

We present a reward‑engineering‑free, online Self‑Improvement procedure that enables robotic foundation models to sample-efficiently improve their policies, and autonomously practice and acquire skills generalizing far beyond their imitation data.

scaling math and code rl

We extend a new RL-based gradient-informed finetuning method to the task of reward finetuning/alignment for 3D native diffusion models.

Reinforcement Learning Successfully Attempt for End-to-End Autonomous Driving in CARLA v2

Training reinforcement learning agents from a single language instruction using vision-language models.

We propose a framework in which a meta-controller learns to coordinate offline learning in 'sleep' phases to maximise reward in an 'awake' phase, choosing between different actions which correspond to types of offline process in the brain.

We propose SAGE, a self-guided hierarchical exploration framework that trains generalist foundation model web agents without human supervision, achieving state-of-the-art performance

We propose priori knowledge-free algorithms for distributionally robust average-reward estimation, with both finite-sample theoretical guarantees and numerical validation.

Global Convergence with Order-Optimal rate for Average Reward Constrained MDPs with Primal-Dual Natural Actor Critic Algorithm

Prompted Policy Search uses LLMs for RL policy search, outperforming common methods across diverse tasks.

We provide a statistically rigorous guidelines for training interactive, multi-step LLM agents, exploring optimal compute allocation, generalization, and hyperparameter settings.

We derive no-regret guarantees for Thompson sampling in episodic reinforcement learning with Gaussian process modelling.

We propose VIKI-Bench and VIKI-R to advance visual-driven cooperation in embodied multi-agent systems.

We introduce VolleyBots, a new testbed where multiple drones cooperate and compete in the sport of volleyball under realistic physical dynamics.

Unsupervised Environment Design to create automatic curricula over world model-generated environments, showing strong generalization on unseen procedural tasks while training exclusively within offline learned world models.

We use Laplacian representation to improve exploration for reinforcement learning agents.

Encouraging model in model-based reinforcement learning to converge to flatter minima in the loss landscape will result in better downstream policies

A progressive instruction evolution framework that iteratively unlock LVLMs' reasoning ability

We propose a data augmentation method for zero-shot generalization in reinforcement learning and provide a theoretical and empirical analysis of the method.

BioReason introduces a novel DNA-LLM architecture where the LLM directly processes genomic information, achieving superior, interpretable multi-step biological reasoning and accelerating mechanistic discovery.

A framework that synthesizes a tuple of optimal control policies for multi-agent systems that maximizes the probability of satisfying a desired hyperproperty.

We propose an algorithm to recover the correct generation order from textual data and solve reasoning and planning problems adaptively without order annotations.

We propose an adversarial policy learning framework for humanoid robots, aiming for precise motion tracking of the upper body and robust locomotion of the lower body.

We propose a novel value function learning scheme for hierarchical policy in offline GCRL

We propose novel model-free RL and FRL algorithms, which simultaneously achieves the best-known near-optimal regret, a low burn-in cost and a logarithmic policy switching cost or communication cost.

We introduce an RL algorithm leveraging reparameterization and distance-based diversity regularization to train intractable multimodal policies for diversity-critical tasks.

We built a generalist clinical foundation model across both time series and vision modalities with a novel RL training algorithm named DRPO.

We stabilize gradients for training increasingly deep reinforcement learning agents by using a second-order optimizer and residual connections

We introduce a novel reasoning paradigm -- pixel-space reasoning. We identified the learning trap when cultivating this ability and proposed a curiosity-driven RL approach to address it.

Long-RL enables RL on hour-long videos on a single A100; LongVILA-R1-7B supports 8,192 frames and scores 65.1%/71.1% on VideoMME.

We propose methods for studying behavior and neural dynamics in deep RL agents based on neuroscience methods and demonstrate them in a naturalistic simulated foraging environment

To incorporate reward shaping approach into multi-task reinforcement learning, we propose a Centralized Reward Agent based MTRL framework (CenRA) to share and transfer knowledge across multiple tasks.

We propose Q-Insight, a reinforcement learning-based IQA approach using limited annotations, achieving superior performance and enhanced visual reasoning capabilities.

This work presents the first asymptotically correct simultaneous confidence region for off-policy evaluation in reinforcement learning.

We introduce STITCH-OPE, a guided-diffusion framework for off-policy evaluation that stitches short behavior-conditioned sub-trajectories, uses negative-behavior guidance to correct distribution shift, and outperforms baselines across all metrics.

NoisyRollout boosts VLM reasoning by mixing clean and noisy inputs during RL, improving generalization with no extra cost.

We prove an ensemble of policies distilled on a diverse dataset improves generalisation in reinforcement learning.

A 3DGS-based Reinforcement Learning training paradigm for end-to-end autonomous driving

We show that penalizing certain CoT reasoning makes LLMs learn encoding schemes that generalize to unseen examples.

We introduce Forecasting in Non-stationary Offline RL (FORL), a novel framework designed to be robust to passive non-stationarities, leveraging diffusion probabilistic models and time-series forecasting foundation models.

We introduce URB, a standardized benchmark for evaluating MARL in urban routing with autonomous vehicles across 29 real-world traffic networks, revealing that current state-of-the-art algorithms struggle to outperform humans and scale effectively.

We propose a new interactive spatial reasoning environment based on manipulating knots.

We propose a curriculum strategy for guiding the training of agents that operate under strict trajectory constraints during deployment by adaptively tightening constraints based on agent's performance.

We propose a co-evolving reinforcement learning method that jointly optimizes the coder and unit tester without relying on ground-truth code supervision.

We develop a reasoning-induced NR-IQA model via reinforcement learning to rank.

We introduce Mesh-RFT, a novel framework leveraging localized preference optimization with topology-aware scoring to achieve state-of-the-art high-fidelity 3D mesh generation.

Teach multimodal models to think or not to think- this is a question.

We propose Flow-GRPO, the first method to integrate online RL into flow matching models, significantly enhancing text-to-image generation performance.

self-play reasoning RL with no data can achieve SOTA against RL models trained with human data

We analyze what kind of LLMs have large improvement in RL finetuning and propose behavior injection augmentation to prepare the LLMs for RL.

A first lower bounds for regret of decentralized multi-agent-SSP.

We makes an early attempt at fully unsupervised LLM reasoning incentivization.

This work presents planning and learning algorithms for average-cost MDPs with dynamic risk measures.

We propose a general reinforcement learning framework tailored for interleaved multimodal tasks by permutating image sequences to simulate varied positional relationships and explore more spatial and positional diversity

We develop DISCOVER, which enables RL agents to solve substantially more challenging tasks than previous exploration strategies in RL.

We extend autoregressive multi-agent sequence models, including Sable and MAT, to the Offline MARL setting and demonstrate that they significanlty outperform current state-of-the-art methods across a diverse set of benchmarks with up to 50 agents.

Actor-free Q-learning in Continuous Action Spaces by Learning a "Wire-fitted Q-function"

We propose DynamicRAG, which optimizes retrieval-augmented generation by using reinforcement learning to adaptively rerank documents based on LLM's responses as feedback, achieving state-of-the-art results across multiple datasets.

This paper introduces a novel reasoning-based VLM guard model dubbed GuardReasoner-VL

We train RL agents directly from high-level specifications, without reward functions or domain-specific oracles.

We only need one example for RLVR on LLMs to achieve significant improvement on math tasks

We take a probabilistic view of decision making tasks, and show that normalizing flows offer many properties which make them highly performant models in imitation learning, offline, goal conditioned and unsupervised RL.

A novel automaton-based constrained MDP formulation and reinforcement learning algorithm for robot control under task and safety constraints specified via temporal logic.

We propose GenPO, which effectively incorporates invertible diffusion model into on-policy RL, and deals with the challenge of log-likehood computation in diffusion policies.

We propose a novel self-improvement algorithm to teach language models to perform effective search.

We propose a way to create and access episodic memory when training transformer policies with RL on long horizon tasks that require remembering things from the past.

Prove with theory and empirical results that prioritising training on questions with "medium" level of difficulty is beneficial for training reasoning models with RL

We propose a value gradient matching formulation for reward finetuning/alignment for flow matching models with the theory of optimal control, and empirically verify our method on the popular text-to-image flow matching model StableDiffusion3

An LLM-based retrosynthesis planning agent trained end-to-end with Agentic Reinforcement Learning.

We introduce OPHR, a multi-agent reinforcement learning framework that outperforms traditional approaches in volatility trading by dynamically selecting options positions and optimizing hedging strategies.

We propose MTRec, a novel sequential recommendation framework which uses a learned mental reward model to guide the recommendation model to align with users' real preferences.

ViGoRL is a vision-language model trained with reinforcement learning to ground each reasoning step in image coordinates, improving performance on spatial and web-based reasoning tasks through better attention and visual verification.

We present a theoretical framework for policy convergence in RL, which permits convergence of return distribution estimates.

A backtracking method that reverts to safer points during generation, reducing safety violations

We introduce TensorRL-QAS, an improved framework for quantum architecture search that combines tensor network initialization with reinforcement learning.

COGNAC is a benchmark suite for evaluating cooperative multi-agent reinforcement learning on graph-structured control tasks, highlighting the strengths of decentralized approaches.

A complete reimplementation of MiniGrid environments with JAX unlocking 160,000x faster experimentation