This work introduces BlockRank, a method designed to make LLMs efficient and scalable for in-context ranking by aligning their internal attention and inference mechanisms with the structure of the in-context ranking task.

We show that highly accurate LLMs can be learned from training sets consisting entirely of synthetic data and weakly curated data.

We introduce the "DataRater", a meta-learning approach to automatically learn the value of data and use it to improve the compute efficiency of training foundation models.

This paper introduces EUGens: Efficient, Unified and General Dense Layers, capable of accurately approximating general feedforward fully connected layers, but of linear rather than quadratic time complexity.

We introduce a method that allows adversarial optimization to be used in general-sum settings to train more robust and diverse policies.

We present a sharp last-iterate analysis of SGD on smooth convex losses in the interpolation regime, extending prior results beyond linear regression and improving known rates for large, constant stepsizes.

We design a PAC-learner for contextual combinatorial semi-bandits with sparse rewards, with a sample complexity bound that primarily scales with the sparsity parameter rather than the number of arms.

We study a variant of nonuniform PAC learning, where the constants in the learning rate may depend on the marginal distribution, and devise a trichotomy of possible rates.

We introduce a new SGD-based algorithm with delayed projection for training kernel machines that achieves comparable or superior performance while reducing training time from days to under an hour.

We highlight the susceptibility of existing unlearning methods to relearning attacks and analyze the characteristics of robust methods by leveraging the weight-space perspective.

An algorithm for calibrating forecasts of high-dimensional outcomes.

We proposed Federated Optimization with Exact Convergence via Push-pull Strategy (FOCUS) that can converge exactly with a linear rate even under arbitrary client participation scenario.

We study the the limits of differentially private set union (or partition selection) when each user can contribute multiple items.

We prove a purely statistical separation between Transformers and other architectures such as feedforward and recurrent networks, where Transformers are more sample-efficient at learning sparse sequence models.

Influence Distillation is a mathematically justified data selection method for LLM fine-tuning that assigns optimal weights to training samples, achieving performance on par with or better than state-of-the-art while being substantially faster.

A method for constructing an optimal behavior basis for the Option Keyboard, enabling zero-shot identification of optimal solutions for any linear-reward task.

We propose an inference-strategy for long-video QA, which substantially improves the accuracy of a VLM by curating its input context.

We show that adapting vision foundation models using self-supervised fine-tuning with simple object-centric videos substantially improves representation quality without labels.

Context ambiguity hinders privacy assessment performance of frontier and reasoning LLMs. Disambiguating contexts based on the reasoning models provide significantly improves their privacy judgements and reduces their sensitivity to prompts.

A framework and benchmark to evaluate language models' reasoning on imperfect tabular data

Undocumented versions of Meta-World have clouded algorithmic performance. This work strives to disambiguate Meta-World results from the literature, while also providing insights into benchmark design.

We propose a sequential test for auditing differential privacy that detects violations with sample sizes that are orders of magnitude smaller than existing methods.

We prove that stochastic momentum can improve the scaling law exponents over SGD on power-law random features by selecting hyperparameters to properly depend on data dimension or model size.

We design a geometric sampling framework to enhance memory efficiency and precision in long-context LLMs.

We show that learning sparse attention is prone to emerging behaviors during training, and study (theoretically and empirically) how data and model design influence emergence speed.

We develop a framework for noisy submodular maximization which allows us to reuse existing algorithms while retaining their guarantees.

A new framework for analyzing and proving length generalization bounds.

Structured Taxonomy and Evaluation for Professional Video Generation

We develop efficient, differentially private algorithms for bilevel optimization with near-optimal excess risk bounds that nearly match the best rates for standard private optimization.

We propose InvisibleInk for DP synthetic text generation. It can produce of hundreds of tokens at 8 times lesser computational cost at the same privacy/utility trade-offs compared to prior state-of-the-art methods.

We introduce a quantization-free way to train autoregressive transformers for continuous action decision making, improving on discretized action methods.

We refine SVG generation using online reinforcement learning with image reconstruction, semantic, and code-level rewards, boosting accuracy, efficiency, and interpretability.

Introduces a sound embedding benchmark called massive sound embedding benchmark (MSEB)

Robo2VLM is a framework that generates VQA datasets for robotic manipulation using real-world robot trajectories and non-visual sensor data

This papers introduces a Diffusion Transformer (DiT) for sampling 3D molecular conformers.

To achieve personalization in LLMs, we leverage the user model to incorporate a curiosity-based intrinsic reward into multi-turn RLHF.

We build a nearly-perfect predictive model for memorization in diffusion models using theory and controlled experiments.

This paper demonstrates that applying adaptive latent-space constraints in personalized FL algorithms improves performance across a number of challenging benchmark tasks, especially those with significant feature heterogeneity

We propose a choice model based on top-k Mallows model and design algorithms for it

SPRINT trains large reasoning models to dynamically spot and run independent subtasks in parallel, reducing sequential tokens by up to 40% on complex tasks like math without sacrificing accuracy compared to sequential reasoning baselines.

We develop a principled framework using approximate message passing (AMP) to analyze iterative self-retraining of ML models and derive the optimal way to combine the given labels with model predictions at each retraining round.

We introduce Recursive Inference Scaling (RIS), a plug-in technique that exploits language's fractal structure to boost model performance, unlocking a new dimension of inference scaling.

We show that DiLoCo, a method for communication-efficient language model training, exhibits reliable scaling law behavior.

We provide a robust method of directly optimizing the pass at k with reinforcement learning, with theory and real world experiments.

We present regret bounds for adversarial contextual bandits with general function approximation under delayed bandit feedback.

We propose ReStraV, a novel method to detect AI-generated videos by exploiting the geometric "straightness" of video trajectories in neural representation space.

We propose the first Best-of-Both-Worlds algorithm for multi-armed bandits with adversarial delays that matches lower bounds in both stochastic and adversarial settings, significantly improving previous results.

Giving tight bounds for the acess loss in the Agnostic Learning under Targeted Poisoning modle unsing randomized learnes.

LODGE delivers outstanding quality and superior rendering speeds in large-scale 3D scenes, enabling real-time rendering even on mobile devices.

We propose a new framework and set of evaluation criteria to assess the utility of text embeddings used in data selection for pretraining langauge models.

We propose a post-training technique, that uses a hypernetwork to efficiently steer diffusion model initial noise towards distributions favored by reward models, enhancing generation quality at low computational cost.

We address distributional shift among diverse preferences with robust DPO: Wasserstein DPO (WDPO) and Kullback–Leibler DPO (KLDPO). Finite-sample guarantees, tractable gradient-based algorithms for hard DRO objectives, strong empirical robustness.

We show the effectiveness of data-augmentation for mitigating bias due to unobserved confounding, and this motivates the proposal of our novel method for the same.

Improved sample complexity bounds for agnostic binary classification in the tau-based model!

A theoretical framework for analyzing the explainability capabilities of GAMs using computational complexity.

We prove tight asymptotic limits for reconstructing a hidden point from noisy distance queries and give a dimension-based criterion for when metric spaces are (non)-pseudo-finite.

GC4NC is a comprehensive framework for systematically evaluating graph condensation methods in node classification across multiple dimensions, providing key insights to enhance performance, efficiency, and real-world applications.

We introduce a new benchmark for long-term spatial-temporal memory in 3D embodied agent. We propose a novel model with memory fusion technique for enhanced memory capabilities.

We introduce Gatekeeper, a novel loss function that calibrates smaller models in cascade setups to confidently handle easy tasks while deferring complex ones, significantly improving deferral performance across diverse architectures and tasks.

Video generalization models can learn physics-based controls from synthetic data and generalize this control across diverse geometries, settings, and materials.

We provide a formal definition of plasticity under minimal assumptions, and highlight a fundamental connection to empowerment

Debate between AI experts outperforms single-advisor consultancy in helping humans make more accurate factual judgments, especially benefiting those with mainstream beliefs.

We solve the learning dynamics of (a close approximation of) word2vec in closed form, revealing what semantic features are learned.

We present a reward‑engineering‑free, online Self‑Improvement procedure that enables robotic foundation models to sample-efficiently improve their policies, and autonomously practice and acquire skills generalizing far beyond their imitation data.

We propose an informed corrector for masked discrete diffusion that reduces approximation errors, enabling faster sampling and better sample quality in both synthetic and large-scale settings.

We propose a method that unifies deterministic feed‑forward rendering with autoregressive diffusion to synthesize photorealistic novel views from sparse inputs in a single transformer framework.

We propose grafting, a simple approach to materialize new architectures by editing pretrained diffusion transformers. It enables architectural exploration under small compute budgets.

We argue that there are important gaps between what unlearning methods can do and claims about/hopes for what these methods can do for broader impact.

The paper generalizes the utility-first approach in differential privacy to any sequence of private estimators, incurring at most a doubling of the privacy budget and allowing for hyperparameter tuning without additional privacy cost.

Meta-RL with self-supervised predictive coding modules can learn interpretable, task-relevant representations that better approximate Bayes-optimal belief states than black-box meta-RL models across diverse partially observable environments.

We perform large-scale strong MIAs on pre-trained LLMs to clarify the extent of actual privacy risk MIAs pose in this setting.

Easy-to-interpret, unified, tunable bounds on major operational attack risks in privacy-preserving ML and statistical releases that are more accurate than prior methods, using f-DP

Learning proof system dynamics, pruning proof search based on diversity and expected outcome

Scalable online training of neural networks using frequentist filtering with martingale-posterior-inspired posteriors for sequential decision making.

We study learning set functions under one-sided feedback in the PAC learning framework.

We give a nearly-linear time algorithm for privately computing the geometric median, addressing the main open question of Haghifam et al (NeurIPS '24).

We stabilize gradients for training increasingly deep reinforcement learning agents by using a second-order optimizer and residual connections

We mechanistically analyze how large language models solve synthetic propositional logic problems.

We propose a hybrid Splat-Voxel representation that fuses and refines Gaussian Splatting, improving static scene reconstruction and enabling history-aware streaming reconstruction in a zero-shot manner.

We show that transformers with linear width can solve many graph problems using constant depth, revealing a trade-off where increasing width enables shallower, faster models—though some tasks still demand quadratic width.

contextual dynamic pricing with heterogenous buyers using optimistic posterior sampling

We show that language generation in the limit is not closed when we take finite unions of collections of languages.

Risk management processes as a way of improving, assessing, and comparing benchmark reliability result in a benchmark of benchmarks

A Novel Dataset with Demographically Intersectional Visual Evaluations (DIVE) for Pluralistic Alignment of Text-to-Image models

We argue that simulating society with generative agents requires moving beyond behavioral mimicry toward cognitively grounded reasoning that is structured, revisable, and traceable.

We propose a backward-mode AD proxy using only forward passes applying to Hamiltonian recurrent units and stacks thereof (namely, SSMs) with theoretical guarantees and experimental evidence

We propose a diffusion-based approach to sampling from Boltzmann densities based on temperature annealing.

This study reveals that reasoning models achieve superior performance by forming distinct reasoning graphs, characterized by greater cyclicity, larger diameters, and strong small-world properties.

We provide improved upper and lower bounds for contextual pricing with adversarial corruptions.

We introduce a theoretical model for the co-occurrence statistics of words that quantitatively explains the emergence of linear analogies of word embedding in models like Word2Vec and GloVe.

Attack of size quadratic in sketch size (meets upper bound)

A new projection-free algorithm for online convex optimization that relies on a separation oracle and does not require preconditioning of the feasible set.

We introduce a framework for evaluating & improving LLM consistency in simulated human dialogue. Our metrics correlate with human judgments and when used with multi-turn RL, reduce inconsistency across chit-chat, teaching and mental health dialogue.

A backtracking method that reverts to safer points during generation, reducing safety violations

A complete reimplementation of MiniGrid environments with JAX unlocking 160,000x faster experimentation