We introduce a framework for in-context (zero-shot) inference/estimation of drift and diffusion functions underlying SDEs from empirical data of different dimensionalities.

We study the expression and limitations of transformers in performing in-context learning of functions governed by Markovian dynamics.

We demonstrate that the PFN-framework allows for the accurate estimation of causal effects under weakened assumptions.

This paper demonstrates that language models possess metacognitive-like abilities to monitor and control their internal neural activations.

We introduce TabDPT, a tabular foundation model capable of providing highly accurate predictions for unseen tabular datasets with no further training or hyperparameter tuning, and demonstrate scaling in both model and pre-training dataset size.

We show that learning sparse attention is prone to emerging behaviors during training, and study (theoretically and empirically) how data and model design influence emergence speed.

We introduce ZEUS, a transformer pretrained on synthetic datasets to perform zero‐shot, fully unsupervised clustering of tabular data—no fine‐tuning or hyperparameter search needed—and show it matches or outperforms classical and deep baselines.

The paper provides a novel approach to ICL example selection/ordering through the extension of DPO.

We show that shallow linear transformers fail to in-context learn linear dynamical systems, uncovering a distinction between in-context learning over iid and non-iid data; in contrast, transformers with log-depth successfully learn dynamical systems.

We propose an information-theoretical metric that helps determine the optimal order of demonstrations for in-context learning in large language models.

We introduce SMMILE, the first multimodal medical benchmark for evaluating in-context learning abilities of vision-language models.

A state-of-the-art semantics-enriched TabPFNv2-style in-context learner trained on real-world tabular data

We propose BAM-ICL, a novel budgeted adversarial manipulation hijacking attack framework for in-context learning.

The paper presents AnyMDP, a framework for procedurally generating diverse tasks to enhance In-Context Reinforcement Learning (ICRL) scalability, and explores the trade-off between generalization and adaptation efficiency.

We prove that transformers can achieve nearly optimal dynamic regret bounds in non-stationary environments.

LLMs show human-like representational change during associative learning, driven by pair similarity and modulated by prior knowledge.

Answer the question: Does the universal approximation property still hold if the context in in-context learning is restricted to a finite set?

We meta-learn a transformer based in-context learning fMRI visual cortex encoder which can adapt to new human subjects without any fine-tuning

Prompted Policy Search uses LLMs for RL policy search, outperforming common methods across diverse tasks.

We use in-context learning as weak supervision to train a student model that internalizes demonstration-induced latent shifts via adapter tuning, enabling efficient inference with improved generalization.

We introduce TrajICL, a novel in-context learning (ICL) framework for pedestrian trajectory forecasting that enables rapid adaptation without costly fine-tuning.

CausalPFN is a pre-trained transformer that amortizes causal effect estimation: trained once on simulated data-generating processes, it outputs calibrated effects for new observational datasets with zero tuning.

In this paper we introduce a framework to decompose the uncertainty of predictions in in-context learning

We prove that, under appropriate conditions, linear attention is an almost optimal metalearner for linear classification.

We find that it is effective to defend "long-length" jailbreak attacks via efficient "short-length" LLM adversarial training, supporting by both theoretical and empirical evidence.