We present POSSM, a novel architecture that combines input cross-attention with a recurrent state-space model, achieving competitive accuracy, fast inference, and efficient generalization for real-time neural decoding applications.

We propose a novel hierarchical soft mixture-of-experts for 3D medical image segmentation

A novel class of MTPP models inspired by linear Hawkes processes and deep state-space models, that brings linear complexity and sublinear scaling while being highly expressive.

We extend DeltaNet by using products of householders as state-transition matrices allowing us to trade-off expressivity and computational complexity.

Bipartite mutual information in natural text exhibits sub-volume growth; from this, we prove a lower bound on how the history state must scale, setting a necessary condition for architectures to be effective at long-context language modeling.

We propose a parametrisation of SSM transition matrices that enables SSMs to track states of arbitrary finite-state automata while keeping the cost of the parallel scan comparable to that of diagonal SSMs.

distillation of high dimensional LDS to low dimension

The paper derives generalization bounds for selective SSMs using connections to self-attention, showing that spectral properties of the state matrix influence generalization.

We propose joint recall, a novel synthetic task, and hybrid sparse attention with context-dependent sparsity for better sub-quadratic long-context modeling.

Nonlinear systems whose future behavior is not overly sensitive to small perturbations can be efficiently parallelized; whereas unpredictable dynamical systems cannot be efficiently parallelized.

First framework integrates scalp and intracranial EEG to estimate whole-brain networks via state-space models and EM algorithm, outperforming traditional methods and showing cortical-subcortical flows in working memory.

We introduce WaLRUS, a wavelet-based SSM leveraging SaFARi for improved accuracy and stability in modeling non-smooth, transient signals, outperforming traditional HiPPO-based models.