Abbas Rahimi

We introduce a novel, scalable framework to evaluate compositional generalization, leverage it to evaluate more than 5k models, and propose a family of neural models pushing the Pareto frontier on this task.

We propose a parametrisation of SSM transition matrices that enables SSMs to track states of arbitrary finite-state automata while keeping the cost of the parallel scan comparable to that of diagonal SSMs.

We train analog foundation models that are robust to noise present in analog in-memory computing hardware and demonstrate accuracy comparable to models trained with 4-bit weight and 8-bit static input quantization.