We propose CATransformer, the first carbon-aware co-optimization framework for Transformer-based models and hardware accelerators.

We present Neural Attention Search (NAtS), an end-to-end learnable sparse transformer

We design a training-free neural architecture search method for Mamba2.