Fine-tuning speech models jointly with brain responses from multiple participants improves brain alignment (even on novel datasets) and decreases the amount of fMRI data needed to predict new participants.