We propose CLAdapter to refine and adapt the rich representations learned from large-scale data for transfer across diverse, data-limited scientific downstream domains.