ChemOrch: Empowering LLMs with Chemical Intelligence via Groundbreaking Synthetic Instructions

Yue Huang, Zhengzhe Jiang, Xiaonan Luo, Kehan Guo, Haomin Zhuang, Yujun Zhou, Zhengqing Yuan, Xiaoqi Sun, Jules Schleinitz, Yanbo Wang, Shuhao Zhang, Mihir Surve, Nitesh V Chawla, Olaf Wiest, Xiangliang Zhang

University of Notre Dame· MIT· California Institute of Technology· MBZUAI· CMU

Chemistry Large Language Model Synthetic Data

⋅ NeurIPS ⋅ OpenReview

Abstract

Empowering large language models (LLMs) with chemical intelligence remains a challenge due to the scarcity of high-quality, domain-specific instruction-response datasets and the misalignment of existing synthetic data generation pipelines with the inherently hierarchical and rule-governed structure of chemical information.

To address this, we propose ChemOrch, a framework that synthesizes chemically grounded instruction–response pairs through a two-stage process: task-controlled instruction generation and tool-aware response construction. ChemOrch enables controllable diversity and levels of difficulty for the generated tasks and ensures response precision through tool planning & distillation, and tool-based self-repair mechanisms.

The effectiveness of ChemOrch is evaluated based on:

the high quality of generated instruction data, demonstrating superior diversity and strong alignment with chemical constraints;
the dynamic generation of evaluation tasks that more effectively reveal LLM weaknesses in chemistry; and
the significant improvement of LLM chemistry capabilities when the generated instruction data are used for fine-tuning.

Our work thus represents a critical step toward scalable and verifiable chemical intelligence in LLMs. The code is available at https://anonymous.4open.science/r/ChemOrch-854A.