We build a LM-based system that can outperfrom expert AI researchers in predicting the outcomes of empirical AI research ideas, without running actual experiments.