Neural Rule Lists: Learning Discretizations, Rules, and Order in One Go

Interpretable machine learning is essential in high-stakes domains like healthcare. Rule lists are apopular choice due to their transparency and accuracy, but learning them effectively remains a challenge. Existing methods require feature pre-discretization, constrain rule complexity or ordering, or struggleto scale.

We present NeuRules, a novel end-to-end framework that overcomes these limitations. At itscore, NeuRules transforms the inherently combinatorial task of rule list learning into a differentiableoptimization problem, enabling gradient-based learning. It simultaneously discovers feature conditions,assembles them into conjunctive rules, and determines their order—without pre-processing or manualconstraints.

A key contribution here is a gradient shaping technique that steers learning toward sparserules with strong predictive performance. To produce ordered lists, we introduce a differentiablerelaxationthat, through simulated annealing, converges to a strict rule list. Extensive experiments showthat NeuRules consistently outperforms combinatorial and neural baselines on binary aswell asmulti-class classification tasks across a wide range of datasets.