Challenges in interpretability of additive models

Abstract

We review generalized additive models as a type of ‘transparent’ model that has recently seen renewed interest in the deep learning community as neural additive models. We highlight multiple types of nonidentifiability in this model class and discuss challenges in interpretability, arguing for restraint when claiming ‘interpretability’ or ‘suitability for safety-critical applications’ of such models.