Bayesian Smoothing and Feature Selection via Variational Automatic Relevance Determination

Abstract: This study introduces Variational Automatic Relevance Determination (VARD), a novel approach for fitting sparse additive regression models in high-dimensional settings. VARD stands out by independently assessing the smoothness of each feature while precisely determining whether its contribution to the response is zero, linear, or nonlinear. Additionally, we present an efficient coordinate descent algorithm for implementing VARD. Empirical evaluations on both simulated and real-world datasets demonstrate VARD’s superior performance compared to alternative variable selection methods for additive models.

This work is in collaboration with Zihe Liu and Diptarka Saha.