Quadratic Distance in Model Selection

• Department of Biostatistics, SPHHP & Jacobs School of Medicine, University at Buffalo, Buffalo, NY, USA [markatou@buffalo.edu]

1 Abstract

Statistical distances, divergencies and similar quantities have a large history and play a fundamental role in statistics, machine learning and associated scientific disciplines. In this talk, we focus on the role of quadratic distances, a special class of statistical distances, on robust model selection and propose a method called Quadratic Information Criterion (QIC). This method is derived as a suitable estimator of the relative quadratic risk, the definition of which is based on the concept of quadratic distance between two probability distributions.

We discuss the construction of QIC and show that, for specific values of the tuning parameter, QIC is equivalent to AIC and asymptotically equivalent to BIC. Using oracle inequalities in the regression case, we propose a specific form of QIC for practical use. Simulation results and application of this criterion to real data sets indicate better performance than BIC in small data sets and equivalent to BIC performance in larger data sets in the regression context. Furthermore, the QIC clearly outperforms both AIC and AIC${}_c$, the corrected AIC.