Large Deviations and Bahadur Efficiency for General Divergence-based Estimators

• Department of Mathematical Sciences, University of Copenhagen, Denmark [collamore@math.ku.dk]

• Department of Statistics, George Mason University, Fairfax, VA, U.S.A. [avidyash@gmu.edu]

Keywords: Divergence measures, Hellinger distance, large deviations, rare event probabilities.

1 Abstract

Minimum divergence estimators have been proposed as alternatives to maximum likelihood methods due to their favorable robustness properties [cf., e.g., Beran, 1977, Lindsay, 1994]. The objective of this work is to study, from a general perspective, the rare event probabilities associated with these divergence-based inferential methods. Namely, given a parametric family of distributions $\{f_{\theta }:\theta \in \mathrm{\Theta }\}$ and a true density $g$, our objective is to study rare events of the form $\{{\widehat{\theta }}_n\in C\}$ for sets $C\subset \mathrm{\Theta }$, where ${\widehat{\theta }}_n\to {\theta }_g\notin C$. In the one-dimensional setting, we derive estimates which state that, for $\theta >{\theta }_g$,

$$\underset{n\to \mathrm{\infty }}{lim}\frac{1}{n}\log 𝐏\left({\widehat{\theta }}_n\ge \theta \right)=-\underset{p\in \Re }{inf}\text{KL}(p,g),$$

where $\Re$ is a subset of the space of densities (dependent on the choice of the divergence measure); KL$(\cdot )$ denotes the Kullback-Leibler distance between the densities $p$ and $g$; and this limit is understood in the sense of large deviation theory. Here, the right-hand side represents the large deviation “rate function” describing the exponential decay rate of the given probability. We also extend these results to the multidimensional setting. Finally, under further assumptions, we utilize these estimates to establish Bahadur efficiency for general divergence measures, extending and complimenting results of Bahadur [1967] and Arcones [2006].