Asymptotic and robustness properties of reweighted posteriors

• Department of Statistics, Columbia University, New York, USA [jm5692@columbia.edu]

• Department of Statistics, Columbia University, New York, USA [marco.avella@columbia.edu]

• Department of Statistics, Columbia University, New York, USA [cynthia.rush@columbia.edu]

Keywords: $\alpha$-posteriors – Bernstein-von Mises theory – Influence function – Local asymptotic normality – Model misspecification – Robust probabilistic modeling

1 Abstract

In recent years, there has been a line of work in Bayesian statistics and probabilistic machine learning that has studied $\alpha$-posteriors. The $\alpha$-posteriors (also known as fractional or power posteriors) are proportional to the product of the prior density and the $\alpha$-power of the model likelihood. For instance, marusic_medina investigates the robustness to misspecification of $\alpha$-posteriors and their variational approximations. In their work, they establish Bernstein-von Mises (BvM) theorems in total variation distance for $\alpha$-posteriors and for their variational approximations. The main assumption in that work is a stochastically local asymptotic normality (LAN) condition, which is also considered in marusic_vaart.

In our work, we consider reweighted posteriors with multiple parameters ${\alpha }_1,\mathrm{\dots },{\alpha }_n$, one for each of the data points, instead of one tempering parameter $\alpha$ for the whole likelihood. This is similar to the robust probabilistic modeling framework for Bayesian data reweighting introduced in marusic_wang. We introduce a weighted stochastically local asymptotic normality condition that allows us to derive a BvM theorem under model misspecification. It shows that our tempered posterior concentrates around the weighted maximum likelihood estimator (MLE) instead of the standard MLE as in marusic_medina. We connect these posterior reweighting approaches to the well-studied M-estimation procedures in frequentist settings. We obtain asymptotic results similar to those in marusic_miller but a under weaker LAN-type assumption for M-estimators. We use the derived BvM statement to better understand the concentration of the reweighted posteriors and the robustness of the posterior mean estimators by means of their influence functions in the spirit of marusic_disparities.