Robustly Valid Inference for M- and Z-estimators

• Department of Statistics and Data Science, Carnegie Mellon University, Pittsburgh, PA, USA, 15218.

Traditional methods of inference (hypothesis tests or confidence sets) are based on limiting distributions of M- and Z-estimators. While the most commonly observed limiting distribution is normal, it is by no means the only one. Furthermore, even in cases where the pointwise limiting distributions are normal, one might have non-normal limits for contiguous data generating processes. This happens for example in the case of median estimation if the underlying Lebesgue density is zero at the median. In these “irregular” problems, traditional methods of inference fail. In this talk, I will present two methods of constructing confidence sets that are more robustly valid. For M-estimands defined as

$$\theta (P):={\mathrm{argmin}}_{\theta \in \mathrm{\Theta }}{𝔼}_P[\mathrm{\ell }(Z,\theta )],$$

we construct a confidence set of the form

$${\widehat{\mathrm{CI}}}_{n,\alpha }:=\{\theta \in \mathrm{\Theta }:\frac{1}{n}\sum _{i=1}^n\{\mathrm{\ell }(Z_i,\theta )-\mathrm{\ell }(Z_i,\widehat{\theta })\}\le {\widehat{\gamma }}_{n,\alpha }(\theta )\},$$

where $\widehat{\theta }$ is constructed independently of the data $Z_1,\mathrm{\dots },Z_n$ (for example, by sample splitting). Here ${\widehat{\gamma }}_{n,\alpha }(\theta )$ is a data-driven quantity that can depend on $\theta$. We show that the validity of this confidence set does not depend on the limiting distribution of $\widehat{\theta }$ and on the dimension/complexity of $\mathrm{\Theta }$. This confidence set is extensively studied in a recent manuscript available at https://arxiv.org/abs/2501.07772.

For Z-estimands defined as solution to the equation ${𝔼}_P[\psi (Z,\theta )]=0$, we construct a confidence set of the form

$${\widehat{\mathrm{CI}}}_{n,\alpha }:=\{\theta \in \mathrm{\Theta }:\underset{a\in 𝒜}{\max }\frac{\left|{\sum }_{i=1}^n{a}^{\top }\psi (Z,\theta )\right|}{\sqrt{{\sum }_{i=1}^n{(a^{\top }\psi (Z_i,\theta ))}^2}}\le {\kappa }_{\alpha ,𝒜}\},$$

for an arbitrary subset $𝒜$ of the unit sphere. Here ${\kappa }_{\alpha ,𝒜}$ is also a data-driven quantity. We show validity of this confidence set irrespective of the asymptotics of the Z-estimator. This confidence set is extensively studied in a recent manuscript available at https://arxiv.org/abs/2407.12278.