Sparse Unbiased Estimating Equations via Likelihood Score Alignment

G. Bertagnolli${}^a$, Z. Huang${}^b$ and D. Ferrari${}^a$

${}^a$Free University of Bozen-Bolzano, IT, ${}^b$RMIT University, Melbourne, AU

Estimating equations are central to statistical inference, underpinning likelihood- and moment-based methods. Consider a parametric model $ℳ=f(\cdot ;\theta ):\theta \in \mathrm{\Theta }\subseteq {ℝ}^p$ and a sample $(Y^{(1)},\mathrm{\dots },Y^{(n)})$. We study inference for $\theta$ in high-dimensional regimes ($p\ge n$), based on an unbiased estimating function $S:\mathrm{\Theta }\times 𝒴\to {ℝ}^p$ satisfying $𝔼\theta [S(\theta ,Y)]=0$ and the associated system $\sum i=1^{n}S(\theta ,Y^{(i)})=0$. Such functions include moment conditions and composite likelihood scores. In high dimensions, the system is typically ill-conditioned, motivating a sparsity assumption: the true parameter ${\theta }_0$ has support $A_0=j:{\theta }_{0j}\ne 0$ with $|A_0|\ll p$. Building on optimal estimating function theory [1, 2], we look for an optimal estimating function $\tilde{S}=W_{0}S$ in the class of linear transformations of $S$, where $W_0$ is the minimiser of the convex criterion

$${ℒ}_{\lambda }(W)=\frac{1}{2}\mathrm{tr}\left(W{\mathrm{\Sigma }}_S{W}^{\top }\right)-\mathrm{tr}\left(H_S{W}^{\top }\right)+{\mathrm{pen}}_{\lambda }(W).$$

with ${\mathrm{\Sigma }}_S$, $H_S$ being the variability and sensitivity matrices of $S$ respectively. The penality ${\mathrm{pen}}_{\lambda }(W)$ enforces sparsity directly at the estimating function level, also providing a link between the sparsity pattern of $W$ and the active set $A_0$. Under standard conditions for penalised models, we establish selection consistency and asymptotic normality on the estimated active set. We also propose a blockwise proximal scoring algorithm for efficient computation of $(\widehat{W},\widehat{\theta })$, and illustrate the method on problems including linear regression and sparse multinomial models.

Keywords: High-dimensional statistics, Estimating equations, Sparsity