Distribution-Weighted Least Squares Regression Estimator. An application in Mendelian Randomization

• Departamento de Estadística, IO, CN, Universidad Nacional de Educación a Distancia (UNED), Madrid, Spain [agar-per@ccia.uned.es]

• Statistical Genetics Research Group, Institute of Medical Biometry, Heidelberg University, Heidelberg, Germany [lorenzo@imbi.uni.heidelberg.de]

• Programa de Doctorado en Ciencias, Universidad Nacional de Educación a Distancia (UNED), Madrid, Spain [jlorenzo12@alumno.uned.es]

• This paper is part of the PhD thesis of the second author under the supervision of the first one

Keywords: Robust Regression – Saddlepoint Approximation – Mendelian Randomization

1 Introduction

Trimmed least squares (LTS) regression estimator Rousseeuw [1984] is, probably, one of the most widely used estimators in robust regression. However, it is not possible to obtain a closed-form expression for it, and its computation could be difficult as it has a non-differentiable objective function. LTS gives 0 weights to large values of standard residuals and 1 weights to small values. In this contributed paper we consider a weighted least squares regression estimator, DWLS, where the weights are not so extreme, but are the probabilities that the corresponding studentized residual is an outlier, i.e. an unlikely observation, when we consider a scale or location contaminated normal model. We obtain a closed-form expression of this new estimator, which has good robustness properties. Based on DWLS, we define a novel robust Inverse Variance Weighted (IVW) estimator for Mendelian Randomisation studies. We study its main properties and include some examples.

2 Main results

In the classical Linear Regression Model with $p$ covariates,

$$𝐲=𝐱𝜷+𝐞$$

where errors follow a normal distribution, $e_i|{𝐱}_i\rightsquigarrow N(0,{\sigma }^2),i=1,\mathrm{\dots },n$, the residuals are defined as

$$r_i=r_i(𝜷)=y_i-{𝐱}_i𝜷$$

and the classical least squares estimator is obtained as

$${\widehat{\beta }}_{LS}={\text{min}}_{𝜷\in {ℝ}^p}\sum _{i=1}^n{r}_i^2(𝜷)$$

Outliers are usually detected using the Studentized Residuals for Regression Diagnostics

$$S{R}_i=\frac{r_i}{\widehat{\sigma }\sqrt{1-h_{ii}}}$$

where $h_{ii}$ are the elements of the diagonal of the hat matrix $𝐇=𝐱{\left({𝐱}^t𝐱\right)}^{-1}{𝐱}^t$.

We know that $S{R}_i$ are iid variables that follow a student $t$ distribution with $n-p-1$ degrees of freedom, if homoscedasticity and normality are satisfied.

Within the class of weighted least squares estimators

$${\text{min}}_{𝜷\in {ℝ}^p}\sum _{i=1}^n{w}_i{r}_i^2(𝜷)$$

considering that $e_i|{𝐱}_i\rightsquigarrow (1-ϵ)N(0,{\sigma }^2)+ϵN(\mu ,k^2{\sigma }^2),i=1,\mathrm{\dots },n$, we propose as weights

$$w_i(t)=P\{|S{R}_i|>t\}$$

for the distribution-weighted least squares regression line, DWLS, defined as

$${\widehat{\beta }}_{DWLS}={\text{min}}_{𝜷\in {ℝ}^p}\sum _{i=1}^n{w}_i{r}_i^2(𝜷)$$

for which the lower the probability of obtaining the ith residual, the lower it weights on the regression line.

Assuming the contaminated normal distribution for the errors mentioned above and with a von Mises expansion for the functional tail probability

$$P_F\{T_n>t\}\simeq P_G\{T_n>t\}+\int \text{TAIF}(x;t;T_n,G)𝑑F(x)$$

and a Lugannani and Rice approximation for the Tail Area Influence Function TAIF, Field and Ronchetti [1985], we obtain, for a scale contaminated normal model, the weights

$w_i$ $\simeq$ $P\{t_{n-p-1}>t\}+ϵA_1\{(A_2-{\displaystyle \frac{3t^2+1}{4(t^2-1)}}){[t^2-k^2(t^2-1)]}^{-1/2}$

$+{\displaystyle \frac{3t^2-1}{t^2-1}}{\displaystyle \frac{k^2}{2}}{[t^2-k^2(t^2-1)]}^{-3/2}$

$-{\displaystyle \frac{3k^4}{4}}{[t^2-k^2(t^2-1)]}^{-5/2}$

$+{\displaystyle \frac{t^2(1-k^2)}{t^2-1}}-A_2\}$

and, for a location contaminated normal model, the weights

$$w_i\simeq P\{t_{n-p-1}>t\}+ϵA_1\{(e^{{\mu }^2(t^2-1)/2}-1)(A_2+\frac{t^2{\mu }^2}{t^2-1})$$

$$-e^{{\mu }^2(t^2-1)/2}\frac{{\mu }^4{t}^4}{4}\}$$

DWLS estimator is regression, scale, and affine equivariant and has the highest possible finite sample breakdown point:

$$\frac{[(n+1)/2]}{n}\mathit{\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} }\text{if }p=1$$

$$\frac{[(n-p)/2]+1}{n}\mathit{\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} }\text{if }p>1$$

3 Mendelian Randomization

Mendelian randomisation is a method of inference for assessing the causal effect of an exposure $X$ on an outcome $Y$. Nevertheless, to analyze the significance of the slope in the classical regression line of $Y$ on $X$ is a too simple method because there are a lot of covariates, than just $X$, to explain $Y$. To solve this problem, $L$ instrumental variables $Z$ (DNA markers related to $X$) are used, obtaining a ratio estimator ${\widehat{\beta }}_{R_j}$ for each DNA marker, $j=1,\mathrm{\dots },L$. Combining all ratio estimators (all DNA markers), we obtain the classical IVW estimator, which has a break point equal to 0. Since the weights in the IVW estimator are the inverse of the variance of ${\widehat{\beta }}_{R_j}$, and IVW is the slope of the classical regression line through the origin of $U$ on $V$ where $U_j={\widehat{\beta }}_{Y_j}/{\sigma }_{Y_j}$ and $V_j={\widehat{\beta }}_{X_j}/{\sigma }_{Y_j}$, we propose a robust IVW estimator in which the classical regression line is replaced by the robust DWLS regression line. We will conclude the talk with an application of this new estimator to the real data set investigated in.