Statistical inference for smoothed quantile regression with streaming data

• Department of Mathematical and Statistical Sciences, University of Alberta, Edmonton, Canada [jinhan3@ualberta.ca; bei1@ualberta.ca; lkong@ualberta.ca]

• Zhongtai Securities Institute for Financial Studies, Shandong University, Jinan, China

  [yanxiaodong@xjtu.edu.cn]

Keywords: Streaming data – Quantile regression – Smoothing – Confidence interval – High-dimensional data

1 Abstract and Motivation

In this paper, we tackle the problem of conducting valid statistical inference for quantile regression with streaming data. The main difficulties are that the quantile regression loss function is non-smooth and it is often infeasible to store the entire dataset in memory, rendering traditional methodologies ineffective. We introduce a fully online updating method for statistical inference in smoothed quantile regression with streaming data to overcome these issues. Our main contributions are twofold. First, for low-dimensional data, we present an incremental updating algorithm to obtain the smoothed quantile regression estimator with the streaming data set. The proposed estimator allows us to construct asymptotically exact statistical inference procedures. Second, within the realm of high-dimensional data, we develop an online debiased lasso procedure to accommodate the special sparse structure of streaming data. The proposed online debiased approach is updated with only the current data and summary statistics of historical data and corrects an approximation error term from online updating with streaming data. Furthermore, theoretical results such as estimation consistency and asymptotic normality are established to justify its validity in both settings. Our findings are supported by simulation studies and illustrated through real data applications.

2 Contributions

We introduce an online framework designed for estimation and statistical inference in quantile regression models, tailored specifically for streaming data and utilizing a convolution-type smoothing technique. Our key contributions are as follows:

• In the low-dimensional setting, we introduce an online renewable estimator that allows for both estimation and inference to be updated in real time using current data alongside summary statistics from previous data. Under mild regularity conditions, the proposed online renewable estimator achieves asymptotic equivalence with the offline oracle estimator, which utilizes the full dataset. This advancement underscores the effectiveness of our approach in dynamically adapting to new information without sacrificing the accuracy and reliability typically associated with offline analysis.

• In high-dimensional data analysis, regularization is crucial, with ${\mathrm{\ell }}_1$-regularization being a natural choice. However, the lasso estimator lacks a tractable limiting distribution [Zhao and Yu, 2006, Bühlmann and Van De Geer, 2011], limiting its use in inference. To address this, we propose an online debiased lasso estimator tailored for streaming data, with online lasso estimation as a by-product. Our method is simple, efficient, and requires only the current data batch and summary statistics, avoiding full dataset re-access. Under mild regularity conditions, we establish oracle inequalities for the online lasso estimator and asymptotic normality for the online debiased estimator, enabling valid statistical inference.

Our work differs from Han et al. [2021] and Luo et al. [2023] in key ways. First, we focus on quantile regression (QR), which models the dependence of response quantiles on predictors, whereas their methods address linear and generalized linear models. Second, unlike high-dimensional generalized linear models that assume a twice-differentiable loss function with local strong convexity (e.g., squared or logistic loss), QR’s check loss is non-differentiable at some points and lacks strong convexity, posing both theoretical and optimization challenges. Lastly, we derive explicit upper bounds for online lasso estimators with sub-Gaussian covariates, providing a clearer distinction between our oracle inequalities and traditional results [Van de Geer, 2008, Huang et al., 2013].

3 Methodology

Let ${\widetilde{𝜷}}_{h_B}^{\ast }={\arg \min }_{𝜷\in {ℝ}^p}{\sum }_{b=1}^B{n}_b{\widehat{Q}}_h(𝜷;{𝒟}_b)/N_B$ denotes the oracle smoothed QR estimator, we propose the following approximation procedure to obtain the online lasso estimator,

${\widehat{𝜷}}_{h_B}=$ $\underset{𝜷\in {ℝ}^p}{\arg \min }\{ℒ(𝜷)+{\lambda }_B\parallel {𝜷\parallel }_1\}$

$\equiv$ $\underset{𝜷\in {ℝ}^p}{\arg \min }\left[{\displaystyle \frac{1}{N_B}}\left\{{\displaystyle \frac{1}{2}}{(𝜷-{\widehat{𝜷}}_{h_{B-1}})}^{\top }{\tilde{Q}}_{B-1}(𝜷-{\widehat{𝜷}}_{h_{B-1}})+n_B{\widehat{Q}}_{h_B}(𝜷;{𝒟}_B)\right\}+{\lambda }_B{\parallel 𝜷\parallel }_1\right],$

where ${\tilde{Q}}_{B-1}={\sum }_{b=1}^{B-1}{n}_b{\widehat{Q}}_{h_b}^{(2)}({\widehat{𝜷}}_{h_b};{𝒟}_b)$ denotes the aggregated information matrix and ${\widehat{𝜷}}_{h_1}={\widehat{𝜷}}_{h_1}^{\ast }$.

The summarized estimation procedure of the whole online lasso is showed in Figure 1. When a new data stream ${𝒟}_B=\{{𝒚}_B,{𝑿}_B\}$ arrives, we update the online smoothed lasso estimator ${\widehat{𝜷}}_{h_B}$ via the proximal gradient descent algorithm based on the historical summary statistics $\{{\widehat{𝜷}}_{h_{B-1}},{\tilde{Q}}_{B-1}\}$. Meanwhile, the previous cumulative dataset that produces the previous online lasso estimator ${\widehat{𝜷}}_{h_{B-1}}$ that can be regarded as the training set and the newly arrived data set ${𝒟}_B$ can be used as the testing set, we then choose tuning parameter ${\lambda }_B$ via minimizing the quantile prediction error.

Figure 1: Flowchart of the online lasso estimation procedure.