CRUSH: Causal Regularization Under Shifted Heterogeneity

A. Berarducci${}^a$, M. Lembo${}^a$, V. Vinciotti${}^b$ and E. C.Wit${}^a$

${}^a$USI Università della Svizzera Italiana, ${}^b$University of Trento,

Keywords: Causal Inference, Invariant Causal Prediction, Out of Sample Risk Minimization.

Prediction is one of the most important uses of statistical methods. However, predictive systems typically struggle when the environment in which they were trained changes. The central research question is to identify and estimate stable models when only arbitrary learning environments are available. Existing approaches, such as Causal Dantzig (Rothenhäusler et al., 2018) and Causal Regularization (Kania and Wit, 2025), directly identify the causal parameter as the unique solution whose residual moments remain equal across environments. However, when perturbations are weak or the shift-induced difference matrix is ill-conditioned, this may lead to unstable estimates. This work extends Causal Regularization, obtaining risk guarantees under heterogeneous shifts with few assumptions. Causal Regularization Under Shifted Heterogeneity (CRUSH) is a generalization of Causal Regularization that operates with two shifted environments, without assuming that either is unperturbed. In general, the risk difference becomes non-convex and may exhibit saddle-point geometry. We show that the causal parameter ${\beta }_{CP}$ is characterized as a stationary point of the generalized risk difference. Furthermore, defining a sieve of out-of-sample distributions, we derive a worst-risk decomposition in each of these increasing sets of data environments with a closed-form solution for $\widehat{\beta }$ that interpolates between the OLS and the causal parameter while guaranteeing uniqueness even under non-aligned shifts. Our results show that even though the estimator was trained on a limited set of in-sample data environments, we can obtain prediction stability in a large set of perturbed or non-stationary data environments with minimal assumptions on the shifts.