High-Dimensional Nonparametric Optimization of Scaled Bregman Distances

M. Broniatowski1 and W. Stummer2
  • 1

    LPSM, Sorbonne Université, Paris, France [michel.broniatowski@sorbonne-universite.fr]

  • 2

    Corresponding Presenter/Speaker of this talk; Department of Mathematics, Friedrich-Alexander-Universität Erlangen–Nürnberg (FAU), Erlangen, Germany [stummer@math.fau.de]

The constrained minimization (respectively maximization) of directed distances (i.e. divergences) is a fundamental task in statistics (e.g. for minimum distance estimations and goodness-of-fit analyses) as well as in the adjacent fields of machine learning, artificial intelligence, information theory, signal processing and pattern recognition. In this talk, we focus on the Scaled Bregman Distances (cf. Stummer [2007], Stummer and Vajda [2012]), which cover both the omnipresent class of Csiszar f-divergences/disparities (e.g. Kullback-Leibler divergence, Peason chisquare divergence, Hellinger distance, GNED) as well as the omnipresent class of Bregman distances (e.g. density power divergences of Basu et al. [1998], (squared) L2-distance, Bregman exponential divergence). Some robustness properties of Scaled Bregman Distances have been extensively studied in Kißlinger and Stummer [2016], see also the illuminating 3D illustrations in Roensch and Stummer [2017].

We present how to solve high-dimensional constrained optimization problems of Scaled Bregman Distances, in terms of the dimension-free precise bare-simulation method developed in Broniatowski and Stummer [2023, 2024]; for this, almost no assumptions (like convexity) on the set of constraints are needed, which makes this approach e.g. applicable to non-parametric or semi-parametric minimum-distance estimations.

References

  • Basu et al. [1998] A. Basu, I. R. Harris, N. L. Hjort, and M.C. Jones. Robust and efficient estimation by minimizing a density power divergence. Biometrika, 85(3):549–559, 1998.
  • Broniatowski and Stummer [2023] M. Broniatowski and W. Stummer. A precise bare simulation approach to the minimization of some distances. I. Foundations. IEEE Trans. Inf. Theory, 69(5):3062–3120, 2023.
  • Broniatowski and Stummer [2024] M. Broniatowski and W. Stummer. A precise bare simulation approach to the minimization of some distances. II. Further foundations. arXiv:2402.08478, pages 1–77, 2024.
  • Kißlinger and Stummer [2016] A.-L. Kißlinger and W. Stummer. Robust statistical engineering by means of scaled Bregman distances. In C. Agostinelli, A. Basu, P. Filzmoser, and D. Mukherjee, editors, Recent Advances in Robust Statistics – Theory and Applications, pages 81–113, Springer: New Delhi, 2016.
  • Roensch and Stummer [2017] B. Roensch and W. Stummer. 3D insights to some divergences for robust statistics and machine learning. In F. Nielsen and F. Barbaresco, editors, Geometric Science of Information GSI 2017, volume LNCS 10589, pages 460–469, Springer International Publishing: Cham, 2017.
  • Stummer [2007] W. Stummer. Some Bregman distances between financial diffusion processes. Proc. Appl. Math. Mech., 7(1):1050503–1050504, 2007.
  • Stummer and Vajda [2012] W. Stummer and I. Vajda. On Bregman distances and divergences of probability measures. IEEE Trans. Inf. Theory, 58(3):1277–1288, 2012.