Robust Matrix Factorization

• Indian Statistical Institute, Kolkata, India [ayanbasu@isical.ac.in]

Keywords: Density power divergence – Singular value decomposition – Principal components analysis

Matrix factorization is an important mathematical tool which has many applications in practical problems. Most standard available techniques for matrix factorization have poor robustness properties. On the other hand, the few robust methods available in this context either sacrifice a large part of their model efficiency or involve high complexity leading to computational roadblocks. Here we present two particular procedures – having different objectives – requiring efficient matrix factorization techniques. For this purpose we make use of robust minimum divergence methods, based, in particular on the density power divergence (Basu et al., 1998), and describe the development of the corresponding methods. These methods, although eventually associated with different objectives, follow the same sequence of alternating regression procedures, apart from making use of the same divergence.

The first item considered here is the problem of robust singular value decomposition. The traditional method for performing singular value decomposition (SVD) of a data matrix makes use of the least squares principle, which is known to be highly sensitive to anomalous data. As an application, we will consider the problem of background modelling of video surveillance data. In the presence of camera tampering this cannot be reliably undertaken by the classical SVD. We will consider a novel robust singular value decomposition technique based on the minimum density power divergence estimator. We will also propose a fast and scalable algorithm based on alternating weighted regression to obtain the relevant estimate.

The second item will involve a robust principal components analysis based on the density power divergence. Principal component analysis (PCA) is a popular statistical technique used primarily for dimensionality reduction. However, it is also adversely affected by the presence of anomalous observations. The robust PCA algorithm, again relying on alternating regressions, leads to a procedure which has a rich theory, has high breakdown independently of the data dimension, and is computationally accessible, a combination of properties which few other procedures match.

We will also give suitable indications for the theoretical properties of the estimators, and give some illustrations to demonstrate the performance of these procedures. Fuller accounts of these methods are available in Roy et al. (2024a, b).