Minimum Covariance Determinant Estimator and Outlier Detection for Interval-valued Data

• CEMAT & Department of Mathematics, Instituto Superior Técnico, Lisbon, Portugal
  [catarinapadrela@tecnico.ulisboa.pt, rosario.oliveira@tecnico.ulisboa.pt]

• Faculdade de Economia, Universidade do Porto & LIAAD-INESC TEC, Porto, Portugal
  [mpbrito@fep.up.pt]

• CAMGSD & Department of Mathematics, Instituto Superior Técnico, Lisbon, Portugal
  [lina.oliveira@tecnico.ulisboa.pt]

Keywords: Symbolic Data Analysis – Minimum Covariance Determinant Estimator – Mallows’ Distance – Outlier Detection

The increasing demand for analysing large datasets has led to the development of Symbolic Data Analysis, which focuses on modelling complex data structures. Symbolic data preserves inherent variability in the data while simultaneously reducing the dataset size. Interval-valued data is one of the most common symbolic data types, where an interval of real numbers is recorded for each variable in each unit. Nevertheless, this framework presents theoretical and methodological challenges requiring innovative solutions.

The location and scale of interval-valued random vectors can be obtained using the barycentre approach based on the Mallows’ distance [Irpino and Verde, 2015, Oliveira et al., 2024]. However, as in conventional data analysis, these (classical) estimates are highly sensitive to anomalous data points present in real-life datasets, resulting in the need for robust methods.

To address this issue, we develop robust estimators for location and scale, extending the Minimum Covariance Determinant (MCD) estimator [Rousseeuw and van Driessen, 1999] to interval-valued data. We start by formulating the optimization problem and showing that the objective function is concave [Boyd and Vandenberghe, 2004]. This enables us to employ the Majorization-Minorization algorithm [Lange, 2016] with Taylor’s expansion to derive the MCD estimator for interval-valued data. The resulting MCD algorithm yields a robust distance, which is used to detect anomalous observations. This can be done by setting appropriate cut-off values or by leveraging the farness concept [Raymaekers et al., 2022].

Finally, we assess the performance of the interval-valued MCD estimator and the outlier detection method using both synthetic and real datasets. In the simulation study, we compare the proposed method with the classical estimators across varying contamination levels. Our findings indicate that the interval-valued MCD estimator effectively estimates the symbolic covariance matrix even with $20\%$ contamination, whereas the classical estimator fails. Additionally, our method achieves high accuracy in identifying anomalous observations, making it a valuable tool for robust symbolic data analysis in real-world applications.