Robust Anomaly Detection via Order Statistical Methods

• Department of Mathematics and Statistics, Mississippi State University, Mississippi State, Mississippi, USA [jwoody@math.msstate.edu,kab1167@msstate.edu]

• Department of Statistical Sciences and Operations Research, Virginia Commonwealth University, Richmond, Virginia, USA [qlu2@vcu.edu]

Keywords: Anomaly Detection – Forensic Statistics – Robust Statistic

1 Motivation

This talk is motivated by recent results in forensic finance and forensic statistics regarding the Paycheck Protection Program (PPP) which was part of the United States’ response to the Covid-19 Pandemic. Recent studies show that clustering in digital transaction amounts [Woody et al., 2024, Griffin et al., 2023] is linked to misreporting and fraud in various governmental programs. This study develops a rigorous methodology to investigate clustering patterns observed in digital transactions in complex data environments. The methodology is robust to departures from any assumed statistical distribution for digital transaction amounts.

2 Methods

Let $\{X_i\}$ for $i=1,2,\mathrm{\dots },n$ denote a sample of $n$ transaction amounts housed by a governmental institution. We assume that the transactions follow a Gamma distribution with density given by

$$f(x;\alpha ,\beta )=\frac{{\beta }^{\alpha }}{\mathrm{\Gamma }(\alpha )}{x}^{\alpha -1}{e}^{-\beta x}\mathit{\hspace{1em}}\text{ for }x>0$$ (1)

and $\alpha$, $\beta >0$. The term $F(x;\alpha ,\beta )$ will represent the gamma cumulative distribution function (CDF) having parameters $\alpha$ and $\beta$. Then define

$$U_i=\widehat{F}(X_i),\text{ for }i=1,2,\mathrm{\dots },n.$$ (2)

When $\widehat{\alpha }$ and $\widehat{\beta }$ are obtained via MLE estimation. However, the distributional assumptions will never accommodate reality in all cases. The CDF transforms in (2) of the data to obtain “uniform-like” nearly independent and nearly identically distributed random variables. The phrase uniform-like acknowledges that the gamma density does not capture the distribution of the transactions; however, as we will show, the gamma fits are robust faulty model assumptions in the ensuing methodology.

The novel mechanism to “filter” uniform-like random variables into clustered and non-clustered transaction data utilizes a familiar result from order statistics. Let ${\{W_i\}}_{i=1}^n$ be an i.i.d. sequence of uniformly distributed random variables on $(0,1)$. Let $W_{(i)}$ be the $i$th order statistic, whereby

and we set $W_{(n+1)}=1$. The inequalities are strict with probability 1. A classic result dating back to Malmquist [1950] states that for $i=1,2,\mathrm{\dots },n$

$$V_i={\left(\frac{W_{(i)}}{W_{(i+1)}}\right)}^i$$ (3)

are i.i.d. such that $V_i$ is uniformly distributed on (0,1). The sequence $\{V_i\}$ compels the methodology. We replace the uniform $W_i$s in (3) with the uniform-like $U_i$s from (2).

The idea is as follows: whenever an alien clustering or atypical concentrations are present, successive $U_{(i)}$s will be much closer together than expected. Subsequently, the associated $V_i$s will be close to unity. The task is then to formalize an approach to statistically test if too many $V_i$s are too close to unity. Interestingly, the $V_i$s are surprisingly robust to $U_i$s departure from a ”uniform-like” distribution.

A segmented density is fit to the $V_i$s:

(4)

where $v\in (0,1)$ and parameters ${\theta }_2\in (0,1)$ and ${\theta }_1\in (0,1)$. Note that when ${\theta }_1={\theta }_2$, (4) reduces to the density of a uniformly distributed random variable on $(0,1)$. Novel statistical theory and practical results are presented.