Generative Adversarial Network Methods with Hellinger Loss

• Department of Statistical Sciences, University of Padua, Padua, Italy [giovanni.saraceno@unipd.it]

• Department of Statistics, George Mason University, Fairfax, VA, USA [avidyash@gmu.edu]

• Department of Mathematics, University of Trento, Trento, Italy [claudio.agostinelli@unitn.it]

Over the past few years, deep learning models, particularly deep neural networks, have revolutionized various machine learning applications. Generative modeling stands out as a major application area, where models are trained to learn and mimic the underlying data distribution, thereby enabling the generation of new synthetic data. Generative Adversarial Networks (GANs), introduced by Goodfellow et al. [2014], have emerged as state-of-the-art generative models, particularly in image analysis, where they operate via an adversarial process involving a generator, which learns to produce realistic samples, and a discriminator, which distinguishes between real and generated data, guiding the generator’s improvement. Despite the success of GANs, the mathematical and statistical properties underpinning their performance are not well understood. Key unknowns include GAN’s ability to approximate the target distribution, the impact of the discriminators on this approximation quality, and the statistical properties of the loss function. This work aims to address these knowledge gaps by proposing a version of GANs with the objective functions inspired by the Hellinger distance between two density functions. We investigate the role of the loss function for prediction and generation using GANs. Our findings provide insight into the statistical properties of the proposed model.