Bayesian Machine Learning
This is a short course of the Mathematics for daTa scieNce study plan
 |
|
Bio
Sara Wade is a Reader (Associate Prof.) in Statistics and Data Science at the School of Mathematics, University of Edinburgh. Before this, she was a Harrison Early Career Assistant Professor at the Department of Statistics, University of Warwick (2015-2018). She was a post-doc in Machine Learning at the Computational and Biological Learning Laboratory, University of Cambridge working with Prof. Zoubin Ghahramani (2012-2015). In January 2013, she earned her PhD in Statistics from Bocconi University, under the supervision of Prof. Sonia Petrone and Prof. Stephen Walker. Her research interests include statistics, machine learning, and Bayesian analysis, with a focus on flexible methodology and efficient inference for complex data.Course Description
Machine learning is rapidly becoming one of the most important areas of general practice, research, and activity in computer science. This is reflected in the tremendous scale and growth of research devoted to the subject and the active recruitment in and employment of machine learning in industry and government. Machine learning algorithms provide impressive predictive accuracy through expressive and highly-complex models. To effectively cope with the over-parametrization of the models, training algorithms must incorporate regularization terms, which can generally be interpreted as prior distributions, an integral component of the Bayesian approach to machine learning. Moreover, the Bayesian framework naturally propagates uncertainty along the entire inferential and decision process to create an ensemble of machine learning models. This provides better uncertainty quantification with improved calibration and robustness, to alleviate issues of deep learning methods, such as overconfident predictions and susceptibility to adversarial attacks Szegedy et al (2013).
In this course, I will provide an introduction to Bayesian machine learning, starting with methods for high-dimensional regression, such as ridge and LASSO, and connecting them to their Bayesian counterparts. Variational Inference is widely-used in the machine learning community for fast, approximate Bayesian inference, and I will give an overview of variational inference, along with examples for high-dimensional regression. Next, I will provide an introduction to Bayesian neural networks and Gaussian process, and discuss how to construct Gaussian processes as the limit of infinitely-wide neural networks.
List of topics
- High-dimensional regression: ridge and LASSO and Bayesian shrinkage priors
- Variational inference
- Bayesian neural networks
- Gaussian processes
References
- James, G.M., Witten, D., Hastie, T, & Tibshirani, R. (2023) An Introduction to Statistical Learning, available with applications in R or Python.
- Murphy, K. (2022). Probabilsitic Machine Learning
- Blei, D. M., Kucukelbir, A., & McAuliffe, J. D. (2017). Variational inference: A review for statisticians. Journal of the American Statistical Association, 112(518), 859-877.
- Neal, R. M. (2012). Bayesian learning for neural networks.
- Williams, C. K., & Rasmussen, C. E. (2006). Gaussian processes for machine learning. Amari, S. I. (1997). Information geometry. Contemporary Mathematics, 203, 81-96.
- Jospin, L. V., Laga, H., Boussaid, F., Buntine, W., & Bennamoun, M. (2022). Hands-on Bayesian neural networks—A tutorial for deep learning users. IEEE Computational Intelligence Magazine, 17(2), 29-48.
Schedule
- Monday 20 May 2024, 14.30-16.30 room A105
- Tuesday 21 May 2024, 11.30-13.30 room A210
- Wednesday 22 May 2024, 14.30-16.30 room A222
- Thursday 23 May 2024, 08.30-10.30 room A215
Details
- Venue: Polo Scientifico e Tecnologico F. Ferrari
- Language: English
- The participation is free. Please send an email to Prof. Claudio Agostinelli to confirm your participation.
- For further information, please contact Prof. Claudio Agostinelli
Material
The material of the course will be available here