Multiple Testing and Beyond: From Error Control to Post-hoc Inference
This is a short course of the Mathematics for daTa scieNce study plan
![]() |
|
Bio
Course Description
This short course provides an overview of the main principles and modern developments in multiple hypothesis testing. Starting from classical methods for controlling the Family-Wise Error Rate (FWER) and the False Discovery Rate (FDR), we will introduce the general framework of closed testing and discuss its extensions to post-hoc inference. Particular attention will be given to the estimation of True Discovery Proportions (TDP) through both parametric and nonparametric approaches (ARI and pARI). The final part of the course will include practical demonstrations in R, with examples from neuroimaging (fMRI, EEG) and genomic data analysis.
Program Outline
- Introduction to Multiple Testing and Error Control Methods
Overview of multiple hypothesis testing; comparison between Family-Wise Error Rate (FWER) and False Discovery Rate (FDR). Discussion of main procedures: Bonferroni (single-step), Holm (step-down), closed testing, Benjamini–Hochberg, Benjamini–Yekutieli, and Storey’s approach based on FDP estimation
-
Post-hoc Inference for True Discovery Proportions (TDP) Parametric and nonparametric critical vector-based approaches: All-Resolution Inference and permutation-based All-Resolution Inference, Notip.
-
R Applications
Implementation and examples on real-world data: functional MRI, EEG, and genomic studies.
Main References
- Goeman, J. J., & Solari, A. (2014). Multiple hypothesis testing in genomics. Statistics in Medicine, 33(11), 1946-1978
- Goeman, J. J., Hemerik, J., & Solari, A. (2021). Only closed testing procedures are admissible for controlling false discovery proportions. Annals of Statistics, 49(2), 1218-1238.
- Goeman, J. J., & Solari, A. (2011). Multiple testing for exploratory research. Statistical Science, 26(4), 584-597.
- Andreella, A., Hemerik, J., Finos, L., Weeda, W., & Goeman, J. (2023). Permutation-based true discovery proportions for functional magnetic resonance imaging cluster analysis. Statistics in Medicine, 42(14), 2311-2340.
- Andreella, A., Vesely, A., Weeda, W., & Goeman, J. (2024). Selective inference for fMRI cluster-wise analysis, issues, and recommendations for critical vector selection: A comment on Blain et al. Imaging Neuroscience, 2, 1-7.
- Blain, A., Thirion, B., & Neuvial, P. (2022). Notip: Non-parametric true discovery proportion control for brain imaging. NeuroImage, 260, 119492.
Schedule
- Thursday, 05 February 2026, 15:00-17:00, room A203 @ Povo1
- Friday, 06 February 2026, 10:30-12:30, room A203 @ Povo1
Details
- Language: English
- For further information, please contact Prof. Claudio Agostinelli
