Casual Inference for Clinical Trails:11 1 The two most often misspelled words during my term as Biometrics Editor
A Spellchecker’s Guide to Randomization Tests in Complex Settings

William Fisher Rosenberger1
  • 1

    Department of Statistics, George Mason University, U. S. A. [wrosenbe@gmu.edu]

Keywords: Clinical trials – Randomization-based inference – R. A. Fisher

1 Abstract

Randomization-based inference is perhaps the earliest form of robust inference, dating from Neyman and Fisher in the 1930s (e.g., Welch [1937]). Any inference procedure from a randomized experiment that assumes random sampling from a population ignores Fisherian principles regarding the analysis of designed experiments. And clinical trials are the quintessential designed experiment. Since we hear quite often about the sacred regulatory issue of strict type I error rate control, it is surprising that a statistical technique guaranteed to do just that would be so easily dismissed or ignored. More recently, the explosion of research on causal inference and conformal inference has focused on randomization-based inference, even in settings that are not randomized.

We first present the difference between a randomization test and a permutation test; the former being based on the reference set of all possible randomization trajectories, and the latter on the basis of permutations of the data independent of any design components (Rosenberger and Lachin [2016]). We then describe Fisher’s randomization test based on a data set of Darwin, presented in the original Design of Experiments book (Fisher [1935]).

We discuss these issues and demonstrate that randomization tests can be used for more complex settings, such as multiple (>2) treatment comparisons, confidence interval estimation, adjusted treatment effects via generalized linear models, sequential stopping, analyses with missing outcome data, and subgroup analyses. It is interesting to note that the only cohort of statisticians not excited about randomization tests in this age of causal inference are the designers and conductors of randomized clinical trials! I will conclude with a few historical notes about Fisher and de Finetti.

Some parts of this talk were presented at the 2024 Fisher Memorial Lecture at the University of Cambridge (Rosenberger [2025]).

References

  • Rosenberger and Lachin [2016] Rosenberger, W. F. and Lachin, J. M. Randomization in Clinical Trials: Theory and Practice, John Wiley & Sons, Hoboken, 2016.
  • Fisher [1935] Fisher, R. A. The Design of Experiments, Hafner, London, 1935.
  • Welch [1937] Welch, B. L. On the z-test in randomized blocks and Latin squares. Biometrika, 6(29):21–52, 1937.
  • Rosenberger [2025] Rosenberger, W. F. The 41st Fisher Memorial Lecture. From Fisher to CARA: the evolution of randomization and randomization-based inference. Journal of the Royal Statistical Society, Series A, in press.