Validation of Medical ML-Models on non-iid Datasets

In the domain of medical image analysis for pathology (histopathology), there is a huge potential for applying modern machine learned image analysis models on tasks like segmentation of cell nuclei or classification of nuclei by cell type. In scientific studies, these models seem to perform well and therefore, one would like to apply them in medical practice. One of the many requirements for that is a very careful and well executed validation of their generalization performance on benchmark datasets. However, because of the very expensive procedure of obtaining images for a validation set and several factors that create interdependencies between the images that violate the common iid assumption of many machine learning approaches, specific care must be taken when validating such models.

The goal of this thesis is to explore the effects of specific violations of the iid-assumption with respect to validating the generalization performance of machine learned models for histopathological images.


  • Very solid knowledge in statistics and validation of machine learning for image based tasks (Machine Learning 1, Data Science or similar courses)
  • Interest in bridging the gap between theoretically oriented research and solid practical outcomes
  • Good programming skills in Python or languages with strong deep learning frameworks
  • Knowledge about the specific medical domain is helpful, but not mandatory


  • Homeyer, A. et. Al. (2021). Artificial Intelligence in Pathology: From Prototype to Product. Journal of Pathology Informatics (DOI 10.4103/jpi.jpi_84_20)
  • He, Y. Shen, Z. Cui, P. (2021). Towards non-iid image classification: A dataset and baselines. Pattern Recognition Volume 110 ISSN 0031-3203
  • Geirhos, R. et. Al. (2020) – Shortcut Learning in Deep Neural Networks