ACROBAT 2023 Banner
Join

ACROBAT

Welcome to the AutomatiC Registration Of Breast cAncer Tissue (ACROBAT) challenge website! The ACROBAT challenge will be held in conjunction with MICCAI 2023. We are looking forward to your contribution and hope to see you in Vancouver!

Overview

The ACROBAT challenge aims to advance the development of whole-slide-image (WSI) registration algorithms that can align WSIs of  breast cancer tissue sections that were stained with immunohistochemistry (IHC) or haematoxylin and eosin (H&E). In order to evaluate usefulness of developed methods on real-world data, all slides in this challenge originate from routine diagnostic workflows. 

Motivation

The ACROBAT challenge aims to advance the development of whole-slide-image (WSI) registration algorithms that can align WSIs of tissue sections from the same tumour that were stained with immunohistochemistry (IHC) or haematoxylin and eosin (H&E). In order to evaluate usefulness of developed methods on real-world data, all slides in this challenge originate from routine diagnostic workflows.

Registration across multiple stains can be an enabling technology both for research and diagnostics. There are e.g. several studies [1]–[4] that use IHC as a local label to train subsequent models to predict from H&E WSIs. In diagnostic settings, it can be of critical importance to overlay WSIs of differently stained tissue e.g. to evaluate resection borders for invasive cancer. In the case of residual tumor in resection margins, additional treatment such as another surgery might be appropriate.

Unfortunately, WSI registration is particularly challenging due to the gigapixel scale of these images. Furthermore, a few micrometers thin tissue sections easily deform during sample preparation and it is possible that parts of the tissue in WSIs of the same sample are not contained in other WSIs of that same sample, particularly if sections are not consecutive. The appearance of tissue can also differ considerably depending on the stain. Multi-stain WSI registration is therefore an active field of research and has previously been addressed in the ANHIR[5] challenge and more recently the ACROBAT 2022 challenge, which was held in conjunction with MICCAI 2022. In the first ACROBAT challenge, the objective was to register WSIs of tissue sections that were stained with one of the four routine diagnostic IHC stains ER, PGR, HER2 and KI67 to corresponding WSIs of tissue sections stained with H&E. Details of the data set are available in [6]. The first round of the ACROBAT challenge was successful in establishing the state-of-the-art in WSI registration on WSIs from routine diagnostic slides. The top- performing methods exceeded our expectations both with regards to accuracy, as well as to robustness. The code for most of these methods is available online. A particularly interesting finding from the first round of the ACROBAT challenge was that conceptually very different registration methods can perform similarly well in this task.

Now, we would like to propose a second round of the challenge that aims to investigate how well these different methods perform under domain shifts. More specifically, we would like to estimate how well methods generalise to unseen IHC stains that were not part of the previously published data. To facilitate the ACROBAT 2023 challenge, we will annotate a new test set consisting of undisclosed and unpublished IHC stains for 200 test cases, each consisting of two WSIs that are to be registered, including IHC-IHC and IHC-HE image pairs. Instead of registered landmarks, participants will this time be asked to submit docker containers of their methods. Participants will therefore be completely blinded to the test data, which will enable an investigation of robustness of the registration methods under domain shift and unknown conditions.

TLDR

Objective: Evaluation of algorithms that register a pair of WSIs. 

Data: Cases are primary breast cancer patients. All WSIs originate from routine diagnostic workflows. 750 cases with 1 H&E and 1-4 matched routine IHC (ER, PGR, HER2, KI67) WSIs for algorithm development, 100 cases for validation. Each validation case consists of 1 H&E and 1 IHC WSI from the four routine stains. Registered validation set landmarks can be submitted to the leaderboard for automated performance quantification. 

ACROBAT 2022: 300 cases for testing. Each case consists of 1 H&E and 1 IHC WSI from the four routine stains. Please contact the challenge organizers to upon submitting registered test set landmarks in order to receive quantitative feedback. Submissions must contain registered landmarks, as well as a link to a publicly available algorithm description, e.g. on arxiv.org. 

ACROBAT 2023: 200 cases for testing. Each case consists of a pair of WSIs, including IHC-HE and IHC-IHC image pairs. The test set will include undisclosed IHC stains that are not part of the training or validation data. Participants must contain a docker container that registers image pairs and returns transformed test set landmarks. 

References

[1] W. Bulten et al., “Epithelium segmentation using deep learning in H&E-stained prostate specimens with immunohistochemistry as reference standard,” Sci. Rep., vol. 9, no. 1, p. 864, Jan. 2019.

[2] D. Tellez et al., “Whole-Slide Mitosis Detection in H&E Breast Histology Using PHH3 as a Reference to Train Distilled Stain-Invariant Convolutional Networks,” IEEE Trans. Med. Imaging, Mar. 2018, doi: 10.1109/TMI.2018.2820199.

[3] P. Gamble et al., “Determining breast cancer biomarker status and associated morphological features using deep learning,” Communications Medicine, vol. 1, no. 1, pp. 1–12, Jul. 2021.

[4] M. Valkonen et al., “Cytokeratin-Supervised Deep Learning for Automatic Recognition of Epithelial Cells in Breast Cancers Stained for ER, PR, and Ki-67,” IEEE Trans. Med. Imaging, vol. 39, no. 2, pp. 534–542, Feb. 2020.

[5] J. Borovec et al., “ANHIR: Automatic Non-Rigid Histological Image Registration Challenge,” IEEE Trans. Med. Imaging, vol. 39, no. 10, pp. 3042–3052, Oct. 2020.

[6] P. Weitz et al., “ACROBAT -- a multi-stain breast cancer histological whole-slide-image data set from routine diagnostics for computational pathology,” arXiv [eess.IV], Nov. 24, 2022. [Online]. Available: http://arxiv.org/abs/2211.13621