Deng, R.; Liu, Q.; Cui, C.; Yao, T.; Xiong, J.; Bao, S.; Li, H.; Yin, M.; Wang, Y.; Zhao, S.; Tang, Y.; Yang, H.; Huo, Y. “HATs: Hierarchical Adaptive Taxonomy Segmentation for Panoramic Pathology Image Analysis.” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Volume 15004 LNCS, 2024, pp. 155-166, DOI: 10.1007/978-3-031-72083-3_15.
Segmenting large, detailed images of tissue samples in computational pathology is very challenging because tissues have complex structures at different scales. For example, in kidney pathology, you have larger regions like the cortex and medulla, as well as smaller structures like glomeruli, tubules, blood vessels, and various cell types.
To tackle this, we developed a new method called Hierarchical Adaptive Taxonomy Segmentation (HATs). HATs is designed to accurately identify and label these different kidney structures in large panoramic images by using a detailed understanding of kidney anatomy. The key features of HATs include:
- A special technique that uses the spatial relationships between 15 different tissue types to create a flexible model that can be applied to different scales, from large regions down to individual cells.
- A simplified way of representing these tissue structures in a matrix format, making it easier to process the whole panoramic image.
- Integration of a new AI model (EfficientSAM) to help extract features from the images without needing manual inputs, making it more adaptable and efficient.
Our tests showed that HATs effectively combines clinical knowledge and imaging techniques to accurately segment over 15 types of tissue structures. The code for HATs is available at https://github.com/hrlblab/HATs.
Fig. 1.
Knowledge transformation from kidney anatomy to a hierarchical taxonomy tree. This figure demonstrates the transformation of intricate clinical anatomical relationships within the kidney into a hierarchical taxonomy tree. (a) Pathologists examine histopathology in accordance with kidney anatomy. (b) This study revisits kidney anatomy using a hierarchical semantic taxonomy for panoramic segmentation, covering 15 classes across regions, units, and cells. The tree incorporates spatial relationships into a semi-supervised learning paradigm and uses hierarchical scale information as prior knowledge to weigh the relationship between classes.