3D Mitochondria Instance Segmentation with Spatio-Temporal Transformers

Medical Image Computing and Computer Assisted Intervention – MICCAI 2023, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Volume 14227 LNCS

Abstract

Accurate 3D mitochondria instance segmentation in electron microscopy (EM) is a challenging problem and serves as a prerequisite to empirically analyze their distributions and morphology. Most existing approaches employ 3D convolutions to obtain representative features. However, these convolution-based approaches struggle to effectively capture long-range dependencies in the volume mitochondria data, due to their limited local receptive field. To address this, we propose a hybrid encoder-decoder framework based on a split spatio-temporal attention module that efficiently computes spatial and temporal self-attentions in parallel, which are later fused through a deformable convolution. Further, we introduce a semantic foreground-background adversarial loss during training that aids in delineating the region of mitochondria instances from the background clutter. Our extensive experiments on three benchmarks, Lucchi, MitoEM-R and MitoEM-H, reveal the benefits of the proposed contributions achieving state-of-the-art results on all three datasets. Our code and models are available at https://github.com/OmkarThawakar/STT-UNET.

Description

Keywords

Electron Microscopy, Hybrid CNN-Transformers, Mitochondria instance segmentation, Spatio-Temporal Transformer

DOI

10.1007/978-3-031-43993-3_59

Citation

Thawakar, O, Anwer, R M, Laaksonen, J, Reiner, O, Shah, M & Khan, F S 2023, 3D Mitochondria Instance Segmentation with Spatio-Temporal Transformers . in H Greenspan, H Greenspan, A Madabhushi, P Mousavi, S Salcudean, J Duncan, T Syeda-Mahmood & R Taylor (eds), Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 : Proceedings of 26th International Conference . Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14227 LNCS, Springer, pp. 613-623, International Conference on Medical Image Computing and Computer Assisted Intervention, Vancouver, British Columbia, Canada, 08/10/2023 . https://doi.org/10.1007/978-3-031-43993-3_59

Permanent link to this item

https://urn.fi/URN:NBN:fi:aalto-202401041120

Collections

[article-cris] Perustieteiden korkeakoulu / SCI

Show all metadata

3D Mitochondria Instance Segmentation with Spatio-Temporal Transformers

Access rights

URL

Journal Title

Journal ISSN

Volume Title

Authors

Date

Department

Major/Subject

Mcode

Degree programme

Language

Pages

Series