While CNN-based methods have been the cornerstone of medical image segmentation due to their promising performance and robustness,they suffer from limitations in capturing long-range dependencies.Transformer-based app...While CNN-based methods have been the cornerstone of medical image segmentation due to their promising performance and robustness,they suffer from limitations in capturing long-range dependencies.Transformer-based approaches are currently prevailing since they enlarge the receptive field to model global contextual correlations.To further extract rich representations,some extensions of U-Net employ multi-scale feature extraction and fusion modules to obtain improved performance.Inspired by this idea,we propose TransCeption for medical image segmentation,a pure transformer-based U-shaped network incorporating an inception-like module in the encoder and adopting a contextual bridge for better feature fusion.The design proposed in this work is based on three core principles.(i)The patch merging module in the encoder is redesigned to use ResInception Patch Merging(RIPM).The Multi-Branch(MB)transformer has the same number of branches as the outputs of RIPM.Combining the two modules enables the model to capture a multi-scale representation within a single stage.(ii)We apply an Intra-stage Feature Fusion(IFF)module following the MB transformer to enhance the aggregation of feature maps from all branches and particularly focus on the interaction between the different channels at all scales.(iii)In contrast to a bridge that only contains tokenwise self-attention,we propose a Dual Transformer Bridge that also includes channel-wise self-attention to exploit correlations between scales at different stages from a dual perspective.Extensive experiments on multi-organ and skin lesion segmentation tasks show the superiority of TransCeption to previous work.The code is publicly available on GitHub.展开更多
Magnetic resonance imaging(MRI)is one of the most prevalent imaging modalities used for diagnosis,treatment planning,and outcome control in various medical conditions.MRI sequences provide physicians with the ability ...Magnetic resonance imaging(MRI)is one of the most prevalent imaging modalities used for diagnosis,treatment planning,and outcome control in various medical conditions.MRI sequences provide physicians with the ability to view and monitor tissues at multiple contrasts within a single scan and serve as input for automated systems to perform downstream tasks.However,in clinical practice,there is usually no concise set of identically acquired sequences for a whole group of patients.As a consequence,medical professionals and automated systems both face difficulties due to the lack of complementary information from such missing sequences.This problem is well known in computer vision,particularly in medical image processing tasks such as tumor segmentation,tissue classification,and image generation.With the aim of helping researchers,this literature review examines a significant number of recent approaches that attempt to mitigate these problems.Basic techniques such as early synthesis methods,as well as later approaches that deploy deep learning,such as common latent space models,knowledge distillation networks,mutual information maximization,and generative adversarial networks(GANs)are examined in detail.We investigate the novelty,strengths,and weaknesses of the aforementioned strategies.Moreover,using a case study on the segmentation task,our survey offers quantitative benchmarks to further analyze the effectiveness of these methods for addressing the missing modalities challenge.Furthermore,a discussion offers possible future research directions.展开更多
基金funded by the German Research Foundation(Deutsche Forschungsgemeinschaft,DFG)project number 455548460.
文摘While CNN-based methods have been the cornerstone of medical image segmentation due to their promising performance and robustness,they suffer from limitations in capturing long-range dependencies.Transformer-based approaches are currently prevailing since they enlarge the receptive field to model global contextual correlations.To further extract rich representations,some extensions of U-Net employ multi-scale feature extraction and fusion modules to obtain improved performance.Inspired by this idea,we propose TransCeption for medical image segmentation,a pure transformer-based U-shaped network incorporating an inception-like module in the encoder and adopting a contextual bridge for better feature fusion.The design proposed in this work is based on three core principles.(i)The patch merging module in the encoder is redesigned to use ResInception Patch Merging(RIPM).The Multi-Branch(MB)transformer has the same number of branches as the outputs of RIPM.Combining the two modules enables the model to capture a multi-scale representation within a single stage.(ii)We apply an Intra-stage Feature Fusion(IFF)module following the MB transformer to enhance the aggregation of feature maps from all branches and particularly focus on the interaction between the different channels at all scales.(iii)In contrast to a bridge that only contains tokenwise self-attention,we propose a Dual Transformer Bridge that also includes channel-wise self-attention to exploit correlations between scales at different stages from a dual perspective.Extensive experiments on multi-organ and skin lesion segmentation tasks show the superiority of TransCeption to previous work.The code is publicly available on GitHub.
基金funded by the German Research Foundation(Deutsche Forschungsgemeinschaft,DFG)under project numbers 191948804 and 455548460.
文摘Magnetic resonance imaging(MRI)is one of the most prevalent imaging modalities used for diagnosis,treatment planning,and outcome control in various medical conditions.MRI sequences provide physicians with the ability to view and monitor tissues at multiple contrasts within a single scan and serve as input for automated systems to perform downstream tasks.However,in clinical practice,there is usually no concise set of identically acquired sequences for a whole group of patients.As a consequence,medical professionals and automated systems both face difficulties due to the lack of complementary information from such missing sequences.This problem is well known in computer vision,particularly in medical image processing tasks such as tumor segmentation,tissue classification,and image generation.With the aim of helping researchers,this literature review examines a significant number of recent approaches that attempt to mitigate these problems.Basic techniques such as early synthesis methods,as well as later approaches that deploy deep learning,such as common latent space models,knowledge distillation networks,mutual information maximization,and generative adversarial networks(GANs)are examined in detail.We investigate the novelty,strengths,and weaknesses of the aforementioned strategies.Moreover,using a case study on the segmentation task,our survey offers quantitative benchmarks to further analyze the effectiveness of these methods for addressing the missing modalities challenge.Furthermore,a discussion offers possible future research directions.
