Semantic segmentation for mixed scenes of aerial remote sensing and road traffic is one of the key technologies for visual perception of flying cars.The State-of-the-Art(SOTA)semantic segmentation methods have made re...Semantic segmentation for mixed scenes of aerial remote sensing and road traffic is one of the key technologies for visual perception of flying cars.The State-of-the-Art(SOTA)semantic segmentation methods have made remarkable achievements in both fine-grained segmentation and real-time performance.However,when faced with the huge differences in scale and semantic categories brought about by the mixed scenes of aerial remote sensing and road traffic,they still face great challenges and there is little related research.Addressing the above issue,this paper proposes a semantic segmentation model specifically for mixed datasets of aerial remote sensing and road traffic scenes.First,a novel decoding-recoding multi-scale feature iterative refinement structure is proposed,which utilizes the re-integration and continuous enhancement of multi-scale information to effectively deal with the huge scale differences between cross-domain scenes,while using a fully convolutional structure to ensure the lightweight and real-time requirements.Second,a welldesigned cross-window attention mechanism combined with a global information integration decoding block forms an enhanced global context perception,which can effectively capture the long-range dependencies and multi-scale global context information of different scenes,thereby achieving fine-grained semantic segmentation.The proposed method is tested on a large-scale mixed dataset of aerial remote sensing and road traffic scenes.The results confirm that it can effectively deal with the problem of large-scale differences in cross-domain scenes.Its segmentation accuracy surpasses that of the SOTA methods,which meets the real-time requirements.展开更多
The self-attention mechanism of Transformers,which captures long-range contextual information,has demonstrated significant potential in image segmentation.However,their ability to learn local,contextual relationships ...The self-attention mechanism of Transformers,which captures long-range contextual information,has demonstrated significant potential in image segmentation.However,their ability to learn local,contextual relationships between pixels requires further improvement.Previous methods face challenges in efficiently managing multi-scale fea-tures of different granularities from the encoder backbone,leaving room for improvement in their global representation and feature extraction capabilities.To address these challenges,we propose a novel Decoder with Multi-Head Feature Receptors(DMHFR),which receives multi-scale features from the encoder backbone and organizes them into three feature groups with different granularities:coarse,fine-grained,and full set.These groups are subsequently processed by Multi-Head Feature Receptors(MHFRs)after feature capture and modeling operations.MHFRs include two Three-Head Feature Receptors(THFRs)and one Four-Head Feature Receptor(FHFR).Each group of features is passed through these MHFRs and then fed into axial transformers,which help the model capture long-range dependencies within the features.The three MHFRs produce three distinct feature outputs.The output from the FHFR serves as auxiliary auxiliary features in the prediction head,and the prediction output and their losses will eventually be aggregated.Experimental results show that the Transformer using DMHFR outperforms 15 state of the arts(SOTA)methods on five public datasets.Specifically,it achieved significant improvements in mean DICE scores over the classic Parallel Reverse Attention Network(PraNet)method,with gains of 4.1%,2.2%,1.4%,8.9%,and 16.3%on the CVC-ClinicDB,Kvasir-SEG,CVC-T,CVC-ColonDB,and ETIS-LaribPolypDB datasets,respectively.展开更多
Semantic segmentation of remote sensing images is a critical research area in the field of remote sensing.Despite the success of Convolutional Neural Networks(CNNs),they often fail to capture inter-layer feature relat...Semantic segmentation of remote sensing images is a critical research area in the field of remote sensing.Despite the success of Convolutional Neural Networks(CNNs),they often fail to capture inter-layer feature relationships and fully leverage contextual information,leading to the loss of important details.Additionally,due to significant intraclass variation and small inter-class differences in remote sensing images,CNNs may experience class confusion.To address these issues,we propose a novel Category-Guided Feature Collaborative Learning Network(CG-FCLNet),which enables fine-grained feature extraction and adaptive fusion.Specifically,we design a Feature Collaborative Learning Module(FCLM)to facilitate the tight interaction of multi-scale features.We also introduce a Scale-Aware Fusion Module(SAFM),which iteratively fuses features from different layers using a spatial attention mechanism,enabling deeper feature fusion.Furthermore,we design a Category-Guided Module(CGM)to extract category-aware information that guides feature fusion,ensuring that the fused featuresmore accurately reflect the semantic information of each category,thereby improving detailed segmentation.The experimental results show that CG-FCLNet achieves a Mean Intersection over Union(mIoU)of 83.46%,an mF1 of 90.87%,and an Overall Accuracy(OA)of 91.34% on the Vaihingen dataset.On the Potsdam dataset,it achieves a mIoU of 86.54%,an mF1 of 92.65%,and an OA of 91.29%.These results highlight the superior performance of CG-FCLNet compared to existing state-of-the-art methods.展开更多
The application of transformer networks and feature fusion models in medical image segmentation has aroused considerable attention within the academic circle.Nevertheless,two main obstacles persist:(1)the restrictions...The application of transformer networks and feature fusion models in medical image segmentation has aroused considerable attention within the academic circle.Nevertheless,two main obstacles persist:(1)the restrictions of the Transformer network in dealing with locally detailed features,and(2)the considerable loss of feature information in current feature fusion modules.To solve these issues,this study initially presents a refined feature extraction approach,employing a double-branch feature extraction network to capture complex multi-scale local and global information from images.Subsequently,we proposed a low-loss feature fusion method-Multi-branch Feature Fusion Enhancement Module(MFFEM),which realizes effective feature fusion with minimal loss.Simultaneously,the cross-layer cross-attention fusion module(CLCA)is adopted to further achieve adequate feature fusion by enhancing the interaction between encoders and decoders of various scales.Finally,the feasibility of our method was verified using the Synapse and ACDC datasets,demonstrating its competitiveness.The average DSC(%)was 83.62 and 91.99 respectively,and the average HD95(mm)was reduced to 19.55 and 1.15 respectively.展开更多
An approach for color image segmentation is proposed based on the contributions of color features to segmentation rather than the choice of a particular color space. The determination of effective color features depen...An approach for color image segmentation is proposed based on the contributions of color features to segmentation rather than the choice of a particular color space. The determination of effective color features depends on the analysis of various color features from each tested color image via the designed feature encoding. It is different from the pervious methods where self organized feature map (SOFM) is used for constructing the feature encoding so that the feature encoding can self organize the effective features for different color images. Fuzzy clustering is applied for the final segmentation when the well suited color features and the initial parameter are available. The proposed method has been applied in segmenting different types of color images and the experimental results show that it outperforms the classical clustering method. The study shows that the feature encoding approach offers great promise in automating and optimizing the segmentation of color images.展开更多
Camouflaged Object Detection(COD)aims to identify objects that share highly similar patterns—such as texture,intensity,and color—with their surrounding environment.Due to their intrinsic resemblance to the backgroun...Camouflaged Object Detection(COD)aims to identify objects that share highly similar patterns—such as texture,intensity,and color—with their surrounding environment.Due to their intrinsic resemblance to the background,camouflaged objects often exhibit vague boundaries and varying scales,making it challenging to accurately locate targets and delineate their indistinct edges.To address this,we propose a novel camouflaged object detection network called Edge-Guided and Multi-scale Fusion Network(EGMFNet),which leverages edge-guided multi-scale integration for enhanced performance.The model incorporates two innovative components:a Multi-scale Fusion Module(MSFM)and an Edge-Guided Attention Module(EGA).These designs exploit multi-scale features to uncover subtle cues between candidate objects and the background while emphasizing camouflaged object boundaries.Moreover,recognizing the rich contextual information in fused features,we introduce a Dual-Branch Global Context Module(DGCM)to refine features using extensive global context,thereby generatingmore informative representations.Experimental results on four benchmark datasets demonstrate that EGMFNet outperforms state-of-the-art methods across five evaluation metrics.Specifically,on COD10K,our EGMFNet-P improves F_(β)by 4.8 points and reduces mean absolute error(MAE)by 0.006 compared with ZoomNeXt;on NC4K,it achieves a 3.6-point increase in F_(β).OnCAMO and CHAMELEON,it obtains 4.5-point increases in F_(β),respectively.These consistent gains substantiate the superiority and robustness of EGMFNet.展开更多
The accurate segmentation of deep gray matter nuclei is critical for neuropathological research,disease diagnosis and treatment.Existing methods employ the supervised learning training approach,which requires large la...The accurate segmentation of deep gray matter nuclei is critical for neuropathological research,disease diagnosis and treatment.Existing methods employ the supervised learning training approach,which requires large labeled datasets.It is challenging and time-consuming to obtain such datasets for medical image analysis.In addition,these methods based on convolutional neural networks(CNNs)only achieve suboptimal performance due to the locality of convolutional operations.Vision Transformers(ViTs)efficiently model long-range dependencies and thus have the potentiality to outperform these methods in segmentation tasks.To address these issues,we propose a novel hybrid network based on self-supervised pre-training for deep gray matter nuclei segmentation.Specifically,we present a CNN-Transformer hybrid network(CTNet),whose encoder consists of 3D CNN and ViT to learn local spatial-detailed features and global semantic information.A self-supervised learning(SSL)approach that integrates rotation prediction and masked feature reconstruction is proposed to pre-train the CTNet,enabling the model to learn valuable visual representations from unlabeled data.We evaluate the effectiveness of our method on 3T and 7T human brain MRI datasets.The results demonstrate that our CTNet achieves better performance than other comparison models and our pre-training strategy outperforms other advanced self-supervised methods.When the training set has only one sample,our pre-trained CTNet enhances segmentation performance,showing an 8.4%improvement in Dice similarity coefficient(DSC)compared to the randomly initialized CTNet.展开更多
The application of deep learning in fabric defect detection has become increasingly widespread.To address false positives and false negatives in fabric roll seam detection,and to improve automation efficiency and prod...The application of deep learning in fabric defect detection has become increasingly widespread.To address false positives and false negatives in fabric roll seam detection,and to improve automation efficiency and product quality,we propose the Multi-scale Context DeepLabV3+(MSC-DeepLabV3+),a semantic segmentation network designed for fabric roll seam detection,based on DeepLabV3+.The model improvements include enhancing the backbone performance through optimization of the UIB-MobileNetV2 network;designing the Dynamic Atrous and Sliding-window Fusion(DASF)module to improve adaptability to multi-scale seam structures with dynamic dilation rates and a sliding-window mechanism;and utilizing the Progressive Low-level Feature Fusion(PLFF)module to progressively restore seam boundary details via shallow feature fusion.Additionally,an enhanced 3-SE attention mechanism is employed,replacing the direct concatenation operation.Experimental results show thatMSCDeepLabV3+outperforms classical and recent segmentation models.Compared to DeepLabV3+with an Xception backbone,MSC-DeepLabV3+achieves a mean intersection over union(mIoU)of 92.30%and the boundary Fscore(BF)of 92.54%,representing improvements of 3.04%and 3.14%,respectively.Moreover,the model complexity is significantly reduced,with the model parameters(params)decreasing to 3.44M and Frames Per Second(FPS)increasing from 101 to 273,demonstrating its potential for deployment in resource-constrained industrial scenarios.展开更多
Visible-infrared Cross-modality Person Re-identification(VI-ReID)is a critical technology in smart public facilities such as cities,campuses and libraries.It aims to match pedestrians in visible light and infrared ima...Visible-infrared Cross-modality Person Re-identification(VI-ReID)is a critical technology in smart public facilities such as cities,campuses and libraries.It aims to match pedestrians in visible light and infrared images for video surveillance,which poses a challenge in exploring cross-modal shared information accurately and efficiently.Therefore,multi-granularity feature learning methods have been applied in VI-ReID to extract potential multi-granularity semantic information related to pedestrian body structure attributes.However,existing research mainly uses traditional dual-stream fusion networks and overlooks the core of cross-modal learning networks,the fusion module.This paper introduces a novel network called the Augmented Deep Multi-Granularity Pose-Aware Feature Fusion Network(ADMPFF-Net),incorporating the Multi-Granularity Pose-Aware Feature Fusion(MPFF)module to generate discriminative representations.MPFF efficiently explores and learns global and local features with multi-level semantic information by inserting disentangling and duplicating blocks into the fusion module of the backbone network.ADMPFF-Net also provides a new perspective for designing multi-granularity learning networks.By incorporating the multi-granularity feature disentanglement(mGFD)and posture information segmentation(pIS)strategies,it extracts more representative features concerning body structure information.The Local Information Enhancement(LIE)module augments high-performance features in VI-ReID,and the multi-granularity joint loss supervises model training for objective feature learning.Experimental results on two public datasets show that ADMPFF-Net efficiently constructs pedestrian feature representations and enhances the accuracy of VI-ReID.展开更多
The semantic segmentation methods based on CNN have made great progress,but there are still some shortcomings in the application of remote sensing images segmentation,such as the small receptive field can not effectiv...The semantic segmentation methods based on CNN have made great progress,but there are still some shortcomings in the application of remote sensing images segmentation,such as the small receptive field can not effectively capture global context.In order to solve this problem,this paper proposes a hybrid model based on ResNet50 and swin transformer to directly capture long-range dependence,which fuses features through Cross Feature Modulation Module(CFMM).Experimental results on two publicly available datasets,Vaihingen and Potsdam,are mIoU of 70.27%and 76.63%,respectively.Thus,CFM-UNet can maintain a high segmentation performance compared with other competitive networks.展开更多
Whale optimization algorithm(WOA)tends to fall into the local optimum and fails to converge quickly in solving complex problems.To address the shortcomings,an improved WOA(QGBWOA)is proposed in this work.First,quasi-o...Whale optimization algorithm(WOA)tends to fall into the local optimum and fails to converge quickly in solving complex problems.To address the shortcomings,an improved WOA(QGBWOA)is proposed in this work.First,quasi-opposition-based learning is introduced to enhance the ability of WOA to search for optimal solutions.Second,a Gaussian barebone mechanism is embedded to promote diversity and expand the scope of the solution space in WOA.To verify the advantages of QGBWOA,comparison experiments between QGBWOA and its comparison peers were carried out on CEC 2014 with dimensions 10,30,50,and 100 and on CEC 2020 test with dimension 30.Furthermore,the performance results were tested using Wilcoxon signed-rank(WS),Friedman test,and post hoc statistical tests for statistical analysis.Convergence accuracy and speed are remarkably improved,as shown by experimental results.Finally,feature selection and multi-threshold image segmentation applications are demonstrated to validate the ability of QGBWOA to solve complex real-world problems.QGBWOA proves its superiority over compared algorithms in feature selection and multi-threshold image segmentation by performing several evaluation metrics.展开更多
Medical image segmentation is an important application field of computer vision in medical image processing.Due to the close location and high similarity of different organs in medical images,the current segmentation ...Medical image segmentation is an important application field of computer vision in medical image processing.Due to the close location and high similarity of different organs in medical images,the current segmentation algorithms have problems with mis-segmentation and poor edge segmentation.To address these challenges,we propose a medical image segmentation network(AF-Net)based on attention mechanism and feature fusion,which can effectively capture global information while focusing the network on the object area.In this approach,we add dual attention blocks(DA-block)to the backbone network,which comprises parallel channels and spatial attention branches,to adaptively calibrate and weigh features.Secondly,the multi-scale feature fusion block(MFF-block)is proposed to obtain feature maps of different receptive domains and get multi-scale information with less computational consumption.Finally,to restore the locations and shapes of organs,we adopt the global feature fusion blocks(GFF-block)to fuse high-level and low-level information,which can obtain accurate pixel positioning.We evaluate our method on multiple datasets(the aorta and lungs dataset),and the experimental results achieve 94.0%in mIoU and 96.3%in DICE,showing that our approach performs better than U-Net and other state-of-art methods.展开更多
Diabetic retinopathy (DR) is a complication of diabetesmellitus thatappears in the retina. Clinitians use retina images to detect DR pathologicalsigns related to the occlusion of tiny blood vessels. Such occlusion bri...Diabetic retinopathy (DR) is a complication of diabetesmellitus thatappears in the retina. Clinitians use retina images to detect DR pathologicalsigns related to the occlusion of tiny blood vessels. Such occlusion brings adegenerative cycle between the breaking off and the new generation of thinnerand weaker blood vessels. This research aims to develop a suitable retinalvasculature segmentation method for improving retinal screening proceduresby means of computer-aided diagnosis systems. The blood vessel segmentationmethodology relies on an effective feature selection based on SequentialForward Selection, using the error rate of a decision tree classifier in theevaluation function. Subsequently, the classification process is performed bythree alternative approaches: artificial neural networks, decision trees andsupport vector machines. The proposed methodology is validated on threepublicly accessible datasets and a private one provided by Hospital Sant Joanof Reus. In all cases we obtain an average accuracy above 96% with a sensitivityof 72% in the blood vessel segmentation process. Compared with the state-ofthe-art, our approach achieves the same performance as other methods thatneed more computational power.Our method significantly reduces the numberof features used in the segmentation process from 20 to 5 dimensions. Theimplementation of the three classifiers confirmed that the five selected featureshave a good effectiveness, independently of the classification algorithm.展开更多
Video data are composed of multimodal information streams including visual, auditory and textual streams, so an approach of story segmentation for news video using multimodal analysis is described in this paper. The p...Video data are composed of multimodal information streams including visual, auditory and textual streams, so an approach of story segmentation for news video using multimodal analysis is described in this paper. The proposed approach detects the topic-caption frames, and integrates them with silence clips detection results, as well as shot segmentation results to locate the news story boundaries. The integration of audio-visual features and text information overcomes the weakness of the approach using only image analysis techniques. On test data with 135 400 frames, when the boundaries between news stories are detected, the accuracy rate 85.8% and the recall rate 97.5% are obtained. The experimental results show the approach is valid and robust.展开更多
Multimodal lung tumor medical images can provide anatomical and functional information for the same lesion.Such as Positron Emission Computed Tomography(PET),Computed Tomography(CT),and PET-CT.How to utilize the lesio...Multimodal lung tumor medical images can provide anatomical and functional information for the same lesion.Such as Positron Emission Computed Tomography(PET),Computed Tomography(CT),and PET-CT.How to utilize the lesion anatomical and functional information effectively and improve the network segmentation performance are key questions.To solve the problem,the Saliency Feature-Guided Interactive Feature Enhancement Lung Tumor Segmentation Network(Guide-YNet)is proposed in this paper.Firstly,a double-encoder single-decoder U-Net is used as the backbone in this model,a single-coder single-decoder U-Net is used to generate the saliency guided feature using PET image and transmit it into the skip connection of the backbone,and the high sensitivity of PET images to tumors is used to guide the network to accurately locate lesions.Secondly,a Cross Scale Feature Enhancement Module(CSFEM)is designed to extract multi-scale fusion features after downsampling.Thirdly,a Cross-Layer Interactive Feature Enhancement Module(CIFEM)is designed in the encoder to enhance the spatial position information and semantic information.Finally,a Cross-Dimension Cross-Layer Feature Enhancement Module(CCFEM)is proposed in the decoder,which effectively extractsmultimodal image features through global attention and multi-dimension local attention.The proposed method is verified on the lung multimodal medical image datasets,and the results showthat theMean Intersection overUnion(MIoU),Accuracy(Acc),Dice Similarity Coefficient(Dice),Volumetric overlap error(Voe),Relative volume difference(Rvd)of the proposed method on lung lesion segmentation are 87.27%,93.08%,97.77%,95.92%,89.28%,and 88.68%,respectively.It is of great significance for computer-aided diagnosis.展开更多
Automatic segmentation of liver in medical images is challenging on the aspects of accuracy, automation and robustness. A crucial stage of the liver segmentation is the selection of the image features for the segmenta...Automatic segmentation of liver in medical images is challenging on the aspects of accuracy, automation and robustness. A crucial stage of the liver segmentation is the selection of the image features for the segmentation. This paper presents an accurate liver segmentation algorithm. The approach starts with a texture analysis which results in an optimal set of texture features including high order statistical texture features and anatomical structural features. Then, it creates liver distribution image by classifying the original image pixelwisely using support vector machines. Lastly, it uses a group of morphological operations to locate the liver organ accurately in the image. The novelty of the approach is resided in the fact that the features are so selected that both local and global texture distributions are considered, which is important in liver organ segmentation where neighbouring tissues and organs have similar greyscale distributions. Experiment results of liver segmentation on CT images using the proposed method are presented with performance validation and discussion.展开更多
An efficient algorithm for facial features extractions is proposed. The facial features we segment are the two eyes, nose and mouth. The algorithm is based on an improved Gabor wavelets edge detector, morphological ap...An efficient algorithm for facial features extractions is proposed. The facial features we segment are the two eyes, nose and mouth. The algorithm is based on an improved Gabor wavelets edge detector, morphological approach to detect the face region and facial features regions, and an improved T-shape face mask to locate the extract location of facial features. The experimental results show that the proposed method is robust against facial expression, illumination, and can be also effective if the person wearing glasses, and so on.展开更多
To address the current issues of inaccurate segmentation and the limited applicability of segmentation methods for building facades in point clouds, we propose a facade segmentation algorithm based on optimal dual-sca...To address the current issues of inaccurate segmentation and the limited applicability of segmentation methods for building facades in point clouds, we propose a facade segmentation algorithm based on optimal dual-scale feature descriptors. First, we select the optimal dual-scale descriptors from a range of feature descriptors. Next, we segment the facade according to the threshold value of the chosen optimal dual-scale descriptors. Finally, we use RANSAC (Random Sample Consensus) to fit the segmented surface and optimize the fitting result. Experimental results show that, compared to commonly used facade segmentation algorithms, the proposed method yields more accurate segmentation results, providing a robust data foundation for subsequent 3D model reconstruction of buildings.展开更多
Objective and quantitative assessment of skin conditions is essential for cosmeceutical studies and research on skin aging and skin regeneration.Various handcraft-based image processing methods have been proposed to e...Objective and quantitative assessment of skin conditions is essential for cosmeceutical studies and research on skin aging and skin regeneration.Various handcraft-based image processing methods have been proposed to evaluate skin conditions objectively,but they have unavoidable disadvantages when used to analyze skin features accurately.This study proposes a hybrid segmentation scheme consisting of Deeplab v3+with an Inception-ResNet-v2 backbone,LightGBM,and morphological processing(MP)to overcome the shortcomings of handcraft-based approaches.First,we apply Deeplab v3+with an Inception-ResNet-v2 backbone for pixel segmentation of skin wrinkles and cells.Then,LightGBM and MP are used to enhance the pixel segmentation quality.Finally,we determine several skin features based on the results of wrinkle and cell segmentation.Our proposed segmentation scheme achieved a mean accuracy of 0.854,mean of intersection over union of 0.749,and mean boundary F1 score of 0.852,which achieved 1.1%,6.7%,and 14.8%improvement over the panoptic-based semantic segmentation method,respectively.展开更多
基金supported by the National Key Research and Development of China(No.2022YFB2503400).
文摘Semantic segmentation for mixed scenes of aerial remote sensing and road traffic is one of the key technologies for visual perception of flying cars.The State-of-the-Art(SOTA)semantic segmentation methods have made remarkable achievements in both fine-grained segmentation and real-time performance.However,when faced with the huge differences in scale and semantic categories brought about by the mixed scenes of aerial remote sensing and road traffic,they still face great challenges and there is little related research.Addressing the above issue,this paper proposes a semantic segmentation model specifically for mixed datasets of aerial remote sensing and road traffic scenes.First,a novel decoding-recoding multi-scale feature iterative refinement structure is proposed,which utilizes the re-integration and continuous enhancement of multi-scale information to effectively deal with the huge scale differences between cross-domain scenes,while using a fully convolutional structure to ensure the lightweight and real-time requirements.Second,a welldesigned cross-window attention mechanism combined with a global information integration decoding block forms an enhanced global context perception,which can effectively capture the long-range dependencies and multi-scale global context information of different scenes,thereby achieving fine-grained semantic segmentation.The proposed method is tested on a large-scale mixed dataset of aerial remote sensing and road traffic scenes.The results confirm that it can effectively deal with the problem of large-scale differences in cross-domain scenes.Its segmentation accuracy surpasses that of the SOTA methods,which meets the real-time requirements.
基金supported by Xiamen Medical and Health Guidance Project in 2021(No.3502Z20214ZD1070)supported by a grant from Guangxi Key Laboratory of Machine Vision and Intelligent Control,China(No.2023B02).
文摘The self-attention mechanism of Transformers,which captures long-range contextual information,has demonstrated significant potential in image segmentation.However,their ability to learn local,contextual relationships between pixels requires further improvement.Previous methods face challenges in efficiently managing multi-scale fea-tures of different granularities from the encoder backbone,leaving room for improvement in their global representation and feature extraction capabilities.To address these challenges,we propose a novel Decoder with Multi-Head Feature Receptors(DMHFR),which receives multi-scale features from the encoder backbone and organizes them into three feature groups with different granularities:coarse,fine-grained,and full set.These groups are subsequently processed by Multi-Head Feature Receptors(MHFRs)after feature capture and modeling operations.MHFRs include two Three-Head Feature Receptors(THFRs)and one Four-Head Feature Receptor(FHFR).Each group of features is passed through these MHFRs and then fed into axial transformers,which help the model capture long-range dependencies within the features.The three MHFRs produce three distinct feature outputs.The output from the FHFR serves as auxiliary auxiliary features in the prediction head,and the prediction output and their losses will eventually be aggregated.Experimental results show that the Transformer using DMHFR outperforms 15 state of the arts(SOTA)methods on five public datasets.Specifically,it achieved significant improvements in mean DICE scores over the classic Parallel Reverse Attention Network(PraNet)method,with gains of 4.1%,2.2%,1.4%,8.9%,and 16.3%on the CVC-ClinicDB,Kvasir-SEG,CVC-T,CVC-ColonDB,and ETIS-LaribPolypDB datasets,respectively.
基金funded by National Natural Science Foundation of China(61603245).
文摘Semantic segmentation of remote sensing images is a critical research area in the field of remote sensing.Despite the success of Convolutional Neural Networks(CNNs),they often fail to capture inter-layer feature relationships and fully leverage contextual information,leading to the loss of important details.Additionally,due to significant intraclass variation and small inter-class differences in remote sensing images,CNNs may experience class confusion.To address these issues,we propose a novel Category-Guided Feature Collaborative Learning Network(CG-FCLNet),which enables fine-grained feature extraction and adaptive fusion.Specifically,we design a Feature Collaborative Learning Module(FCLM)to facilitate the tight interaction of multi-scale features.We also introduce a Scale-Aware Fusion Module(SAFM),which iteratively fuses features from different layers using a spatial attention mechanism,enabling deeper feature fusion.Furthermore,we design a Category-Guided Module(CGM)to extract category-aware information that guides feature fusion,ensuring that the fused featuresmore accurately reflect the semantic information of each category,thereby improving detailed segmentation.The experimental results show that CG-FCLNet achieves a Mean Intersection over Union(mIoU)of 83.46%,an mF1 of 90.87%,and an Overall Accuracy(OA)of 91.34% on the Vaihingen dataset.On the Potsdam dataset,it achieves a mIoU of 86.54%,an mF1 of 92.65%,and an OA of 91.29%.These results highlight the superior performance of CG-FCLNet compared to existing state-of-the-art methods.
基金funded by the Henan Science and Technology research project(222103810042)Support by the open project of scientific research platform of grain information processing center of Henan University of Technology(KFJJ-2021-108)+1 种基金Support by the innovative funds plan of Henan University of Technology(2021ZKCJ14)Henan University of Technology Youth Backbone Teacher Program.
文摘The application of transformer networks and feature fusion models in medical image segmentation has aroused considerable attention within the academic circle.Nevertheless,two main obstacles persist:(1)the restrictions of the Transformer network in dealing with locally detailed features,and(2)the considerable loss of feature information in current feature fusion modules.To solve these issues,this study initially presents a refined feature extraction approach,employing a double-branch feature extraction network to capture complex multi-scale local and global information from images.Subsequently,we proposed a low-loss feature fusion method-Multi-branch Feature Fusion Enhancement Module(MFFEM),which realizes effective feature fusion with minimal loss.Simultaneously,the cross-layer cross-attention fusion module(CLCA)is adopted to further achieve adequate feature fusion by enhancing the interaction between encoders and decoders of various scales.Finally,the feasibility of our method was verified using the Synapse and ACDC datasets,demonstrating its competitiveness.The average DSC(%)was 83.62 and 91.99 respectively,and the average HD95(mm)was reduced to 19.55 and 1.15 respectively.
文摘An approach for color image segmentation is proposed based on the contributions of color features to segmentation rather than the choice of a particular color space. The determination of effective color features depends on the analysis of various color features from each tested color image via the designed feature encoding. It is different from the pervious methods where self organized feature map (SOFM) is used for constructing the feature encoding so that the feature encoding can self organize the effective features for different color images. Fuzzy clustering is applied for the final segmentation when the well suited color features and the initial parameter are available. The proposed method has been applied in segmenting different types of color images and the experimental results show that it outperforms the classical clustering method. The study shows that the feature encoding approach offers great promise in automating and optimizing the segmentation of color images.
基金financially supported byChongqingUniversity of Technology Graduate Innovation Foundation(Grant No.gzlcx20253267).
文摘Camouflaged Object Detection(COD)aims to identify objects that share highly similar patterns—such as texture,intensity,and color—with their surrounding environment.Due to their intrinsic resemblance to the background,camouflaged objects often exhibit vague boundaries and varying scales,making it challenging to accurately locate targets and delineate their indistinct edges.To address this,we propose a novel camouflaged object detection network called Edge-Guided and Multi-scale Fusion Network(EGMFNet),which leverages edge-guided multi-scale integration for enhanced performance.The model incorporates two innovative components:a Multi-scale Fusion Module(MSFM)and an Edge-Guided Attention Module(EGA).These designs exploit multi-scale features to uncover subtle cues between candidate objects and the background while emphasizing camouflaged object boundaries.Moreover,recognizing the rich contextual information in fused features,we introduce a Dual-Branch Global Context Module(DGCM)to refine features using extensive global context,thereby generatingmore informative representations.Experimental results on four benchmark datasets demonstrate that EGMFNet outperforms state-of-the-art methods across five evaluation metrics.Specifically,on COD10K,our EGMFNet-P improves F_(β)by 4.8 points and reduces mean absolute error(MAE)by 0.006 compared with ZoomNeXt;on NC4K,it achieves a 3.6-point increase in F_(β).OnCAMO and CHAMELEON,it obtains 4.5-point increases in F_(β),respectively.These consistent gains substantiate the superiority and robustness of EGMFNet.
基金supported in part by the National Natural Science Foundation of China under Grant 62071405the National Natural Science Foundation of China under Grant 12175189.
文摘The accurate segmentation of deep gray matter nuclei is critical for neuropathological research,disease diagnosis and treatment.Existing methods employ the supervised learning training approach,which requires large labeled datasets.It is challenging and time-consuming to obtain such datasets for medical image analysis.In addition,these methods based on convolutional neural networks(CNNs)only achieve suboptimal performance due to the locality of convolutional operations.Vision Transformers(ViTs)efficiently model long-range dependencies and thus have the potentiality to outperform these methods in segmentation tasks.To address these issues,we propose a novel hybrid network based on self-supervised pre-training for deep gray matter nuclei segmentation.Specifically,we present a CNN-Transformer hybrid network(CTNet),whose encoder consists of 3D CNN and ViT to learn local spatial-detailed features and global semantic information.A self-supervised learning(SSL)approach that integrates rotation prediction and masked feature reconstruction is proposed to pre-train the CTNet,enabling the model to learn valuable visual representations from unlabeled data.We evaluate the effectiveness of our method on 3T and 7T human brain MRI datasets.The results demonstrate that our CTNet achieves better performance than other comparison models and our pre-training strategy outperforms other advanced self-supervised methods.When the training set has only one sample,our pre-trained CTNet enhances segmentation performance,showing an 8.4%improvement in Dice similarity coefficient(DSC)compared to the randomly initialized CTNet.
文摘The application of deep learning in fabric defect detection has become increasingly widespread.To address false positives and false negatives in fabric roll seam detection,and to improve automation efficiency and product quality,we propose the Multi-scale Context DeepLabV3+(MSC-DeepLabV3+),a semantic segmentation network designed for fabric roll seam detection,based on DeepLabV3+.The model improvements include enhancing the backbone performance through optimization of the UIB-MobileNetV2 network;designing the Dynamic Atrous and Sliding-window Fusion(DASF)module to improve adaptability to multi-scale seam structures with dynamic dilation rates and a sliding-window mechanism;and utilizing the Progressive Low-level Feature Fusion(PLFF)module to progressively restore seam boundary details via shallow feature fusion.Additionally,an enhanced 3-SE attention mechanism is employed,replacing the direct concatenation operation.Experimental results show thatMSCDeepLabV3+outperforms classical and recent segmentation models.Compared to DeepLabV3+with an Xception backbone,MSC-DeepLabV3+achieves a mean intersection over union(mIoU)of 92.30%and the boundary Fscore(BF)of 92.54%,representing improvements of 3.04%and 3.14%,respectively.Moreover,the model complexity is significantly reduced,with the model parameters(params)decreasing to 3.44M and Frames Per Second(FPS)increasing from 101 to 273,demonstrating its potential for deployment in resource-constrained industrial scenarios.
基金supported in part by the National Natural Science Foundation of China under Grant 62177029,62307025in part by the Startup Foundation for Introducing Talent of Nanjing University of Posts and Communications under Grant NY221041in part by the General Project of The Natural Science Foundation of Jiangsu Higher Education Institution of China 22KJB520025,23KJD580.
文摘Visible-infrared Cross-modality Person Re-identification(VI-ReID)is a critical technology in smart public facilities such as cities,campuses and libraries.It aims to match pedestrians in visible light and infrared images for video surveillance,which poses a challenge in exploring cross-modal shared information accurately and efficiently.Therefore,multi-granularity feature learning methods have been applied in VI-ReID to extract potential multi-granularity semantic information related to pedestrian body structure attributes.However,existing research mainly uses traditional dual-stream fusion networks and overlooks the core of cross-modal learning networks,the fusion module.This paper introduces a novel network called the Augmented Deep Multi-Granularity Pose-Aware Feature Fusion Network(ADMPFF-Net),incorporating the Multi-Granularity Pose-Aware Feature Fusion(MPFF)module to generate discriminative representations.MPFF efficiently explores and learns global and local features with multi-level semantic information by inserting disentangling and duplicating blocks into the fusion module of the backbone network.ADMPFF-Net also provides a new perspective for designing multi-granularity learning networks.By incorporating the multi-granularity feature disentanglement(mGFD)and posture information segmentation(pIS)strategies,it extracts more representative features concerning body structure information.The Local Information Enhancement(LIE)module augments high-performance features in VI-ReID,and the multi-granularity joint loss supervises model training for objective feature learning.Experimental results on two public datasets show that ADMPFF-Net efficiently constructs pedestrian feature representations and enhances the accuracy of VI-ReID.
基金Young Innovative Talents Project of Guangdong Ordinary Universities(No.2022KQNCX225)School-level Teaching and Research Project of Guangzhou City Polytechnic(No.2022xky046)。
文摘The semantic segmentation methods based on CNN have made great progress,but there are still some shortcomings in the application of remote sensing images segmentation,such as the small receptive field can not effectively capture global context.In order to solve this problem,this paper proposes a hybrid model based on ResNet50 and swin transformer to directly capture long-range dependence,which fuses features through Cross Feature Modulation Module(CFMM).Experimental results on two publicly available datasets,Vaihingen and Potsdam,are mIoU of 70.27%and 76.63%,respectively.Thus,CFM-UNet can maintain a high segmentation performance compared with other competitive networks.
基金the Zhejiang Provincial Natural Science Foundation of China(no.LZ21F020001)the Basic Scientific Research Program of Wenzhou(no.S20220018).
文摘Whale optimization algorithm(WOA)tends to fall into the local optimum and fails to converge quickly in solving complex problems.To address the shortcomings,an improved WOA(QGBWOA)is proposed in this work.First,quasi-opposition-based learning is introduced to enhance the ability of WOA to search for optimal solutions.Second,a Gaussian barebone mechanism is embedded to promote diversity and expand the scope of the solution space in WOA.To verify the advantages of QGBWOA,comparison experiments between QGBWOA and its comparison peers were carried out on CEC 2014 with dimensions 10,30,50,and 100 and on CEC 2020 test with dimension 30.Furthermore,the performance results were tested using Wilcoxon signed-rank(WS),Friedman test,and post hoc statistical tests for statistical analysis.Convergence accuracy and speed are remarkably improved,as shown by experimental results.Finally,feature selection and multi-threshold image segmentation applications are demonstrated to validate the ability of QGBWOA to solve complex real-world problems.QGBWOA proves its superiority over compared algorithms in feature selection and multi-threshold image segmentation by performing several evaluation metrics.
基金This work was supported in part by the National Natural Science Foundation of China under Grant 61772561,author J.Q,http://www.nsfc.gov.cn/in part by the Science Research Projects of Hunan Provincial Education Department under Grant 18A174,author X.X,http://kxjsc.gov.hnedu.cn/+5 种基金in part by the Science Research Projects of Hunan Provincial Education Department under Grant 19B584,author Y.T,http://kxjsc.gov.hnedu.cn/in part by the Natural Science Foundation of Hunan Province(No.2020JJ4140),author Y.T,http://kjt.hunan.gov.cn/in part by the Natural Science Foundation of Hunan Province(No.2020JJ4141),author X.X,http://kjt.hunan.gov.cn/in part by the Key Research and Development Plan of Hunan Province under Grant 2019SK2022,author Y.T,http://kjt.hunan.gov.cn/in part by the Key Research and Development Plan of Hunan Province under Grant CX20200730,author G.H,http://kjt.hunan.gov.cn/in part by the Graduate Science and Technology Innovation Fund Project of Central South University of Forestry and Technology under Grant CX20202038,author G.H,http://jwc.csuft.edu.cn/.
文摘Medical image segmentation is an important application field of computer vision in medical image processing.Due to the close location and high similarity of different organs in medical images,the current segmentation algorithms have problems with mis-segmentation and poor edge segmentation.To address these challenges,we propose a medical image segmentation network(AF-Net)based on attention mechanism and feature fusion,which can effectively capture global information while focusing the network on the object area.In this approach,we add dual attention blocks(DA-block)to the backbone network,which comprises parallel channels and spatial attention branches,to adaptively calibrate and weigh features.Secondly,the multi-scale feature fusion block(MFF-block)is proposed to obtain feature maps of different receptive domains and get multi-scale information with less computational consumption.Finally,to restore the locations and shapes of organs,we adopt the global feature fusion blocks(GFF-block)to fuse high-level and low-level information,which can obtain accurate pixel positioning.We evaluate our method on multiple datasets(the aorta and lungs dataset),and the experimental results achieve 94.0%in mIoU and 96.3%in DICE,showing that our approach performs better than U-Net and other state-of-art methods.
基金This work has been funded by the research project PI18/00169 from Instituto de Salud Carlos III&FEDER funds.University Rovira i.Virgili also provided funds with Project 2019PFR-B2-61.
文摘Diabetic retinopathy (DR) is a complication of diabetesmellitus thatappears in the retina. Clinitians use retina images to detect DR pathologicalsigns related to the occlusion of tiny blood vessels. Such occlusion brings adegenerative cycle between the breaking off and the new generation of thinnerand weaker blood vessels. This research aims to develop a suitable retinalvasculature segmentation method for improving retinal screening proceduresby means of computer-aided diagnosis systems. The blood vessel segmentationmethodology relies on an effective feature selection based on SequentialForward Selection, using the error rate of a decision tree classifier in theevaluation function. Subsequently, the classification process is performed bythree alternative approaches: artificial neural networks, decision trees andsupport vector machines. The proposed methodology is validated on threepublicly accessible datasets and a private one provided by Hospital Sant Joanof Reus. In all cases we obtain an average accuracy above 96% with a sensitivityof 72% in the blood vessel segmentation process. Compared with the state-ofthe-art, our approach achieves the same performance as other methods thatneed more computational power.Our method significantly reduces the numberof features used in the segmentation process from 20 to 5 dimensions. Theimplementation of the three classifiers confirmed that the five selected featureshave a good effectiveness, independently of the classification algorithm.
文摘Video data are composed of multimodal information streams including visual, auditory and textual streams, so an approach of story segmentation for news video using multimodal analysis is described in this paper. The proposed approach detects the topic-caption frames, and integrates them with silence clips detection results, as well as shot segmentation results to locate the news story boundaries. The integration of audio-visual features and text information overcomes the weakness of the approach using only image analysis techniques. On test data with 135 400 frames, when the boundaries between news stories are detected, the accuracy rate 85.8% and the recall rate 97.5% are obtained. The experimental results show the approach is valid and robust.
基金supported in part by the National Natural Science Foundation of China(Grant No.62062003)Natural Science Foundation of Ningxia(Grant No.2023AAC03293).
文摘Multimodal lung tumor medical images can provide anatomical and functional information for the same lesion.Such as Positron Emission Computed Tomography(PET),Computed Tomography(CT),and PET-CT.How to utilize the lesion anatomical and functional information effectively and improve the network segmentation performance are key questions.To solve the problem,the Saliency Feature-Guided Interactive Feature Enhancement Lung Tumor Segmentation Network(Guide-YNet)is proposed in this paper.Firstly,a double-encoder single-decoder U-Net is used as the backbone in this model,a single-coder single-decoder U-Net is used to generate the saliency guided feature using PET image and transmit it into the skip connection of the backbone,and the high sensitivity of PET images to tumors is used to guide the network to accurately locate lesions.Secondly,a Cross Scale Feature Enhancement Module(CSFEM)is designed to extract multi-scale fusion features after downsampling.Thirdly,a Cross-Layer Interactive Feature Enhancement Module(CIFEM)is designed in the encoder to enhance the spatial position information and semantic information.Finally,a Cross-Dimension Cross-Layer Feature Enhancement Module(CCFEM)is proposed in the decoder,which effectively extractsmultimodal image features through global attention and multi-dimension local attention.The proposed method is verified on the lung multimodal medical image datasets,and the results showthat theMean Intersection overUnion(MIoU),Accuracy(Acc),Dice Similarity Coefficient(Dice),Volumetric overlap error(Voe),Relative volume difference(Rvd)of the proposed method on lung lesion segmentation are 87.27%,93.08%,97.77%,95.92%,89.28%,and 88.68%,respectively.It is of great significance for computer-aided diagnosis.
文摘Automatic segmentation of liver in medical images is challenging on the aspects of accuracy, automation and robustness. A crucial stage of the liver segmentation is the selection of the image features for the segmentation. This paper presents an accurate liver segmentation algorithm. The approach starts with a texture analysis which results in an optimal set of texture features including high order statistical texture features and anatomical structural features. Then, it creates liver distribution image by classifying the original image pixelwisely using support vector machines. Lastly, it uses a group of morphological operations to locate the liver organ accurately in the image. The novelty of the approach is resided in the fact that the features are so selected that both local and global texture distributions are considered, which is important in liver organ segmentation where neighbouring tissues and organs have similar greyscale distributions. Experiment results of liver segmentation on CT images using the proposed method are presented with performance validation and discussion.
基金Sponsored by the National Natural Science Foundation of China (60772066)
文摘An efficient algorithm for facial features extractions is proposed. The facial features we segment are the two eyes, nose and mouth. The algorithm is based on an improved Gabor wavelets edge detector, morphological approach to detect the face region and facial features regions, and an improved T-shape face mask to locate the extract location of facial features. The experimental results show that the proposed method is robust against facial expression, illumination, and can be also effective if the person wearing glasses, and so on.
文摘To address the current issues of inaccurate segmentation and the limited applicability of segmentation methods for building facades in point clouds, we propose a facade segmentation algorithm based on optimal dual-scale feature descriptors. First, we select the optimal dual-scale descriptors from a range of feature descriptors. Next, we segment the facade according to the threshold value of the chosen optimal dual-scale descriptors. Finally, we use RANSAC (Random Sample Consensus) to fit the segmented surface and optimize the fitting result. Experimental results show that, compared to commonly used facade segmentation algorithms, the proposed method yields more accurate segmentation results, providing a robust data foundation for subsequent 3D model reconstruction of buildings.
基金This work was supported by the National Research Foundation of Korea(NRF)grant funded by the Korea government(MSIT)(No.2020R1F1A1074885)was supported by the Brain Korea 21 Project in 2021(No.4199990114242).
文摘Objective and quantitative assessment of skin conditions is essential for cosmeceutical studies and research on skin aging and skin regeneration.Various handcraft-based image processing methods have been proposed to evaluate skin conditions objectively,but they have unavoidable disadvantages when used to analyze skin features accurately.This study proposes a hybrid segmentation scheme consisting of Deeplab v3+with an Inception-ResNet-v2 backbone,LightGBM,and morphological processing(MP)to overcome the shortcomings of handcraft-based approaches.First,we apply Deeplab v3+with an Inception-ResNet-v2 backbone for pixel segmentation of skin wrinkles and cells.Then,LightGBM and MP are used to enhance the pixel segmentation quality.Finally,we determine several skin features based on the results of wrinkle and cell segmentation.Our proposed segmentation scheme achieved a mean accuracy of 0.854,mean of intersection over union of 0.749,and mean boundary F1 score of 0.852,which achieved 1.1%,6.7%,and 14.8%improvement over the panoptic-based semantic segmentation method,respectively.
