Weakly Supervised Semantic Segmentation(WSSS),which relies only on image-level labels,has attracted significant attention for its cost-effectiveness and scalability.Existing methods mainly enhance inter-class distinct...Weakly Supervised Semantic Segmentation(WSSS),which relies only on image-level labels,has attracted significant attention for its cost-effectiveness and scalability.Existing methods mainly enhance inter-class distinctions and employ data augmentation to mitigate semantic ambiguity and reduce spurious activations.However,they often neglect the complex contextual dependencies among image patches,resulting in incomplete local representations and limited segmentation accuracy.To address these issues,we propose the Context Patch Fusion with Class Token Enhancement(CPF-CTE)framework,which exploits contextual relations among patches to enrich feature repre-sentations and improve segmentation.At its core,the Contextual-Fusion Bidirectional Long Short-Term Memory(CF-BiLSTM)module captures spatial dependencies between patches and enables bidirectional information flow,yield-ing a more comprehensive understanding of spatial correlations.This strengthens feature learning and segmentation robustness.Moreover,we introduce learnable class tokens that dynamically encode and refine class-specific semantics,enhancing discriminative capability.By effectively integrating spatial and semantic cues,CPF-CTE produces richer and more accurate representations of image content.Extensive experiments on PASCAL VOC 2012 and MS COCO 2014 validate that CPF-CTE consistently surpasses prior WSSS methods.展开更多
Quantitative analysis of aluminum-silicon(Al-Si)alloy microstructure is crucial for evaluating and controlling alloy performance.Conventional analysis methods rely on manual segmentation,which is inefficient and subje...Quantitative analysis of aluminum-silicon(Al-Si)alloy microstructure is crucial for evaluating and controlling alloy performance.Conventional analysis methods rely on manual segmentation,which is inefficient and subjective,while fully supervised deep learning approaches require extensive and expensive pixel-level annotated data.Furthermore,existing semi-supervised methods still face challenges in handling the adhesion of adjacent primary silicon particles and effectively utilizing consistency in unlabeled data.To address these issues,this paper proposes a novel semi-supervised framework for Al-Si alloy microstructure image segmentation.First,we introduce a Rotational Uncertainty Correction Strategy(RUCS).This strategy employs multi-angle rotational perturbations andMonte Carlo sampling to assess prediction consistency,generating a pixel-wise confidence weight map.By integrating this map into the loss function,the model dynamically focuses on high-confidence regions,thereby improving generalization ability while reducing manual annotation pressure.Second,we design a Boundary EnhancementModule(BEM)to strengthen boundary feature extraction through erosion difference and multi-scale dilated convolutions.This module guides the model to focus on the boundary regions of adjacent particles,effectively resolving particle adhesion and improving segmentation accuracy.Systematic experiments were conducted on the Aluminum-Silicon Alloy Microstructure Dataset(ASAD).Results indicate that the proposed method performs exceptionally well with scarce labeled data.Specifically,using only 5%labeled data,our method improves the Jaccard index and Adjusted Rand Index(ARI)by 2.84 and 1.57 percentage points,respectively,and reduces the Variation of Information(VI)by 8.65 compared to stateof-the-art semi-supervised models,approaching the performance levels of 10%labeled data.These results demonstrate that the proposed method significantly enhances the accuracy and robustness of quantitative microstructure analysis while reducing annotation costs.展开更多
Advanced traffic monitoring systems encounter substantial challenges in vehicle detection and classification due to the limitations of conventional methods,which often demand extensive computational resources and stru...Advanced traffic monitoring systems encounter substantial challenges in vehicle detection and classification due to the limitations of conventional methods,which often demand extensive computational resources and struggle with diverse data acquisition techniques.This research presents a novel approach for vehicle classification and recognition in aerial image sequences,integrating multiple advanced techniques to enhance detection accuracy.The proposed model begins with preprocessing using Multiscale Retinex(MSR)to enhance image quality,followed by Expectation-Maximization(EM)Segmentation for precise foreground object identification.Vehicle detection is performed using the state-of-the-art YOLOv10 framework,while feature extraction incorporates Maximally Stable Extremal Regions(MSER),Dense Scale-Invariant Feature Transform(Dense SIFT),and Zernike Moments Features to capture distinct object characteristics.Feature optimization is further refined through a Hybrid Swarm-based Optimization algorithm,ensuring optimal feature selection for improved classification performance.The final classification is conducted using a Vision Transformer,leveraging its robust learning capabilities for enhanced accuracy.Experimental evaluations on benchmark datasets,including UAVDT and the Unmanned Aerial Vehicle Intruder Dataset(UAVID),demonstrate the superiority of the proposed approach,achieving an accuracy of 94.40%on UAVDT and 93.57%on UAVID.The results highlight the efficacy of the model in significantly enhancing vehicle detection and classification in aerial imagery,outperforming existing methodologies and offering a statistically validated improvement for intelligent traffic monitoring systems compared to existing approaches.展开更多
Weakly supervised semantic segmentation(WSSS)is a tricky task,which only provides category information for segmentation prediction.Thus,the key stage of WSSS is to generate the pseudo labels.For convolutional neural n...Weakly supervised semantic segmentation(WSSS)is a tricky task,which only provides category information for segmentation prediction.Thus,the key stage of WSSS is to generate the pseudo labels.For convolutional neural network(CNN)based methods,in which class activation mapping(CAM)is proposed to obtain the pseudo labels,and only concentrates on the most discriminative parts.Recently,transformer-based methods utilize attention map from the multi-headed self-attention(MHSA)module to predict pseudo labels,which usually contain obvious background noise and incoherent object area.To solve the above problems,we use the Conformer as our backbone,which is a parallel network based on convolutional neural network(CNN)and Transformer.The two branches generate pseudo labels and refine them independently,and can effectively combine the advantages of CNN and Transformer.However,the parallel structure is not close enough in the information communication.Thus,parallel structure can result in poor details about pseudo labels,and the background noise still exists.To alleviate this problem,we propose enhancing convolution CAM(ECCAM)model,which have three improved modules based on enhancing convolution,including deeper stem(DStem),convolutional feed-forward network(CFFN)and feature coupling unit with convolution(FCUConv).The ECCAM could make Conformer have tighter interaction between CNN and Transformer branches.After experimental verification,the improved modules we propose can help the network perceive more local information from images,making the final segmentation results more refined.Compared with similar architecture,our modules greatly improve the semantic segmentation performance and achieve70.2%mean intersection over union(mIoU)on the PASCAL VOC 2012 dataset.展开更多
Satellite image segmentation plays a crucial role in remote sensing,supporting applications such as environmental monitoring,land use analysis,and disaster management.However,traditional segmentation methods often rel...Satellite image segmentation plays a crucial role in remote sensing,supporting applications such as environmental monitoring,land use analysis,and disaster management.However,traditional segmentation methods often rely on large amounts of labeled data,which are costly and time-consuming to obtain,especially in largescale or dynamic environments.To address this challenge,we propose the Semi-Supervised Multi-View Picture Fuzzy Clustering(SS-MPFC)algorithm,which improves segmentation accuracy and robustness,particularly in complex and uncertain remote sensing scenarios.SS-MPFC unifies three paradigms:semi-supervised learning,multi-view clustering,and picture fuzzy set theory.This integration allows the model to effectively utilize a small number of labeled samples,fuse complementary information from multiple data views,and handle the ambiguity and uncertainty inherent in satellite imagery.We design a novel objective function that jointly incorporates picture fuzzy membership functions across multiple views of the data,and embeds pairwise semi-supervised constraints(must-link and cannot-link)directly into the clustering process to enhance segmentation accuracy.Experiments conducted on several benchmark satellite datasets demonstrate that SS-MPFC significantly outperforms existing state-of-the-art methods in segmentation accuracy,noise robustness,and semantic interpretability.On the Augsburg dataset,SS-MPFC achieves a Purity of 0.8158 and an Accuracy of 0.6860,highlighting its outstanding robustness and efficiency.These results demonstrate that SSMPFC offers a scalable and effective solution for real-world satellite-based monitoring systems,particularly in scenarios where rapid annotation is infeasible,such as wildfire tracking,agricultural monitoring,and dynamic urban mapping.展开更多
The accurate segmentation of deep gray matter nuclei is critical for neuropathological research,disease diagnosis and treatment.Existing methods employ the supervised learning training approach,which requires large la...The accurate segmentation of deep gray matter nuclei is critical for neuropathological research,disease diagnosis and treatment.Existing methods employ the supervised learning training approach,which requires large labeled datasets.It is challenging and time-consuming to obtain such datasets for medical image analysis.In addition,these methods based on convolutional neural networks(CNNs)only achieve suboptimal performance due to the locality of convolutional operations.Vision Transformers(ViTs)efficiently model long-range dependencies and thus have the potentiality to outperform these methods in segmentation tasks.To address these issues,we propose a novel hybrid network based on self-supervised pre-training for deep gray matter nuclei segmentation.Specifically,we present a CNN-Transformer hybrid network(CTNet),whose encoder consists of 3D CNN and ViT to learn local spatial-detailed features and global semantic information.A self-supervised learning(SSL)approach that integrates rotation prediction and masked feature reconstruction is proposed to pre-train the CTNet,enabling the model to learn valuable visual representations from unlabeled data.We evaluate the effectiveness of our method on 3T and 7T human brain MRI datasets.The results demonstrate that our CTNet achieves better performance than other comparison models and our pre-training strategy outperforms other advanced self-supervised methods.When the training set has only one sample,our pre-trained CTNet enhances segmentation performance,showing an 8.4%improvement in Dice similarity coefficient(DSC)compared to the randomly initialized CTNet.展开更多
To avoid the laborious annotation process for dense prediction tasks like semantic segmentation,unsupervised domain adaptation(UDA)methods have been proposed to leverage the abundant annotations from a source domain,s...To avoid the laborious annotation process for dense prediction tasks like semantic segmentation,unsupervised domain adaptation(UDA)methods have been proposed to leverage the abundant annotations from a source domain,such as virtual world(e.g.,3D games),and adapt models to the target domain(the real world)by narrowing the domain discrepancies.However,because of the large domain gap,directly aligning two distinct domains without considering the intermediates leads to inefficient alignment and inferior adaptation.To address this issue,we propose a novel learnable evolutionary Category Intermediates(CIs)guided UDA model named Leci,which enables the information transfer between the two domains via two processes,i.e.,Distilling and Blending.Starting from a random initialization,the CIs learn shared category-wise semantics automatically from two domains in the Distilling process.Then,the learned semantics in the CIs are sent back to blend the domain features through a residual attentive fusion(RAF)module,such that the categorywise features of both domains shift towards each other.As the CIs progressively and consistently learn from the varying feature distributions during training,they are evolutionary to guide the model to achieve category-wise feature alignment.Experiments on both GTA5 and SYNTHIA datasets demonstrate Leci's superiority over prior representative methods.展开更多
Existing semi-supervisedmedical image segmentation algorithms use copy-paste data augmentation to correct the labeled-unlabeled data distribution mismatch.However,current copy-paste methods have three limitations:(1)t...Existing semi-supervisedmedical image segmentation algorithms use copy-paste data augmentation to correct the labeled-unlabeled data distribution mismatch.However,current copy-paste methods have three limitations:(1)training the model solely with copy-paste mixed pictures from labeled and unlabeled input loses a lot of labeled information;(2)low-quality pseudo-labels can cause confirmation bias in pseudo-supervised learning on unlabeled data;(3)the segmentation performance in low-contrast and local regions is less than optimal.We design a Stochastic Augmentation-Based Dual-Teaching Auxiliary Training Strategy(SADT),which enhances feature diversity and learns high-quality features to overcome these problems.To be more precise,SADT trains the Student Network by using pseudo-label-based training from Teacher Network 1 and supervised learning with labeled data,which prevents the loss of rare labeled data.We introduce a bi-directional copy-pastemask with progressive high-entropy filtering to reduce data distribution disparities and mitigate confirmation bias in pseudo-supervision.For the mixed images,Deep-Shallow Spatial Contrastive Learning(DSSCL)is proposed in the feature spaces of Teacher Network 2 and the Student Network to improve the segmentation capabilities in low-contrast and local areas.In this procedure,the features retrieved by the Student Network are subjected to a random feature perturbation technique.On two openly available datasets,extensive trials show that our proposed SADT performs much better than the state-ofthe-art semi-supervised medical segmentation techniques.Using only 10%of the labeled data for training,SADT was able to acquire a Dice score of 90.10%on the ACDC(Automatic Cardiac Diagnosis Challenge)dataset.展开更多
Organoids possess immense potential for unraveling the intricate functions of human tissues and facilitating preclinical disease treatment.Their applications span from high-throughput drug screening to the modeling of...Organoids possess immense potential for unraveling the intricate functions of human tissues and facilitating preclinical disease treatment.Their applications span from high-throughput drug screening to the modeling of complex diseases,with some even achieving clinical translation.Changes in the overall size,shape,boundary,and other morphological features of organoids provide a noninvasive method for assessing organoid drug sensitivity.However,the precise segmentation of organoids in bright-field microscopy images is made difficult by the complexity of the organoid morphology and interference,including overlapping organoids,bubbles,dust particles,and cell fragments.This paper introduces the precision organoid segmentation technique(POST),which is a deep-learning algorithm for segmenting challenging organoids under simple bright-field imaging conditions.Unlike existing methods,POST accurately segments each organoid and eliminates various artifacts encountered during organoid culturing and imaging.Furthermore,it is sensitive to and aligns with measurements of organoid activity in drug sensitivity experiments.POST is expected to be a valuable tool for drug screening using organoids owing to its capability of automatically and rapidly eliminating interfering substances and thereby streamlining the organoid analysis and drug screening process.展开更多
Segmentation tasks require multiple annotation work which is time-consuming and labour-intensive.How to make full use of unlabelled data to assist in training deep learning models has been a research hotspot in recent...Segmentation tasks require multiple annotation work which is time-consuming and labour-intensive.How to make full use of unlabelled data to assist in training deep learning models has been a research hotspot in recent years.This paper takes instrument segmentation in endoscopic surgery as the background to explore how to use unlabelled data for semi-supervised learning more reasonably and effectively.An adaptive gradient correction method based on the degree of perturbation is proposed to improve segmentation accuracy.This paper integrates the recently popular segment anything model(SAM)with semi-supervised learning,taking full advantage of the large model to enhance the zero-shot ability of the model.Experimental results demonstrate the superior performance of the proposed segmentation strategy compared to traditional semi-supervised segmentation methods,achieving a 2.56% improvement in mean intersection over union(mIoU).The visual segmentation results show that incorporation of SAM significantly enhances our method,resulting in more accurate segmentation boundaries.展开更多
Medical image segmentation is a crucial preliminary step for a number of downstream diagnosis tasks.As deep convolutional neural networks successfully promote the development of computer vision,it is possible to make ...Medical image segmentation is a crucial preliminary step for a number of downstream diagnosis tasks.As deep convolutional neural networks successfully promote the development of computer vision,it is possible to make medical image segmentation a semi-automatic procedure by applying deep convolutional neural networks to finding the contours of regions of interest that are then revised by radiologists.However,supervised learning necessitates large annotated data,which are difficult to acquire especially for medical images.Self-supervised learning is able to take advantage of unlabeled data and provide good initialization to be finetuned for downstream tasks with limited annotations.Considering that most self-supervised learning especially contrastive learning methods are tailored to natural image classification and entail expensive GPU resources,we propose a novel and simple pretext-based self-supervised learning method that exploits the value of positional information in volumetric medical images.Specifically,we regard spatial coordinates as pseudo labels and pretrain the model by predicting positions of randomly sampled 2D slices in volumetric medical images.Experiments on four semantic segmentation datasets demonstrate the superiority of our method over other self-supervised learning methods in both semi-supervised learning and transfer learning settings.Codes are available at https://github.com/alienzyj/PPos.展开更多
Indoor visual localization relies heavily on image retrieval to ascertain location information.However,the widespread presence and high consistency of floor patterns across different images of-ten lead to the extracti...Indoor visual localization relies heavily on image retrieval to ascertain location information.However,the widespread presence and high consistency of floor patterns across different images of-ten lead to the extraction of numerous repetitive features,thereby reducing the accuracy of image retrieval.This article proposes an indoor visual localization method based on semantic segmentation and adaptive weight fusion to address the issue of ground texture interference with retrieval results.During the positioning process,an indoor semantic segmentation model is established.Semantic segmentation technology is applied to accurately delineate the ground portion of the images.Fea-ture extraction is performed on both the original database and the ground-segmented database.The vector of locally aggregated descriptors(VLAD)algorithm is then used to convert image features into a fixed-length feature representation,which improves the efficiency of image retrieval.Simul-taneously,a method for adaptive weight optimization in similarity calculation is proposed,using a-daptive weights to compute similarity for different regional features,thereby improving the accuracy of image retrieval.The experimental results indicate that this method significantly reduces ground interference and effectively utilizes ground information,thereby improving the accuracy of image retrieval.展开更多
Existing weakly supervised semantic segmentation(WSSS)methods based on image-level labels always rely on class activation maps(CAMs),which measure the relationships between features and classifiers.However,CAMs only f...Existing weakly supervised semantic segmentation(WSSS)methods based on image-level labels always rely on class activation maps(CAMs),which measure the relationships between features and classifiers.However,CAMs only focus on the most discriminative regions of images,resulting in their poor coverage performance.We attribute this to the deficiency in the recognition ability of a single classifier and the negative impacts caused by magnitudes during the CAMs normalisation process.To address the aforementioned issues,we propose to construct selective multiple classifiers(SMC).During the training process,we extract multiple prototypes for each class and store them in the corresponding memory bank.These prototypes are divided into foreground and background prototypes,with the former used to identify foreground objects and the latter aimed at preventing the false activation of background pixels.As for the inference stage,multiple prototypes are adaptively selected from the memory bank for each image as SMC.Subsequently,CAMs are generated by measuring the angle between SMC and features.We enhance the recognition ability of classifiers by adaptively constructing multiple classifiers for each image,while only relying on angle measurement to generate CAMs can alleviate the suppression phenomenon caused by magnitudes.Furthermore,SMC can be integrated into other WSSS approaches to help generate better CAMs.Extensive experiments conducted on standard WSSS benchmarks such as PASCAL VOC 2012 and MS COCO 2014 demonstrate the superiority of our proposed method.展开更多
Techniques in deep learning have significantly boosted the accuracy and productivity of computer vision segmentation tasks.This article offers an intriguing architecture for semantic,instance,and panoptic segmentation...Techniques in deep learning have significantly boosted the accuracy and productivity of computer vision segmentation tasks.This article offers an intriguing architecture for semantic,instance,and panoptic segmentation using EfficientNet-B7 and Bidirectional Feature Pyramid Networks(Bi-FPN).When implemented in place of the EfficientNet-B5 backbone,EfficientNet-B7 strengthens the model’s feature extraction capabilities and is far more appropriate for real-world applications.By ensuring superior multi-scale feature fusion,Bi-FPN integration enhances the segmentation of complex objects across various urban environments.The design suggested is examined on rigorous datasets,encompassing Cityscapes,Common Objects in Context,KITTI Karlsruhe Institute of Technology and Toyota Technological Institute,and Indian Driving Dataset,which replicate numerous real-world driving conditions.During extensive training,validation,and testing,the model showcases major gains in segmentation accuracy and surpasses state-of-the-art performance in semantic,instance,and panoptic segmentation tasks.Outperforming present methods,the recommended approach generates noteworthy gains in Panoptic Quality:+0.4%on Cityscapes,+0.2%on COCO,+1.7%on KITTI,and+0.4%on IDD.These changes show just how efficient it is in various driving circumstances and datasets.This study emphasizes the potential of EfficientNet-B7 and Bi-FPN to provide dependable,high-precision segmentation in computer vision applications,primarily autonomous driving.The research results suggest that this framework efficiently tackles the constraints of practical situations while delivering a robust solution for high-performance tasks involving segmentation.展开更多
In the realm of medical image segmentation,particularly in cardiac magnetic resonance imaging(MRI),achieving robust performance with limited annotated data is a significant challenge.Performance often degrades when fa...In the realm of medical image segmentation,particularly in cardiac magnetic resonance imaging(MRI),achieving robust performance with limited annotated data is a significant challenge.Performance often degrades when faced with testing scenarios from unknown domains.To address this problem,this paper proposes a novel semi-supervised approach for cardiac magnetic resonance image segmentation,aiming to enhance predictive capabilities and domain generalization(DG).This paper establishes an MT-like model utilizing pseudo-labeling and consistency regularization from semi-supervised learning,and integrates uncertainty estimation to improve the accuracy of pseudo-labels.Additionally,to tackle the challenge of domain generalization,a data manipulation strategy is introduced,extracting spatial and content-related information from images across different domains,enriching the dataset with a multi-domain perspective.This papers method is meticulously evaluated on the publicly available cardiac magnetic resonance imaging dataset M&Ms,validating its effectiveness.Comparative analyses against various methods highlight the out-standing performance of this papers approach,demonstrating its capability to segment cardiac magnetic resonance images in previously unseen domains even with limited annotated data.展开更多
The primary challenge in weakly supervised semantic segmentation is effectively leveraging weak annotations while minimizing the performance gap compared to fully supervised methods.End-to-end model designs have gaine...The primary challenge in weakly supervised semantic segmentation is effectively leveraging weak annotations while minimizing the performance gap compared to fully supervised methods.End-to-end model designs have gained significant attention for improving training efficiency.Most current algorithms rely on Convolutional Neural Networks(CNNs)for feature extraction.Although CNNs are proficient at capturing local features,they often struggle with global context,leading to incomplete and false Class Activation Mapping(CAM).To address these limitations,this work proposes a Contextual Prototype-Based End-to-End Weakly Supervised Semantic Segmentation(CPEWS)model,which improves feature extraction by utilizing the Vision Transformer(ViT).By incorporating its intermediate feature layers to preserve semantic information,this work introduces the Intermediate Supervised Module(ISM)to supervise the final layer’s output,reducing boundary ambiguity and mitigating issues related to incomplete activation.Additionally,the Contextual Prototype Module(CPM)generates class-specific prototypes,while the proposed Prototype Discrimination Loss and Superclass Suppression Loss guide the network’s training,(LPDL)(LSSL)effectively addressing false activation without the need for extra supervision.The CPEWS model proposed in this paper achieves state-of-the-art performance in end-to-end weakly supervised semantic segmentation without additional supervision.The validation set and test set Mean Intersection over Union(MIoU)of PASCAL VOC 2012 dataset achieved 69.8%and 72.6%,respectively.Compared with ToCo(pre trained weight ImageNet-1k),MIoU on the test set is 2.1%higher.In addition,MIoU reached 41.4%on the validation set of the MS COCO 2014 dataset.展开更多
Medical image segmentation is a crucial task in clinical applications.However,obtaining labeled data for medical images is often challenging.This has led to the appeal of semi-supervised learning(SSL),a technique adep...Medical image segmentation is a crucial task in clinical applications.However,obtaining labeled data for medical images is often challenging.This has led to the appeal of semi-supervised learning(SSL),a technique adept at leveraging a modest amount of labeled data.Nonetheless,most prevailing SSL segmentation methods for medical images either rely on the single consistency training method or directly fine-tune SSL methods designed for natural images.In this paper,we propose an innovative semi-supervised method called multi-consistency training(MCT)for medical image segmentation.Our approach transcends the constraints of prior methodologies by considering consistency from a dual perspective:output consistency across different up-sampling methods and output consistency of the same data within the same network under various perturbations to the intermediate features.We design distinct semi-supervised loss regression methods for these two types of consistencies.To enhance the application of our MCT model,we also develop a dedicated decoder as the core of our neural network.Thorough experiments were conducted on the polyp dataset and the dental dataset,rigorously compared against other SSL methods.Experimental results demonstrate the superiority of our approach,achieving higher segmentation accuracy.Moreover,comprehensive ablation studies and insightful discussion substantiate the efficacy of our approach in navigating the intricacies of medical image segmentation.展开更多
Due to the inability of manufacturing a single monolithic mirror at the 10-meter scales,segmented mirrors have become indispensable tools in modern astronomical research.However,to match the imaging performance of the...Due to the inability of manufacturing a single monolithic mirror at the 10-meter scales,segmented mirrors have become indispensable tools in modern astronomical research.However,to match the imaging performance of the monolithic counterpart,the sub-mirrors must maintain precise co-phasing.Piston error critically degrades segmented mirror imaging quality,necessitating efficient and precise detection.To ad-dress the limitations that the conventional circular-aperture diffraction with two-wavelength algorithm is sus-ceptible to decentration errors,and the traditional convolutional neural networks(CNNs)struggle to capture global features under large-range piston errors due to their restricted local receptive fields,this paper pro-poses a method that integrates extended Young’s interference principles with a Vision Transformer(ViT)to detect piston error.By suppressing decentration error interference through two symmetrically arranged aper-tures and extending the measurement range to±7.95μm via a two-wavelength(589 nm/600 nm)algorithm.This approach exploits ViT’s self-attention mechanism to model global characteristics of interference fringes.Unlike CNNs constrained by local convolutional kernels,the ViT significantly improves sensitivity to inter-ferogram periodicity.The simulation results demonstrate that the proposed method achieves a measurement accuracy of 5 nm(0.0083λ0)across the range of±7.95μm,while maintaining an accuracy exceeding 95%in the presence of Gaussian noise(SNR≥15 dB),Poisson noise(λ≥9 photons/pixel),and sub-mirror gap er-ror(Egap≤0.2)interference.Moreover,the detection speed shows significant improvement compared to the cross-correlation algorithm.This study establishes an accurate,robust framework for segmented mirror error detection,advancing high-precision astronomical observation.展开更多
Detailed individual tree crown segmentation is highly relevant for the detection and monitoring of Fraxinus excelsior L.trees affected by ash dieback,a major threat to common ash populations across Europe.In this stud...Detailed individual tree crown segmentation is highly relevant for the detection and monitoring of Fraxinus excelsior L.trees affected by ash dieback,a major threat to common ash populations across Europe.In this study,both fine and coarse crown segmentation methods were applied to close-range multispectral UAV imagery.The fine tree crown segmentation method utilized a novel unsupervised machine learning approach based on a blended NIR-NDVI image,whereas the coarse segmentation relied on the segment anything model(SAM).Both methods successfully delineated tree crown outlines,however,only the fine segmentation accurately captured internal canopy gaps.Despite these structural differences,mean NDVI values calculated per tree crown revealed no significant differences between the two approaches,indicating that coarse segmentation is sufficient for mean vegetation index assessments.Nevertheless,the fine segmentation revealed increased heterogeneity in NDVI values in more severely damaged trees,underscoring its value for detailed structural and health analyses.Furthermore,the fine segmentation workflow proved transferable to both individual UAV images and orthophotos from broader UAV surveys.For applications focused on structural integrity and spatial variation in canopy health,the fine segmentation approach is recommended.展开更多
AIM:To construct an intelligent segmentation scheme for precise localization of central serous chorioretinopathy(CSC)leakage points,thereby enabling ophthalmologists to deliver accurate laser treatment without navigat...AIM:To construct an intelligent segmentation scheme for precise localization of central serous chorioretinopathy(CSC)leakage points,thereby enabling ophthalmologists to deliver accurate laser treatment without navigational laser equipment.METHODS:A dataset with dual labels(point-level and pixel-level)was first established based on fundus fluorescein angiography(FFA)images of CSC and subsequently divided into training(102 images),validation(40 images),and test(40 images)datasets.An intelligent segmentation method was then developed,based on the You Only Look Once version 8 Pose Estimation(YOLOv8-Pose)model and segment anything model(SAM),to segment CSC leakage points.Next,the YOLOv8-Pose model was trained for 200 epochs,and the best-performing model was selected to form the optimal combination with SAM.Additionally,the classic five types of U-Net series models[i.e.,U-Net,recurrent residual U-Net(R2U-Net),attention U-Net(AttU-Net),recurrent residual attention U-Net(R2AttUNet),and nested U-Net(UNet^(++))]were initialized with three random seeds and trained for 200 epochs,resulting in a total of 15 baseline models for comparison.Finally,based on the metrics including Dice similarity coefficient(DICE),intersection over union(IoU),precision,recall,precisionrecall(PR)curve,and receiver operating characteristic(ROC)curve,the proposed method was compared with baseline models through quantitative and qualitative experiments for leakage point segmentation,thereby demonstrating its effectiveness.RESULTS:With the increase of training epochs,the mAP50-95,Recall,and precision of the YOLOv8-Pose model showed a significant increase and tended to stabilize,and it achieved a preliminary localization success rate of 90%(i.e.,36 images)for CSC leakage points in 40 test images.Using manually expert-annotated pixel-level labels as the ground truth,the proposed method achieved outcomes with a DICE of 57.13%,an IoU of 45.31%,a precision of 45.91%,a recall of 93.57%,an area under the PR curve(AUC-PR)of 0.78 and an area under the ROC curve(AUC-ROC)of 0.97,which enables more accurate segmentation of CSC leakage points.CONCLUSION:By combining the precise localization capability of the YOLOv8-Pose model with the robust and flexible segmentation ability of SAM,the proposed method not only demonstrates the effectiveness of the YOLOv8-Pose model in detecting keypoint coordinates of CSC leakage points from the perspective of application innovation but also establishes a novel approach for accurate segmentation of CSC leakage points through the“detect-then-segment”strategy,thereby providing a potential auxiliary means for the automatic and precise realtime localization of leakage points during traditional laser photocoagulation for CSC.展开更多
文摘Weakly Supervised Semantic Segmentation(WSSS),which relies only on image-level labels,has attracted significant attention for its cost-effectiveness and scalability.Existing methods mainly enhance inter-class distinctions and employ data augmentation to mitigate semantic ambiguity and reduce spurious activations.However,they often neglect the complex contextual dependencies among image patches,resulting in incomplete local representations and limited segmentation accuracy.To address these issues,we propose the Context Patch Fusion with Class Token Enhancement(CPF-CTE)framework,which exploits contextual relations among patches to enrich feature repre-sentations and improve segmentation.At its core,the Contextual-Fusion Bidirectional Long Short-Term Memory(CF-BiLSTM)module captures spatial dependencies between patches and enables bidirectional information flow,yield-ing a more comprehensive understanding of spatial correlations.This strengthens feature learning and segmentation robustness.Moreover,we introduce learnable class tokens that dynamically encode and refine class-specific semantics,enhancing discriminative capability.By effectively integrating spatial and semantic cues,CPF-CTE produces richer and more accurate representations of image content.Extensive experiments on PASCAL VOC 2012 and MS COCO 2014 validate that CPF-CTE consistently surpasses prior WSSS methods.
基金funded by the National Natural Science Foundation of China (52061020).
文摘Quantitative analysis of aluminum-silicon(Al-Si)alloy microstructure is crucial for evaluating and controlling alloy performance.Conventional analysis methods rely on manual segmentation,which is inefficient and subjective,while fully supervised deep learning approaches require extensive and expensive pixel-level annotated data.Furthermore,existing semi-supervised methods still face challenges in handling the adhesion of adjacent primary silicon particles and effectively utilizing consistency in unlabeled data.To address these issues,this paper proposes a novel semi-supervised framework for Al-Si alloy microstructure image segmentation.First,we introduce a Rotational Uncertainty Correction Strategy(RUCS).This strategy employs multi-angle rotational perturbations andMonte Carlo sampling to assess prediction consistency,generating a pixel-wise confidence weight map.By integrating this map into the loss function,the model dynamically focuses on high-confidence regions,thereby improving generalization ability while reducing manual annotation pressure.Second,we design a Boundary EnhancementModule(BEM)to strengthen boundary feature extraction through erosion difference and multi-scale dilated convolutions.This module guides the model to focus on the boundary regions of adjacent particles,effectively resolving particle adhesion and improving segmentation accuracy.Systematic experiments were conducted on the Aluminum-Silicon Alloy Microstructure Dataset(ASAD).Results indicate that the proposed method performs exceptionally well with scarce labeled data.Specifically,using only 5%labeled data,our method improves the Jaccard index and Adjusted Rand Index(ARI)by 2.84 and 1.57 percentage points,respectively,and reduces the Variation of Information(VI)by 8.65 compared to stateof-the-art semi-supervised models,approaching the performance levels of 10%labeled data.These results demonstrate that the proposed method significantly enhances the accuracy and robustness of quantitative microstructure analysis while reducing annotation costs.
文摘Advanced traffic monitoring systems encounter substantial challenges in vehicle detection and classification due to the limitations of conventional methods,which often demand extensive computational resources and struggle with diverse data acquisition techniques.This research presents a novel approach for vehicle classification and recognition in aerial image sequences,integrating multiple advanced techniques to enhance detection accuracy.The proposed model begins with preprocessing using Multiscale Retinex(MSR)to enhance image quality,followed by Expectation-Maximization(EM)Segmentation for precise foreground object identification.Vehicle detection is performed using the state-of-the-art YOLOv10 framework,while feature extraction incorporates Maximally Stable Extremal Regions(MSER),Dense Scale-Invariant Feature Transform(Dense SIFT),and Zernike Moments Features to capture distinct object characteristics.Feature optimization is further refined through a Hybrid Swarm-based Optimization algorithm,ensuring optimal feature selection for improved classification performance.The final classification is conducted using a Vision Transformer,leveraging its robust learning capabilities for enhanced accuracy.Experimental evaluations on benchmark datasets,including UAVDT and the Unmanned Aerial Vehicle Intruder Dataset(UAVID),demonstrate the superiority of the proposed approach,achieving an accuracy of 94.40%on UAVDT and 93.57%on UAVID.The results highlight the efficacy of the model in significantly enhancing vehicle detection and classification in aerial imagery,outperforming existing methodologies and offering a statistically validated improvement for intelligent traffic monitoring systems compared to existing approaches.
文摘Weakly supervised semantic segmentation(WSSS)is a tricky task,which only provides category information for segmentation prediction.Thus,the key stage of WSSS is to generate the pseudo labels.For convolutional neural network(CNN)based methods,in which class activation mapping(CAM)is proposed to obtain the pseudo labels,and only concentrates on the most discriminative parts.Recently,transformer-based methods utilize attention map from the multi-headed self-attention(MHSA)module to predict pseudo labels,which usually contain obvious background noise and incoherent object area.To solve the above problems,we use the Conformer as our backbone,which is a parallel network based on convolutional neural network(CNN)and Transformer.The two branches generate pseudo labels and refine them independently,and can effectively combine the advantages of CNN and Transformer.However,the parallel structure is not close enough in the information communication.Thus,parallel structure can result in poor details about pseudo labels,and the background noise still exists.To alleviate this problem,we propose enhancing convolution CAM(ECCAM)model,which have three improved modules based on enhancing convolution,including deeper stem(DStem),convolutional feed-forward network(CFFN)and feature coupling unit with convolution(FCUConv).The ECCAM could make Conformer have tighter interaction between CNN and Transformer branches.After experimental verification,the improved modules we propose can help the network perceive more local information from images,making the final segmentation results more refined.Compared with similar architecture,our modules greatly improve the semantic segmentation performance and achieve70.2%mean intersection over union(mIoU)on the PASCAL VOC 2012 dataset.
基金funded by the Research Project:THTETN.05/24-25,VietnamAcademy of Science and Technology.
文摘Satellite image segmentation plays a crucial role in remote sensing,supporting applications such as environmental monitoring,land use analysis,and disaster management.However,traditional segmentation methods often rely on large amounts of labeled data,which are costly and time-consuming to obtain,especially in largescale or dynamic environments.To address this challenge,we propose the Semi-Supervised Multi-View Picture Fuzzy Clustering(SS-MPFC)algorithm,which improves segmentation accuracy and robustness,particularly in complex and uncertain remote sensing scenarios.SS-MPFC unifies three paradigms:semi-supervised learning,multi-view clustering,and picture fuzzy set theory.This integration allows the model to effectively utilize a small number of labeled samples,fuse complementary information from multiple data views,and handle the ambiguity and uncertainty inherent in satellite imagery.We design a novel objective function that jointly incorporates picture fuzzy membership functions across multiple views of the data,and embeds pairwise semi-supervised constraints(must-link and cannot-link)directly into the clustering process to enhance segmentation accuracy.Experiments conducted on several benchmark satellite datasets demonstrate that SS-MPFC significantly outperforms existing state-of-the-art methods in segmentation accuracy,noise robustness,and semantic interpretability.On the Augsburg dataset,SS-MPFC achieves a Purity of 0.8158 and an Accuracy of 0.6860,highlighting its outstanding robustness and efficiency.These results demonstrate that SSMPFC offers a scalable and effective solution for real-world satellite-based monitoring systems,particularly in scenarios where rapid annotation is infeasible,such as wildfire tracking,agricultural monitoring,and dynamic urban mapping.
基金supported in part by the National Natural Science Foundation of China under Grant 62071405the National Natural Science Foundation of China under Grant 12175189.
文摘The accurate segmentation of deep gray matter nuclei is critical for neuropathological research,disease diagnosis and treatment.Existing methods employ the supervised learning training approach,which requires large labeled datasets.It is challenging and time-consuming to obtain such datasets for medical image analysis.In addition,these methods based on convolutional neural networks(CNNs)only achieve suboptimal performance due to the locality of convolutional operations.Vision Transformers(ViTs)efficiently model long-range dependencies and thus have the potentiality to outperform these methods in segmentation tasks.To address these issues,we propose a novel hybrid network based on self-supervised pre-training for deep gray matter nuclei segmentation.Specifically,we present a CNN-Transformer hybrid network(CTNet),whose encoder consists of 3D CNN and ViT to learn local spatial-detailed features and global semantic information.A self-supervised learning(SSL)approach that integrates rotation prediction and masked feature reconstruction is proposed to pre-train the CTNet,enabling the model to learn valuable visual representations from unlabeled data.We evaluate the effectiveness of our method on 3T and 7T human brain MRI datasets.The results demonstrate that our CTNet achieves better performance than other comparison models and our pre-training strategy outperforms other advanced self-supervised methods.When the training set has only one sample,our pre-trained CTNet enhances segmentation performance,showing an 8.4%improvement in Dice similarity coefficient(DSC)compared to the randomly initialized CTNet.
基金Australian Research Council Project(FL-170100117).
文摘To avoid the laborious annotation process for dense prediction tasks like semantic segmentation,unsupervised domain adaptation(UDA)methods have been proposed to leverage the abundant annotations from a source domain,such as virtual world(e.g.,3D games),and adapt models to the target domain(the real world)by narrowing the domain discrepancies.However,because of the large domain gap,directly aligning two distinct domains without considering the intermediates leads to inefficient alignment and inferior adaptation.To address this issue,we propose a novel learnable evolutionary Category Intermediates(CIs)guided UDA model named Leci,which enables the information transfer between the two domains via two processes,i.e.,Distilling and Blending.Starting from a random initialization,the CIs learn shared category-wise semantics automatically from two domains in the Distilling process.Then,the learned semantics in the CIs are sent back to blend the domain features through a residual attentive fusion(RAF)module,such that the categorywise features of both domains shift towards each other.As the CIs progressively and consistently learn from the varying feature distributions during training,they are evolutionary to guide the model to achieve category-wise feature alignment.Experiments on both GTA5 and SYNTHIA datasets demonstrate Leci's superiority over prior representative methods.
基金supported by the Natural Science Foundation of China(No.41804112,author:Chengyun Song).
文摘Existing semi-supervisedmedical image segmentation algorithms use copy-paste data augmentation to correct the labeled-unlabeled data distribution mismatch.However,current copy-paste methods have three limitations:(1)training the model solely with copy-paste mixed pictures from labeled and unlabeled input loses a lot of labeled information;(2)low-quality pseudo-labels can cause confirmation bias in pseudo-supervised learning on unlabeled data;(3)the segmentation performance in low-contrast and local regions is less than optimal.We design a Stochastic Augmentation-Based Dual-Teaching Auxiliary Training Strategy(SADT),which enhances feature diversity and learns high-quality features to overcome these problems.To be more precise,SADT trains the Student Network by using pseudo-label-based training from Teacher Network 1 and supervised learning with labeled data,which prevents the loss of rare labeled data.We introduce a bi-directional copy-pastemask with progressive high-entropy filtering to reduce data distribution disparities and mitigate confirmation bias in pseudo-supervision.For the mixed images,Deep-Shallow Spatial Contrastive Learning(DSSCL)is proposed in the feature spaces of Teacher Network 2 and the Student Network to improve the segmentation capabilities in low-contrast and local areas.In this procedure,the features retrieved by the Student Network are subjected to a random feature perturbation technique.On two openly available datasets,extensive trials show that our proposed SADT performs much better than the state-ofthe-art semi-supervised medical segmentation techniques.Using only 10%of the labeled data for training,SADT was able to acquire a Dice score of 90.10%on the ACDC(Automatic Cardiac Diagnosis Challenge)dataset.
基金supported by the National Key R&D Program of China(No.2022YFC2504403)the National Natural Science Foundation of China(No.62172202)+1 种基金the Experiment Project of China Manned Space Program(No.HYZHXM01019)the Fundamental Research Funds for the Central Universities from Southeast University(No.3207032101C3)。
文摘Organoids possess immense potential for unraveling the intricate functions of human tissues and facilitating preclinical disease treatment.Their applications span from high-throughput drug screening to the modeling of complex diseases,with some even achieving clinical translation.Changes in the overall size,shape,boundary,and other morphological features of organoids provide a noninvasive method for assessing organoid drug sensitivity.However,the precise segmentation of organoids in bright-field microscopy images is made difficult by the complexity of the organoid morphology and interference,including overlapping organoids,bubbles,dust particles,and cell fragments.This paper introduces the precision organoid segmentation technique(POST),which is a deep-learning algorithm for segmenting challenging organoids under simple bright-field imaging conditions.Unlike existing methods,POST accurately segments each organoid and eliminates various artifacts encountered during organoid culturing and imaging.Furthermore,it is sensitive to and aligns with measurements of organoid activity in drug sensitivity experiments.POST is expected to be a valuable tool for drug screening using organoids owing to its capability of automatically and rapidly eliminating interfering substances and thereby streamlining the organoid analysis and drug screening process.
基金supported by the National Key R and D Program of China(Grant No.2023YFB4706300).
文摘Segmentation tasks require multiple annotation work which is time-consuming and labour-intensive.How to make full use of unlabelled data to assist in training deep learning models has been a research hotspot in recent years.This paper takes instrument segmentation in endoscopic surgery as the background to explore how to use unlabelled data for semi-supervised learning more reasonably and effectively.An adaptive gradient correction method based on the degree of perturbation is proposed to improve segmentation accuracy.This paper integrates the recently popular segment anything model(SAM)with semi-supervised learning,taking full advantage of the large model to enhance the zero-shot ability of the model.Experimental results demonstrate the superior performance of the proposed segmentation strategy compared to traditional semi-supervised segmentation methods,achieving a 2.56% improvement in mean intersection over union(mIoU).The visual segmentation results show that incorporation of SAM significantly enhances our method,resulting in more accurate segmentation boundaries.
基金the Major Research Plan of the National Natural Science Foundation of China(No.92059206)。
文摘Medical image segmentation is a crucial preliminary step for a number of downstream diagnosis tasks.As deep convolutional neural networks successfully promote the development of computer vision,it is possible to make medical image segmentation a semi-automatic procedure by applying deep convolutional neural networks to finding the contours of regions of interest that are then revised by radiologists.However,supervised learning necessitates large annotated data,which are difficult to acquire especially for medical images.Self-supervised learning is able to take advantage of unlabeled data and provide good initialization to be finetuned for downstream tasks with limited annotations.Considering that most self-supervised learning especially contrastive learning methods are tailored to natural image classification and entail expensive GPU resources,we propose a novel and simple pretext-based self-supervised learning method that exploits the value of positional information in volumetric medical images.Specifically,we regard spatial coordinates as pseudo labels and pretrain the model by predicting positions of randomly sampled 2D slices in volumetric medical images.Experiments on four semantic segmentation datasets demonstrate the superiority of our method over other self-supervised learning methods in both semi-supervised learning and transfer learning settings.Codes are available at https://github.com/alienzyj/PPos.
基金Supported by the National Natural Science Foundation of China(No.61971162,61771186)the Natural Science Foundation of Heilongjiang Province(No.PL2024F025)+2 种基金the Open Research Fund of National Mobile Communications Research Laboratory Southeast University(No.2023D07)the Outstanding Youth Program of Natural Science Foundation of Heilongjiang Province(No.YQ2020F012)the Funda-mental Scientific Research Funds of Heilongjiang Province(No.2022-KYYWF-1050).
文摘Indoor visual localization relies heavily on image retrieval to ascertain location information.However,the widespread presence and high consistency of floor patterns across different images of-ten lead to the extraction of numerous repetitive features,thereby reducing the accuracy of image retrieval.This article proposes an indoor visual localization method based on semantic segmentation and adaptive weight fusion to address the issue of ground texture interference with retrieval results.During the positioning process,an indoor semantic segmentation model is established.Semantic segmentation technology is applied to accurately delineate the ground portion of the images.Fea-ture extraction is performed on both the original database and the ground-segmented database.The vector of locally aggregated descriptors(VLAD)algorithm is then used to convert image features into a fixed-length feature representation,which improves the efficiency of image retrieval.Simul-taneously,a method for adaptive weight optimization in similarity calculation is proposed,using a-daptive weights to compute similarity for different regional features,thereby improving the accuracy of image retrieval.The experimental results indicate that this method significantly reduces ground interference and effectively utilizes ground information,thereby improving the accuracy of image retrieval.
基金supported by the National Natural Science Foundation of China(Grants 62176097,61433007)Fundamental Research Funds for the Central Universities(Grant 2019kfyXKJC024)the 111 Project on Computational Intelligence and Intelligent Control(Grant B18024).
文摘Existing weakly supervised semantic segmentation(WSSS)methods based on image-level labels always rely on class activation maps(CAMs),which measure the relationships between features and classifiers.However,CAMs only focus on the most discriminative regions of images,resulting in their poor coverage performance.We attribute this to the deficiency in the recognition ability of a single classifier and the negative impacts caused by magnitudes during the CAMs normalisation process.To address the aforementioned issues,we propose to construct selective multiple classifiers(SMC).During the training process,we extract multiple prototypes for each class and store them in the corresponding memory bank.These prototypes are divided into foreground and background prototypes,with the former used to identify foreground objects and the latter aimed at preventing the false activation of background pixels.As for the inference stage,multiple prototypes are adaptively selected from the memory bank for each image as SMC.Subsequently,CAMs are generated by measuring the angle between SMC and features.We enhance the recognition ability of classifiers by adaptively constructing multiple classifiers for each image,while only relying on angle measurement to generate CAMs can alleviate the suppression phenomenon caused by magnitudes.Furthermore,SMC can be integrated into other WSSS approaches to help generate better CAMs.Extensive experiments conducted on standard WSSS benchmarks such as PASCAL VOC 2012 and MS COCO 2014 demonstrate the superiority of our proposed method.
文摘Techniques in deep learning have significantly boosted the accuracy and productivity of computer vision segmentation tasks.This article offers an intriguing architecture for semantic,instance,and panoptic segmentation using EfficientNet-B7 and Bidirectional Feature Pyramid Networks(Bi-FPN).When implemented in place of the EfficientNet-B5 backbone,EfficientNet-B7 strengthens the model’s feature extraction capabilities and is far more appropriate for real-world applications.By ensuring superior multi-scale feature fusion,Bi-FPN integration enhances the segmentation of complex objects across various urban environments.The design suggested is examined on rigorous datasets,encompassing Cityscapes,Common Objects in Context,KITTI Karlsruhe Institute of Technology and Toyota Technological Institute,and Indian Driving Dataset,which replicate numerous real-world driving conditions.During extensive training,validation,and testing,the model showcases major gains in segmentation accuracy and surpasses state-of-the-art performance in semantic,instance,and panoptic segmentation tasks.Outperforming present methods,the recommended approach generates noteworthy gains in Panoptic Quality:+0.4%on Cityscapes,+0.2%on COCO,+1.7%on KITTI,and+0.4%on IDD.These changes show just how efficient it is in various driving circumstances and datasets.This study emphasizes the potential of EfficientNet-B7 and Bi-FPN to provide dependable,high-precision segmentation in computer vision applications,primarily autonomous driving.The research results suggest that this framework efficiently tackles the constraints of practical situations while delivering a robust solution for high-performance tasks involving segmentation.
基金Supported by the National Natural Science Foundation of China(No.62001313)the Key Project of Liaoning Provincial Department of Science and Technology(No.2021JH2/10300134,2022JH1/10500004)。
文摘In the realm of medical image segmentation,particularly in cardiac magnetic resonance imaging(MRI),achieving robust performance with limited annotated data is a significant challenge.Performance often degrades when faced with testing scenarios from unknown domains.To address this problem,this paper proposes a novel semi-supervised approach for cardiac magnetic resonance image segmentation,aiming to enhance predictive capabilities and domain generalization(DG).This paper establishes an MT-like model utilizing pseudo-labeling and consistency regularization from semi-supervised learning,and integrates uncertainty estimation to improve the accuracy of pseudo-labels.Additionally,to tackle the challenge of domain generalization,a data manipulation strategy is introduced,extracting spatial and content-related information from images across different domains,enriching the dataset with a multi-domain perspective.This papers method is meticulously evaluated on the publicly available cardiac magnetic resonance imaging dataset M&Ms,validating its effectiveness.Comparative analyses against various methods highlight the out-standing performance of this papers approach,demonstrating its capability to segment cardiac magnetic resonance images in previously unseen domains even with limited annotated data.
基金funding from the following sources:National Natural Science Foundation of China(U1904119)Research Programs of Henan Science and Technology Department(232102210054)+3 种基金Chongqing Natural Science Foundation(CSTB2023NSCQ-MSX0070)Henan Province Key Research and Development Project(231111212000)Aviation Science Foundation(20230001055002)supported by Henan Center for Outstanding Overseas Scientists(GZS2022011).
文摘The primary challenge in weakly supervised semantic segmentation is effectively leveraging weak annotations while minimizing the performance gap compared to fully supervised methods.End-to-end model designs have gained significant attention for improving training efficiency.Most current algorithms rely on Convolutional Neural Networks(CNNs)for feature extraction.Although CNNs are proficient at capturing local features,they often struggle with global context,leading to incomplete and false Class Activation Mapping(CAM).To address these limitations,this work proposes a Contextual Prototype-Based End-to-End Weakly Supervised Semantic Segmentation(CPEWS)model,which improves feature extraction by utilizing the Vision Transformer(ViT).By incorporating its intermediate feature layers to preserve semantic information,this work introduces the Intermediate Supervised Module(ISM)to supervise the final layer’s output,reducing boundary ambiguity and mitigating issues related to incomplete activation.Additionally,the Contextual Prototype Module(CPM)generates class-specific prototypes,while the proposed Prototype Discrimination Loss and Superclass Suppression Loss guide the network’s training,(LPDL)(LSSL)effectively addressing false activation without the need for extra supervision.The CPEWS model proposed in this paper achieves state-of-the-art performance in end-to-end weakly supervised semantic segmentation without additional supervision.The validation set and test set Mean Intersection over Union(MIoU)of PASCAL VOC 2012 dataset achieved 69.8%and 72.6%,respectively.Compared with ToCo(pre trained weight ImageNet-1k),MIoU on the test set is 2.1%higher.In addition,MIoU reached 41.4%on the validation set of the MS COCO 2014 dataset.
基金the Innovation Program of Shanghai Industrial Synergy(No.XTCX-KJ-2023-2-12)。
文摘Medical image segmentation is a crucial task in clinical applications.However,obtaining labeled data for medical images is often challenging.This has led to the appeal of semi-supervised learning(SSL),a technique adept at leveraging a modest amount of labeled data.Nonetheless,most prevailing SSL segmentation methods for medical images either rely on the single consistency training method or directly fine-tune SSL methods designed for natural images.In this paper,we propose an innovative semi-supervised method called multi-consistency training(MCT)for medical image segmentation.Our approach transcends the constraints of prior methodologies by considering consistency from a dual perspective:output consistency across different up-sampling methods and output consistency of the same data within the same network under various perturbations to the intermediate features.We design distinct semi-supervised loss regression methods for these two types of consistencies.To enhance the application of our MCT model,we also develop a dedicated decoder as the core of our neural network.Thorough experiments were conducted on the polyp dataset and the dental dataset,rigorously compared against other SSL methods.Experimental results demonstrate the superiority of our approach,achieving higher segmentation accuracy.Moreover,comprehensive ablation studies and insightful discussion substantiate the efficacy of our approach in navigating the intricacies of medical image segmentation.
文摘Due to the inability of manufacturing a single monolithic mirror at the 10-meter scales,segmented mirrors have become indispensable tools in modern astronomical research.However,to match the imaging performance of the monolithic counterpart,the sub-mirrors must maintain precise co-phasing.Piston error critically degrades segmented mirror imaging quality,necessitating efficient and precise detection.To ad-dress the limitations that the conventional circular-aperture diffraction with two-wavelength algorithm is sus-ceptible to decentration errors,and the traditional convolutional neural networks(CNNs)struggle to capture global features under large-range piston errors due to their restricted local receptive fields,this paper pro-poses a method that integrates extended Young’s interference principles with a Vision Transformer(ViT)to detect piston error.By suppressing decentration error interference through two symmetrically arranged aper-tures and extending the measurement range to±7.95μm via a two-wavelength(589 nm/600 nm)algorithm.This approach exploits ViT’s self-attention mechanism to model global characteristics of interference fringes.Unlike CNNs constrained by local convolutional kernels,the ViT significantly improves sensitivity to inter-ferogram periodicity.The simulation results demonstrate that the proposed method achieves a measurement accuracy of 5 nm(0.0083λ0)across the range of±7.95μm,while maintaining an accuracy exceeding 95%in the presence of Gaussian noise(SNR≥15 dB),Poisson noise(λ≥9 photons/pixel),and sub-mirror gap er-ror(Egap≤0.2)interference.Moreover,the detection speed shows significant improvement compared to the cross-correlation algorithm.This study establishes an accurate,robust framework for segmented mirror error detection,advancing high-precision astronomical observation.
基金This study was conducted within the project FraxVir“Detection,characterisation and analyses of the occurrence of viruses and ash dieback in special stands of Fraxinus excelsior-a supplementary study to the FraxForFuture demonstration project”and receives funding via the Waldklimafonds(WKF)funded by the German Federal Ministry of Food and Agriculture(BMEL)and Federal Ministry for the Environment,Nature Conservation,Nuclear Safety and Consumer Protection(BMUV)administrated by the Agency for Renewable Resources(FNR)under grant agreement 2220WK40A4.
文摘Detailed individual tree crown segmentation is highly relevant for the detection and monitoring of Fraxinus excelsior L.trees affected by ash dieback,a major threat to common ash populations across Europe.In this study,both fine and coarse crown segmentation methods were applied to close-range multispectral UAV imagery.The fine tree crown segmentation method utilized a novel unsupervised machine learning approach based on a blended NIR-NDVI image,whereas the coarse segmentation relied on the segment anything model(SAM).Both methods successfully delineated tree crown outlines,however,only the fine segmentation accurately captured internal canopy gaps.Despite these structural differences,mean NDVI values calculated per tree crown revealed no significant differences between the two approaches,indicating that coarse segmentation is sufficient for mean vegetation index assessments.Nevertheless,the fine segmentation revealed increased heterogeneity in NDVI values in more severely damaged trees,underscoring its value for detailed structural and health analyses.Furthermore,the fine segmentation workflow proved transferable to both individual UAV images and orthophotos from broader UAV surveys.For applications focused on structural integrity and spatial variation in canopy health,the fine segmentation approach is recommended.
基金Supported by the Shenzhen Science and Technology Program(No.JCYJ20240813152704006)the National Natural Science Foundation of China(No.62401259)+2 种基金the Fundamental Research Funds for the Central Universities(No.NZ2024036)the Postdoctoral Fellowship Program of CPSF(No.GZC20242228)High Performance Computing Platform of Nanjing University of Aeronautics and Astronautics。
文摘AIM:To construct an intelligent segmentation scheme for precise localization of central serous chorioretinopathy(CSC)leakage points,thereby enabling ophthalmologists to deliver accurate laser treatment without navigational laser equipment.METHODS:A dataset with dual labels(point-level and pixel-level)was first established based on fundus fluorescein angiography(FFA)images of CSC and subsequently divided into training(102 images),validation(40 images),and test(40 images)datasets.An intelligent segmentation method was then developed,based on the You Only Look Once version 8 Pose Estimation(YOLOv8-Pose)model and segment anything model(SAM),to segment CSC leakage points.Next,the YOLOv8-Pose model was trained for 200 epochs,and the best-performing model was selected to form the optimal combination with SAM.Additionally,the classic five types of U-Net series models[i.e.,U-Net,recurrent residual U-Net(R2U-Net),attention U-Net(AttU-Net),recurrent residual attention U-Net(R2AttUNet),and nested U-Net(UNet^(++))]were initialized with three random seeds and trained for 200 epochs,resulting in a total of 15 baseline models for comparison.Finally,based on the metrics including Dice similarity coefficient(DICE),intersection over union(IoU),precision,recall,precisionrecall(PR)curve,and receiver operating characteristic(ROC)curve,the proposed method was compared with baseline models through quantitative and qualitative experiments for leakage point segmentation,thereby demonstrating its effectiveness.RESULTS:With the increase of training epochs,the mAP50-95,Recall,and precision of the YOLOv8-Pose model showed a significant increase and tended to stabilize,and it achieved a preliminary localization success rate of 90%(i.e.,36 images)for CSC leakage points in 40 test images.Using manually expert-annotated pixel-level labels as the ground truth,the proposed method achieved outcomes with a DICE of 57.13%,an IoU of 45.31%,a precision of 45.91%,a recall of 93.57%,an area under the PR curve(AUC-PR)of 0.78 and an area under the ROC curve(AUC-ROC)of 0.97,which enables more accurate segmentation of CSC leakage points.CONCLUSION:By combining the precise localization capability of the YOLOv8-Pose model with the robust and flexible segmentation ability of SAM,the proposed method not only demonstrates the effectiveness of the YOLOv8-Pose model in detecting keypoint coordinates of CSC leakage points from the perspective of application innovation but also establishes a novel approach for accurate segmentation of CSC leakage points through the“detect-then-segment”strategy,thereby providing a potential auxiliary means for the automatic and precise realtime localization of leakage points during traditional laser photocoagulation for CSC.
