Text semantic extraction has been envisioned as a promising solution to improve the data transmission efficiency with the limited radio resources for the autonomous interactions among machines and things in the future...Text semantic extraction has been envisioned as a promising solution to improve the data transmission efficiency with the limited radio resources for the autonomous interactions among machines and things in the future sixth-generation(6G)wireless networks.In this paper,we propose a Chinese text semantic extraction model,namely T-Pointer,to improve the quality of semantic extraction by integrating the Transformer with the pointer-generator network.The proposed T-Pointer model consists of a semantic encoder and a semantic decoder.In the encoding stage,we use the multi-head attention mechanism of the Transformer to extract semantic features from the input Chinese text.In the decoding stage,we first use the Transformer to extract multi-level global text features.Then,we introduce the pointer-generator network model to directly copy the keyword information from the source text.The simulation results demonstrate that the T-Pointer model can improve the bilingual evaluation understudy(BLEU)and recalloriented understudy for gisting evaluation(ROUGE)by 14.69%and 14.87%on average in comparison with the state-of-the-art models,respectively.Also,we implement the T-Pointer model on a semantic communication system based on the universal software radio peripheral(USRP)platform.The result shows that the packet delay of semantic transmission can be reduced by 52.05%on average,compared to traditional information transmission.展开更多
This paper presents an intelligent patrol and security robot integrating 2D LiDAR and RGB-D vision sensors to achieve semantic simultaneous localization and mapping(SLAM),real-time object recognition,and dynamic obsta...This paper presents an intelligent patrol and security robot integrating 2D LiDAR and RGB-D vision sensors to achieve semantic simultaneous localization and mapping(SLAM),real-time object recognition,and dynamic obstacle avoidance.The system employs the YOLOv7 deep-learning framework for semantic detection and SLAM for localization and mapping,fusing geometric and visual data to build a high-fidelity 2D semantic map.This map enables the robot to identify and project object information for improved situational awareness.Experimental results show that object recognition reached 95.4%mAP@0.5.Semantic completeness increased from 68.7%(single view)to 94.1%(multi-view)with an average position error of 3.1 cm.During navigation,the robot achieved 98.0%reliability,avoided moving obstacles in 90.0%of encounters,and replanned paths in 0.42 s on average.The integration of LiDAR-based SLAMwith deep-learning–driven semantic perception establishes a robust foundation for intelligent,adaptive,and safe robotic navigation in dynamic environments.展开更多
Weakly Supervised Semantic Segmentation(WSSS),which relies only on image-level labels,has attracted significant attention for its cost-effectiveness and scalability.Existing methods mainly enhance inter-class distinct...Weakly Supervised Semantic Segmentation(WSSS),which relies only on image-level labels,has attracted significant attention for its cost-effectiveness and scalability.Existing methods mainly enhance inter-class distinctions and employ data augmentation to mitigate semantic ambiguity and reduce spurious activations.However,they often neglect the complex contextual dependencies among image patches,resulting in incomplete local representations and limited segmentation accuracy.To address these issues,we propose the Context Patch Fusion with Class Token Enhancement(CPF-CTE)framework,which exploits contextual relations among patches to enrich feature repre-sentations and improve segmentation.At its core,the Contextual-Fusion Bidirectional Long Short-Term Memory(CF-BiLSTM)module captures spatial dependencies between patches and enables bidirectional information flow,yield-ing a more comprehensive understanding of spatial correlations.This strengthens feature learning and segmentation robustness.Moreover,we introduce learnable class tokens that dynamically encode and refine class-specific semantics,enhancing discriminative capability.By effectively integrating spatial and semantic cues,CPF-CTE produces richer and more accurate representations of image content.Extensive experiments on PASCAL VOC 2012 and MS COCO 2014 validate that CPF-CTE consistently surpasses prior WSSS methods.展开更多
Regular detection of pavement cracks is essential for infrastructure maintenance.However,existing methods often ignore the challenges such as the continuous evolution of crack features between video frames and the dif...Regular detection of pavement cracks is essential for infrastructure maintenance.However,existing methods often ignore the challenges such as the continuous evolution of crack features between video frames and the difficulty of defect quantification.To this end,this paper proposes an integrated framework for pavement crack detection,segmentation,tracking and counting based on Transformer.Firstly,we design theVitSeg-Det network,which is an integrated detection and segmentation network that can accurately locate and segment tiny cracks in complex scenes.Second,the TransTra-Count system is developed to automatically count the number of defects by combining defect tracking with width estimation.Finally,we conduct experimental verification on three datasets.The results show that the proposed method is superior to the existing deep learning methods in detection accuracy.In addition,the actual scene video test shows that the framework can accurately label the defect location and output the number of defects in real time.展开更多
This study aimed to enhance the performance of semantic segmentation for autonomous driving by improving the 2DPASS model.Two novel improvements were proposed and implemented in this paper:dynamically adjusting the lo...This study aimed to enhance the performance of semantic segmentation for autonomous driving by improving the 2DPASS model.Two novel improvements were proposed and implemented in this paper:dynamically adjusting the loss function ratio and integrating an attention mechanism(CBAM).First,the loss function weights were adjusted dynamically.The grid search method is used for deciding the best ratio of 7:3.It gives greater emphasis to the cross-entropy loss,which resulted in better segmentation performance.Second,CBAM was applied at different layers of the 2Dencoder.Heatmap analysis revealed that introducing it after the second block of 2D image encoding produced the most effective enhancement of important feature representation.The training epoch was chosen for optimizing the best value by experiments,which improved model convergence and overall accuracy.To evaluate the proposed approach,experiments were conducted based on the SemanticKITTI database.The results showed that the improved model achieved higher segmentation accuracy by 64.31%,improved 11.47% in mIoU compared with the conventional 2DPASS model(baseline:52.84%).It was more effective at detecting small and distant objects and clearly identifying boundaries between different classes.Issues such as noise and variations in data distribution affected its accuracy,indicating the need for further refinement.Overall,the proposed improvements to the 2DPASS model demonstrated the potential to advance semantic segmentation technology and contributed to a more reliable perception of complex,dynamic environments in autonomous vehicles.Accurate segmentation enhances the vehicle’s ability to distinguish different objects,and this improvement directly supports safer navigation,robust decision-making,and efficient path planning,making it highly applicable to real-world deployment of autonomous systems in urban and highway settings.展开更多
Advanced traffic monitoring systems encounter substantial challenges in vehicle detection and classification due to the limitations of conventional methods,which often demand extensive computational resources and stru...Advanced traffic monitoring systems encounter substantial challenges in vehicle detection and classification due to the limitations of conventional methods,which often demand extensive computational resources and struggle with diverse data acquisition techniques.This research presents a novel approach for vehicle classification and recognition in aerial image sequences,integrating multiple advanced techniques to enhance detection accuracy.The proposed model begins with preprocessing using Multiscale Retinex(MSR)to enhance image quality,followed by Expectation-Maximization(EM)Segmentation for precise foreground object identification.Vehicle detection is performed using the state-of-the-art YOLOv10 framework,while feature extraction incorporates Maximally Stable Extremal Regions(MSER),Dense Scale-Invariant Feature Transform(Dense SIFT),and Zernike Moments Features to capture distinct object characteristics.Feature optimization is further refined through a Hybrid Swarm-based Optimization algorithm,ensuring optimal feature selection for improved classification performance.The final classification is conducted using a Vision Transformer,leveraging its robust learning capabilities for enhanced accuracy.Experimental evaluations on benchmark datasets,including UAVDT and the Unmanned Aerial Vehicle Intruder Dataset(UAVID),demonstrate the superiority of the proposed approach,achieving an accuracy of 94.40%on UAVDT and 93.57%on UAVID.The results highlight the efficacy of the model in significantly enhancing vehicle detection and classification in aerial imagery,outperforming existing methodologies and offering a statistically validated improvement for intelligent traffic monitoring systems compared to existing approaches.展开更多
In the age of big data,ensuring data privacy while enabling efficient encrypted data retrieval has become a critical challenge.Traditional searchable encryption schemes face difficulties in handling complex semantic q...In the age of big data,ensuring data privacy while enabling efficient encrypted data retrieval has become a critical challenge.Traditional searchable encryption schemes face difficulties in handling complex semantic queries.Additionally,they typically rely on honest but curious cloud servers,which introduces the risk of repudiation.Furthermore,the combined operations of search and verification increase system load,thereby reducing performance.Traditional verification mechanisms,which rely on complex hash constructions,suffer from low verification efficiency.To address these challenges,this paper proposes a blockchain-based contextual semantic-aware ciphertext retrieval scheme with efficient verification.Building on existing single and multi-keyword search methods,the scheme uses vector models to semantically train the dataset,enabling it to retain semantic information and achieve context-aware encrypted retrieval,significantly improving search accuracy.Additionally,a blockchain-based updatable master-slave chain storage model is designed,where the master chain stores encrypted keyword indexes and the slave chain stores verification information generated by zero-knowledge proofs,thus balancing system load while improving search and verification efficiency.Finally,an improved non-interactive zero-knowledge proof mechanism is introduced,reducing the computational complexity of verification and ensuring efficient validation of search results.Experimental results demonstrate that the proposed scheme offers stronger security,balanced overhead,and higher search verification efficiency.展开更多
Urban traffic generates massive and diverse data,yet most systems remain fragmented.Current approaches to congestion management suffer from weak data consistency and poor scalability.This study addresses this gap by p...Urban traffic generates massive and diverse data,yet most systems remain fragmented.Current approaches to congestion management suffer from weak data consistency and poor scalability.This study addresses this gap by proposing the Urban Traffic Congestion Unified Metadata Model(UTC-UMM).The goal is to provide a standardized and extensible framework for describing,extracting,and storing multisource traffic data in smart cities.The model defines a two-tier specification that organizes nine core traffic resource classes.It employs an eXtensible Markup Language(XML)Schema that connects general elements with resource-specific elements.This design ensures both syntactic and semantic interoperability across siloed datasets.Extension principles allow new elements or constraints to be introducedwithout breaking backward compatibility.Adistributed pipeline is implemented usingHadoop Distributed File System(HDFS)and HBase.It integrates computer vision for video and natural language processing for text to automate metadata extraction.Optimized row-key designs enable low-latency queries.Performance is tested with the Yahoo!Cloud Serving Benchmark(YCSB),which shows linear scalability and high throughput.The results demonstrate that UTC-UMM can unify heterogeneous traffic data while supporting real-time analytics.The discussion highlights its potential to improve data reuse,portability,and scalability in urban congestion studies.Future research will explore integration with association rulemining and advanced knowledge representation to capture richer spatiotemporal traffic patterns.展开更多
This article studies the problem of image segmentation-based semantic communication in autonomous driving.In real traffic scenes,the detecting of objects(e.g.,vehicles and pedestrians)is more important to guarantee dr...This article studies the problem of image segmentation-based semantic communication in autonomous driving.In real traffic scenes,the detecting of objects(e.g.,vehicles and pedestrians)is more important to guarantee driving safety,which is always ignored in existing works.Therefore,we propose a vehicular image segmentation-oriented semantic communication system,termed VIS-SemCom,focusing on transmitting and recovering image semantic features of high-important objects to reduce transmission redundancy.First,we develop a semantic codec based on Swin Transformer architecture,which expands the perceptual field thus improving the segmentation accuracy.To highlight the important objects'accuracy,we propose a multi-scale semantic extraction method by assigning the number of Swin Transformer blocks for diverse resolution semantic features.Also,an importance-aware loss incorporating important levels is devised,and an online hard example mining(OHEM)strategy is proposed to handle small sample issues in the dataset.Finally,experimental results demonstrate that the proposed VIS-SemCom can achieve a significant mean intersection over union(mIoU)performance in the SNR regions,a reduction of transmitted data volume by about 60%at 60%mIoU,and improve the segmentation accuracy of important objects,compared to baseline image communication.展开更多
Low-light image enhancement(LLIE)remains challenging due to underexposure,color distortion,and amplified noise introduced during illumination correction.Existing deep learning–based methods typically apply uniform en...Low-light image enhancement(LLIE)remains challenging due to underexposure,color distortion,and amplified noise introduced during illumination correction.Existing deep learning–based methods typically apply uniform enhancement across the entire image,which overlooks scene semantics and often leads to texture degradation or unnatural color reproduction.To overcome these limitations,we propose a Semantic-Guided Visual Mamba Network(SGVMNet)that unifies semantic reasoning,state-space modeling,and mixture-of-experts routing for adaptive illumination correction.SGVMNet comprises three key components:(1)a semantic modulation module(SMM)that extracts scene-aware semantic priors from pretrained multimodal models—Large Language and Vision Assistant(LLaVA)and Contrastive Language–Image Pretraining(CLIP)—and injects them hierarchically into the feature stream;(2)aMixture-of-Experts State-Space Feature EnhancementModule(MoE-SSMFEM)that dynamically selects informative channels and activates specialized state-space experts for efficient global–local illumination modeling;and(3)a Text-Guided Mixture Mamba Block(TGMB)that fuses semantic priors and visual features through bidirectional state propagation.Experimental results demonstrate that on the low-light(LOL)dataset,SGVMNet outperforms other state-of-the-art methods in both quantitative and qualitative evaluations,and it also maintains low computational complexity with fast inference speed.On LOLv2-Syn,SGVMNet achieves 26.512 dB PSNR and 0.935 SSIM,outperforming RetinexFormer by 0.61 dB.On LOLv1,SGVMNet attains 26.50 dB PSNR and 0.863 SSIM.Furthermore,experiments on multiple unpaired real-world datasets further validate the superiority of SGVMNet,showing that the model not only exhibits strong cross-scene generalization ability but also effectively preserves semantic consistency and visual naturalness.展开更多
Chinese abbreviations improve communicative efficiency by extracting key components from longer expressions.They are widely used in both daily communication and professional domains.However,existing abbreviation gener...Chinese abbreviations improve communicative efficiency by extracting key components from longer expressions.They are widely used in both daily communication and professional domains.However,existing abbreviation generation methods still face two major challenges.First,sequence-labeling-based approaches often neglect contextual meaning by making binary decisions at the character level,leading to abbreviations that fail to capture semantic completeness.Second,generation-basedmethods rely heavily on a single decoding process,which frequently produces correct abbreviations but ranks them lower due to inadequate semantic evaluation.To address these limitations,we propose a novel two-stage frameworkwithGeneration–Iterative Optimization forAbbreviation(GIOA).In the first stage,we design aChain-of-Thought prompting strategy and incorporate definitional and situational contexts to generate multiple abbreviation candidates.In the second stage,we introduce a Semantic Preservation Dynamic Adjustment mechanism that alternates between character-level importance estimation and semantic restoration to optimize candidate ranking.Experiments on two public benchmark datasets show that our method outperforms existing state-of-the-art approaches,achieving Hit@1 improvements of 15.15%and 13.01%,respectively,while maintaining consistent results in Hit@3.展开更多
High-resolution remote sensing images(HRSIs)are now an essential data source for gathering surface information due to advancements in remote sensing data capture technologies.However,their significant scale changes an...High-resolution remote sensing images(HRSIs)are now an essential data source for gathering surface information due to advancements in remote sensing data capture technologies.However,their significant scale changes and wealth of spatial details pose challenges for semantic segmentation.While convolutional neural networks(CNNs)excel at capturing local features,they are limited in modeling long-range dependencies.Conversely,transformers utilize multihead self-attention to integrate global context effectively,but this approach often incurs a high computational cost.This paper proposes a global-local multiscale context network(GLMCNet)to extract both global and local multiscale contextual information from HRSIs.A detail-enhanced filtering module(DEFM)is proposed at the end of the encoder to refine the encoder outputs further,thereby enhancing the key details extracted by the encoder and effectively suppressing redundant information.In addition,a global-local multiscale transformer block(GLMTB)is proposed in the decoding stage to enable the modeling of rich multiscale global and local information.We also design a stair fusion mechanism to transmit deep semantic information from deep to shallow layers progressively.Finally,we propose the semantic awareness enhancement module(SAEM),which further enhances the representation of multiscale semantic features through spatial attention and covariance channel attention.Extensive ablation analyses and comparative experiments were conducted to evaluate the performance of the proposed method.Specifically,our method achieved a mean Intersection over Union(mIoU)of 86.89%on the ISPRS Potsdam dataset and 84.34%on the ISPRS Vaihingen dataset,outperforming existing models such as ABCNet and BANet.展开更多
Deep learning-based methods have become alternatives to traditional numerical weather prediction systems,offering faster computation and the ability to utilize large historical datasets.However,the application of deep...Deep learning-based methods have become alternatives to traditional numerical weather prediction systems,offering faster computation and the ability to utilize large historical datasets.However,the application of deep learning to medium-range regional weather forecasting with limited data remains a significant challenge.In this work,three key solutions are proposed:(1)motivated by the need to improve model performance in data-scarce regional forecasting scenarios,the authors innovatively apply semantic segmentation models,to better capture spatiotemporal features and improve prediction accuracy;(2)recognizing the challenge of overfitting and the inability of traditional noise-based data augmentation methods to effectively enhance model robustness,a novel learnable Gaussian noise mechanism is introduced that allows the model to adaptively optimize perturbations for different locations,ensuring more effective learning;and(3)to address the issue of error accumulation in autoregressive prediction,as well as the challenge of learning difficulty and the lack of intermediate data utilization in one-shot prediction,the authors propose a cascade prediction approach that effectively resolves these problems while significantly improving model forecasting performance.The method achieves a competitive result in The East China Regional AI Medium Range Weather Forecasting Competition.Ablation experiments further validate the effectiveness of each component,highlighting their contributions to enhancing prediction performance.展开更多
[Objective]Leaf diseases significantly affect both the yield and quality of tea throughout the year.To address the issue of inadequate segmentation finesse in the current tea spot segmentation models,a novel diagnosis...[Objective]Leaf diseases significantly affect both the yield and quality of tea throughout the year.To address the issue of inadequate segmentation finesse in the current tea spot segmentation models,a novel diagnosis of the severity of tea spots was proposed in this research,designated as MDC-U-Net3+,to enhance segmentation accuracy on the base framework of U-Net3+.[Methods]Multi-scale feature fusion module(MSFFM)was incorporated into the backbone network of U-Net3+to obtain feature information across multiple receptive fields of diseased spots,thereby reducing the loss of features within the encoder.Dual multi-scale attention(DMSA)was incorporated into the skip connection process to mitigate the segmentation boundary ambiguity issue.This integration facilitates the comprehensive fusion of fine-grained and coarse-grained semantic information at full scale.Furthermore,the segmented mask image was subjected to conditional random fields(CRF)to enhance the optimization of the segmentation results[Results and Discussions]The improved model MDC-U-Net3+achieved a mean pixel accuracy(mPA)of 94.92%,accompanied by a mean Intersection over Union(mIoU)ratio of 90.9%.When compared to the mPA and mIoU of U-Net3+,MDC-U-Net3+model showed improvements of 1.85 and 2.12 percentage points,respectively.These results illustrated a more effective segmentation performance than that achieved by other classical semantic segmentation models.[Conclusions]The methodology presented herein could provide data support for automated disease detection and precise medication,consequently reducing the losses associated with tea diseases.展开更多
Semantic segmentation for mixed scenes of aerial remote sensing and road traffic is one of the key technologies for visual perception of flying cars.The State-of-the-Art(SOTA)semantic segmentation methods have made re...Semantic segmentation for mixed scenes of aerial remote sensing and road traffic is one of the key technologies for visual perception of flying cars.The State-of-the-Art(SOTA)semantic segmentation methods have made remarkable achievements in both fine-grained segmentation and real-time performance.However,when faced with the huge differences in scale and semantic categories brought about by the mixed scenes of aerial remote sensing and road traffic,they still face great challenges and there is little related research.Addressing the above issue,this paper proposes a semantic segmentation model specifically for mixed datasets of aerial remote sensing and road traffic scenes.First,a novel decoding-recoding multi-scale feature iterative refinement structure is proposed,which utilizes the re-integration and continuous enhancement of multi-scale information to effectively deal with the huge scale differences between cross-domain scenes,while using a fully convolutional structure to ensure the lightweight and real-time requirements.Second,a welldesigned cross-window attention mechanism combined with a global information integration decoding block forms an enhanced global context perception,which can effectively capture the long-range dependencies and multi-scale global context information of different scenes,thereby achieving fine-grained semantic segmentation.The proposed method is tested on a large-scale mixed dataset of aerial remote sensing and road traffic scenes.The results confirm that it can effectively deal with the problem of large-scale differences in cross-domain scenes.Its segmentation accuracy surpasses that of the SOTA methods,which meets the real-time requirements.展开更多
In image analysis,high-precision semantic segmentation predominantly relies on supervised learning.Despite significant advancements driven by deep learning techniques,challenges such as class imbalance and dynamic per...In image analysis,high-precision semantic segmentation predominantly relies on supervised learning.Despite significant advancements driven by deep learning techniques,challenges such as class imbalance and dynamic performance evaluation persist.Traditional weighting methods,often based on pre-statistical class counting,tend to overemphasize certain classes while neglecting others,particularly rare sample categories.Approaches like focal loss and other rare-sample segmentation techniques introduce multiple hyperparameters that require manual tuning,leading to increased experimental costs due to their instability.This paper proposes a novel CAWASeg framework to address these limitations.Our approach leverages Grad-CAM technology to generate class activation maps,identifying key feature regions that the model focuses on during decision-making.We introduce a Comprehensive Segmentation Performance Score(CSPS)to dynamically evaluate model performance by converting these activation maps into pseudo mask and comparing them with Ground Truth.Additionally,we design two adaptive weights for each class:a Basic Weight(BW)and a Ratio Weight(RW),which the model adjusts during training based on real-time feedback.Extensive experiments on the COCO-Stuff,CityScapes,and ADE20k datasets demonstrate that our CAWASeg framework significantly improves segmentation performance for rare sample categories while enhancing overall segmentation accuracy.The proposed method offers a robust and efficient solution for addressing class imbalance in semantic segmentation tasks.展开更多
Accurate prediction of environmental temperature is pivotal for promoting sustainable crop growth.At present,the most effective temperature sensing and prediction system is the Agricultural Internet of Things(AIoT),wh...Accurate prediction of environmental temperature is pivotal for promoting sustainable crop growth.At present,the most effective temperature sensing and prediction system is the Agricultural Internet of Things(AIoT),which deploys a large number of sensors to collect meteorological data and transmits them to the cloud server for prediction.However,this procedure is computationally and communicationally expensive for resourceconstrained AIoT.Recently,Semantic Communication(SC)has shown potential in efficient data transmission,but existing methods overlook the repetitive semantic information whilst sensing data,bringing additional overheads.With the resource-constraint nature of AIoT in mind,we propose the Semantic Communication-enabled Cognitive Agriculture Framework(SC-CAF)for delivering accurate temperature predictions.The proposed SC-CAF incorporates an intelligent analysis layer that performs the temperature prediction and model training and distribution,while a semantic layer transmitting the semantic information extracted from raw data based on the download model,ultimately to reduce communication overheads in AIoT.Furthermore,we propose a novel model called the Light Temperature Semantic Communication(LTSC)by adopting skip-attention and semantic compressor to avoid unnecessary computation and repetitive information,thereby addressing the semantic redundancy issues in sensing data.We also develop a Semantic-based Model Compression(SCMC)algorithm to alleviate the computation and bandwidth burden,enabling AIoT to explore the extensive usage of SC.Experimental results demonstrate that the proposed SC-CAF achieves the lowest prediction error while reducing Floating Point Operations(FLOPs)by 95.88%,memory requirements by 78.30%,Graphics Processing Unit(GPU)power by 50.77%,and time latency by 84.44%,outperforming notable state-of-the-art methods.展开更多
The application of deep learning in fabric defect detection has become increasingly widespread.To address false positives and false negatives in fabric roll seam detection,and to improve automation efficiency and prod...The application of deep learning in fabric defect detection has become increasingly widespread.To address false positives and false negatives in fabric roll seam detection,and to improve automation efficiency and product quality,we propose the Multi-scale Context DeepLabV3+(MSC-DeepLabV3+),a semantic segmentation network designed for fabric roll seam detection,based on DeepLabV3+.The model improvements include enhancing the backbone performance through optimization of the UIB-MobileNetV2 network;designing the Dynamic Atrous and Sliding-window Fusion(DASF)module to improve adaptability to multi-scale seam structures with dynamic dilation rates and a sliding-window mechanism;and utilizing the Progressive Low-level Feature Fusion(PLFF)module to progressively restore seam boundary details via shallow feature fusion.Additionally,an enhanced 3-SE attention mechanism is employed,replacing the direct concatenation operation.Experimental results show thatMSCDeepLabV3+outperforms classical and recent segmentation models.Compared to DeepLabV3+with an Xception backbone,MSC-DeepLabV3+achieves a mean intersection over union(mIoU)of 92.30%and the boundary Fscore(BF)of 92.54%,representing improvements of 3.04%and 3.14%,respectively.Moreover,the model complexity is significantly reduced,with the model parameters(params)decreasing to 3.44M and Frames Per Second(FPS)increasing from 101 to 273,demonstrating its potential for deployment in resource-constrained industrial scenarios.展开更多
Weakly supervised semantic segmentation(WSSS)is a tricky task,which only provides category information for segmentation prediction.Thus,the key stage of WSSS is to generate the pseudo labels.For convolutional neural n...Weakly supervised semantic segmentation(WSSS)is a tricky task,which only provides category information for segmentation prediction.Thus,the key stage of WSSS is to generate the pseudo labels.For convolutional neural network(CNN)based methods,in which class activation mapping(CAM)is proposed to obtain the pseudo labels,and only concentrates on the most discriminative parts.Recently,transformer-based methods utilize attention map from the multi-headed self-attention(MHSA)module to predict pseudo labels,which usually contain obvious background noise and incoherent object area.To solve the above problems,we use the Conformer as our backbone,which is a parallel network based on convolutional neural network(CNN)and Transformer.The two branches generate pseudo labels and refine them independently,and can effectively combine the advantages of CNN and Transformer.However,the parallel structure is not close enough in the information communication.Thus,parallel structure can result in poor details about pseudo labels,and the background noise still exists.To alleviate this problem,we propose enhancing convolution CAM(ECCAM)model,which have three improved modules based on enhancing convolution,including deeper stem(DStem),convolutional feed-forward network(CFFN)and feature coupling unit with convolution(FCUConv).The ECCAM could make Conformer have tighter interaction between CNN and Transformer branches.After experimental verification,the improved modules we propose can help the network perceive more local information from images,making the final segmentation results more refined.Compared with similar architecture,our modules greatly improve the semantic segmentation performance and achieve70.2%mean intersection over union(mIoU)on the PASCAL VOC 2012 dataset.展开更多
基金National Natural Science Foundation of China under Grants 62122069,62071431,62072490,62301490Science and Technology Development Fund of Macao,Macao,China under Grant 0158/2022/A+2 种基金Guangdong Basic and Applied Basic Research Foundation(2022A1515011287)MYRG2020-00107-IOTSCFDCT SKL-IOTSC(UM)-2021-2023。
文摘Text semantic extraction has been envisioned as a promising solution to improve the data transmission efficiency with the limited radio resources for the autonomous interactions among machines and things in the future sixth-generation(6G)wireless networks.In this paper,we propose a Chinese text semantic extraction model,namely T-Pointer,to improve the quality of semantic extraction by integrating the Transformer with the pointer-generator network.The proposed T-Pointer model consists of a semantic encoder and a semantic decoder.In the encoding stage,we use the multi-head attention mechanism of the Transformer to extract semantic features from the input Chinese text.In the decoding stage,we first use the Transformer to extract multi-level global text features.Then,we introduce the pointer-generator network model to directly copy the keyword information from the source text.The simulation results demonstrate that the T-Pointer model can improve the bilingual evaluation understudy(BLEU)and recalloriented understudy for gisting evaluation(ROUGE)by 14.69%and 14.87%on average in comparison with the state-of-the-art models,respectively.Also,we implement the T-Pointer model on a semantic communication system based on the universal software radio peripheral(USRP)platform.The result shows that the packet delay of semantic transmission can be reduced by 52.05%on average,compared to traditional information transmission.
基金supported by the National Science and Technology Council of under Grant NSTC 114-2221-E-130-007.
文摘This paper presents an intelligent patrol and security robot integrating 2D LiDAR and RGB-D vision sensors to achieve semantic simultaneous localization and mapping(SLAM),real-time object recognition,and dynamic obstacle avoidance.The system employs the YOLOv7 deep-learning framework for semantic detection and SLAM for localization and mapping,fusing geometric and visual data to build a high-fidelity 2D semantic map.This map enables the robot to identify and project object information for improved situational awareness.Experimental results show that object recognition reached 95.4%mAP@0.5.Semantic completeness increased from 68.7%(single view)to 94.1%(multi-view)with an average position error of 3.1 cm.During navigation,the robot achieved 98.0%reliability,avoided moving obstacles in 90.0%of encounters,and replanned paths in 0.42 s on average.The integration of LiDAR-based SLAMwith deep-learning–driven semantic perception establishes a robust foundation for intelligent,adaptive,and safe robotic navigation in dynamic environments.
文摘Weakly Supervised Semantic Segmentation(WSSS),which relies only on image-level labels,has attracted significant attention for its cost-effectiveness and scalability.Existing methods mainly enhance inter-class distinctions and employ data augmentation to mitigate semantic ambiguity and reduce spurious activations.However,they often neglect the complex contextual dependencies among image patches,resulting in incomplete local representations and limited segmentation accuracy.To address these issues,we propose the Context Patch Fusion with Class Token Enhancement(CPF-CTE)framework,which exploits contextual relations among patches to enrich feature repre-sentations and improve segmentation.At its core,the Contextual-Fusion Bidirectional Long Short-Term Memory(CF-BiLSTM)module captures spatial dependencies between patches and enables bidirectional information flow,yield-ing a more comprehensive understanding of spatial correlations.This strengthens feature learning and segmentation robustness.Moreover,we introduce learnable class tokens that dynamically encode and refine class-specific semantics,enhancing discriminative capability.By effectively integrating spatial and semantic cues,CPF-CTE produces richer and more accurate representations of image content.Extensive experiments on PASCAL VOC 2012 and MS COCO 2014 validate that CPF-CTE consistently surpasses prior WSSS methods.
基金supported in part by the Natural Science Foundation of Shaanxi Province of China under Grant 2024JC-YBQN-0695.
文摘Regular detection of pavement cracks is essential for infrastructure maintenance.However,existing methods often ignore the challenges such as the continuous evolution of crack features between video frames and the difficulty of defect quantification.To this end,this paper proposes an integrated framework for pavement crack detection,segmentation,tracking and counting based on Transformer.Firstly,we design theVitSeg-Det network,which is an integrated detection and segmentation network that can accurately locate and segment tiny cracks in complex scenes.Second,the TransTra-Count system is developed to automatically count the number of defects by combining defect tracking with width estimation.Finally,we conduct experimental verification on three datasets.The results show that the proposed method is superior to the existing deep learning methods in detection accuracy.In addition,the actual scene video test shows that the framework can accurately label the defect location and output the number of defects in real time.
文摘This study aimed to enhance the performance of semantic segmentation for autonomous driving by improving the 2DPASS model.Two novel improvements were proposed and implemented in this paper:dynamically adjusting the loss function ratio and integrating an attention mechanism(CBAM).First,the loss function weights were adjusted dynamically.The grid search method is used for deciding the best ratio of 7:3.It gives greater emphasis to the cross-entropy loss,which resulted in better segmentation performance.Second,CBAM was applied at different layers of the 2Dencoder.Heatmap analysis revealed that introducing it after the second block of 2D image encoding produced the most effective enhancement of important feature representation.The training epoch was chosen for optimizing the best value by experiments,which improved model convergence and overall accuracy.To evaluate the proposed approach,experiments were conducted based on the SemanticKITTI database.The results showed that the improved model achieved higher segmentation accuracy by 64.31%,improved 11.47% in mIoU compared with the conventional 2DPASS model(baseline:52.84%).It was more effective at detecting small and distant objects and clearly identifying boundaries between different classes.Issues such as noise and variations in data distribution affected its accuracy,indicating the need for further refinement.Overall,the proposed improvements to the 2DPASS model demonstrated the potential to advance semantic segmentation technology and contributed to a more reliable perception of complex,dynamic environments in autonomous vehicles.Accurate segmentation enhances the vehicle’s ability to distinguish different objects,and this improvement directly supports safer navigation,robust decision-making,and efficient path planning,making it highly applicable to real-world deployment of autonomous systems in urban and highway settings.
文摘Advanced traffic monitoring systems encounter substantial challenges in vehicle detection and classification due to the limitations of conventional methods,which often demand extensive computational resources and struggle with diverse data acquisition techniques.This research presents a novel approach for vehicle classification and recognition in aerial image sequences,integrating multiple advanced techniques to enhance detection accuracy.The proposed model begins with preprocessing using Multiscale Retinex(MSR)to enhance image quality,followed by Expectation-Maximization(EM)Segmentation for precise foreground object identification.Vehicle detection is performed using the state-of-the-art YOLOv10 framework,while feature extraction incorporates Maximally Stable Extremal Regions(MSER),Dense Scale-Invariant Feature Transform(Dense SIFT),and Zernike Moments Features to capture distinct object characteristics.Feature optimization is further refined through a Hybrid Swarm-based Optimization algorithm,ensuring optimal feature selection for improved classification performance.The final classification is conducted using a Vision Transformer,leveraging its robust learning capabilities for enhanced accuracy.Experimental evaluations on benchmark datasets,including UAVDT and the Unmanned Aerial Vehicle Intruder Dataset(UAVID),demonstrate the superiority of the proposed approach,achieving an accuracy of 94.40%on UAVDT and 93.57%on UAVID.The results highlight the efficacy of the model in significantly enhancing vehicle detection and classification in aerial imagery,outperforming existing methodologies and offering a statistically validated improvement for intelligent traffic monitoring systems compared to existing approaches.
基金supported in part by the National Natural Science Foundation of China under Grant 62262073in part by the Yunnan Provincial Ten Thousand People Program for Young Top Talents under Grant YNWR-QNBJ-2019-237in part by the Yunnan Provincial Major Science and Technology Special Program under Grant 202402AD080002.
文摘In the age of big data,ensuring data privacy while enabling efficient encrypted data retrieval has become a critical challenge.Traditional searchable encryption schemes face difficulties in handling complex semantic queries.Additionally,they typically rely on honest but curious cloud servers,which introduces the risk of repudiation.Furthermore,the combined operations of search and verification increase system load,thereby reducing performance.Traditional verification mechanisms,which rely on complex hash constructions,suffer from low verification efficiency.To address these challenges,this paper proposes a blockchain-based contextual semantic-aware ciphertext retrieval scheme with efficient verification.Building on existing single and multi-keyword search methods,the scheme uses vector models to semantically train the dataset,enabling it to retain semantic information and achieve context-aware encrypted retrieval,significantly improving search accuracy.Additionally,a blockchain-based updatable master-slave chain storage model is designed,where the master chain stores encrypted keyword indexes and the slave chain stores verification information generated by zero-knowledge proofs,thus balancing system load while improving search and verification efficiency.Finally,an improved non-interactive zero-knowledge proof mechanism is introduced,reducing the computational complexity of verification and ensuring efficient validation of search results.Experimental results demonstrate that the proposed scheme offers stronger security,balanced overhead,and higher search verification efficiency.
基金supported by the National Natural Science Foundation of China(Grant No.62172033).
文摘Urban traffic generates massive and diverse data,yet most systems remain fragmented.Current approaches to congestion management suffer from weak data consistency and poor scalability.This study addresses this gap by proposing the Urban Traffic Congestion Unified Metadata Model(UTC-UMM).The goal is to provide a standardized and extensible framework for describing,extracting,and storing multisource traffic data in smart cities.The model defines a two-tier specification that organizes nine core traffic resource classes.It employs an eXtensible Markup Language(XML)Schema that connects general elements with resource-specific elements.This design ensures both syntactic and semantic interoperability across siloed datasets.Extension principles allow new elements or constraints to be introducedwithout breaking backward compatibility.Adistributed pipeline is implemented usingHadoop Distributed File System(HDFS)and HBase.It integrates computer vision for video and natural language processing for text to automate metadata extraction.Optimized row-key designs enable low-latency queries.Performance is tested with the Yahoo!Cloud Serving Benchmark(YCSB),which shows linear scalability and high throughput.The results demonstrate that UTC-UMM can unify heterogeneous traffic data while supporting real-time analytics.The discussion highlights its potential to improve data reuse,portability,and scalability in urban congestion studies.Future research will explore integration with association rulemining and advanced knowledge representation to capture richer spatiotemporal traffic patterns.
基金National Natural Science Foundation of China under Grants No.62171047,U22B2001,62271065,62001051Beijing Natural Science Foundation under Grant L223027BUPT Excellent Ph.D Students Foundation under Grants CX2021114。
文摘This article studies the problem of image segmentation-based semantic communication in autonomous driving.In real traffic scenes,the detecting of objects(e.g.,vehicles and pedestrians)is more important to guarantee driving safety,which is always ignored in existing works.Therefore,we propose a vehicular image segmentation-oriented semantic communication system,termed VIS-SemCom,focusing on transmitting and recovering image semantic features of high-important objects to reduce transmission redundancy.First,we develop a semantic codec based on Swin Transformer architecture,which expands the perceptual field thus improving the segmentation accuracy.To highlight the important objects'accuracy,we propose a multi-scale semantic extraction method by assigning the number of Swin Transformer blocks for diverse resolution semantic features.Also,an importance-aware loss incorporating important levels is devised,and an online hard example mining(OHEM)strategy is proposed to handle small sample issues in the dataset.Finally,experimental results demonstrate that the proposed VIS-SemCom can achieve a significant mean intersection over union(mIoU)performance in the SNR regions,a reduction of transmitted data volume by about 60%at 60%mIoU,and improve the segmentation accuracy of important objects,compared to baseline image communication.
文摘Low-light image enhancement(LLIE)remains challenging due to underexposure,color distortion,and amplified noise introduced during illumination correction.Existing deep learning–based methods typically apply uniform enhancement across the entire image,which overlooks scene semantics and often leads to texture degradation or unnatural color reproduction.To overcome these limitations,we propose a Semantic-Guided Visual Mamba Network(SGVMNet)that unifies semantic reasoning,state-space modeling,and mixture-of-experts routing for adaptive illumination correction.SGVMNet comprises three key components:(1)a semantic modulation module(SMM)that extracts scene-aware semantic priors from pretrained multimodal models—Large Language and Vision Assistant(LLaVA)and Contrastive Language–Image Pretraining(CLIP)—and injects them hierarchically into the feature stream;(2)aMixture-of-Experts State-Space Feature EnhancementModule(MoE-SSMFEM)that dynamically selects informative channels and activates specialized state-space experts for efficient global–local illumination modeling;and(3)a Text-Guided Mixture Mamba Block(TGMB)that fuses semantic priors and visual features through bidirectional state propagation.Experimental results demonstrate that on the low-light(LOL)dataset,SGVMNet outperforms other state-of-the-art methods in both quantitative and qualitative evaluations,and it also maintains low computational complexity with fast inference speed.On LOLv2-Syn,SGVMNet achieves 26.512 dB PSNR and 0.935 SSIM,outperforming RetinexFormer by 0.61 dB.On LOLv1,SGVMNet attains 26.50 dB PSNR and 0.863 SSIM.Furthermore,experiments on multiple unpaired real-world datasets further validate the superiority of SGVMNet,showing that the model not only exhibits strong cross-scene generalization ability but also effectively preserves semantic consistency and visual naturalness.
基金supported by the National Key Research and Development Program of China(2020AAA0109300)the Shanghai Collaborative Innovation Center of data intelligence technology(No.0232-A1-8900-24-13).
文摘Chinese abbreviations improve communicative efficiency by extracting key components from longer expressions.They are widely used in both daily communication and professional domains.However,existing abbreviation generation methods still face two major challenges.First,sequence-labeling-based approaches often neglect contextual meaning by making binary decisions at the character level,leading to abbreviations that fail to capture semantic completeness.Second,generation-basedmethods rely heavily on a single decoding process,which frequently produces correct abbreviations but ranks them lower due to inadequate semantic evaluation.To address these limitations,we propose a novel two-stage frameworkwithGeneration–Iterative Optimization forAbbreviation(GIOA).In the first stage,we design aChain-of-Thought prompting strategy and incorporate definitional and situational contexts to generate multiple abbreviation candidates.In the second stage,we introduce a Semantic Preservation Dynamic Adjustment mechanism that alternates between character-level importance estimation and semantic restoration to optimize candidate ranking.Experiments on two public benchmark datasets show that our method outperforms existing state-of-the-art approaches,achieving Hit@1 improvements of 15.15%and 13.01%,respectively,while maintaining consistent results in Hit@3.
基金provided by the Science Research Project of Hebei Education Department under grant No.BJK2024115.
文摘High-resolution remote sensing images(HRSIs)are now an essential data source for gathering surface information due to advancements in remote sensing data capture technologies.However,their significant scale changes and wealth of spatial details pose challenges for semantic segmentation.While convolutional neural networks(CNNs)excel at capturing local features,they are limited in modeling long-range dependencies.Conversely,transformers utilize multihead self-attention to integrate global context effectively,but this approach often incurs a high computational cost.This paper proposes a global-local multiscale context network(GLMCNet)to extract both global and local multiscale contextual information from HRSIs.A detail-enhanced filtering module(DEFM)is proposed at the end of the encoder to refine the encoder outputs further,thereby enhancing the key details extracted by the encoder and effectively suppressing redundant information.In addition,a global-local multiscale transformer block(GLMTB)is proposed in the decoding stage to enable the modeling of rich multiscale global and local information.We also design a stair fusion mechanism to transmit deep semantic information from deep to shallow layers progressively.Finally,we propose the semantic awareness enhancement module(SAEM),which further enhances the representation of multiscale semantic features through spatial attention and covariance channel attention.Extensive ablation analyses and comparative experiments were conducted to evaluate the performance of the proposed method.Specifically,our method achieved a mean Intersection over Union(mIoU)of 86.89%on the ISPRS Potsdam dataset and 84.34%on the ISPRS Vaihingen dataset,outperforming existing models such as ABCNet and BANet.
基金supported by the National Natural Science Foundation of China[grant number 62376217]the Young Elite Scientists Sponsorship Program by CAST[grant number 2023QNRC001]the Joint Research Project for Meteorological Capacity Improvement[grant number 24NLTSZ003]。
文摘Deep learning-based methods have become alternatives to traditional numerical weather prediction systems,offering faster computation and the ability to utilize large historical datasets.However,the application of deep learning to medium-range regional weather forecasting with limited data remains a significant challenge.In this work,three key solutions are proposed:(1)motivated by the need to improve model performance in data-scarce regional forecasting scenarios,the authors innovatively apply semantic segmentation models,to better capture spatiotemporal features and improve prediction accuracy;(2)recognizing the challenge of overfitting and the inability of traditional noise-based data augmentation methods to effectively enhance model robustness,a novel learnable Gaussian noise mechanism is introduced that allows the model to adaptively optimize perturbations for different locations,ensuring more effective learning;and(3)to address the issue of error accumulation in autoregressive prediction,as well as the challenge of learning difficulty and the lack of intermediate data utilization in one-shot prediction,the authors propose a cascade prediction approach that effectively resolves these problems while significantly improving model forecasting performance.The method achieves a competitive result in The East China Regional AI Medium Range Weather Forecasting Competition.Ablation experiments further validate the effectiveness of each component,highlighting their contributions to enhancing prediction performance.
文摘[Objective]Leaf diseases significantly affect both the yield and quality of tea throughout the year.To address the issue of inadequate segmentation finesse in the current tea spot segmentation models,a novel diagnosis of the severity of tea spots was proposed in this research,designated as MDC-U-Net3+,to enhance segmentation accuracy on the base framework of U-Net3+.[Methods]Multi-scale feature fusion module(MSFFM)was incorporated into the backbone network of U-Net3+to obtain feature information across multiple receptive fields of diseased spots,thereby reducing the loss of features within the encoder.Dual multi-scale attention(DMSA)was incorporated into the skip connection process to mitigate the segmentation boundary ambiguity issue.This integration facilitates the comprehensive fusion of fine-grained and coarse-grained semantic information at full scale.Furthermore,the segmented mask image was subjected to conditional random fields(CRF)to enhance the optimization of the segmentation results[Results and Discussions]The improved model MDC-U-Net3+achieved a mean pixel accuracy(mPA)of 94.92%,accompanied by a mean Intersection over Union(mIoU)ratio of 90.9%.When compared to the mPA and mIoU of U-Net3+,MDC-U-Net3+model showed improvements of 1.85 and 2.12 percentage points,respectively.These results illustrated a more effective segmentation performance than that achieved by other classical semantic segmentation models.[Conclusions]The methodology presented herein could provide data support for automated disease detection and precise medication,consequently reducing the losses associated with tea diseases.
基金supported by the National Key Research and Development of China(No.2022YFB2503400).
文摘Semantic segmentation for mixed scenes of aerial remote sensing and road traffic is one of the key technologies for visual perception of flying cars.The State-of-the-Art(SOTA)semantic segmentation methods have made remarkable achievements in both fine-grained segmentation and real-time performance.However,when faced with the huge differences in scale and semantic categories brought about by the mixed scenes of aerial remote sensing and road traffic,they still face great challenges and there is little related research.Addressing the above issue,this paper proposes a semantic segmentation model specifically for mixed datasets of aerial remote sensing and road traffic scenes.First,a novel decoding-recoding multi-scale feature iterative refinement structure is proposed,which utilizes the re-integration and continuous enhancement of multi-scale information to effectively deal with the huge scale differences between cross-domain scenes,while using a fully convolutional structure to ensure the lightweight and real-time requirements.Second,a welldesigned cross-window attention mechanism combined with a global information integration decoding block forms an enhanced global context perception,which can effectively capture the long-range dependencies and multi-scale global context information of different scenes,thereby achieving fine-grained semantic segmentation.The proposed method is tested on a large-scale mixed dataset of aerial remote sensing and road traffic scenes.The results confirm that it can effectively deal with the problem of large-scale differences in cross-domain scenes.Its segmentation accuracy surpasses that of the SOTA methods,which meets the real-time requirements.
基金supported by the Funds for Central-Guided Local Science and Technology Development(Grant No.202407AC110005)Key Technologies for the Construction of a Whole-Process Intelligent Service System for Neuroendocrine Neoplasm.Supported by 2023 Opening Research Fund of Yunnan Key Laboratory of Digital Communications(YNJTKFB-20230686,YNKLDC-KFKT-202304).
文摘In image analysis,high-precision semantic segmentation predominantly relies on supervised learning.Despite significant advancements driven by deep learning techniques,challenges such as class imbalance and dynamic performance evaluation persist.Traditional weighting methods,often based on pre-statistical class counting,tend to overemphasize certain classes while neglecting others,particularly rare sample categories.Approaches like focal loss and other rare-sample segmentation techniques introduce multiple hyperparameters that require manual tuning,leading to increased experimental costs due to their instability.This paper proposes a novel CAWASeg framework to address these limitations.Our approach leverages Grad-CAM technology to generate class activation maps,identifying key feature regions that the model focuses on during decision-making.We introduce a Comprehensive Segmentation Performance Score(CSPS)to dynamically evaluate model performance by converting these activation maps into pseudo mask and comparing them with Ground Truth.Additionally,we design two adaptive weights for each class:a Basic Weight(BW)and a Ratio Weight(RW),which the model adjusts during training based on real-time feedback.Extensive experiments on the COCO-Stuff,CityScapes,and ADE20k datasets demonstrate that our CAWASeg framework significantly improves segmentation performance for rare sample categories while enhancing overall segmentation accuracy.The proposed method offers a robust and efficient solution for addressing class imbalance in semantic segmentation tasks.
基金supported by the Key Research and Development Project of Hubei Province(No.2024BAB070),China。
文摘Accurate prediction of environmental temperature is pivotal for promoting sustainable crop growth.At present,the most effective temperature sensing and prediction system is the Agricultural Internet of Things(AIoT),which deploys a large number of sensors to collect meteorological data and transmits them to the cloud server for prediction.However,this procedure is computationally and communicationally expensive for resourceconstrained AIoT.Recently,Semantic Communication(SC)has shown potential in efficient data transmission,but existing methods overlook the repetitive semantic information whilst sensing data,bringing additional overheads.With the resource-constraint nature of AIoT in mind,we propose the Semantic Communication-enabled Cognitive Agriculture Framework(SC-CAF)for delivering accurate temperature predictions.The proposed SC-CAF incorporates an intelligent analysis layer that performs the temperature prediction and model training and distribution,while a semantic layer transmitting the semantic information extracted from raw data based on the download model,ultimately to reduce communication overheads in AIoT.Furthermore,we propose a novel model called the Light Temperature Semantic Communication(LTSC)by adopting skip-attention and semantic compressor to avoid unnecessary computation and repetitive information,thereby addressing the semantic redundancy issues in sensing data.We also develop a Semantic-based Model Compression(SCMC)algorithm to alleviate the computation and bandwidth burden,enabling AIoT to explore the extensive usage of SC.Experimental results demonstrate that the proposed SC-CAF achieves the lowest prediction error while reducing Floating Point Operations(FLOPs)by 95.88%,memory requirements by 78.30%,Graphics Processing Unit(GPU)power by 50.77%,and time latency by 84.44%,outperforming notable state-of-the-art methods.
文摘The application of deep learning in fabric defect detection has become increasingly widespread.To address false positives and false negatives in fabric roll seam detection,and to improve automation efficiency and product quality,we propose the Multi-scale Context DeepLabV3+(MSC-DeepLabV3+),a semantic segmentation network designed for fabric roll seam detection,based on DeepLabV3+.The model improvements include enhancing the backbone performance through optimization of the UIB-MobileNetV2 network;designing the Dynamic Atrous and Sliding-window Fusion(DASF)module to improve adaptability to multi-scale seam structures with dynamic dilation rates and a sliding-window mechanism;and utilizing the Progressive Low-level Feature Fusion(PLFF)module to progressively restore seam boundary details via shallow feature fusion.Additionally,an enhanced 3-SE attention mechanism is employed,replacing the direct concatenation operation.Experimental results show thatMSCDeepLabV3+outperforms classical and recent segmentation models.Compared to DeepLabV3+with an Xception backbone,MSC-DeepLabV3+achieves a mean intersection over union(mIoU)of 92.30%and the boundary Fscore(BF)of 92.54%,representing improvements of 3.04%and 3.14%,respectively.Moreover,the model complexity is significantly reduced,with the model parameters(params)decreasing to 3.44M and Frames Per Second(FPS)increasing from 101 to 273,demonstrating its potential for deployment in resource-constrained industrial scenarios.
文摘Weakly supervised semantic segmentation(WSSS)is a tricky task,which only provides category information for segmentation prediction.Thus,the key stage of WSSS is to generate the pseudo labels.For convolutional neural network(CNN)based methods,in which class activation mapping(CAM)is proposed to obtain the pseudo labels,and only concentrates on the most discriminative parts.Recently,transformer-based methods utilize attention map from the multi-headed self-attention(MHSA)module to predict pseudo labels,which usually contain obvious background noise and incoherent object area.To solve the above problems,we use the Conformer as our backbone,which is a parallel network based on convolutional neural network(CNN)and Transformer.The two branches generate pseudo labels and refine them independently,and can effectively combine the advantages of CNN and Transformer.However,the parallel structure is not close enough in the information communication.Thus,parallel structure can result in poor details about pseudo labels,and the background noise still exists.To alleviate this problem,we propose enhancing convolution CAM(ECCAM)model,which have three improved modules based on enhancing convolution,including deeper stem(DStem),convolutional feed-forward network(CFFN)and feature coupling unit with convolution(FCUConv).The ECCAM could make Conformer have tighter interaction between CNN and Transformer branches.After experimental verification,the improved modules we propose can help the network perceive more local information from images,making the final segmentation results more refined.Compared with similar architecture,our modules greatly improve the semantic segmentation performance and achieve70.2%mean intersection over union(mIoU)on the PASCAL VOC 2012 dataset.
