In the study of automatic driving,understanding the road scene is a key to improve driving safety.The semantic segmentation method could divide the image into different areas associated with semantic categories in acc...In the study of automatic driving,understanding the road scene is a key to improve driving safety.The semantic segmentation method could divide the image into different areas associated with semantic categories in accordance with the pixel level,so as to help vehicles to perceive and obtain the surrounding road environment information,which would improve driving safety.Deeplabv3+is the current popular semantic segmentation model.There are phenomena that small targets are missed and similar objects are easily misjudged during its semantic segmentation tasks,which leads to rough segmentation boundary and reduces semantic accuracy.This study focuses on the issue,based on the Deeplabv3+network structure and combined with the attention mechanism,to increase the weight of the segmentation area,and then proposes an improved Deeplabv3+fusion attention mechanism for road scene semantic segmentation method.First,a group of parallel position attention module and channel attention module are introduced on the Deeplabv3+encoding end to capture more spatial context information and high-level semantic information.Then,an attention mechanism is introduced to restore the spatial detail information,and the data shall be normalized in order to accelerate the convergence speed of the model at the decoding end.The effects of model segmentation with different attention-introducing mechanisms are compared and tested on CamVid and Cityscapes datasets.The experimental results show that the mean Intersection over Unons of the improved model segmentation accuracies on the two datasets are boosted by 6.88%and 2.58%,respectively,which is better than using Deeplabv3+.This method does not significantly increase the amount of network calculation and complexity,and has a good balance of speed and accuracy.展开更多
Road marking detection is an important branch in autonomous driving,understanding the road information.In recent years,deep-learning-based semantic segmentation methods for road marking detection have been arising sin...Road marking detection is an important branch in autonomous driving,understanding the road information.In recent years,deep-learning-based semantic segmentation methods for road marking detection have been arising since they can generalize detection result well under complicated environments and hold rich pixel-level semantic information.Nevertheless,the previous methods mostly study the training process of the segmentation network,while omitting the time cost of manually annotating pixel-level data.Besides,the pixel-level semantic segmentation results need to be fitted into more reliable and compact models so that geometrical information of road markings can be explicitly obtained.In order to tackle the above problems,this paper describes a semantic segmentation-based road marking detection method using around view monitoring system.A semiautomatic semantic annotation platform is developed,which exploits an auxiliary segmentation graph to speed up the annotation process while guaranteeing the annotation accuracy.A segmentation-based detection module is also described,which models the semantic segmentation results for the more robust and compact analysis.The proposed detection module is composed of three parts:vote-based segmentation fusion filtering,graph-based road marking clustering,and road-marking fitting.Experiments under various scenarios show that the semantic segmentation-based detection method can achieve accurate,robust,and real-time detection performance.展开更多
In recent years,with the continuous deepening of smart city construction,there have been significant changes and improvements in the field of intelligent transportation.The semantic segmentation of road scenes has imp...In recent years,with the continuous deepening of smart city construction,there have been significant changes and improvements in the field of intelligent transportation.The semantic segmentation of road scenes has important practical significance in the fields of automatic driving,transportation planning,and intelligent transportation systems.However,the current mainstream lightweight semantic segmentation models in road scene segmentation face problems such as poor segmentation performance of small targets and insufficient refinement of segmentation edges.Therefore,this article proposes a lightweight semantic segmentation model based on the LiteSeg model improvement to address these issues.The model uses the lightweight backbone network MobileNet instead of the LiteSeg backbone network to reduce the network parameters and computation,and combines the Coordinate Attention(CA)mechanism to help the network capture long-distance dependencies.At the same time,by combining the dependencies of spatial information and channel information,the Spatial and Channel Network(SCNet)attention mechanism is proposed to improve the feature extraction ability of the model.Finally,a multiscale transposed attention encoding(MTAE)module was proposed to obtain features of different resolutions and perform feature fusion.In this paper,the proposed model is verified on the Cityscapes dataset.The experimental results show that the addition of SCNet and MTAE modules increases the mean Intersection over Union(mIoU)of the original LiteSeg model by 4.69%.On this basis,the backbone network is replaced with MobileNet,and the CA model is added at the same time.At the cost of increasing the minimum model parameters and computing costs,the mIoU of the original LiteSeg model is increased by 2.46%.This article also compares the proposed model with some current lightweight semantic segmentation models,and experiments show that the comprehensive performance of the proposed model is the best,especially in achieving excellent results in small object segmentation.Finally,this article will conduct generalization testing on the KITTI dataset for the proposed model,and the experimental results show that the proposed algorithm has a certain degree of generalization.展开更多
Neuronal soma segmentation plays a crucial role in neuroscience applications.However,the fine structure,such as boundaries,small-volume neuronal somata and fibers,are commonly present in cell images,which pose a chall...Neuronal soma segmentation plays a crucial role in neuroscience applications.However,the fine structure,such as boundaries,small-volume neuronal somata and fibers,are commonly present in cell images,which pose a challenge for accurate segmentation.In this paper,we propose a 3D semantic segmentation network for neuronal soma segmentation to address this issue.Using an encoding-decoding structure,we introduce a Multi-Scale feature extraction and Adaptive Weighting fusion module(MSAW)after each encoding block.The MSAW module can not only emphasize the fine structures via an upsampling strategy,but also provide pixel-wise weights to measure the importance of the multi-scale features.Additionally,a dynamic convolution instead of normal convolution is employed to better adapt the network to input data with different distributions.The proposed MSAW-based semantic segmentation network(MSAW-Net)was evaluated on three neuronal soma images from mouse brain and one neuronal soma image from macaque brain,demonstrating the efficiency of the proposed method.It achieved an F1 score of 91.8%on Fezf2-2A-CreER dataset,97.1%on LSL-H2B-GFP dataset,82.8%on Thy1-EGFP-Mline dataset,and 86.9%on macaque dataset,achieving improvements over the 3D U-Net model by 3.1%,3.3%,3.9%,and 2.3%,respectively.展开更多
Semantic segmentation in street scenes is a crucial technology for autonomous driving to analyze the surrounding environment.In street scenes,issues such as high image resolution caused by a large viewpoints and diffe...Semantic segmentation in street scenes is a crucial technology for autonomous driving to analyze the surrounding environment.In street scenes,issues such as high image resolution caused by a large viewpoints and differences in object scales lead to a decline in real-time performance and difficulties in multi-scale feature extraction.To address this,we propose a bilateral-branch real-time semantic segmentationmethod based on semantic information distillation(BSDNet)for street scene images.The BSDNet consists of a Feature Conversion Convolutional Block(FCB),a Semantic Information Distillation Module(SIDM),and a Deep Aggregation Atrous Convolution Pyramid Pooling(DASP).FCB reduces the semantic gap between the backbone and the semantic branch.SIDM extracts high-quality semantic information fromthe Transformer branch to reduce computational costs.DASP aggregates information lost in atrous convolutions,effectively capturingmulti-scale objects.Extensive experiments conducted on Cityscapes,CamVid,and ADE20K,achieving an accuracy of 81.7% Mean Intersection over Union(mIoU)at 70.6 Frames Per Second(FPS)on Cityscapes,demonstrate that our method achieves a better balance between accuracy and inference speed.展开更多
The detection of crack defects on the walls of road tunnels is a crucial step in the process of ensuring travel safetyand performing routine tunnel maintenance. The automatic and accurate detection of cracks on the su...The detection of crack defects on the walls of road tunnels is a crucial step in the process of ensuring travel safetyand performing routine tunnel maintenance. The automatic and accurate detection of cracks on the surface of roadtunnels is the key to improving the maintenance efficiency of road tunnels. Machine vision technology combinedwith a deep neural network model is an effective means to realize the localization and identification of crackdefects on the surface of road tunnels.We propose a complete set of automatic inspection methods for identifyingcracks on the walls of road tunnels as a solution to the problem of difficulty in identifying cracks during manualmaintenance. First, a set of equipment applied to the real-time acquisition of high-definition images of walls inroad tunnels is designed. Images of walls in road tunnels are acquired based on the designed equipment, whereimages containing crack defects are manually identified and selected. Subsequently, the training and validationsets used to construct the crack inspection model are obtained based on the acquired images, whereas the regionscontaining cracks and the pixels of the cracks are finely labeled. After that, a crack area sensing module is designedbased on the proposed you only look once version 7 model combined with coordinate attention mechanism (CAYOLOV7) network to locate the crack regions in the road tunnel surface images. Only subimages containingcracks are acquired and sent to the multiscale semantic segmentation module for extraction of the pixels to whichthe cracks belong based on the DeepLab V3+ network. The precision and recall of the crack region localizationon the surface of a road tunnel based on our proposed method are 82.4% and 93.8%, respectively. Moreover, themean intersection over union (MIoU) and pixel accuracy (PA) values for achieving pixel-level detection accuracyare 76.84% and 78.29%, respectively. The experimental results on the dataset show that our proposed two-stagedetection method outperforms other state-of-the-art models in crack region localization and detection. Based onour proposedmethod, the images captured on the surface of a road tunnel can complete crack detection at a speed often frames/second, and the detection accuracy can reach 0.25 mm, which meets the requirements for maintenanceof an actual project. The designed CA-YOLO V7 network enables precise localization of the area to which a crackbelongs in images acquired under different environmental and lighting conditions in road tunnels. The improvedDeepLab V3+ network based on lightweighting is able to extract crack morphology in a given region more quicklywhile maintaining segmentation accuracy. The established model combines defect localization and segmentationmodels for the first time, realizing pixel-level defect localization and extraction on the surface of road tunnelsin complex environments, and is capable of determining the actual size of cracks based on the physical coordinatesystemafter camera calibration. The trainedmodelhas highaccuracy andcanbe extendedandapplied to embeddedcomputing devices for the assessment and repair of damaged areas in different types of road tunnels.展开更多
According to the characteristics of the road features,an Encoder-Decoder deep semantic segmentation network is designed for the road extraction of remote sensing images.Firstly,as the features of the road target are r...According to the characteristics of the road features,an Encoder-Decoder deep semantic segmentation network is designed for the road extraction of remote sensing images.Firstly,as the features of the road target are rich in local details and simple in semantic features,an Encoder-Decoder network with shallow layers and high resolution is designed to improve the ability to represent detail information.Secondly,as the road area is a small proportion in remote sensing images,the cross-entropy loss function is improved,which solves the imbalance between positive and negative samples in the training process.Experiments on large road extraction datasets show that the proposed method gets the recall rate 83.9%,precision 82.5%and F1-score 82.9%,which can extract the road targets in remote sensing images completely and accurately.The Encoder-Decoder network designed in this paper performs well in the road extraction task and needs less artificial participation,so it has a good application prospect.展开更多
Significant advancements have been achieved in road surface extraction based on high-resolution remote sensingimage processing. Most current methods rely on fully supervised learning, which necessitates enormous human...Significant advancements have been achieved in road surface extraction based on high-resolution remote sensingimage processing. Most current methods rely on fully supervised learning, which necessitates enormous humaneffort to label the image. Within this field, other research endeavors utilize weakly supervised methods. Theseapproaches aim to reduce the expenses associated with annotation by leveraging sparsely annotated data, such asscribbles. This paper presents a novel technique called a weakly supervised network using scribble-supervised andedge-mask (WSSE-net). This network is a three-branch network architecture, whereby each branch is equippedwith a distinct decoder module dedicated to road extraction tasks. One of the branches is dedicated to generatingedge masks using edge detection algorithms and optimizing road edge details. The other two branches supervise themodel’s training by employing scribble labels and spreading scribble information throughout the image. To addressthe historical flaw that created pseudo-labels that are not updated with network training, we use mixup to blendprediction results dynamically and continually update new pseudo-labels to steer network training. Our solutiondemonstrates efficient operation by simultaneously considering both edge-mask aid and dynamic pseudo-labelsupport. The studies are conducted on three separate road datasets, which consist primarily of high-resolutionremote-sensing satellite photos and drone images. The experimental findings suggest that our methodologyperforms better than advanced scribble-supervised approaches and specific traditional fully supervised methods.展开更多
When existing deep learning models are used for road extraction tasks from high-resolution images,they are easily affected by noise factors such as tree and building occlusion and complex backgrounds,resulting in inco...When existing deep learning models are used for road extraction tasks from high-resolution images,they are easily affected by noise factors such as tree and building occlusion and complex backgrounds,resulting in incomplete road extraction and low accuracy.We propose the introduction of spatial and channel attention modules to the convolutional neural network ConvNeXt.Then,ConvNeXt is used as the backbone network,which cooperates with the perceptual analysis network UPerNet,retains the detection head of the semantic segmentation,and builds a new model ConvNeXt-UPerNet to suppress noise interference.Training on the open-source DeepGlobe and CHN6-CUG datasets and introducing the DiceLoss on the basis of CrossEntropyLoss solves the problem of positive and negative sample imbalance.Experimental results show that the new network model can achieve the following performance on the DeepGlobe dataset:79.40%for precision(Pre),97.93% for accuracy(Acc),69.28% for intersection over union(IoU),and 83.56% for mean intersection over union(MIoU).On the CHN6-CUG dataset,the model achieves the respective values of 78.17%for Pre,97.63%for Acc,65.4% for IoU,and 81.46% for MIoU.Compared with other network models,the fused ConvNeXt-UPerNet model can extract road information better when faced with the influence of noise contained in high-resolution remote sensing images.It also achieves multiscale image feature information with unified perception,ultimately improving the generalization ability of deep learning technology in extracting complex roads from high-resolution remote sensing images.展开更多
There is no unified planning standard for unstructured roads,and the morphological structures of these roads are complex and varied.It is important to maintain a balance between accuracy and speed for unstructured roa...There is no unified planning standard for unstructured roads,and the morphological structures of these roads are complex and varied.It is important to maintain a balance between accuracy and speed for unstructured road extraction models.Unstructured road extraction algorithms based on deep learning have problems such as high model complexity,high computational cost,and the inability to adapt to current edge computing devices.Therefore,it is best to use lightweight network models.Considering the need for lightweight models and the characteristics of unstructured roads with different pattern shapes,such as blocks and strips,a TMB(Triple Multi-Block)feature extraction module is proposed,and the overall structure of the TMBNet network is described.The TMB module was compared with SS-nbt,Non-bottleneck-1D,and other modules via experiments.The feasibility and effectiveness of the TMB module design were proven through experiments and visualizations.The comparison experiment,using multiple convolution kernel categories,proved that the TMB module can improve the segmentation accuracy of the network.The comparison with different semantic segmentation networks demonstrates that the TMBNet network has advantages in terms of unstructured road extraction.展开更多
Determining the navigation line is critical for the automatic navigation of agricultural robots in the farmland.In this research,considering a wheat field as the typical scenario,a novel navigation line extraction alg...Determining the navigation line is critical for the automatic navigation of agricultural robots in the farmland.In this research,considering a wheat field as the typical scenario,a novel navigation line extraction algorithm based on semantic segmentation is proposed.The data containing horizontal parallax,height,and grayscale information(HHG)is constructed by combining re-encoded depth data and red-green-blue(RGB)data.The HHG,RGB,and depth data are used to achieve scene recognition and navigation line extraction for a wheat field.The method includes two main steps.First,the semantic segmentation of the wheat,ground,and background are performed using a fully convolutional network(FCN).Second,the navigation line is fitted in the camera coordinate system on the basis of the semantic segmentation result and the principle of camera pinhole imaging.Our segmentation model is trained using 508 randomly selected images from a data set,and the model is tested on 199 images.When labelled data are used as the reference benchmark,the mean intersection over union(mIoU)of the HHG data is greater than 95%,which is the highest among the three types of data.The semantic segmentation methods based on the RGB and HHG data show higher navigation line extraction accuracy rates(with the absolute value of the angle deviation less than 5)than the compared methods.The mean and standard deviation of the angle deviation of the two methods are within 0.1and 2.0,while the mean and standard deviation of the distance deviation are less than 30 mm and 60 mm,respectively.These values meet the basic requirements of agricultural machinery field navigation.The novelty of this work is the proposal of a navigation line extraction algorithm based on semantic segmentation in wheat fields.This method is high in accuracy and robustness to interference from crop occlusion.展开更多
Geothermal resources are efficient,renewable and clean energy sources,and their reservoirs are usually closely associated with high-temperature regions of the land surface.Current exploration methods primarily involve...Geothermal resources are efficient,renewable and clean energy sources,and their reservoirs are usually closely associated with high-temperature regions of the land surface.Current exploration methods primarily involve migrating traditional geological techniques,which fail to fully use the unique features of geothermal radiation characteristics.Thermal infrared remote-sensing imaging technology can capture and present areas with distinctive surface thermal radiation features,providing considerable significance as a guide for localization prior tofield exploration.In this study,we propose a deep learningebased method for intelligently identifying and segmenting geothermal radiation sources from thermal infrared remote-sensing images,including data preparation and model training.To improve the localization drift and anomalous interference caused by the high complexity of the Earth's surface environment,this study uses a surface temperature retrieval algorithm to calculate the land surface temperature in the research area.The retrieval results are used to train the semantic segmentation model.In addition,a pixel-level geothermal spatial segmentation network(PGSSNet)is proposed to suppress the diffuse thermal radiation and reduce the broad and blurred white areas of images to exact locations.Once the training is completed,the model directly segments and extracts the actual range of thermal radiation sources from subsequent thermal infrared remote-sensing images without temperature retrieval and/or manual calibration.展开更多
This paper presents CW-HRNet,a high-resolution,lightweight crack segmentation network designed to address challenges in complex scenes with slender,deformable,and blurred crack structures.The model incorporates two ke...This paper presents CW-HRNet,a high-resolution,lightweight crack segmentation network designed to address challenges in complex scenes with slender,deformable,and blurred crack structures.The model incorporates two key modules:Constrained Deformable Convolution(CDC),which stabilizes geometric alignment by applying a tanh limiter and learnable scaling factor to the predicted offsets,and the Wavelet Frequency Enhancement Module(WFEM),which decomposes features using Haar wavelets to preserve low-frequency structures while enhancing high-frequency boundaries and textures.Evaluations on the CrackSeg9k benchmark demonstrate CW-HRNet’s superior performance,achieving 82.39%mIoU with only 7.49M parameters and 10.34 GFLOPs,outperforming HrSegNet-B48 by 1.83% in segmentation accuracy with minimal complexity overhead.The model also shows strong cross-dataset generalization,achieving 60.01%mIoU and 66.22%F1 on Asphalt3k without fine-tuning.These results highlight CW-HRNet’s favorable accuracyefficiency trade-off for real-world crack segmentation tasks.展开更多
Accurate vector extraction from design drawings is required first to automatically create 3D models from pixel-level engineering design drawings. However, this task faces the challenges of complicated design shapes as...Accurate vector extraction from design drawings is required first to automatically create 3D models from pixel-level engineering design drawings. However, this task faces the challenges of complicated design shapes as well as cumbersome and cluttered annotations on drawings, which interfere with the vector extraction heavily. In this article, the transmission tower containing the most complex structure is taken as the research object, and a semantic segmentation network is constructed to first segment the shape masks from the pixel-level drawings. Preprocessing and postprocessing are also proposed to ensure the stability and accuracy of the shape mask segmentation. Then, based on the obtained shape masks, a vector extraction network guided by heatmaps is designed to extract structural vectors by fusing the features from node heatmap and skeleton heatmap, respectively. Compared with the state-of-the-art methods, experiment results illustrate that the proposed semantic segmentation method can effectively eliminate the interference of many elements on drawings to segment the shape masks effectively, meanwhile, the model trained by the proposed vector extraction network can accurately extract the vectors such as nodes and line connections, avoiding redundant vector detection. The proposed method lays a solid foundation for automatic 3D model reconstruction and contributes to technological advancements in relevant fields.展开更多
In the management of land resources and the protection of cultivated land,the law enforcement of land satellite images is often used as one of the main means.In recent years,the policies and regulations of the law enf...In the management of land resources and the protection of cultivated land,the law enforcement of land satellite images is often used as one of the main means.In recent years,the policies and regulations of the law enforcement of land satellite images have become more and more strict and been adjusted increasingly frequently,playing a decisive role in preventing excessive non-agricultural and non-food urbanization.In the process of the law enforcement,the extraction of suspected illegal buildings is the most important and time-consuming content.Compared with the traditional deep learning model,fully convolutional networks(FCN)has a great advantage in remote sensing image processing because its input images are not limited by size,and both convolution and deconvolution are independent of the overall size of images.In this paper,an intelligent extraction model of suspected illegal buildings from land satellite images based on deep learning FCN was built.Kaiyuan City,Yunnan Province was taken as an example.The verification results show that the global accuracy of this model was 86.6%in the process of building extraction,and mean intersection over union(mIoU)was 73.6%.This study can provide reference for the extraction of suspected illegal buildings in the law enforcement work of land satellite images,and reduce the tedious manual operation to a certain extent.展开更多
Road geometric design data are a vital input for diverse transportation studies. This information is usually obtained from the road design project. However, these are not always available and the as-built course of th...Road geometric design data are a vital input for diverse transportation studies. This information is usually obtained from the road design project. However, these are not always available and the as-built course of the road may diverge considerably from its projected one, rendering subsequent studies inaccurate or impossible. Moreover, the systematic acquisition of this data for the entire road network of a country or even a state represents a very challenging and laborious task. This study＇s goal was the extraction of geometric design data for the paved segments of the Brazilian federal highway network, containing more than 47,000 km of highways. It presents the details of the method＇s adoption process, the particularities of its application to the dataset and the obtained geometric design information. Additionally, it provides a first overview of the Brazilian federal highway network composition （curves and tangents） and geometry.展开更多
Based on low-altitude remote sensing images,this paper established sample set of typical river vegetation elements and proposed river vegetation extraction technical solution to adaptively extract typical vegetation e...Based on low-altitude remote sensing images,this paper established sample set of typical river vegetation elements and proposed river vegetation extraction technical solution to adaptively extract typical vegetation elements of river basins.The main research of this paper were as follows:(1)a typical vegetation extraction sample set based on low-altitude remote sensing images was established.(2)A low-altitude remote sensing image vegetation extraction model based on the focus perception module was designed to realize the end-to-end automatic extraction of different types of vegetation areas of low-altitude remote sensing images to fully learn the spectral spatial texture information and deep semantic information of the images.(3)By comparison with the baseline method,baseline method with embedded focus perception module showed an improvement in the precision by 7.37%and mIoU by 49.49%.Through visual interpretation and quantitative calculation analysis,the typical river vegetation adaptive extraction network has effectiveness and generalization ability,consistent with the needs of practical applications of vegetation extraction.展开更多
Cultivated land extraction is essential for sustainable development and agriculture.In this paper,the network we propose is based on the encoder-decoder structure,which extracts the semantic segmentation neural networ...Cultivated land extraction is essential for sustainable development and agriculture.In this paper,the network we propose is based on the encoder-decoder structure,which extracts the semantic segmentation neural network of cultivated land from satellite images and uses it for agricultural automation solutions.The encoder consists of two part:the first is the modified Xception,it can used as the feature extraction network,and the second is the atrous convolution,it can used to expand the receptive field and the context information to extract richer feature information.The decoder part uses the conventional upsampling operation to restore the original resolution.In addition,we use the combination of BCE and Loves-hinge as a loss function to optimize the Intersection over Union(IoU).Experimental results show that the proposed network structure can solve the problem of cultivated land extraction in Yinchuan City.展开更多
The development of autonomous vehicles has become one of the greatest research endeavors in recent years. These vehicles rely on many complex systems working in tandem to make decisions. For practical use and safety r...The development of autonomous vehicles has become one of the greatest research endeavors in recent years. These vehicles rely on many complex systems working in tandem to make decisions. For practical use and safety reasons, these systems must not only be accurate, but also quickly detect changes in the surrounding environment. In autonomous vehicle research, the environment perception system is one of the key components of development. Environment perception systems allow the vehicle to understand its surroundings. This is done by using cameras, light detection and ranging (LiDAR), with other sensor systems and modalities. Deep learning computer vision algorithms have been shown to be the strongest tool for translating camera data into accurate and safe traversability decisions regarding the environment surrounding a vehicle. In order for a vehicle to safely traverse an area in real time, these computer vision algorithms must be accurate and have low latency. While much research has studied autonomous driving for traversing well-structured urban environments, limited research exists evaluating perception system improvements in off-road settings. This research aims to investigate the adaptability of several existing deep-learning architectures for semantic segmentation in off-road environments. Previous studies of two Convolutional Neural Network (CNN) architectures are included for comparison with new evaluation of Vision Transformer (ViT) architectures for semantic segmentation. Our results demonstrate viability of ViT architectures for off-road perception systems, having a strong segmentation accuracy, lower inference speed and memory footprint compared to previous results with CNN architectures.展开更多
基金National Natural Science Foundation of China(Nos.61941109,62061023)Distinguished Young Scholars of Gansu Province of China(No.21JR7RA345)。
文摘In the study of automatic driving,understanding the road scene is a key to improve driving safety.The semantic segmentation method could divide the image into different areas associated with semantic categories in accordance with the pixel level,so as to help vehicles to perceive and obtain the surrounding road environment information,which would improve driving safety.Deeplabv3+is the current popular semantic segmentation model.There are phenomena that small targets are missed and similar objects are easily misjudged during its semantic segmentation tasks,which leads to rough segmentation boundary and reduces semantic accuracy.This study focuses on the issue,based on the Deeplabv3+network structure and combined with the attention mechanism,to increase the weight of the segmentation area,and then proposes an improved Deeplabv3+fusion attention mechanism for road scene semantic segmentation method.First,a group of parallel position attention module and channel attention module are introduced on the Deeplabv3+encoding end to capture more spatial context information and high-level semantic information.Then,an attention mechanism is introduced to restore the spatial detail information,and the data shall be normalized in order to accelerate the convergence speed of the model at the decoding end.The effects of model segmentation with different attention-introducing mechanisms are compared and tested on CamVid and Cityscapes datasets.The experimental results show that the mean Intersection over Unons of the improved model segmentation accuracies on the two datasets are boosted by 6.88%and 2.58%,respectively,which is better than using Deeplabv3+.This method does not significantly increase the amount of network calculation and complexity,and has a good balance of speed and accuracy.
基金the National Natural Science Foundation of China(Nos.U1764264 and 61873165)the Shanghai Automotive Industry Science and Technology Development Foundation(No.1807)。
文摘Road marking detection is an important branch in autonomous driving,understanding the road information.In recent years,deep-learning-based semantic segmentation methods for road marking detection have been arising since they can generalize detection result well under complicated environments and hold rich pixel-level semantic information.Nevertheless,the previous methods mostly study the training process of the segmentation network,while omitting the time cost of manually annotating pixel-level data.Besides,the pixel-level semantic segmentation results need to be fitted into more reliable and compact models so that geometrical information of road markings can be explicitly obtained.In order to tackle the above problems,this paper describes a semantic segmentation-based road marking detection method using around view monitoring system.A semiautomatic semantic annotation platform is developed,which exploits an auxiliary segmentation graph to speed up the annotation process while guaranteeing the annotation accuracy.A segmentation-based detection module is also described,which models the semantic segmentation results for the more robust and compact analysis.The proposed detection module is composed of three parts:vote-based segmentation fusion filtering,graph-based road marking clustering,and road-marking fitting.Experiments under various scenarios show that the semantic segmentation-based detection method can achieve accurate,robust,and real-time detection performance.
基金the National Natural Science Foundation of China(No.62063006)the Natural Science Foundation of Guangxi Province(No.2023GXNSFAA026025)+3 种基金to the Innovation Fund of Chinese Universities Industry-University-Research(ID:2021RYC06005)to the Research Project for Young and Middle-Aged Teachers in Guangxi Universities(ID:2020KY15013)to the Special Research Project of Hechi University(ID:2021GCC028)supported by the Project of Outstanding Thousand Young Teachers’Training in Higher Education Institutions of Guangxi,Guangxi Colleges and Universities Key Laboratory of AI and Information Processing(Hechi University),Education Department of Guangxi Zhuang Autonomous Region.
文摘In recent years,with the continuous deepening of smart city construction,there have been significant changes and improvements in the field of intelligent transportation.The semantic segmentation of road scenes has important practical significance in the fields of automatic driving,transportation planning,and intelligent transportation systems.However,the current mainstream lightweight semantic segmentation models in road scene segmentation face problems such as poor segmentation performance of small targets and insufficient refinement of segmentation edges.Therefore,this article proposes a lightweight semantic segmentation model based on the LiteSeg model improvement to address these issues.The model uses the lightweight backbone network MobileNet instead of the LiteSeg backbone network to reduce the network parameters and computation,and combines the Coordinate Attention(CA)mechanism to help the network capture long-distance dependencies.At the same time,by combining the dependencies of spatial information and channel information,the Spatial and Channel Network(SCNet)attention mechanism is proposed to improve the feature extraction ability of the model.Finally,a multiscale transposed attention encoding(MTAE)module was proposed to obtain features of different resolutions and perform feature fusion.In this paper,the proposed model is verified on the Cityscapes dataset.The experimental results show that the addition of SCNet and MTAE modules increases the mean Intersection over Union(mIoU)of the original LiteSeg model by 4.69%.On this basis,the backbone network is replaced with MobileNet,and the CA model is added at the same time.At the cost of increasing the minimum model parameters and computing costs,the mIoU of the original LiteSeg model is increased by 2.46%.This article also compares the proposed model with some current lightweight semantic segmentation models,and experiments show that the comprehensive performance of the proposed model is the best,especially in achieving excellent results in small object segmentation.Finally,this article will conduct generalization testing on the KITTI dataset for the proposed model,and the experimental results show that the proposed algorithm has a certain degree of generalization.
基金supported by the STI2030-Major-Projects(No.2021ZD0200104)the National Natural Science Foundations of China under Grant 61771437.
文摘Neuronal soma segmentation plays a crucial role in neuroscience applications.However,the fine structure,such as boundaries,small-volume neuronal somata and fibers,are commonly present in cell images,which pose a challenge for accurate segmentation.In this paper,we propose a 3D semantic segmentation network for neuronal soma segmentation to address this issue.Using an encoding-decoding structure,we introduce a Multi-Scale feature extraction and Adaptive Weighting fusion module(MSAW)after each encoding block.The MSAW module can not only emphasize the fine structures via an upsampling strategy,but also provide pixel-wise weights to measure the importance of the multi-scale features.Additionally,a dynamic convolution instead of normal convolution is employed to better adapt the network to input data with different distributions.The proposed MSAW-based semantic segmentation network(MSAW-Net)was evaluated on three neuronal soma images from mouse brain and one neuronal soma image from macaque brain,demonstrating the efficiency of the proposed method.It achieved an F1 score of 91.8%on Fezf2-2A-CreER dataset,97.1%on LSL-H2B-GFP dataset,82.8%on Thy1-EGFP-Mline dataset,and 86.9%on macaque dataset,achieving improvements over the 3D U-Net model by 3.1%,3.3%,3.9%,and 2.3%,respectively.
基金supported in part by the National Natural Science Foundation of China[Grant number 62471075]the Major Science and Technology Project Grant of the Chongqing Municipal Education Commission[Grant number KJZD-M202301901]Graduate Innovation Fund of Chongqing[gzlcx20253235].
文摘Semantic segmentation in street scenes is a crucial technology for autonomous driving to analyze the surrounding environment.In street scenes,issues such as high image resolution caused by a large viewpoints and differences in object scales lead to a decline in real-time performance and difficulties in multi-scale feature extraction.To address this,we propose a bilateral-branch real-time semantic segmentationmethod based on semantic information distillation(BSDNet)for street scene images.The BSDNet consists of a Feature Conversion Convolutional Block(FCB),a Semantic Information Distillation Module(SIDM),and a Deep Aggregation Atrous Convolution Pyramid Pooling(DASP).FCB reduces the semantic gap between the backbone and the semantic branch.SIDM extracts high-quality semantic information fromthe Transformer branch to reduce computational costs.DASP aggregates information lost in atrous convolutions,effectively capturingmulti-scale objects.Extensive experiments conducted on Cityscapes,CamVid,and ADE20K,achieving an accuracy of 81.7% Mean Intersection over Union(mIoU)at 70.6 Frames Per Second(FPS)on Cityscapes,demonstrate that our method achieves a better balance between accuracy and inference speed.
基金the Changsha Science and Technology Plan 2004081in part by the Science and Technology Program of Hunan Provincial Department of Transportation 202117in part by the Science and Technology Research and Development Program Project of the China Railway Group Limited 2021-Special-08.
文摘The detection of crack defects on the walls of road tunnels is a crucial step in the process of ensuring travel safetyand performing routine tunnel maintenance. The automatic and accurate detection of cracks on the surface of roadtunnels is the key to improving the maintenance efficiency of road tunnels. Machine vision technology combinedwith a deep neural network model is an effective means to realize the localization and identification of crackdefects on the surface of road tunnels.We propose a complete set of automatic inspection methods for identifyingcracks on the walls of road tunnels as a solution to the problem of difficulty in identifying cracks during manualmaintenance. First, a set of equipment applied to the real-time acquisition of high-definition images of walls inroad tunnels is designed. Images of walls in road tunnels are acquired based on the designed equipment, whereimages containing crack defects are manually identified and selected. Subsequently, the training and validationsets used to construct the crack inspection model are obtained based on the acquired images, whereas the regionscontaining cracks and the pixels of the cracks are finely labeled. After that, a crack area sensing module is designedbased on the proposed you only look once version 7 model combined with coordinate attention mechanism (CAYOLOV7) network to locate the crack regions in the road tunnel surface images. Only subimages containingcracks are acquired and sent to the multiscale semantic segmentation module for extraction of the pixels to whichthe cracks belong based on the DeepLab V3+ network. The precision and recall of the crack region localizationon the surface of a road tunnel based on our proposed method are 82.4% and 93.8%, respectively. Moreover, themean intersection over union (MIoU) and pixel accuracy (PA) values for achieving pixel-level detection accuracyare 76.84% and 78.29%, respectively. The experimental results on the dataset show that our proposed two-stagedetection method outperforms other state-of-the-art models in crack region localization and detection. Based onour proposedmethod, the images captured on the surface of a road tunnel can complete crack detection at a speed often frames/second, and the detection accuracy can reach 0.25 mm, which meets the requirements for maintenanceof an actual project. The designed CA-YOLO V7 network enables precise localization of the area to which a crackbelongs in images acquired under different environmental and lighting conditions in road tunnels. The improvedDeepLab V3+ network based on lightweighting is able to extract crack morphology in a given region more quicklywhile maintaining segmentation accuracy. The established model combines defect localization and segmentationmodels for the first time, realizing pixel-level defect localization and extraction on the surface of road tunnelsin complex environments, and is capable of determining the actual size of cracks based on the physical coordinatesystemafter camera calibration. The trainedmodelhas highaccuracy andcanbe extendedandapplied to embeddedcomputing devices for the assessment and repair of damaged areas in different types of road tunnels.
基金National Natural Science Foundation of China(Nos.61673017,61403398)and Natural Science Foundation of Shaanxi Province(Nos.2017JM6077,2018ZDXM-GY-039)。
文摘According to the characteristics of the road features,an Encoder-Decoder deep semantic segmentation network is designed for the road extraction of remote sensing images.Firstly,as the features of the road target are rich in local details and simple in semantic features,an Encoder-Decoder network with shallow layers and high resolution is designed to improve the ability to represent detail information.Secondly,as the road area is a small proportion in remote sensing images,the cross-entropy loss function is improved,which solves the imbalance between positive and negative samples in the training process.Experiments on large road extraction datasets show that the proposed method gets the recall rate 83.9%,precision 82.5%and F1-score 82.9%,which can extract the road targets in remote sensing images completely and accurately.The Encoder-Decoder network designed in this paper performs well in the road extraction task and needs less artificial participation,so it has a good application prospect.
基金the National Natural Science Foundation of China(42001408,61806097).
文摘Significant advancements have been achieved in road surface extraction based on high-resolution remote sensingimage processing. Most current methods rely on fully supervised learning, which necessitates enormous humaneffort to label the image. Within this field, other research endeavors utilize weakly supervised methods. Theseapproaches aim to reduce the expenses associated with annotation by leveraging sparsely annotated data, such asscribbles. This paper presents a novel technique called a weakly supervised network using scribble-supervised andedge-mask (WSSE-net). This network is a three-branch network architecture, whereby each branch is equippedwith a distinct decoder module dedicated to road extraction tasks. One of the branches is dedicated to generatingedge masks using edge detection algorithms and optimizing road edge details. The other two branches supervise themodel’s training by employing scribble labels and spreading scribble information throughout the image. To addressthe historical flaw that created pseudo-labels that are not updated with network training, we use mixup to blendprediction results dynamically and continually update new pseudo-labels to steer network training. Our solutiondemonstrates efficient operation by simultaneously considering both edge-mask aid and dynamic pseudo-labelsupport. The studies are conducted on three separate road datasets, which consist primarily of high-resolutionremote-sensing satellite photos and drone images. The experimental findings suggest that our methodologyperforms better than advanced scribble-supervised approaches and specific traditional fully supervised methods.
基金This work was supported in part by the Key Project of Natural Science Research of Anhui Provincial Department of Education under Grant KJ2017A416in part by the Fund of National Sensor Network Engineering Technology Research Center(No.NSNC202103).
文摘When existing deep learning models are used for road extraction tasks from high-resolution images,they are easily affected by noise factors such as tree and building occlusion and complex backgrounds,resulting in incomplete road extraction and low accuracy.We propose the introduction of spatial and channel attention modules to the convolutional neural network ConvNeXt.Then,ConvNeXt is used as the backbone network,which cooperates with the perceptual analysis network UPerNet,retains the detection head of the semantic segmentation,and builds a new model ConvNeXt-UPerNet to suppress noise interference.Training on the open-source DeepGlobe and CHN6-CUG datasets and introducing the DiceLoss on the basis of CrossEntropyLoss solves the problem of positive and negative sample imbalance.Experimental results show that the new network model can achieve the following performance on the DeepGlobe dataset:79.40%for precision(Pre),97.93% for accuracy(Acc),69.28% for intersection over union(IoU),and 83.56% for mean intersection over union(MIoU).On the CHN6-CUG dataset,the model achieves the respective values of 78.17%for Pre,97.63%for Acc,65.4% for IoU,and 81.46% for MIoU.Compared with other network models,the fused ConvNeXt-UPerNet model can extract road information better when faced with the influence of noise contained in high-resolution remote sensing images.It also achieves multiscale image feature information with unified perception,ultimately improving the generalization ability of deep learning technology in extracting complex roads from high-resolution remote sensing images.
基金Supported by National Natural Science Foundation of China(Grant Nos.62261160575,61991414,61973036)Technical Field Foundation of the National Defense Science and Technology 173 Program of China(Grant Nos.20220601053,20220601030)。
文摘There is no unified planning standard for unstructured roads,and the morphological structures of these roads are complex and varied.It is important to maintain a balance between accuracy and speed for unstructured road extraction models.Unstructured road extraction algorithms based on deep learning have problems such as high model complexity,high computational cost,and the inability to adapt to current edge computing devices.Therefore,it is best to use lightweight network models.Considering the need for lightweight models and the characteristics of unstructured roads with different pattern shapes,such as blocks and strips,a TMB(Triple Multi-Block)feature extraction module is proposed,and the overall structure of the TMBNet network is described.The TMB module was compared with SS-nbt,Non-bottleneck-1D,and other modules via experiments.The feasibility and effectiveness of the TMB module design were proven through experiments and visualizations.The comparison experiment,using multiple convolution kernel categories,proved that the TMB module can improve the segmentation accuracy of the network.The comparison with different semantic segmentation networks demonstrates that the TMBNet network has advantages in terms of unstructured road extraction.
基金supported by the National Natural Science Foundation of China(No.61503363).
文摘Determining the navigation line is critical for the automatic navigation of agricultural robots in the farmland.In this research,considering a wheat field as the typical scenario,a novel navigation line extraction algorithm based on semantic segmentation is proposed.The data containing horizontal parallax,height,and grayscale information(HHG)is constructed by combining re-encoded depth data and red-green-blue(RGB)data.The HHG,RGB,and depth data are used to achieve scene recognition and navigation line extraction for a wheat field.The method includes two main steps.First,the semantic segmentation of the wheat,ground,and background are performed using a fully convolutional network(FCN).Second,the navigation line is fitted in the camera coordinate system on the basis of the semantic segmentation result and the principle of camera pinhole imaging.Our segmentation model is trained using 508 randomly selected images from a data set,and the model is tested on 199 images.When labelled data are used as the reference benchmark,the mean intersection over union(mIoU)of the HHG data is greater than 95%,which is the highest among the three types of data.The semantic segmentation methods based on the RGB and HHG data show higher navigation line extraction accuracy rates(with the absolute value of the angle deviation less than 5)than the compared methods.The mean and standard deviation of the angle deviation of the two methods are within 0.1and 2.0,while the mean and standard deviation of the distance deviation are less than 30 mm and 60 mm,respectively.These values meet the basic requirements of agricultural machinery field navigation.The novelty of this work is the proposal of a navigation line extraction algorithm based on semantic segmentation in wheat fields.This method is high in accuracy and robustness to interference from crop occlusion.
基金supported by National Natural Science Foundation(NNSF)of China under Grant 41927801.
文摘Geothermal resources are efficient,renewable and clean energy sources,and their reservoirs are usually closely associated with high-temperature regions of the land surface.Current exploration methods primarily involve migrating traditional geological techniques,which fail to fully use the unique features of geothermal radiation characteristics.Thermal infrared remote-sensing imaging technology can capture and present areas with distinctive surface thermal radiation features,providing considerable significance as a guide for localization prior tofield exploration.In this study,we propose a deep learningebased method for intelligently identifying and segmenting geothermal radiation sources from thermal infrared remote-sensing images,including data preparation and model training.To improve the localization drift and anomalous interference caused by the high complexity of the Earth's surface environment,this study uses a surface temperature retrieval algorithm to calculate the land surface temperature in the research area.The retrieval results are used to train the semantic segmentation model.In addition,a pixel-level geothermal spatial segmentation network(PGSSNet)is proposed to suppress the diffuse thermal radiation and reduce the broad and blurred white areas of images to exact locations.Once the training is completed,the model directly segments and extracts the actual range of thermal radiation sources from subsequent thermal infrared remote-sensing images without temperature retrieval and/or manual calibration.
文摘This paper presents CW-HRNet,a high-resolution,lightweight crack segmentation network designed to address challenges in complex scenes with slender,deformable,and blurred crack structures.The model incorporates two key modules:Constrained Deformable Convolution(CDC),which stabilizes geometric alignment by applying a tanh limiter and learnable scaling factor to the predicted offsets,and the Wavelet Frequency Enhancement Module(WFEM),which decomposes features using Haar wavelets to preserve low-frequency structures while enhancing high-frequency boundaries and textures.Evaluations on the CrackSeg9k benchmark demonstrate CW-HRNet’s superior performance,achieving 82.39%mIoU with only 7.49M parameters and 10.34 GFLOPs,outperforming HrSegNet-B48 by 1.83% in segmentation accuracy with minimal complexity overhead.The model also shows strong cross-dataset generalization,achieving 60.01%mIoU and 66.22%F1 on Asphalt3k without fine-tuning.These results highlight CW-HRNet’s favorable accuracyefficiency trade-off for real-world crack segmentation tasks.
基金funded by the Chinese State Grid Jiangsu Electric Power Co.,Ltd.Science and Technology Project Funding,Grant Number J2023031.
文摘Accurate vector extraction from design drawings is required first to automatically create 3D models from pixel-level engineering design drawings. However, this task faces the challenges of complicated design shapes as well as cumbersome and cluttered annotations on drawings, which interfere with the vector extraction heavily. In this article, the transmission tower containing the most complex structure is taken as the research object, and a semantic segmentation network is constructed to first segment the shape masks from the pixel-level drawings. Preprocessing and postprocessing are also proposed to ensure the stability and accuracy of the shape mask segmentation. Then, based on the obtained shape masks, a vector extraction network guided by heatmaps is designed to extract structural vectors by fusing the features from node heatmap and skeleton heatmap, respectively. Compared with the state-of-the-art methods, experiment results illustrate that the proposed semantic segmentation method can effectively eliminate the interference of many elements on drawings to segment the shape masks effectively, meanwhile, the model trained by the proposed vector extraction network can accurately extract the vectors such as nodes and line connections, avoiding redundant vector detection. The proposed method lays a solid foundation for automatic 3D model reconstruction and contributes to technological advancements in relevant fields.
文摘In the management of land resources and the protection of cultivated land,the law enforcement of land satellite images is often used as one of the main means.In recent years,the policies and regulations of the law enforcement of land satellite images have become more and more strict and been adjusted increasingly frequently,playing a decisive role in preventing excessive non-agricultural and non-food urbanization.In the process of the law enforcement,the extraction of suspected illegal buildings is the most important and time-consuming content.Compared with the traditional deep learning model,fully convolutional networks(FCN)has a great advantage in remote sensing image processing because its input images are not limited by size,and both convolution and deconvolution are independent of the overall size of images.In this paper,an intelligent extraction model of suspected illegal buildings from land satellite images based on deep learning FCN was built.Kaiyuan City,Yunnan Province was taken as an example.The verification results show that the global accuracy of this model was 86.6%in the process of building extraction,and mean intersection over union(mIoU)was 73.6%.This study can provide reference for the extraction of suspected illegal buildings in the law enforcement work of land satellite images,and reduce the tedious manual operation to a certain extent.
文摘Road geometric design data are a vital input for diverse transportation studies. This information is usually obtained from the road design project. However, these are not always available and the as-built course of the road may diverge considerably from its projected one, rendering subsequent studies inaccurate or impossible. Moreover, the systematic acquisition of this data for the entire road network of a country or even a state represents a very challenging and laborious task. This study＇s goal was the extraction of geometric design data for the paved segments of the Brazilian federal highway network, containing more than 47,000 km of highways. It presents the details of the method＇s adoption process, the particularities of its application to the dataset and the obtained geometric design information. Additionally, it provides a first overview of the Brazilian federal highway network composition （curves and tangents） and geometry.
文摘Based on low-altitude remote sensing images,this paper established sample set of typical river vegetation elements and proposed river vegetation extraction technical solution to adaptively extract typical vegetation elements of river basins.The main research of this paper were as follows:(1)a typical vegetation extraction sample set based on low-altitude remote sensing images was established.(2)A low-altitude remote sensing image vegetation extraction model based on the focus perception module was designed to realize the end-to-end automatic extraction of different types of vegetation areas of low-altitude remote sensing images to fully learn the spectral spatial texture information and deep semantic information of the images.(3)By comparison with the baseline method,baseline method with embedded focus perception module showed an improvement in the precision by 7.37%and mIoU by 49.49%.Through visual interpretation and quantitative calculation analysis,the typical river vegetation adaptive extraction network has effectiveness and generalization ability,consistent with the needs of practical applications of vegetation extraction.
基金support for this work are as follows:Ningxia Hui Autonomous Region Key Research and Development Program Project:Research and demonstration application of key technologies for intelligent monitoring of spatial planning based on high-scoring remote sensing(Project No.2018YBZD1629).
文摘Cultivated land extraction is essential for sustainable development and agriculture.In this paper,the network we propose is based on the encoder-decoder structure,which extracts the semantic segmentation neural network of cultivated land from satellite images and uses it for agricultural automation solutions.The encoder consists of two part:the first is the modified Xception,it can used as the feature extraction network,and the second is the atrous convolution,it can used to expand the receptive field and the context information to extract richer feature information.The decoder part uses the conventional upsampling operation to restore the original resolution.In addition,we use the combination of BCE and Loves-hinge as a loss function to optimize the Intersection over Union(IoU).Experimental results show that the proposed network structure can solve the problem of cultivated land extraction in Yinchuan City.
文摘The development of autonomous vehicles has become one of the greatest research endeavors in recent years. These vehicles rely on many complex systems working in tandem to make decisions. For practical use and safety reasons, these systems must not only be accurate, but also quickly detect changes in the surrounding environment. In autonomous vehicle research, the environment perception system is one of the key components of development. Environment perception systems allow the vehicle to understand its surroundings. This is done by using cameras, light detection and ranging (LiDAR), with other sensor systems and modalities. Deep learning computer vision algorithms have been shown to be the strongest tool for translating camera data into accurate and safe traversability decisions regarding the environment surrounding a vehicle. In order for a vehicle to safely traverse an area in real time, these computer vision algorithms must be accurate and have low latency. While much research has studied autonomous driving for traversing well-structured urban environments, limited research exists evaluating perception system improvements in off-road settings. This research aims to investigate the adaptability of several existing deep-learning architectures for semantic segmentation in off-road environments. Previous studies of two Convolutional Neural Network (CNN) architectures are included for comparison with new evaluation of Vision Transformer (ViT) architectures for semantic segmentation. Our results demonstrate viability of ViT architectures for off-road perception systems, having a strong segmentation accuracy, lower inference speed and memory footprint compared to previous results with CNN architectures.
