Because pixel values of foggy images are irregularly higher than those of images captured in normal weather(clear images),it is difficult to extract and express their texture.No method has previously been developed to...Because pixel values of foggy images are irregularly higher than those of images captured in normal weather(clear images),it is difficult to extract and express their texture.No method has previously been developed to directly explore the relationship between foggy images and semantic segmentation images.We investigated this relationship and propose a generative adversarial network(GAN)for foggy image semantic segmentation(FISS GAN),which contains two parts:an edge GAN and a semantic segmentation GAN.The edge GAN is designed to generate edge information from foggy images to provide auxiliary information to the semantic segmentation GAN.The semantic segmentation GAN is designed to extract and express the texture of foggy images and generate semantic segmentation images.Experiments on foggy cityscapes datasets and foggy driving datasets indicated that FISS GAN achieved state-of-the-art performance.展开更多
Image semantic segmentation is an essential technique for studying human behavior through image data.This paper proposes an image semantic segmentation method for human behavior research.Firstly,an end-to-end convolut...Image semantic segmentation is an essential technique for studying human behavior through image data.This paper proposes an image semantic segmentation method for human behavior research.Firstly,an end-to-end convolutional neural network architecture is proposed,which consists of a depth-separable jump-connected fully convolutional network and a conditional random field network;then jump-connected convolution is used to classify each pixel in the image,and an image semantic segmentation method based on convolu-tional neural network is proposed;and then a conditional random field network is used to improve the effect of image segmentation of hu-man behavior and a linear modeling and nonlinear modeling method based on the semantic segmentation of conditional random field im-age is proposed.Finally,using the proposed image segmentation network,the input entrepreneurial image data is semantically segmented to obtain the contour features of the person;and the segmentation of the images in the medical field.The experimental results show that the image semantic segmentation method is effective.It is a new way to use image data to study human behavior and can be extended to other research areas.展开更多
Remote sensing images contain a wealth of geospatial information.To accurately identify different geospatial categories and extract relevant data,image semantic segmentation plays a crucial role.In recent years,deep l...Remote sensing images contain a wealth of geospatial information.To accurately identify different geospatial categories and extract relevant data,image semantic segmentation plays a crucial role.In recent years,deep learning technology has brought significant breakthroughs to semantic segmentation of remote sensing images,significantly enhancing its performance.This paper investigates the application of deep learning technologies in remote sensing image semantic segmentation,based on Convolutional Neural Networks(CNN)and Transformer-based semantic segmentation methods.It conducts an in-depth comparison of their structural characteristics and applicable scenarios,summarizes the achievements and shortcomings of existing research,and provides technical references and theoretical support for future studies,thereby contributing to the further development of deep learning technology in the field of remote sensing.Research results indicate that CNN-based semantic segmentation methods still hold advantages in extracting local features and achieving efficient segmentation,whereas Transformers address CNN's limitations in global context modeling and long-range dependency capture.Therefore,the collaborative integration of CNN and Transformers will become an important research direction for enhancing model performance in the future.展开更多
Weed management plays a crucial role in increasing crop yields.Semantic segmentation,which classifies each pixel in an image captured by a camera into categories such as crops,weeds,and background,is a widely used met...Weed management plays a crucial role in increasing crop yields.Semantic segmentation,which classifies each pixel in an image captured by a camera into categories such as crops,weeds,and background,is a widely used method in this context.However,conventional semantic segmentation methods rely solely on pixel information within the camera's field of view(FOV),hindering their ability to detect weeds outside the visible area.This limitation can lead to incomplete weed removal and inefficient herbicide application.Incorporating information beyond the FOV in crop and weed segmentation is therefore essential for effective herbicide usage.Nevertheless,existing research on crop and weed segmentation has largely overlooked this limitation.To address this issue,we propose the knowledge distillation-based outpainting and semantic segmentation network(KDOSS-Net)for crop and weed images,a novel framework that enhances segmentation accuracy by leveraging information beyond the FOV.KDOSS-Net consists of two parts:the object prediction-guided outpainting and semantic segmentation network(OPOSS-Net),which serves as the teacher model by restoring areas outside the FOV and performing semantic segmentation,and the semantic segmentation without outpainting network(SSWO-Net),which serves as the student model,directly performing segmentation without outpainting.Through knowledge distillation(KD),the student model learns from the teacher's outputs,which results in a lightweight yet highly accurate segmentation network that is suitable for deployment on agricultural robots with limited computing power.Experiments on three public datasets-Rice seedling and weed,CWFID,and BoniRob-yielded mean intersection over union(mIOU)scores of 0.6315,0.7101,and 0.7524,respectively.These results demonstrate that KDOSS-Net achieves higher accuracy than existing state-of-the-art(SOTA)segmentation models while significantly reducing computational overhead.Furthermore,the weed information extracted using our method is automatically linked as input to the open-source large language and vision assistant(LLaVA),enabling the development of a system that recommends optimal herbicide strategies tailored to the detected weed class.展开更多
As simulation-informed design gains importance in addressing urban complexity,integrating urban imagery into interactive feedback and decision-making has become increasingly essential.However,this potential remains un...As simulation-informed design gains importance in addressing urban complexity,integrating urban imagery into interactive feedback and decision-making has become increasingly essential.However,this potential remains underused,as urban imagery is often treated as a supporting variable in urban research rather than a core layer of spatial intelligence,hindering informed strategies in city branding,resource allocation,and livability.This study develops a data-driven framework,Street View Search Engine,which integrates urban imagery analysis with interactive exploration to advance human-centered insights into urban visual form.Based on 81,478 street view imagery collected in Hong Kong,China,a dataset comprising 19 visual features was first constructed to represent urban visual information across three categories:physical,impression,and isovist.Subsequently,the machine learning algorithm self-organizing maps was employed to train the dataset,producing a visualized“data landscape”that re-organizes street views according to their visual similarities.Third,building on the data landscape,this study develops the Street View Search Engine framework to conduct three main tasks:define visual foundations,comprehend streetscape morphology,and evaluate regional visual schemes.These tasks combine general-use exploration with research-oriented analysis:a web-based platform was developed to support general-use exploration(http://47.113.226.77/project1/#/),while various data processing methods were employed to enable in-depth professional investigations.By transforming raw data into a visualizable,computable,and interactive urban imagery system,this study paves the way for evidence-based interventions,strategic resource allocation,and greater public engagement in urban planning.展开更多
Coal-rock interface identification technology was pivotal in automatically adjusting the shearer's cutting drum during coal mining.However,it also served as a technical bottleneck hindering the advancement of inte...Coal-rock interface identification technology was pivotal in automatically adjusting the shearer's cutting drum during coal mining.However,it also served as a technical bottleneck hindering the advancement of intelligent coal mining.This study aimed to address the poor accuracy of current coal-rock identification technology on comprehensive working faces,coupled with the limited availability of coal-rock datasets.The loss function of the SegFormer model was enhanced,the model's hyperparameters and learning rate were adjusted,and an automatic recognition method was proposed for coal-rock interfaces based on FL-SegFormer.Additionally,an experimental platform was constructed to simulate the dusty environment during coal-rock cutting by the shearer,enabling the collection of coal-rock test image datasets.The morphology-based algorithms were employed to expand the coal-rock image datasets through image rotation,color dithering,and Gaussian noise injection so as to augment the diversity and applicability of the datasets.As a result,a coal-rock image dataset comprising 8424 samples was generated.The findings demonstrated that the FL-SegFormer model achieved a Mean Intersection over Union(MIoU)and mean pixel accuracy(MPA)of 97.72%and 98.83%,respectively.The FLSegFormer model outperformed other models in terms of recognition accuracy,as evidenced by an MloU exceeding 95.70% of the original image.Furthermore,the FL-SegFormer model using original coal-rock images was validated from No.15205 working face of the Yulin test mine in northern Shaanxi.The calculated average error was only 1.77%,and the model operated at a rate of 46.96 frames per second,meeting the practical application and deployment requirements in underground settings.These results provided a theoretical foundation for achieving automatic and efficient mining with coal mining machines and the intelligent development of coal mines.展开更多
Semantic image parsing, which refers to the pro- cess of decomposing images into semantic regions and constructing the structure representation of the input, has re- cently aroused widespread interest in the field of ...Semantic image parsing, which refers to the pro- cess of decomposing images into semantic regions and constructing the structure representation of the input, has re- cently aroused widespread interest in the field of computer vision. The recent application of deep representation learning has driven this field into a new stage of development. In this paper, we summarize three aspects of the progress of research on semantic image parsing, i.e., category-level semantic segmentation, instance-level semantic segmentation, and beyond segmentation. Specifically, we first review the general frameworks for each task and introduce the relevant variants. The advantages and limitations of each method are also discussed. Moreover, we present a comprehensive comparison of different benchmark datasets and evaluation metrics. Finally, we explore the future trends and challenges of semantic image parsing.展开更多
Ephemeral gullies are widely distributed in the hilly and gully region of the Loess Plateau and play a unique role in the slope gully erosion system.Rapid and accurate identification of ephemeral gullies impacts the d...Ephemeral gullies are widely distributed in the hilly and gully region of the Loess Plateau and play a unique role in the slope gully erosion system.Rapid and accurate identification of ephemeral gullies impacts the distribution law and development trend of soil erosion on the Loess Plateau.Deep learning algorithms can quickly and accurately process large data samples that recognize ephemeral gullies from remote sensing images.Here,we investigated ephemeral gullies in the Zhoutungou watershed in the hilly and gully region of the Loess Plateau in China using satellite and unmanned aerial vehicle images and combined a deep learning image semantic segmentation model to realize automatic recognition and feature extraction.Using Accuracy,Precision,Recall,F1value,and AUC,we compared the ephemeral gully recognition results and accuracy evaluation of U-Net,R2U-Net,and SegNet image semantic segmentation models.The SegNet model was ranked first,followed by the R2U-Net and U-Net models,for ephemeral gully recognition in the hilly and gully region of the Loess Plateau.The ephemeral gully length and width between predicted and measured values had RMSE values of 6.78 m and 0.50 m,respectively,indicating that the model has an excellent recognition effect.This study identified a fast and accurate method for ephemeral gully recognition in the hilly and gully region of the Loess Plateau based on remote sensing images to provide an academic reference and practical guidance for soil erosion monitoring and slope and gully management in the Loess Plateau region.展开更多
基金supported in part by the National Key Research and Development Program of China(2018YFB1305002)the National Natural Science Foundation of China(62006256)+2 种基金the Postdoctoral Science Foundation of China(2020M683050)the Key Research and Development Program of Guangzhou(202007050002)the Fundamental Research Funds for the Central Universities(67000-31610134)。
文摘Because pixel values of foggy images are irregularly higher than those of images captured in normal weather(clear images),it is difficult to extract and express their texture.No method has previously been developed to directly explore the relationship between foggy images and semantic segmentation images.We investigated this relationship and propose a generative adversarial network(GAN)for foggy image semantic segmentation(FISS GAN),which contains two parts:an edge GAN and a semantic segmentation GAN.The edge GAN is designed to generate edge information from foggy images to provide auxiliary information to the semantic segmentation GAN.The semantic segmentation GAN is designed to extract and express the texture of foggy images and generate semantic segmentation images.Experiments on foggy cityscapes datasets and foggy driving datasets indicated that FISS GAN achieved state-of-the-art performance.
基金Supported by the Major Consulting and Research Project of the Chinese Academy of Engineering(2020-CQ-ZD-1)the National Natural Science Foundation of China(72101235)Zhejiang Soft Science Research Program(2023C35012)。
文摘Image semantic segmentation is an essential technique for studying human behavior through image data.This paper proposes an image semantic segmentation method for human behavior research.Firstly,an end-to-end convolutional neural network architecture is proposed,which consists of a depth-separable jump-connected fully convolutional network and a conditional random field network;then jump-connected convolution is used to classify each pixel in the image,and an image semantic segmentation method based on convolu-tional neural network is proposed;and then a conditional random field network is used to improve the effect of image segmentation of hu-man behavior and a linear modeling and nonlinear modeling method based on the semantic segmentation of conditional random field im-age is proposed.Finally,using the proposed image segmentation network,the input entrepreneurial image data is semantically segmented to obtain the contour features of the person;and the segmentation of the images in the medical field.The experimental results show that the image semantic segmentation method is effective.It is a new way to use image data to study human behavior and can be extended to other research areas.
文摘Remote sensing images contain a wealth of geospatial information.To accurately identify different geospatial categories and extract relevant data,image semantic segmentation plays a crucial role.In recent years,deep learning technology has brought significant breakthroughs to semantic segmentation of remote sensing images,significantly enhancing its performance.This paper investigates the application of deep learning technologies in remote sensing image semantic segmentation,based on Convolutional Neural Networks(CNN)and Transformer-based semantic segmentation methods.It conducts an in-depth comparison of their structural characteristics and applicable scenarios,summarizes the achievements and shortcomings of existing research,and provides technical references and theoretical support for future studies,thereby contributing to the further development of deep learning technology in the field of remote sensing.Research results indicate that CNN-based semantic segmentation methods still hold advantages in extracting local features and achieving efficient segmentation,whereas Transformers address CNN's limitations in global context modeling and long-range dependency capture.Therefore,the collaborative integration of CNN and Transformers will become an important research direction for enhancing model performance in the future.
基金This work was supported in part by the Ministry of Science and ICT(MSIT),Korea,through the Information Technology Research Center(ITRC)Support Program under Grant IITP-2025-RS-2020-II201789in part by the Artificial Intelligence Convergence Innovation Human Resources Development Supervised by the Institute of Information&Communications Technology Planning&Evaluation(IITP)under Grant IITP-2025-RS-2023-00254592.
文摘Weed management plays a crucial role in increasing crop yields.Semantic segmentation,which classifies each pixel in an image captured by a camera into categories such as crops,weeds,and background,is a widely used method in this context.However,conventional semantic segmentation methods rely solely on pixel information within the camera's field of view(FOV),hindering their ability to detect weeds outside the visible area.This limitation can lead to incomplete weed removal and inefficient herbicide application.Incorporating information beyond the FOV in crop and weed segmentation is therefore essential for effective herbicide usage.Nevertheless,existing research on crop and weed segmentation has largely overlooked this limitation.To address this issue,we propose the knowledge distillation-based outpainting and semantic segmentation network(KDOSS-Net)for crop and weed images,a novel framework that enhances segmentation accuracy by leveraging information beyond the FOV.KDOSS-Net consists of two parts:the object prediction-guided outpainting and semantic segmentation network(OPOSS-Net),which serves as the teacher model by restoring areas outside the FOV and performing semantic segmentation,and the semantic segmentation without outpainting network(SSWO-Net),which serves as the student model,directly performing segmentation without outpainting.Through knowledge distillation(KD),the student model learns from the teacher's outputs,which results in a lightweight yet highly accurate segmentation network that is suitable for deployment on agricultural robots with limited computing power.Experiments on three public datasets-Rice seedling and weed,CWFID,and BoniRob-yielded mean intersection over union(mIOU)scores of 0.6315,0.7101,and 0.7524,respectively.These results demonstrate that KDOSS-Net achieves higher accuracy than existing state-of-the-art(SOTA)segmentation models while significantly reducing computational overhead.Furthermore,the weed information extracted using our method is automatically linked as input to the open-source large language and vision assistant(LLaVA),enabling the development of a system that recommends optimal herbicide strategies tailored to the detected weed class.
基金supported by the National Natural Science Foundation of China(Grant No.52308015)the Guangdong Basic and Applied Basic Research Foundation(Grant No.2024A1515011214)the Guangzhou Science and Technology Planning Project(Grant No.SL2024A04J01189).
文摘As simulation-informed design gains importance in addressing urban complexity,integrating urban imagery into interactive feedback and decision-making has become increasingly essential.However,this potential remains underused,as urban imagery is often treated as a supporting variable in urban research rather than a core layer of spatial intelligence,hindering informed strategies in city branding,resource allocation,and livability.This study develops a data-driven framework,Street View Search Engine,which integrates urban imagery analysis with interactive exploration to advance human-centered insights into urban visual form.Based on 81,478 street view imagery collected in Hong Kong,China,a dataset comprising 19 visual features was first constructed to represent urban visual information across three categories:physical,impression,and isovist.Subsequently,the machine learning algorithm self-organizing maps was employed to train the dataset,producing a visualized“data landscape”that re-organizes street views according to their visual similarities.Third,building on the data landscape,this study develops the Street View Search Engine framework to conduct three main tasks:define visual foundations,comprehend streetscape morphology,and evaluate regional visual schemes.These tasks combine general-use exploration with research-oriented analysis:a web-based platform was developed to support general-use exploration(http://47.113.226.77/project1/#/),while various data processing methods were employed to enable in-depth professional investigations.By transforming raw data into a visualizable,computable,and interactive urban imagery system,this study paves the way for evidence-based interventions,strategic resource allocation,and greater public engagement in urban planning.
基金funded by the National Natural Science Foundation of China(52004201,52274143,52204153)China Postdoctoral Science Foundation(2021T140551).
文摘Coal-rock interface identification technology was pivotal in automatically adjusting the shearer's cutting drum during coal mining.However,it also served as a technical bottleneck hindering the advancement of intelligent coal mining.This study aimed to address the poor accuracy of current coal-rock identification technology on comprehensive working faces,coupled with the limited availability of coal-rock datasets.The loss function of the SegFormer model was enhanced,the model's hyperparameters and learning rate were adjusted,and an automatic recognition method was proposed for coal-rock interfaces based on FL-SegFormer.Additionally,an experimental platform was constructed to simulate the dusty environment during coal-rock cutting by the shearer,enabling the collection of coal-rock test image datasets.The morphology-based algorithms were employed to expand the coal-rock image datasets through image rotation,color dithering,and Gaussian noise injection so as to augment the diversity and applicability of the datasets.As a result,a coal-rock image dataset comprising 8424 samples was generated.The findings demonstrated that the FL-SegFormer model achieved a Mean Intersection over Union(MIoU)and mean pixel accuracy(MPA)of 97.72%and 98.83%,respectively.The FLSegFormer model outperformed other models in terms of recognition accuracy,as evidenced by an MloU exceeding 95.70% of the original image.Furthermore,the FL-SegFormer model using original coal-rock images was validated from No.15205 working face of the Yulin test mine in northern Shaanxi.The calculated average error was only 1.77%,and the model operated at a rate of 46.96 frames per second,meeting the practical application and deployment requirements in underground settings.These results provided a theoretical foundation for achieving automatic and efficient mining with coal mining machines and the intelligent development of coal mines.
基金This work was supported by the National Science Fund for Excellent Young Scholars （61622214）, the National Natural Science Foundation of China （Grant Nos. 61702565 and 61622214）, Guangdong Natural Science Foundation Project for Research Teams （2017A030312006）, and was also sponsored by CCF-Tencent Open Research Fund.
文摘Semantic image parsing, which refers to the pro- cess of decomposing images into semantic regions and constructing the structure representation of the input, has re- cently aroused widespread interest in the field of computer vision. The recent application of deep representation learning has driven this field into a new stage of development. In this paper, we summarize three aspects of the progress of research on semantic image parsing, i.e., category-level semantic segmentation, instance-level semantic segmentation, and beyond segmentation. Specifically, we first review the general frameworks for each task and introduce the relevant variants. The advantages and limitations of each method are also discussed. Moreover, we present a comprehensive comparison of different benchmark datasets and evaluation metrics. Finally, we explore the future trends and challenges of semantic image parsing.
基金This research was supported by the National Natural Science Foundation of China(41977064)the Fundamental Research Funds for the Central Universities(2452021158+1 种基金2452021036)the 111 Project of the Ministry of Education and the State Administration of Foreign Experts Affairs(B12007)。
文摘Ephemeral gullies are widely distributed in the hilly and gully region of the Loess Plateau and play a unique role in the slope gully erosion system.Rapid and accurate identification of ephemeral gullies impacts the distribution law and development trend of soil erosion on the Loess Plateau.Deep learning algorithms can quickly and accurately process large data samples that recognize ephemeral gullies from remote sensing images.Here,we investigated ephemeral gullies in the Zhoutungou watershed in the hilly and gully region of the Loess Plateau in China using satellite and unmanned aerial vehicle images and combined a deep learning image semantic segmentation model to realize automatic recognition and feature extraction.Using Accuracy,Precision,Recall,F1value,and AUC,we compared the ephemeral gully recognition results and accuracy evaluation of U-Net,R2U-Net,and SegNet image semantic segmentation models.The SegNet model was ranked first,followed by the R2U-Net and U-Net models,for ephemeral gully recognition in the hilly and gully region of the Loess Plateau.The ephemeral gully length and width between predicted and measured values had RMSE values of 6.78 m and 0.50 m,respectively,indicating that the model has an excellent recognition effect.This study identified a fast and accurate method for ephemeral gully recognition in the hilly and gully region of the Loess Plateau based on remote sensing images to provide an academic reference and practical guidance for soil erosion monitoring and slope and gully management in the Loess Plateau region.
