Due to strong vibrations and the impact on high-speed vehicles,road potholes may cause discomfort to passengers,damaging the durability of the suspension and the integrity of the cargo.Therefore,a road pothole detecti...Due to strong vibrations and the impact on high-speed vehicles,road potholes may cause discomfort to passengers,damaging the durability of the suspension and the integrity of the cargo.Therefore,a road pothole detection system is proposed,which uses deep learning and binocular vision for precise detection in the front.6,848 road surface pothole condition recognition datasets were constructed using a vehicle mounted binocular camera.The group attention shuffle block(GASB)is designed to enhance the expression of channel and spatial feature information in road images,while improving the existing shuffling network(ShuffleNetv2).This establishes a ShuffleNetv2(GASB-ShuffleNetv2)model based on mixed attention for recognizing the state of road potholes.The experimental results show that the model has better accuracy than the basic model and can effectively detect road potholes.In addition,we replaced the ordinary convolution in the CenterNet feature extraction network with pyramid convolution with multiple receptive fields,and designed a feature fusion module in the same network to fuse low-level and high-level features related to holes,thus establishing a PF-CenterNet that combines pyramid convolution with feature fusion to detect areas containing road potholes.A pothole distance estimation model based on binocular vision was established by analyzing the stereo ranging model and semi-global block matching algorithm.After parameter calibration,images were rectified and stereo-matched to generate a disparity map.This map was optimized using a weighted least squares filter to fill blank areas.The 3D coordinates are then calculated based on the disparity provided with distance information.Finally,vehicle experiments were conducted to verify the effectiveness of the algorithm in meeting detection requirements while considering long-range perception accuracy.The experimental results show that the system can meet the needs of unmanned vehicles,enabling them to perceive potholes in advance,thereby issuing timely warnings to drivers.展开更多
Computer vision algorithms have been utilized for 3-D road imaging and pothole detection for over two decades.Nonetheless,there is a lack of systematic survey articles on state-of-the-art(SoTA)computer vision techniqu...Computer vision algorithms have been utilized for 3-D road imaging and pothole detection for over two decades.Nonetheless,there is a lack of systematic survey articles on state-of-the-art(SoTA)computer vision techniques,especially deep learningmodels,developed to tackle these problems.This article first introduces the sensing systems employed for 2-D and 3-D road data acquisition,including camera(s),laser scanners and Microsoft Kinect.It then comprehensively reviews the SoTA computer vision algorithms,including(1)classical 2-D image processing,(2)3-D point cloud modelling and segmentation and(3)machine/deep learning,developed for road pothole detection.The article also discusses the existing challenges and future development trends of computer vision-based road pothole detection approaches:classical 2-D image processing-based and 3-D point cloud modelling and segmentation-based approaches have already become history;and convolutional neural networks(CNNs)have demonstrated compelling road pothole detection results and are promising to break the bottleneck with future advances in self/un-supervised learning for multi-modal semantic segmentation.We believe that this survey can serve as practical guidance for developing the next-generation road condition assessment systems.展开更多
The increasing global population at a rapid pace makes road trafficdense;managing such massive traffic is challenging. In developing countrieslike Pakistan, road traffic accidents (RTA) have the highest mortality perc...The increasing global population at a rapid pace makes road trafficdense;managing such massive traffic is challenging. In developing countrieslike Pakistan, road traffic accidents (RTA) have the highest mortality percentageamong other Asian countries. The main reasons for RTAs are roadcracks and potholes. Understanding the need for an automated system forthe detection of cracks and potholes, this study proposes a decision supportsystem (DSS) for an autonomous road information system for smart citydevelopment with the use of deep learning. The proposed DSS works in layerswhere initially the image of roads is captured and coordinates attached to theimage with the help of global positioning system (GPS), communicated tothe decision layer to find about the cracks and potholes in the roads, andeventually, that information is passed to the road management informationsystem, which gives information to drivers and the maintenance department.For the decision layer, we projected a CNN-based model for pothole crackdetection (PCD). Aimed at training, a K-fold cross-validation strategy wasused where the value of K was set to 10. The training of PCD was completedwith a self-collected dataset consisting of 6000 images from Pakistani roads.The proposed PCD achieved 98% of precision, 97% recall, and accuracy whiletesting on unseen images. The results produced by our model are higher thanthe existing model in terms of performance and computational cost, whichproves its significance.展开更多
基金supported in part by the Shaanxi Provincial Innovation Team Projects(grant number 2025RS-CXTD-015)in part by the Key Research and Development Program of Shaanxi Province(grant number 2024CY2-GJHX-31).
文摘Due to strong vibrations and the impact on high-speed vehicles,road potholes may cause discomfort to passengers,damaging the durability of the suspension and the integrity of the cargo.Therefore,a road pothole detection system is proposed,which uses deep learning and binocular vision for precise detection in the front.6,848 road surface pothole condition recognition datasets were constructed using a vehicle mounted binocular camera.The group attention shuffle block(GASB)is designed to enhance the expression of channel and spatial feature information in road images,while improving the existing shuffling network(ShuffleNetv2).This establishes a ShuffleNetv2(GASB-ShuffleNetv2)model based on mixed attention for recognizing the state of road potholes.The experimental results show that the model has better accuracy than the basic model and can effectively detect road potholes.In addition,we replaced the ordinary convolution in the CenterNet feature extraction network with pyramid convolution with multiple receptive fields,and designed a feature fusion module in the same network to fuse low-level and high-level features related to holes,thus establishing a PF-CenterNet that combines pyramid convolution with feature fusion to detect areas containing road potholes.A pothole distance estimation model based on binocular vision was established by analyzing the stereo ranging model and semi-global block matching algorithm.After parameter calibration,images were rectified and stereo-matched to generate a disparity map.This map was optimized using a weighted least squares filter to fill blank areas.The 3D coordinates are then calculated based on the disparity provided with distance information.Finally,vehicle experiments were conducted to verify the effectiveness of the algorithm in meeting detection requirements while considering long-range perception accuracy.The experimental results show that the system can meet the needs of unmanned vehicles,enabling them to perceive potholes in advance,thereby issuing timely warnings to drivers.
基金the National Key R&D Program of China(Grant No.2020AAA0108100)the Fundamental Research Funds for the Central Universities(Grant Nos.22120220184,22120220214 and 2022-5-YB-08)the Shanghai Municipal Science and Technology Major Project(Grant No.2021SHZDZX0100).
文摘Computer vision algorithms have been utilized for 3-D road imaging and pothole detection for over two decades.Nonetheless,there is a lack of systematic survey articles on state-of-the-art(SoTA)computer vision techniques,especially deep learningmodels,developed to tackle these problems.This article first introduces the sensing systems employed for 2-D and 3-D road data acquisition,including camera(s),laser scanners and Microsoft Kinect.It then comprehensively reviews the SoTA computer vision algorithms,including(1)classical 2-D image processing,(2)3-D point cloud modelling and segmentation and(3)machine/deep learning,developed for road pothole detection.The article also discusses the existing challenges and future development trends of computer vision-based road pothole detection approaches:classical 2-D image processing-based and 3-D point cloud modelling and segmentation-based approaches have already become history;and convolutional neural networks(CNNs)have demonstrated compelling road pothole detection results and are promising to break the bottleneck with future advances in self/un-supervised learning for multi-modal semantic segmentation.We believe that this survey can serve as practical guidance for developing the next-generation road condition assessment systems.
基金Hunan Provincial Science and Technology Innovation Leader Project,Grant/Award Number:2021RC4025National Natural ScienceFoundation of China,Grant/Award Number:51808209Hunan Provincial Innovation Foundation for Postgraduate,Grant/Award Number:QL20210106.
文摘The increasing global population at a rapid pace makes road trafficdense;managing such massive traffic is challenging. In developing countrieslike Pakistan, road traffic accidents (RTA) have the highest mortality percentageamong other Asian countries. The main reasons for RTAs are roadcracks and potholes. Understanding the need for an automated system forthe detection of cracks and potholes, this study proposes a decision supportsystem (DSS) for an autonomous road information system for smart citydevelopment with the use of deep learning. The proposed DSS works in layerswhere initially the image of roads is captured and coordinates attached to theimage with the help of global positioning system (GPS), communicated tothe decision layer to find about the cracks and potholes in the roads, andeventually, that information is passed to the road management informationsystem, which gives information to drivers and the maintenance department.For the decision layer, we projected a CNN-based model for pothole crackdetection (PCD). Aimed at training, a K-fold cross-validation strategy wasused where the value of K was set to 10. The training of PCD was completedwith a self-collected dataset consisting of 6000 images from Pakistani roads.The proposed PCD achieved 98% of precision, 97% recall, and accuracy whiletesting on unseen images. The results produced by our model are higher thanthe existing model in terms of performance and computational cost, whichproves its significance.
