Stable and reliable perception capability is the basis for the safety of autonomous driving,and pedestrian detection is one of the key tasks for vehicle-mounted sensors to perceive the environment.In order to make ful...Stable and reliable perception capability is the basis for the safety of autonomous driving,and pedestrian detection is one of the key tasks for vehicle-mounted sensors to perceive the environment.In order to make full use of the complementarity of vehicle cameras and lidars,we make improvements on the basis of the EPNet algorthm,and a pedestrian detection method based on prefusion of point cloud and image data is proposed.Use the bidirectional cascaded feature fusion module to achieve more information exchange between image and point cloud data,and obtain more comprehensive fusion features;design a consistency loss function to enhance the correlation between location confidence and category confidence and improve model detection accuracy.Validated on KITTI and other datasets,the detection result of pedestrian s can reach 84%mAP,4.49%higher than the EPNet on difficult pedestrian samples.Compared with a single visual sensor,the proposed method has a better detection effect on objects affected by shadow or longer distance.Finally,the model is accelerated based on the TensorRT custom plug-in and uses CUDA to improve the effciencyof multimodal data pre-processing and post-processing.Deployed on the Nvidia Jetson Orin edge computing device,the model runs at 10 frames per second,and the inference speed is increased by about 60%,laying the foundation for the application of algorithm engineering.展开更多
基金partially supported by the Hunan Association for Science and Technology Talent Support Project(Grant No.2022TJN14)the Postgraduate Scientific Research Innovation Project of Central South University(Grant No.2023XQLHO67).
文摘Stable and reliable perception capability is the basis for the safety of autonomous driving,and pedestrian detection is one of the key tasks for vehicle-mounted sensors to perceive the environment.In order to make full use of the complementarity of vehicle cameras and lidars,we make improvements on the basis of the EPNet algorthm,and a pedestrian detection method based on prefusion of point cloud and image data is proposed.Use the bidirectional cascaded feature fusion module to achieve more information exchange between image and point cloud data,and obtain more comprehensive fusion features;design a consistency loss function to enhance the correlation between location confidence and category confidence and improve model detection accuracy.Validated on KITTI and other datasets,the detection result of pedestrian s can reach 84%mAP,4.49%higher than the EPNet on difficult pedestrian samples.Compared with a single visual sensor,the proposed method has a better detection effect on objects affected by shadow or longer distance.Finally,the model is accelerated based on the TensorRT custom plug-in and uses CUDA to improve the effciencyof multimodal data pre-processing and post-processing.Deployed on the Nvidia Jetson Orin edge computing device,the model runs at 10 frames per second,and the inference speed is increased by about 60%,laying the foundation for the application of algorithm engineering.
