As the number and complexity of sensors in autonomous vehicles continue to rise,multimodal fusionbased object detection algorithms are increasingly being used to detect 3D environmental information,significantly advan...As the number and complexity of sensors in autonomous vehicles continue to rise,multimodal fusionbased object detection algorithms are increasingly being used to detect 3D environmental information,significantly advancing the development of perception technology in autonomous driving.To further promote the development of fusion algorithms and improve detection performance,this paper discusses the advantages and recent advancements of multimodal fusion-based object detection algorithms.Starting fromsingle-modal sensor detection,the paper provides a detailed overview of typical sensors used in autonomous driving and introduces object detection methods based on images and point clouds.For image-based detection methods,they are categorized into monocular detection and binocular detection based on different input types.For point cloud-based detection methods,they are classified into projection-based,voxel-based,point cluster-based,pillar-based,and graph structure-based approaches based on the technical pathways for processing point cloud features.Additionally,multimodal fusion algorithms are divided into Camera-LiDAR fusion,Camera-Radar fusion,Camera-LiDAR-Radar fusion,and other sensor fusion methods based on the types of sensors involved.Furthermore,the paper identifies five key future research directions in this field,aiming to provide insights for researchers engaged in multimodal fusion-based object detection algorithms and to encourage broader attention to the research and application of multimodal fusion-based object detection.展开更多
Task scheduling in cloud computing environments is a multi-objective optimization problem, which is NP hard. It is also a challenging problem to find an appropriate trade-off among resource utilization, energy consump...Task scheduling in cloud computing environments is a multi-objective optimization problem, which is NP hard. It is also a challenging problem to find an appropriate trade-off among resource utilization, energy consumption and Quality of Service(QoS) requirements under the changing environment and diverse tasks. Considering both processing time and transmission time, a PSO-based Adaptive Multi-objective Task Scheduling(AMTS) Strategy is proposed in this paper. First, the task scheduling problem is formulated. Then, a task scheduling policy is advanced to get the optimal resource utilization, task completion time, average cost and average energy consumption. In order to maintain the particle diversity, the adaptive acceleration coefficient is adopted. Experimental results show that the improved PSO algorithm can obtain quasi-optimal solutions for the cloud task scheduling problem.展开更多
An object oriented multi robotic graphic simulation environment is described in this paper. Object oriented programming is used to model the physical objects of the robotic workcell in the form of software objects ...An object oriented multi robotic graphic simulation environment is described in this paper. Object oriented programming is used to model the physical objects of the robotic workcell in the form of software objects or classes. The virtual objects are defined to provide the user with a user friendly interface including realistic graphic simulation and clarify the software architecture. The programming method of associating the task object with active object effectively increases the software reusability, maintainability and modifiability. Task level programming is also demonstrated through a multi robot welding task that allows the user to concentrate on the most important aspects of the tasks. The multi thread programming technique is used to simulate the interaction of multiple tasks. Finally, a virtual test is carried out in the graphic simulation environment to observe design and program errors and fix them before downloading the software to the real workcell.展开更多
Purpose： It is used for judging the advantages and disadvantages of information technology foundation course teaching in health vocational colleges. Method： In teaching, it takes the two classes of 2012 grade nursin...Purpose： It is used for judging the advantages and disadvantages of information technology foundation course teaching in health vocational colleges. Method： In teaching, it takes the two classes of 2012 grade nursing major as the experiment object. The comparison class adopts traditonal and speaking-practice combination teaching method and the experiment class adopts task-driving teaching method. When the semester finishes, it conducts testing andd questionnaire survey, collecting the relevant data, analyzing the changes of students in the aspects of performance, learning interest and attitude, autonomous learning consciousness and ability after experiment class adopting new teaching methods. Result： The exam performance of experiment class is obviously higher than the comparison class, and the experiment class has an obvious improvement in the aspects of learning interest, autonomous learning consciousness and ability, and the difference has statistical significance. Conclusion： The task driving teaching method is suitable for the status of information foundation teaching in health vocational colleges, which improves students＇ performance significantly and is good for students＇ learning interest and enthusiasm, obtaining good classroom effect. Also, it makes students＇ autonomous learning consciousness and ability improve greatly.展开更多
The basic theory of YOLO series object detection algorithms is discussed, the dangerous driving behavior dataset is collected and produced, and then the YOLOv7 network is introduced in detail, the deep separable convo...The basic theory of YOLO series object detection algorithms is discussed, the dangerous driving behavior dataset is collected and produced, and then the YOLOv7 network is introduced in detail, the deep separable convolution and CA attention mechanism are introduced, the YOLOv7 bounding box loss function and clustering algorithm are optimized, and the DB-YOLOv7 network structure is constructed. In the first stage of the experiment, the PASCAL VOC public dataset was utilized for pre-training. A comparative analysis was conducted to assess the recognition accuracy and inference time before and after the proposed improvements. The experimental results demonstrated an increase of 1.4% in the average recognition accuracy, alongside a reduction in the inference time by 4 ms. Subsequently, a model for the recognition of dangerous driving behaviors was trained using a specialized dangerous driving behavior dataset. A series of experiments were performed to evaluate the efficacy of the DB-YOLOv7 algorithm in this context. The findings indicate a significant enhancement in detection performance, with a 4% improvement in accuracy compared to the baseline network. Furthermore, the model’s inference time was reduced by 20%, from 25 ms to 20 ms. These results substantiate the effectiveness of the DB-YOLOv7 recognition algorithm for detecting dangerous driving behaviors, providing comprehensive validation of its practical applicability.展开更多
In the broader field of mechanical technology,and particularly in the context of self-driving vehicles,cameras and Light Detection and Ranging(LiDAR)sensors provide complementary modalities that hold significant poten...In the broader field of mechanical technology,and particularly in the context of self-driving vehicles,cameras and Light Detection and Ranging(LiDAR)sensors provide complementary modalities that hold significant potential for sensor fusion.However,directly merging multi-sensor data through point projection often results in information loss due to quantization,and managing the differing data formats from multiple sensors remains a persistent challenge.To address these issues,we propose a new fusion method that leverages continuous convolution,point-pooling,and a learned Multilayer Perceptron(MLP)to achieve superior detection performance.Our approach integrates the segmentation mask with raw LiDAR points rather than relying on projected points,effectively avoiding quantization loss.Additionally,when retrieving corresponding semantic information from images through point cloud projection,we employ linear interpolation and upsample the image feature maps to mitigate quantization loss.We employ nearest-neighbor search and continuous convolution to seamlessly fuse data from different formats.Moreover,we integrate pooling and aggregation operations,which serve as conceptual extensions of convolution,and are specifically designed to reconcile the inherent disparities among these data representations.Our detection network operates in two stages:in the first stage,preliminary proposals and segmentation features are generated;in the second stage,we refine the fusion results together with the segmentation mask to yield the final prediction.Notably,in our approach,the image network is used solely to provide semantic information,serving to enhance the point cloud features.Extensive experiments on the Karlsruhe Institute of Technology and Toyota Technological Institute(KITTI)dataset demonstrate the effectiveness of our approach,which achieves both high precision and robust performance in 3D object detection tasks.展开更多
复杂场景下的环境感知对自动驾驶安全至关重要.为提升低光照雾天条件下车辆的感知能力,本文首先基于晴朗天气下的KITTI数据集,引入一种结合深度信息的大气散射模型,用于模拟生成真实的低光照雾天场景数据.随后,在YOLOv11(You only look ...复杂场景下的环境感知对自动驾驶安全至关重要.为提升低光照雾天条件下车辆的感知能力,本文首先基于晴朗天气下的KITTI数据集,引入一种结合深度信息的大气散射模型,用于模拟生成真实的低光照雾天场景数据.随后,在YOLOv11(You only look once)框架中设计了多层通道融合模块MLCFM(Multi-layer channel fusion module),通过对通道的分割与重组,增强了各层次特征的提取能力;接着利用语义重要性驱动的动态多尺度检测头,实现了更强的多尺度感知效果;最后,采用ATSS(Adaptive training sample selection)策略自适应地优化正负样本分配,以进一步提升小目标检测性能.在增强KITTI数据集上的实验结果表明,改进后网络在Car、Cyclist、Pedestrian三类目标上的检测精度分别提高了2.2个百分点、11.8个百分点和7.8个百分点,总体的mAP@0.5提升了7.3个百分点,并通过可视化分析与消融实验进一步验证了各模块在复杂环境下提升检测性能的有效性.展开更多
基金funded by the Yangtze River Delta Science and Technology Innovation Community Joint Research Project(2023CSJGG1600)the Natural Science Foundation of Anhui Province(2208085MF173)Wuhu“ChiZhu Light”Major Science and Technology Project(2023ZD01,2023ZD03).
文摘As the number and complexity of sensors in autonomous vehicles continue to rise,multimodal fusionbased object detection algorithms are increasingly being used to detect 3D environmental information,significantly advancing the development of perception technology in autonomous driving.To further promote the development of fusion algorithms and improve detection performance,this paper discusses the advantages and recent advancements of multimodal fusion-based object detection algorithms.Starting fromsingle-modal sensor detection,the paper provides a detailed overview of typical sensors used in autonomous driving and introduces object detection methods based on images and point clouds.For image-based detection methods,they are categorized into monocular detection and binocular detection based on different input types.For point cloud-based detection methods,they are classified into projection-based,voxel-based,point cluster-based,pillar-based,and graph structure-based approaches based on the technical pathways for processing point cloud features.Additionally,multimodal fusion algorithms are divided into Camera-LiDAR fusion,Camera-Radar fusion,Camera-LiDAR-Radar fusion,and other sensor fusion methods based on the types of sensors involved.Furthermore,the paper identifies five key future research directions in this field,aiming to provide insights for researchers engaged in multimodal fusion-based object detection algorithms and to encourage broader attention to the research and application of multimodal fusion-based object detection.
基金partially been sponsored by the National Science Foundation of China(No.61572355,61272093,610172063)Tianjin Research Program of Application Foundation and Advanced Technology under grant No.15JCYBJC15700
文摘Task scheduling in cloud computing environments is a multi-objective optimization problem, which is NP hard. It is also a challenging problem to find an appropriate trade-off among resource utilization, energy consumption and Quality of Service(QoS) requirements under the changing environment and diverse tasks. Considering both processing time and transmission time, a PSO-based Adaptive Multi-objective Task Scheduling(AMTS) Strategy is proposed in this paper. First, the task scheduling problem is formulated. Then, a task scheduling policy is advanced to get the optimal resource utilization, task completion time, average cost and average energy consumption. In order to maintain the particle diversity, the adaptive acceleration coefficient is adopted. Experimental results show that the improved PSO algorithm can obtain quasi-optimal solutions for the cloud task scheduling problem.
文摘An object oriented multi robotic graphic simulation environment is described in this paper. Object oriented programming is used to model the physical objects of the robotic workcell in the form of software objects or classes. The virtual objects are defined to provide the user with a user friendly interface including realistic graphic simulation and clarify the software architecture. The programming method of associating the task object with active object effectively increases the software reusability, maintainability and modifiability. Task level programming is also demonstrated through a multi robot welding task that allows the user to concentrate on the most important aspects of the tasks. The multi thread programming technique is used to simulate the interaction of multiple tasks. Finally, a virtual test is carried out in the graphic simulation environment to observe design and program errors and fix them before downloading the software to the real workcell.
文摘Purpose： It is used for judging the advantages and disadvantages of information technology foundation course teaching in health vocational colleges. Method： In teaching, it takes the two classes of 2012 grade nursing major as the experiment object. The comparison class adopts traditonal and speaking-practice combination teaching method and the experiment class adopts task-driving teaching method. When the semester finishes, it conducts testing andd questionnaire survey, collecting the relevant data, analyzing the changes of students in the aspects of performance, learning interest and attitude, autonomous learning consciousness and ability after experiment class adopting new teaching methods. Result： The exam performance of experiment class is obviously higher than the comparison class, and the experiment class has an obvious improvement in the aspects of learning interest, autonomous learning consciousness and ability, and the difference has statistical significance. Conclusion： The task driving teaching method is suitable for the status of information foundation teaching in health vocational colleges, which improves students＇ performance significantly and is good for students＇ learning interest and enthusiasm, obtaining good classroom effect. Also, it makes students＇ autonomous learning consciousness and ability improve greatly.
文摘The basic theory of YOLO series object detection algorithms is discussed, the dangerous driving behavior dataset is collected and produced, and then the YOLOv7 network is introduced in detail, the deep separable convolution and CA attention mechanism are introduced, the YOLOv7 bounding box loss function and clustering algorithm are optimized, and the DB-YOLOv7 network structure is constructed. In the first stage of the experiment, the PASCAL VOC public dataset was utilized for pre-training. A comparative analysis was conducted to assess the recognition accuracy and inference time before and after the proposed improvements. The experimental results demonstrated an increase of 1.4% in the average recognition accuracy, alongside a reduction in the inference time by 4 ms. Subsequently, a model for the recognition of dangerous driving behaviors was trained using a specialized dangerous driving behavior dataset. A series of experiments were performed to evaluate the efficacy of the DB-YOLOv7 algorithm in this context. The findings indicate a significant enhancement in detection performance, with a 4% improvement in accuracy compared to the baseline network. Furthermore, the model’s inference time was reduced by 20%, from 25 ms to 20 ms. These results substantiate the effectiveness of the DB-YOLOv7 recognition algorithm for detecting dangerous driving behaviors, providing comprehensive validation of its practical applicability.
文摘In the broader field of mechanical technology,and particularly in the context of self-driving vehicles,cameras and Light Detection and Ranging(LiDAR)sensors provide complementary modalities that hold significant potential for sensor fusion.However,directly merging multi-sensor data through point projection often results in information loss due to quantization,and managing the differing data formats from multiple sensors remains a persistent challenge.To address these issues,we propose a new fusion method that leverages continuous convolution,point-pooling,and a learned Multilayer Perceptron(MLP)to achieve superior detection performance.Our approach integrates the segmentation mask with raw LiDAR points rather than relying on projected points,effectively avoiding quantization loss.Additionally,when retrieving corresponding semantic information from images through point cloud projection,we employ linear interpolation and upsample the image feature maps to mitigate quantization loss.We employ nearest-neighbor search and continuous convolution to seamlessly fuse data from different formats.Moreover,we integrate pooling and aggregation operations,which serve as conceptual extensions of convolution,and are specifically designed to reconcile the inherent disparities among these data representations.Our detection network operates in two stages:in the first stage,preliminary proposals and segmentation features are generated;in the second stage,we refine the fusion results together with the segmentation mask to yield the final prediction.Notably,in our approach,the image network is used solely to provide semantic information,serving to enhance the point cloud features.Extensive experiments on the Karlsruhe Institute of Technology and Toyota Technological Institute(KITTI)dataset demonstrate the effectiveness of our approach,which achieves both high precision and robust performance in 3D object detection tasks.
文摘复杂场景下的环境感知对自动驾驶安全至关重要.为提升低光照雾天条件下车辆的感知能力,本文首先基于晴朗天气下的KITTI数据集,引入一种结合深度信息的大气散射模型,用于模拟生成真实的低光照雾天场景数据.随后,在YOLOv11(You only look once)框架中设计了多层通道融合模块MLCFM(Multi-layer channel fusion module),通过对通道的分割与重组,增强了各层次特征的提取能力;接着利用语义重要性驱动的动态多尺度检测头,实现了更强的多尺度感知效果;最后,采用ATSS(Adaptive training sample selection)策略自适应地优化正负样本分配,以进一步提升小目标检测性能.在增强KITTI数据集上的实验结果表明,改进后网络在Car、Cyclist、Pedestrian三类目标上的检测精度分别提高了2.2个百分点、11.8个百分点和7.8个百分点,总体的mAP@0.5提升了7.3个百分点,并通过可视化分析与消融实验进一步验证了各模块在复杂环境下提升检测性能的有效性.
