The objective of this work is to develop an innovative system(ROSGPT)that merges large language models(LLMs)with the robot operating system(ROS),facilitating natural language voice control of mobile robots.This integr...The objective of this work is to develop an innovative system(ROSGPT)that merges large language models(LLMs)with the robot operating system(ROS),facilitating natural language voice control of mobile robots.This integration aims to bridge the gap between human-robot interaction(HRI)and artificial intelligence(AI).ROSGPT integrates several subsystems,including speech recognition,prompt engineering,LLM and ROS,enabling seamless control of robots through human voice or text commands.The LLM component is optimized,with its performance refined from the open-source Llama2 model through fine-tuning and quantization procedures.Through extensive experiments conducted in both real-world and virtual environments,ROSGPT demonstrates its efficacy in meeting user requirements and delivering user-friendly interactive experiences.The system demonstrates versatility and adaptability through its ability to comprehend diverse user commands and execute corresponding tasks with precision and reliability,thereby showcasing its potential for various practical applications in robotics and AI.The demonstration video can be viewed at https://iklxo6z9yv.feishu.cn/docx/Lux3dmTDxoZ5YnxWJTZcxUCWnTh.展开更多
The course ROS Robot Programming Practice is designed to equip students with both the foundational principles and practical applications of the Robot Operating System(ROS)through structured instruction and hands-on ex...The course ROS Robot Programming Practice is designed to equip students with both the foundational principles and practical applications of the Robot Operating System(ROS)through structured instruction and hands-on exercises.This paper summarizes the course’s pedagogical approach and the insights gained from its implementation,focusing on key areas such as the practical teaching platform,instructional system,teaching methods,and evaluation mechanisms.The practical teaching platform incorporates robotic arms and mobile robots,along with commonly used sensors,to engage students and stimulate their interest in robotics.A well-structured teaching system is employed,guiding students through a series of experiments-ranging from basic tasks to modeling and simulation experiments,sensor experiments,mobile robot tasks,robotic arm exercises,and comprehensive project practices.Regarding teaching methods,a blended learning approach and progressive instructional model are utilized to ensure active student participation.This approach follows a logical progression,starting with fundamental ROS knowledge,advancing to robot applications,and culminating in comprehensive project-based practice.In terms of evaluation,a dual approach combining process and outcome assessments is employed,ensuring that students’performance is evaluated comprehensively through various metrics.展开更多
An efficient algorithm for path planning is crucial for guiding autonomous surface vehicles(ASVs)through designated waypoints.However,current evaluations of ASV path planning mainly focus on comparing total path lengt...An efficient algorithm for path planning is crucial for guiding autonomous surface vehicles(ASVs)through designated waypoints.However,current evaluations of ASV path planning mainly focus on comparing total path lengths,using temporal models to estimate travel time,idealized integration of global and local motion planners,and omission of external environmental disturbances.These rudimentary criteria cannot adequately capture real-world operations.To address these shortcomings,this study introduces a simulation framework for evaluating navigation modules designed for ASVs.The proposed framework is implemented on a prototype ASV using the Robot Operating System(ROS)and the Gazebo simulation platform.The implementation processes replicated satellite images with the extended Kalman filter technique to acquire localized location data.Cost minimization for global trajectories is achieved through the application of Dijkstra and A*algorithms,while local obstacle avoidance is managed by the dynamic window approach algorithm.The results demonstrate the distinctions and intricacies of the metrics provided by the proposed simulation framework compared with the rudimentary criteria commonly utilized in conventional path planning works.展开更多
Currently, due to the detrimental effects on surface finish and machining system, chatter has been one crucial factor restricting robotic drilling operations, which improve both quality and efficiency of aviation manu...Currently, due to the detrimental effects on surface finish and machining system, chatter has been one crucial factor restricting robotic drilling operations, which improve both quality and efficiency of aviation manufacturing. Based on the matrix notch filter and fast wavelet packet decomposition, this paper presents a novel pre-generated matrix-based real-time chatter monitoring method for robotic drilling. Taking vibration characteristics of robotic drilling into account, the matrix notch filter is designed to eliminate the interference of spindle-related components on the measured vibration signal. Then, the fast wavelet packet decomposition is presented to decompose the filtered signal into several equidistant frequency bands, and the energy of each sub-band is obtained. Finally, the energy entropy which characterizes inhomogeneity of energy distribution is utilized as the feature to recognize chatter on-line, and the effectiveness of the presented algorithm is validated by extensive experimental data. The results show that the proposed algorithm can effectively detect chatter before it is fully developed. Moreover, since both filtering and decomposition of signal are implemented by the pre-generated matrices, calculation for an energy entropy of vibration signal with 512 samples takes only about 0.690 ms. Consequently, the proposed method achieves real-time chatter monitoring for robotic drilling, which is essential for subsequent chatter suppression.展开更多
Traditional cochlear implantation surgery has problems such as high surgical accuracy requirement and large trauma,which cause the difficulty of the operation and the high requirements for doctors,so that only a few d...Traditional cochlear implantation surgery has problems such as high surgical accuracy requirement and large trauma,which cause the difficulty of the operation and the high requirements for doctors,so that only a few doctors can complete the operation independently.However,there is no research on robotic cochlear implantation in China.In response to this problem,a robotic cochlear implantation system is proposed.The robot is controlled by robot operating system(ROS).A simulation environment for the overall surgery is established on the ROS based on the real surgery environment.Through the analysis of the kinematics and the motion planning algorithm of the manipulator,an appropriate motion mode is designed to control the motion of the manipulator,and perform the surgery under the simulation environment.A simple and feasible method of navigation is proposed,and through the model experiment,the feasibility of robotic cochlear implantation surgery is verified.展开更多
在停车场、隧道中GPS、Wi-Fi信号受限的情况下,提出一种基于激光雷达的车辆自主定位方法。采用激光雷达SLAM(simultaneous localization and mapping)算法,通过三维激光雷达点云匹配获取车辆的估计位姿;根据图优化方法和非线性优化方法...在停车场、隧道中GPS、Wi-Fi信号受限的情况下,提出一种基于激光雷达的车辆自主定位方法。采用激光雷达SLAM(simultaneous localization and mapping)算法,通过三维激光雷达点云匹配获取车辆的估计位姿;根据图优化方法和非线性优化方法,对所有位姿进行后端调整,进而得到分辨率可控的环境信息平面栅格地图;基于蒙特卡洛方法,采用粒子滤波器进行实时车辆定位,并提出了粒子采样的一种改善方式,实现了较高精度的激光雷达自主定位。结果表明:粒子滤波器能够有效地实现车辆在停车场等无GPS环境下的定位,定位精度在10 cm之内。展开更多
Despite the advances in automated vulnerability detection approaches,security vulnerabilities caused by design flaws in software systems are continuously appearing in real-world systems.Such security design flaws can ...Despite the advances in automated vulnerability detection approaches,security vulnerabilities caused by design flaws in software systems are continuously appearing in real-world systems.Such security design flaws can bring unrestricted and misimplemented behaviors of a system and can lead to fatal vulnerabilities such as remote code execution or sensitive data leakage.Therefore,it is an essential task to discover unrestricted and misimplemented behaviors of a system.However,it is a daunting task for security experts to discover such vulnerabilities in advance because it is timeconsuming and error-prone to analyze the whole code in detail.Also,most of the existing vulnerability detection approaches still focus on detecting memory corruption bugs because these bugs are the dominant root cause of software vulnerabilities.This paper proposes SMINER,a novel approach that discovers vulnerabilities caused by unrestricted and misimplemented behaviors.SMINER first collects unit test cases for the target system from the official repository.Next,preprocess the collected code fragments.SMINER uses pre-processed data to show the security policies that can occur on the target system and creates a test case for security policy testing.To demonstrate the effectiveness of SMINER,this paper evaluates SMINER against Robot Operating System(ROS),a real-world system used for intelligent robots in Amazon and controlling satellites in National Aeronautics and Space Administration(NASA).From the evaluation,we discovered two real-world vulnerabilities in ROS.展开更多
With the continuous development of robotics and artificial intelligence,robots are being increasingly used in various applications.For traditional navigation algorithms,such as Dijkstra and A*,many dynamic scenarios i...With the continuous development of robotics and artificial intelligence,robots are being increasingly used in various applications.For traditional navigation algorithms,such as Dijkstra and A*,many dynamic scenarios in life are difficult to cope with.To solve the navigation problem of complex dynamic scenes,we present an improved reinforcement-learning-based algorithm for local path planning that allows it to perform well even when more dynamic obstacles are present.The method applies the gmapping algorithm as the upper layer input and uses reinforcement learning methods as the output.The algorithm enhances the robots’ability to actively avoid obstacles while retaining the adaptability of traditional methods.展开更多
Urban Air Mobility(UAM)is an emerging aviation sector which the goal is to transform air transportation with safe,on-demand air travel for both passengers and cargo.UAM flight planning strategically separates flows of...Urban Air Mobility(UAM)is an emerging aviation sector which the goal is to transform air transportation with safe,on-demand air travel for both passengers and cargo.UAM flight planning strategically separates flows of aircraft on intersecting routes vertically by allocating distinct flight levels to them,and aircraft are required to maintain the flight level when crossing the intersection.However,there is a possibility that an aircraft may fail to maintain the assigned flight level,leading to a potential conflict at intersections.This paper aims to address conflicts at intersections in the context of UAM,focusing on decentralized conflict detection and resolution.A novel approach is developed to facilitate information exchange among UAM components,including the provider of services to UAM,UAM operators,and the pilot in command.A receding horizon trajectory planning approach is proposed for the execution of conflict resolution,optimizing trajectory planning by eliminating potential problems and challenges associated with geometric approaches.The proposed trajectory planner considers the model and constraints of UAM aircraft,offering optimal solutions for safe separation at UAM airspace intersections.The significance of the proposed planning framework is demonstrated through simulations considering conflict at intersections by communicating the UAM components through request and replay services and generating resolution maneuvers on-the-fly for each aircraft involved in the conflict.展开更多
基金National Natural Science Foundation of China(No.61601112)。
文摘The objective of this work is to develop an innovative system(ROSGPT)that merges large language models(LLMs)with the robot operating system(ROS),facilitating natural language voice control of mobile robots.This integration aims to bridge the gap between human-robot interaction(HRI)and artificial intelligence(AI).ROSGPT integrates several subsystems,including speech recognition,prompt engineering,LLM and ROS,enabling seamless control of robots through human voice or text commands.The LLM component is optimized,with its performance refined from the open-source Llama2 model through fine-tuning and quantization procedures.Through extensive experiments conducted in both real-world and virtual environments,ROSGPT demonstrates its efficacy in meeting user requirements and delivering user-friendly interactive experiences.The system demonstrates versatility and adaptability through its ability to comprehend diverse user commands and execute corresponding tasks with precision and reliability,thereby showcasing its potential for various practical applications in robotics and AI.The demonstration video can be viewed at https://iklxo6z9yv.feishu.cn/docx/Lux3dmTDxoZ5YnxWJTZcxUCWnTh.
文摘The course ROS Robot Programming Practice is designed to equip students with both the foundational principles and practical applications of the Robot Operating System(ROS)through structured instruction and hands-on exercises.This paper summarizes the course’s pedagogical approach and the insights gained from its implementation,focusing on key areas such as the practical teaching platform,instructional system,teaching methods,and evaluation mechanisms.The practical teaching platform incorporates robotic arms and mobile robots,along with commonly used sensors,to engage students and stimulate their interest in robotics.A well-structured teaching system is employed,guiding students through a series of experiments-ranging from basic tasks to modeling and simulation experiments,sensor experiments,mobile robot tasks,robotic arm exercises,and comprehensive project practices.Regarding teaching methods,a blended learning approach and progressive instructional model are utilized to ensure active student participation.This approach follows a logical progression,starting with fundamental ROS knowledge,advancing to robot applications,and culminating in comprehensive project-based practice.In terms of evaluation,a dual approach combining process and outcome assessments is employed,ensuring that students’performance is evaluated comprehensively through various metrics.
基金Supported by the funding from RMIT Internal Research Grant R1.
文摘An efficient algorithm for path planning is crucial for guiding autonomous surface vehicles(ASVs)through designated waypoints.However,current evaluations of ASV path planning mainly focus on comparing total path lengths,using temporal models to estimate travel time,idealized integration of global and local motion planners,and omission of external environmental disturbances.These rudimentary criteria cannot adequately capture real-world operations.To address these shortcomings,this study introduces a simulation framework for evaluating navigation modules designed for ASVs.The proposed framework is implemented on a prototype ASV using the Robot Operating System(ROS)and the Gazebo simulation platform.The implementation processes replicated satellite images with the extended Kalman filter technique to acquire localized location data.Cost minimization for global trajectories is achieved through the application of Dijkstra and A*algorithms,while local obstacle avoidance is managed by the dynamic window approach algorithm.The results demonstrate the distinctions and intricacies of the metrics provided by the proposed simulation framework compared with the rudimentary criteria commonly utilized in conventional path planning works.
基金supported by the National Key R&D Program of China (No. 2017YFB1302601 and 2018YFB1702503)
文摘Currently, due to the detrimental effects on surface finish and machining system, chatter has been one crucial factor restricting robotic drilling operations, which improve both quality and efficiency of aviation manufacturing. Based on the matrix notch filter and fast wavelet packet decomposition, this paper presents a novel pre-generated matrix-based real-time chatter monitoring method for robotic drilling. Taking vibration characteristics of robotic drilling into account, the matrix notch filter is designed to eliminate the interference of spindle-related components on the measured vibration signal. Then, the fast wavelet packet decomposition is presented to decompose the filtered signal into several equidistant frequency bands, and the energy of each sub-band is obtained. Finally, the energy entropy which characterizes inhomogeneity of energy distribution is utilized as the feature to recognize chatter on-line, and the effectiveness of the presented algorithm is validated by extensive experimental data. The results show that the proposed algorithm can effectively detect chatter before it is fully developed. Moreover, since both filtering and decomposition of signal are implemented by the pre-generated matrices, calculation for an energy entropy of vibration signal with 512 samples takes only about 0.690 ms. Consequently, the proposed method achieves real-time chatter monitoring for robotic drilling, which is essential for subsequent chatter suppression.
基金the National Natural Science Foundation of China(Nos.61973211,62133009,51911540479 and M-0221)the Science and Technology Commission of Shanghai Municipality(Nos.21550714200 and 20DZ2220400)+1 种基金the Research Project of Institute of Medical Robotics of Shanghai Jiao Tong Universitythe Interdisciplinary Program of Shanghai Jiao Tong University(Nos.YG2017ZD03 and ZH2018QNB31)。
文摘Traditional cochlear implantation surgery has problems such as high surgical accuracy requirement and large trauma,which cause the difficulty of the operation and the high requirements for doctors,so that only a few doctors can complete the operation independently.However,there is no research on robotic cochlear implantation in China.In response to this problem,a robotic cochlear implantation system is proposed.The robot is controlled by robot operating system(ROS).A simulation environment for the overall surgery is established on the ROS based on the real surgery environment.Through the analysis of the kinematics and the motion planning algorithm of the manipulator,an appropriate motion mode is designed to control the motion of the manipulator,and perform the surgery under the simulation environment.A simple and feasible method of navigation is proposed,and through the model experiment,the feasibility of robotic cochlear implantation surgery is verified.
文摘在停车场、隧道中GPS、Wi-Fi信号受限的情况下,提出一种基于激光雷达的车辆自主定位方法。采用激光雷达SLAM(simultaneous localization and mapping)算法,通过三维激光雷达点云匹配获取车辆的估计位姿;根据图优化方法和非线性优化方法,对所有位姿进行后端调整,进而得到分辨率可控的环境信息平面栅格地图;基于蒙特卡洛方法,采用粒子滤波器进行实时车辆定位,并提出了粒子采样的一种改善方式,实现了较高精度的激光雷达自主定位。结果表明:粒子滤波器能够有效地实现车辆在停车场等无GPS环境下的定位,定位精度在10 cm之内。
基金This work was supported in part by the National Research Foundation of Korea(NRF)funded by the Ministry of Science and ICT(MSIT)Future Planning under Grant NRF-2020R1A2C2014336 and Grant NRF-2021R1A4A1029650.
文摘Despite the advances in automated vulnerability detection approaches,security vulnerabilities caused by design flaws in software systems are continuously appearing in real-world systems.Such security design flaws can bring unrestricted and misimplemented behaviors of a system and can lead to fatal vulnerabilities such as remote code execution or sensitive data leakage.Therefore,it is an essential task to discover unrestricted and misimplemented behaviors of a system.However,it is a daunting task for security experts to discover such vulnerabilities in advance because it is timeconsuming and error-prone to analyze the whole code in detail.Also,most of the existing vulnerability detection approaches still focus on detecting memory corruption bugs because these bugs are the dominant root cause of software vulnerabilities.This paper proposes SMINER,a novel approach that discovers vulnerabilities caused by unrestricted and misimplemented behaviors.SMINER first collects unit test cases for the target system from the official repository.Next,preprocess the collected code fragments.SMINER uses pre-processed data to show the security policies that can occur on the target system and creates a test case for security policy testing.To demonstrate the effectiveness of SMINER,this paper evaluates SMINER against Robot Operating System(ROS),a real-world system used for intelligent robots in Amazon and controlling satellites in National Aeronautics and Space Administration(NASA).From the evaluation,we discovered two real-world vulnerabilities in ROS.
基金supported in part by the National Key Research and Development Project of China(No.2019YFB2102500)the Natural Science Foundation of Hebei Province(No.F2018201115).
文摘With the continuous development of robotics and artificial intelligence,robots are being increasingly used in various applications.For traditional navigation algorithms,such as Dijkstra and A*,many dynamic scenarios in life are difficult to cope with.To solve the navigation problem of complex dynamic scenes,we present an improved reinforcement-learning-based algorithm for local path planning that allows it to perform well even when more dynamic obstacles are present.The method applies the gmapping algorithm as the upper layer input and uses reinforcement learning methods as the output.The algorithm enhances the robots’ability to actively avoid obstacles while retaining the adaptability of traditional methods.
基金support from the NASA University Leadership Initiative(ULI)under agreement number 2 CFR 200.514 Grant Number 80NSSC20M0161the U.S.Department of Transportation(USDOT)University Transportation Program(UTC)under Grant No.69A3552348304.
文摘Urban Air Mobility(UAM)is an emerging aviation sector which the goal is to transform air transportation with safe,on-demand air travel for both passengers and cargo.UAM flight planning strategically separates flows of aircraft on intersecting routes vertically by allocating distinct flight levels to them,and aircraft are required to maintain the flight level when crossing the intersection.However,there is a possibility that an aircraft may fail to maintain the assigned flight level,leading to a potential conflict at intersections.This paper aims to address conflicts at intersections in the context of UAM,focusing on decentralized conflict detection and resolution.A novel approach is developed to facilitate information exchange among UAM components,including the provider of services to UAM,UAM operators,and the pilot in command.A receding horizon trajectory planning approach is proposed for the execution of conflict resolution,optimizing trajectory planning by eliminating potential problems and challenges associated with geometric approaches.The proposed trajectory planner considers the model and constraints of UAM aircraft,offering optimal solutions for safe separation at UAM airspace intersections.The significance of the proposed planning framework is demonstrated through simulations considering conflict at intersections by communicating the UAM components through request and replay services and generating resolution maneuvers on-the-fly for each aircraft involved in the conflict.
