The objective of this work is to develop an innovative system(ROSGPT)that merges large language models(LLMs)with the robot operating system(ROS),facilitating natural language voice control of mobile robots.This integr...The objective of this work is to develop an innovative system(ROSGPT)that merges large language models(LLMs)with the robot operating system(ROS),facilitating natural language voice control of mobile robots.This integration aims to bridge the gap between human-robot interaction(HRI)and artificial intelligence(AI).ROSGPT integrates several subsystems,including speech recognition,prompt engineering,LLM and ROS,enabling seamless control of robots through human voice or text commands.The LLM component is optimized,with its performance refined from the open-source Llama2 model through fine-tuning and quantization procedures.Through extensive experiments conducted in both real-world and virtual environments,ROSGPT demonstrates its efficacy in meeting user requirements and delivering user-friendly interactive experiences.The system demonstrates versatility and adaptability through its ability to comprehend diverse user commands and execute corresponding tasks with precision and reliability,thereby showcasing its potential for various practical applications in robotics and AI.The demonstration video can be viewed at https://iklxo6z9yv.feishu.cn/docx/Lux3dmTDxoZ5YnxWJTZcxUCWnTh.展开更多
Traditional cochlear implantation surgery has problems such as high surgical accuracy requirement and large trauma,which cause the difficulty of the operation and the high requirements for doctors,so that only a few d...Traditional cochlear implantation surgery has problems such as high surgical accuracy requirement and large trauma,which cause the difficulty of the operation and the high requirements for doctors,so that only a few doctors can complete the operation independently.However,there is no research on robotic cochlear implantation in China.In response to this problem,a robotic cochlear implantation system is proposed.The robot is controlled by robot operating system(ROS).A simulation environment for the overall surgery is established on the ROS based on the real surgery environment.Through the analysis of the kinematics and the motion planning algorithm of the manipulator,an appropriate motion mode is designed to control the motion of the manipulator,and perform the surgery under the simulation environment.A simple and feasible method of navigation is proposed,and through the model experiment,the feasibility of robotic cochlear implantation surgery is verified.展开更多
在停车场、隧道中GPS、Wi-Fi信号受限的情况下,提出一种基于激光雷达的车辆自主定位方法。采用激光雷达SLAM(simultaneous localization and mapping)算法,通过三维激光雷达点云匹配获取车辆的估计位姿;根据图优化方法和非线性优化方法...在停车场、隧道中GPS、Wi-Fi信号受限的情况下,提出一种基于激光雷达的车辆自主定位方法。采用激光雷达SLAM(simultaneous localization and mapping)算法,通过三维激光雷达点云匹配获取车辆的估计位姿;根据图优化方法和非线性优化方法,对所有位姿进行后端调整,进而得到分辨率可控的环境信息平面栅格地图;基于蒙特卡洛方法,采用粒子滤波器进行实时车辆定位,并提出了粒子采样的一种改善方式,实现了较高精度的激光雷达自主定位。结果表明:粒子滤波器能够有效地实现车辆在停车场等无GPS环境下的定位,定位精度在10 cm之内。展开更多
With the continuous development of robotics and artificial intelligence,robots are being increasingly used in various applications.For traditional navigation algorithms,such as Dijkstra and A*,many dynamic scenarios i...With the continuous development of robotics and artificial intelligence,robots are being increasingly used in various applications.For traditional navigation algorithms,such as Dijkstra and A*,many dynamic scenarios in life are difficult to cope with.To solve the navigation problem of complex dynamic scenes,we present an improved reinforcement-learning-based algorithm for local path planning that allows it to perform well even when more dynamic obstacles are present.The method applies the gmapping algorithm as the upper layer input and uses reinforcement learning methods as the output.The algorithm enhances the robots’ability to actively avoid obstacles while retaining the adaptability of traditional methods.展开更多
Urban Air Mobility(UAM)is an emerging aviation sector which the goal is to transform air transportation with safe,on-demand air travel for both passengers and cargo.UAM flight planning strategically separates flows of...Urban Air Mobility(UAM)is an emerging aviation sector which the goal is to transform air transportation with safe,on-demand air travel for both passengers and cargo.UAM flight planning strategically separates flows of aircraft on intersecting routes vertically by allocating distinct flight levels to them,and aircraft are required to maintain the flight level when crossing the intersection.However,there is a possibility that an aircraft may fail to maintain the assigned flight level,leading to a potential conflict at intersections.This paper aims to address conflicts at intersections in the context of UAM,focusing on decentralized conflict detection and resolution.A novel approach is developed to facilitate information exchange among UAM components,including the provider of services to UAM,UAM operators,and the pilot in command.A receding horizon trajectory planning approach is proposed for the execution of conflict resolution,optimizing trajectory planning by eliminating potential problems and challenges associated with geometric approaches.The proposed trajectory planner considers the model and constraints of UAM aircraft,offering optimal solutions for safe separation at UAM airspace intersections.The significance of the proposed planning framework is demonstrated through simulations considering conflict at intersections by communicating the UAM components through request and replay services and generating resolution maneuvers on-the-fly for each aircraft involved in the conflict.展开更多
基金National Natural Science Foundation of China(No.61601112)。
文摘The objective of this work is to develop an innovative system(ROSGPT)that merges large language models(LLMs)with the robot operating system(ROS),facilitating natural language voice control of mobile robots.This integration aims to bridge the gap between human-robot interaction(HRI)and artificial intelligence(AI).ROSGPT integrates several subsystems,including speech recognition,prompt engineering,LLM and ROS,enabling seamless control of robots through human voice or text commands.The LLM component is optimized,with its performance refined from the open-source Llama2 model through fine-tuning and quantization procedures.Through extensive experiments conducted in both real-world and virtual environments,ROSGPT demonstrates its efficacy in meeting user requirements and delivering user-friendly interactive experiences.The system demonstrates versatility and adaptability through its ability to comprehend diverse user commands and execute corresponding tasks with precision and reliability,thereby showcasing its potential for various practical applications in robotics and AI.The demonstration video can be viewed at https://iklxo6z9yv.feishu.cn/docx/Lux3dmTDxoZ5YnxWJTZcxUCWnTh.
基金the National Natural Science Foundation of China(Nos.61973211,62133009,51911540479 and M-0221)the Science and Technology Commission of Shanghai Municipality(Nos.21550714200 and 20DZ2220400)+1 种基金the Research Project of Institute of Medical Robotics of Shanghai Jiao Tong Universitythe Interdisciplinary Program of Shanghai Jiao Tong University(Nos.YG2017ZD03 and ZH2018QNB31)。
文摘Traditional cochlear implantation surgery has problems such as high surgical accuracy requirement and large trauma,which cause the difficulty of the operation and the high requirements for doctors,so that only a few doctors can complete the operation independently.However,there is no research on robotic cochlear implantation in China.In response to this problem,a robotic cochlear implantation system is proposed.The robot is controlled by robot operating system(ROS).A simulation environment for the overall surgery is established on the ROS based on the real surgery environment.Through the analysis of the kinematics and the motion planning algorithm of the manipulator,an appropriate motion mode is designed to control the motion of the manipulator,and perform the surgery under the simulation environment.A simple and feasible method of navigation is proposed,and through the model experiment,the feasibility of robotic cochlear implantation surgery is verified.
文摘在停车场、隧道中GPS、Wi-Fi信号受限的情况下,提出一种基于激光雷达的车辆自主定位方法。采用激光雷达SLAM(simultaneous localization and mapping)算法,通过三维激光雷达点云匹配获取车辆的估计位姿;根据图优化方法和非线性优化方法,对所有位姿进行后端调整,进而得到分辨率可控的环境信息平面栅格地图;基于蒙特卡洛方法,采用粒子滤波器进行实时车辆定位,并提出了粒子采样的一种改善方式,实现了较高精度的激光雷达自主定位。结果表明:粒子滤波器能够有效地实现车辆在停车场等无GPS环境下的定位,定位精度在10 cm之内。
基金supported in part by the National Key Research and Development Project of China(No.2019YFB2102500)the Natural Science Foundation of Hebei Province(No.F2018201115).
文摘With the continuous development of robotics and artificial intelligence,robots are being increasingly used in various applications.For traditional navigation algorithms,such as Dijkstra and A*,many dynamic scenarios in life are difficult to cope with.To solve the navigation problem of complex dynamic scenes,we present an improved reinforcement-learning-based algorithm for local path planning that allows it to perform well even when more dynamic obstacles are present.The method applies the gmapping algorithm as the upper layer input and uses reinforcement learning methods as the output.The algorithm enhances the robots’ability to actively avoid obstacles while retaining the adaptability of traditional methods.
基金support from the NASA University Leadership Initiative(ULI)under agreement number 2 CFR 200.514 Grant Number 80NSSC20M0161the U.S.Department of Transportation(USDOT)University Transportation Program(UTC)under Grant No.69A3552348304.
文摘Urban Air Mobility(UAM)is an emerging aviation sector which the goal is to transform air transportation with safe,on-demand air travel for both passengers and cargo.UAM flight planning strategically separates flows of aircraft on intersecting routes vertically by allocating distinct flight levels to them,and aircraft are required to maintain the flight level when crossing the intersection.However,there is a possibility that an aircraft may fail to maintain the assigned flight level,leading to a potential conflict at intersections.This paper aims to address conflicts at intersections in the context of UAM,focusing on decentralized conflict detection and resolution.A novel approach is developed to facilitate information exchange among UAM components,including the provider of services to UAM,UAM operators,and the pilot in command.A receding horizon trajectory planning approach is proposed for the execution of conflict resolution,optimizing trajectory planning by eliminating potential problems and challenges associated with geometric approaches.The proposed trajectory planner considers the model and constraints of UAM aircraft,offering optimal solutions for safe separation at UAM airspace intersections.The significance of the proposed planning framework is demonstrated through simulations considering conflict at intersections by communicating the UAM components through request and replay services and generating resolution maneuvers on-the-fly for each aircraft involved in the conflict.
