The objective of this work is to develop an innovative system(ROSGPT)that merges large language models(LLMs)with the robot operating system(ROS),facilitating natural language voice control of mobile robots.This integr...The objective of this work is to develop an innovative system(ROSGPT)that merges large language models(LLMs)with the robot operating system(ROS),facilitating natural language voice control of mobile robots.This integration aims to bridge the gap between human-robot interaction(HRI)and artificial intelligence(AI).ROSGPT integrates several subsystems,including speech recognition,prompt engineering,LLM and ROS,enabling seamless control of robots through human voice or text commands.The LLM component is optimized,with its performance refined from the open-source Llama2 model through fine-tuning and quantization procedures.Through extensive experiments conducted in both real-world and virtual environments,ROSGPT demonstrates its efficacy in meeting user requirements and delivering user-friendly interactive experiences.The system demonstrates versatility and adaptability through its ability to comprehend diverse user commands and execute corresponding tasks with precision and reliability,thereby showcasing its potential for various practical applications in robotics and AI.The demonstration video can be viewed at https://iklxo6z9yv.feishu.cn/docx/Lux3dmTDxoZ5YnxWJTZcxUCWnTh.展开更多
Traditional cochlear implantation surgery has problems such as high surgical accuracy requirement and large trauma,which cause the difficulty of the operation and the high requirements for doctors,so that only a few d...Traditional cochlear implantation surgery has problems such as high surgical accuracy requirement and large trauma,which cause the difficulty of the operation and the high requirements for doctors,so that only a few doctors can complete the operation independently.However,there is no research on robotic cochlear implantation in China.In response to this problem,a robotic cochlear implantation system is proposed.The robot is controlled by robot operating system(ROS).A simulation environment for the overall surgery is established on the ROS based on the real surgery environment.Through the analysis of the kinematics and the motion planning algorithm of the manipulator,an appropriate motion mode is designed to control the motion of the manipulator,and perform the surgery under the simulation environment.A simple and feasible method of navigation is proposed,and through the model experiment,the feasibility of robotic cochlear implantation surgery is verified.展开更多
探讨基于机器人操作系统(Robot Operating System,ROS)的智能挖掘机控制系统设计方案。构建一个模块化的系统架构,包含感知模块、决策模块、路径规划模块、控制模块以及仿真模块。通过深入比较快速扩展随机树(Rapidly-exploring Random ...探讨基于机器人操作系统(Robot Operating System,ROS)的智能挖掘机控制系统设计方案。构建一个模块化的系统架构,包含感知模块、决策模块、路径规划模块、控制模块以及仿真模块。通过深入比较快速扩展随机树(Rapidly-exploring Random Tree,RRT)算法和RRT-connect算法,发现RRT-connect算法更适合挖掘机运动规划,所设计的系统在复杂动态环境下展现出良好的规划效率和运行稳定性,为智能挖掘机的实际应用提供了可靠的技术支持。展开更多
针对快速搜索随机树(rapidly-exploring random tree,RRT)算法的随机采样特征导致的收敛速度慢、路径冗余度高、采样点利用率低问题,给出一种新的解决方法。首先,根据图复杂度公式,计算出图的复杂度后确定目标偏执概率,建立偏置概率自...针对快速搜索随机树(rapidly-exploring random tree,RRT)算法的随机采样特征导致的收敛速度慢、路径冗余度高、采样点利用率低问题,给出一种新的解决方法。首先,根据图复杂度公式,计算出图的复杂度后确定目标偏执概率,建立偏置概率自适应模型;其次,在首次规划好路线后,路径中仍存在一些不必要的拐点与棱角,针对传统路径裁剪依赖局部搜索策略,可能导致次优解生成,提出PRM-Dijkstra(probabilistic roadmap-dijkstra)算法对路径进行裁剪,将改进RRT算法生成的树节点利用PRM算法相互连接起来,通过Dijkstra算法计算出一条最优路径;最后,改进RRT算法与PRM-Dijkstra种算法优势相结合,在保证有一条路径的前提下,最大概率的寻找最优路径。通过复杂图下仿真避障实验,结果显示:改进RRT算法在节点生成数量与规划用时相较传统RRT算法平均减少80%,相较于Goal-bias RRT算法均减少40%。并通过机器人操作系统(robot operating system,ROS)下的MoveIt!集成开发平台进行现实环境下避障实验,验证了算法的可行性与有效性。展开更多
基金National Natural Science Foundation of China(No.61601112)。
文摘The objective of this work is to develop an innovative system(ROSGPT)that merges large language models(LLMs)with the robot operating system(ROS),facilitating natural language voice control of mobile robots.This integration aims to bridge the gap between human-robot interaction(HRI)and artificial intelligence(AI).ROSGPT integrates several subsystems,including speech recognition,prompt engineering,LLM and ROS,enabling seamless control of robots through human voice or text commands.The LLM component is optimized,with its performance refined from the open-source Llama2 model through fine-tuning and quantization procedures.Through extensive experiments conducted in both real-world and virtual environments,ROSGPT demonstrates its efficacy in meeting user requirements and delivering user-friendly interactive experiences.The system demonstrates versatility and adaptability through its ability to comprehend diverse user commands and execute corresponding tasks with precision and reliability,thereby showcasing its potential for various practical applications in robotics and AI.The demonstration video can be viewed at https://iklxo6z9yv.feishu.cn/docx/Lux3dmTDxoZ5YnxWJTZcxUCWnTh.
基金the National Natural Science Foundation of China(Nos.61973211,62133009,51911540479 and M-0221)the Science and Technology Commission of Shanghai Municipality(Nos.21550714200 and 20DZ2220400)+1 种基金the Research Project of Institute of Medical Robotics of Shanghai Jiao Tong Universitythe Interdisciplinary Program of Shanghai Jiao Tong University(Nos.YG2017ZD03 and ZH2018QNB31)。
文摘Traditional cochlear implantation surgery has problems such as high surgical accuracy requirement and large trauma,which cause the difficulty of the operation and the high requirements for doctors,so that only a few doctors can complete the operation independently.However,there is no research on robotic cochlear implantation in China.In response to this problem,a robotic cochlear implantation system is proposed.The robot is controlled by robot operating system(ROS).A simulation environment for the overall surgery is established on the ROS based on the real surgery environment.Through the analysis of the kinematics and the motion planning algorithm of the manipulator,an appropriate motion mode is designed to control the motion of the manipulator,and perform the surgery under the simulation environment.A simple and feasible method of navigation is proposed,and through the model experiment,the feasibility of robotic cochlear implantation surgery is verified.
文摘探讨基于机器人操作系统(Robot Operating System,ROS)的智能挖掘机控制系统设计方案。构建一个模块化的系统架构,包含感知模块、决策模块、路径规划模块、控制模块以及仿真模块。通过深入比较快速扩展随机树(Rapidly-exploring Random Tree,RRT)算法和RRT-connect算法,发现RRT-connect算法更适合挖掘机运动规划,所设计的系统在复杂动态环境下展现出良好的规划效率和运行稳定性,为智能挖掘机的实际应用提供了可靠的技术支持。
