This article describes a pilot study aiming at generating social interactions between a humanoid robot and adolescents with autism spectrum disorder (ASD), through the practice of a gesture imitation game. The partici...This article describes a pilot study aiming at generating social interactions between a humanoid robot and adolescents with autism spectrum disorder (ASD), through the practice of a gesture imitation game. The participants were a 17-year-old young lady with ASD and intellectual deficit, and a control participant: a preadolescent with ASD but no intellectual deficit (Asperger syndrome). The game is comprised of four phases: greetings, pairing, imitation, and closing. Field educators were involved, playing specific roles: visual or physical inciter. The use of a robot allows for catching the participants’ attention, playing the imitation game for a longer period of time than with a human partner, and preventing the game partner’s negative facial expressions resulting from tiredness, impatience, or boredom. The participants’ behavior was observed in terms of initial approach towards the robot, positioning relative to the robot in terms of distance and orientation, reactions to the robot’s voice or moves, signs of happiness, and imitation attempts. Results suggest a more and more natural approach towards the robot during the sessions, as well as a higher level of social interaction, based on the variations of the parameters listed above. We use these preliminary results to draw the next steps of our research work as well as identify further perspectives, with this aim in mind: improving social interactions with adolescents with ASD and intellectual deficit, allowing for better integration of these people into our societies.展开更多
The objective of this work is to develop an innovative system(ROSGPT)that merges large language models(LLMs)with the robot operating system(ROS),facilitating natural language voice control of mobile robots.This integr...The objective of this work is to develop an innovative system(ROSGPT)that merges large language models(LLMs)with the robot operating system(ROS),facilitating natural language voice control of mobile robots.This integration aims to bridge the gap between human-robot interaction(HRI)and artificial intelligence(AI).ROSGPT integrates several subsystems,including speech recognition,prompt engineering,LLM and ROS,enabling seamless control of robots through human voice or text commands.The LLM component is optimized,with its performance refined from the open-source Llama2 model through fine-tuning and quantization procedures.Through extensive experiments conducted in both real-world and virtual environments,ROSGPT demonstrates its efficacy in meeting user requirements and delivering user-friendly interactive experiences.The system demonstrates versatility and adaptability through its ability to comprehend diverse user commands and execute corresponding tasks with precision and reliability,thereby showcasing its potential for various practical applications in robotics and AI.The demonstration video can be viewed at https://iklxo6z9yv.feishu.cn/docx/Lux3dmTDxoZ5YnxWJTZcxUCWnTh.展开更多
基于语音识别设计了针对脑瘫患儿的数字语音训练系统。应用人机交互(HRI)技术与仿生机械手动作控制相结合达到提升脑瘫(CP)康复训练效果的目的。该系统中控制器采用Arduino MEGA 2560为主控制器,显示屏LCD1602作为人机交互数据显示界面...基于语音识别设计了针对脑瘫患儿的数字语音训练系统。应用人机交互(HRI)技术与仿生机械手动作控制相结合达到提升脑瘫(CP)康复训练效果的目的。该系统中控制器采用Arduino MEGA 2560为主控制器,显示屏LCD1602作为人机交互数据显示界面,通过LD3320语音芯片实现人机语音交互功能。人机交互功能是通过Labview环境展开,可实现人、机器人的手势与动作的实时交互训练与评价。该系统可训练脑瘫患者反应能力、语言表述能力以及认识手势动作数字动作能力,为提升脑瘫康复训练系统提供关键技术。展开更多
影像分割是面向对象影像分析中的重要步骤。为了提高高分辨率遥感影像(high-resolution remote sensing image,HRI)分割算法的性能,提出一种新的影像分割算法,包含种子确定、基于种子区域生长(seeded region growing,SRG)的过分割(advan...影像分割是面向对象影像分析中的重要步骤。为了提高高分辨率遥感影像(high-resolution remote sensing image,HRI)分割算法的性能,提出一种新的影像分割算法,包含种子确定、基于种子区域生长(seeded region growing,SRG)的过分割(advanced SRG,ASRG)和层次区域生长(hierarchical region growing,HRG)3个步骤。利用Gabor纹理特征定义纹理均匀性,将种子自动放置在HRI中同一纹理组成区域的中心位置;在SRG阶段,将HRI光谱信息与斑块形状信息相结合,提出了一种新的合并规则,以提高SRG过分割的精度与分割结果中各个斑块排列的紧凑性;在HRG阶段,提出了一种自适应的阈值,可以更好地保持多尺度分割的特性;在实验部分,采用3景HRI验证了上述方法。利用监督的影像分割评价方法定量评价了该方法的分割精度,并与另外2种主流的遥感影像分割算法进行了对比。结果表明,该方法可以得到令人满意的分割效果。展开更多
文摘This article describes a pilot study aiming at generating social interactions between a humanoid robot and adolescents with autism spectrum disorder (ASD), through the practice of a gesture imitation game. The participants were a 17-year-old young lady with ASD and intellectual deficit, and a control participant: a preadolescent with ASD but no intellectual deficit (Asperger syndrome). The game is comprised of four phases: greetings, pairing, imitation, and closing. Field educators were involved, playing specific roles: visual or physical inciter. The use of a robot allows for catching the participants’ attention, playing the imitation game for a longer period of time than with a human partner, and preventing the game partner’s negative facial expressions resulting from tiredness, impatience, or boredom. The participants’ behavior was observed in terms of initial approach towards the robot, positioning relative to the robot in terms of distance and orientation, reactions to the robot’s voice or moves, signs of happiness, and imitation attempts. Results suggest a more and more natural approach towards the robot during the sessions, as well as a higher level of social interaction, based on the variations of the parameters listed above. We use these preliminary results to draw the next steps of our research work as well as identify further perspectives, with this aim in mind: improving social interactions with adolescents with ASD and intellectual deficit, allowing for better integration of these people into our societies.
基金National Natural Science Foundation of China(No.61601112)。
文摘The objective of this work is to develop an innovative system(ROSGPT)that merges large language models(LLMs)with the robot operating system(ROS),facilitating natural language voice control of mobile robots.This integration aims to bridge the gap between human-robot interaction(HRI)and artificial intelligence(AI).ROSGPT integrates several subsystems,including speech recognition,prompt engineering,LLM and ROS,enabling seamless control of robots through human voice or text commands.The LLM component is optimized,with its performance refined from the open-source Llama2 model through fine-tuning and quantization procedures.Through extensive experiments conducted in both real-world and virtual environments,ROSGPT demonstrates its efficacy in meeting user requirements and delivering user-friendly interactive experiences.The system demonstrates versatility and adaptability through its ability to comprehend diverse user commands and execute corresponding tasks with precision and reliability,thereby showcasing its potential for various practical applications in robotics and AI.The demonstration video can be viewed at https://iklxo6z9yv.feishu.cn/docx/Lux3dmTDxoZ5YnxWJTZcxUCWnTh.
文摘基于语音识别设计了针对脑瘫患儿的数字语音训练系统。应用人机交互(HRI)技术与仿生机械手动作控制相结合达到提升脑瘫(CP)康复训练效果的目的。该系统中控制器采用Arduino MEGA 2560为主控制器,显示屏LCD1602作为人机交互数据显示界面,通过LD3320语音芯片实现人机语音交互功能。人机交互功能是通过Labview环境展开,可实现人、机器人的手势与动作的实时交互训练与评价。该系统可训练脑瘫患者反应能力、语言表述能力以及认识手势动作数字动作能力,为提升脑瘫康复训练系统提供关键技术。
文摘影像分割是面向对象影像分析中的重要步骤。为了提高高分辨率遥感影像(high-resolution remote sensing image,HRI)分割算法的性能,提出一种新的影像分割算法,包含种子确定、基于种子区域生长(seeded region growing,SRG)的过分割(advanced SRG,ASRG)和层次区域生长(hierarchical region growing,HRG)3个步骤。利用Gabor纹理特征定义纹理均匀性,将种子自动放置在HRI中同一纹理组成区域的中心位置;在SRG阶段,将HRI光谱信息与斑块形状信息相结合,提出了一种新的合并规则,以提高SRG过分割的精度与分割结果中各个斑块排列的紧凑性;在HRG阶段,提出了一种自适应的阈值,可以更好地保持多尺度分割的特性;在实验部分,采用3景HRI验证了上述方法。利用监督的影像分割评价方法定量评价了该方法的分割精度,并与另外2种主流的遥感影像分割算法进行了对比。结果表明,该方法可以得到令人满意的分割效果。
