Learning from demonstration is widely regarded as a promising paradigm for robots to acquire diverse skills.Other than the artificial learning from observation-action pairs for machines,humans can learn to imitate in ...Learning from demonstration is widely regarded as a promising paradigm for robots to acquire diverse skills.Other than the artificial learning from observation-action pairs for machines,humans can learn to imitate in a more versatile and effective manner:acquiring skills through mere“observation”.Video to Command task is widely perceived as a promising approach for task-based learning,which yet faces two key challenges:(1)High redundancy and low frame rate of fine-grained action sequences make it difficult to manipulate objects robustly and accurately.(2)Video to Command models often prioritize accuracy and richness of output commands over physical capabilities,leading to impractical or unsafe instructions for robots.This article presents a novel Video to Command framework that employs multiple data associations and physical constraints.First,we introduce an object-level appearancecontrasting multiple data association strategy to effectively associate manipulated objects in visually complex environments,capturing dynamic changes in video content.Then,we propose a multi-task Video to Command model that utilizes object-level video content changes to compile expert demonstrations into manipulation commands.Finally,a multi-task hybrid loss function is proposed to train a Video to Command model that adheres to the constraints of the physical world and manipulation tasks.Our method achieved over 10%on BLEU_N,METEOR,ROUGE_L,and CIDEr compared to the up-to-date methods.The dual-arm robot prototype was established to demonstrate the whole process of learning from an expert demonstration of multiple skills and then executing the tasks by a robot.展开更多
森林火灾能对森林资源造成巨大的危害,因此发现林区火源并进行科学决策具有重要意义.本系统数据主要包括1∶1万地形图、1∶1万森林资源二类调查数据、扑火队分布数据、瞭望台站数据、扑火物资储备数据、林业局管理机构分布数据、研究区2...森林火灾能对森林资源造成巨大的危害,因此发现林区火源并进行科学决策具有重要意义.本系统数据主要包括1∶1万地形图、1∶1万森林资源二类调查数据、扑火队分布数据、瞭望台站数据、扑火物资储备数据、林业局管理机构分布数据、研究区2010年landsat TM遥感影像图等;系统功能主要包括数据的处理和管理、森林火灾监测、最佳扑火路径分析、防火扑火预案制定和决策、森林火险等级评价、林火趋势模拟和预测、灾害损失评估、相关图表和报告制作等.系统采用Microsoft Visual Studio 2005(C#)作为开发语言,基于Arc Engine 9.3组件进行二次开发,三维地图显示采用skyline Terra Explorer开发平台,数据库采用SQL Server 2005数据库管理系统,利用Arc SDE 9.3空间数据库引擎对空间数据进行管理.展开更多
基金Supported by Zhejiang Provincial Key Research and Development Program(Grant No.2021C04015)。
文摘Learning from demonstration is widely regarded as a promising paradigm for robots to acquire diverse skills.Other than the artificial learning from observation-action pairs for machines,humans can learn to imitate in a more versatile and effective manner:acquiring skills through mere“observation”.Video to Command task is widely perceived as a promising approach for task-based learning,which yet faces two key challenges:(1)High redundancy and low frame rate of fine-grained action sequences make it difficult to manipulate objects robustly and accurately.(2)Video to Command models often prioritize accuracy and richness of output commands over physical capabilities,leading to impractical or unsafe instructions for robots.This article presents a novel Video to Command framework that employs multiple data associations and physical constraints.First,we introduce an object-level appearancecontrasting multiple data association strategy to effectively associate manipulated objects in visually complex environments,capturing dynamic changes in video content.Then,we propose a multi-task Video to Command model that utilizes object-level video content changes to compile expert demonstrations into manipulation commands.Finally,a multi-task hybrid loss function is proposed to train a Video to Command model that adheres to the constraints of the physical world and manipulation tasks.Our method achieved over 10%on BLEU_N,METEOR,ROUGE_L,and CIDEr compared to the up-to-date methods.The dual-arm robot prototype was established to demonstrate the whole process of learning from an expert demonstration of multiple skills and then executing the tasks by a robot.
文摘森林火灾能对森林资源造成巨大的危害,因此发现林区火源并进行科学决策具有重要意义.本系统数据主要包括1∶1万地形图、1∶1万森林资源二类调查数据、扑火队分布数据、瞭望台站数据、扑火物资储备数据、林业局管理机构分布数据、研究区2010年landsat TM遥感影像图等;系统功能主要包括数据的处理和管理、森林火灾监测、最佳扑火路径分析、防火扑火预案制定和决策、森林火险等级评价、林火趋势模拟和预测、灾害损失评估、相关图表和报告制作等.系统采用Microsoft Visual Studio 2005(C#)作为开发语言,基于Arc Engine 9.3组件进行二次开发,三维地图显示采用skyline Terra Explorer开发平台,数据库采用SQL Server 2005数据库管理系统,利用Arc SDE 9.3空间数据库引擎对空间数据进行管理.
