The integration of Human-Robot Collaboration(HRC)into Virtual Reality(VR)technology is transforming industries by enhancing workforce skills,improving safety,and optimizing operational processes and efficiency through...The integration of Human-Robot Collaboration(HRC)into Virtual Reality(VR)technology is transforming industries by enhancing workforce skills,improving safety,and optimizing operational processes and efficiency through realistic simulations of industry-specific scenarios.Despite the growing adoption of VR integrated with HRC,comprehensive reviews of current research in HRC-VR within the construction and manufacturing fields are lacking.This review examines the latest advances in designing and implementing HRC using VR technology in these industries.The aim is to address the application domains of HRC-VR,types of robots used,VR setups,and software solutions used.To achieve this,a systematic literature review using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses methodology was conducted on the Web of Science and Google Scholar databases,analyzing 383 articles and selecting 53 papers that met the established selection criteria.The findings emphasize a significant focus on enhancing human-robot interaction with a trend toward using immersive VR experiences and interactive 3D content creation tools.However,the integration of HRC with VR,especially in the dynamic construction environment,presents unique challenges and opportunities for future research,including developing more realistic simulations and adaptable robot systems.This paper offers insights for researchers,practitioners,educators,industry professionals,and policymakers interested in leveraging the integration of HRC with VR in construction and manufacturing industries.展开更多
Despite the gradual transformation of traditional manufacturing by the Human-Robot Collaboration Assembly(HRCA),challenges remain in the robot’s ability to understand and predict human assembly intentions.This study ...Despite the gradual transformation of traditional manufacturing by the Human-Robot Collaboration Assembly(HRCA),challenges remain in the robot’s ability to understand and predict human assembly intentions.This study aims to enhance the robot’s comprehension and prediction capabilities of operator assembly intentions by capturing and analyzing operator behavior and movements.We propose a video feature extraction method based on the Temporal Shift Module Network(TSM-ResNet50)to extract spatiotemporal features from assembly videos and differentiate various assembly actions using feature differences between video frames.Furthermore,we construct an action recognition and segmentation model based on the Refined-Multi-Scale Temporal Convolutional Network(Refined-MS-TCN)to identify assembly action intervals and accurately acquire action categories.Experiments on our self-built reducer assembly action dataset demonstrate that our network can classify assembly actions frame by frame,achieving an accuracy rate of 83%.Additionally,we develop a HiddenMarkovModel(HMM)integrated with assembly task constraints to predict operator assembly intentions based on the probability transition matrix and assembly task constraints.The experimental results show that our method for predicting operator assembly intentions can achieve an accuracy of 90.6%,which is a 13.3%improvement over the HMM without task constraints.展开更多
The wearable exoskeleton system is a typical strongly coupled human-robotic system.Human-robotic is the environment for each other.The two support each other and compete with each other.Achieving high human-robotic co...The wearable exoskeleton system is a typical strongly coupled human-robotic system.Human-robotic is the environment for each other.The two support each other and compete with each other.Achieving high human-robotic compatibility is the most critical technology for wearable systems.Full structural compatibility can improve the intrinsic safety of the exoskeleton,and precise intention understanding and motion control can improve the comfort of the exoskeleton.This paper first designs a physiologically functional bionic lower limb exoskeleton based on the study of bone and joint functional anatomy and analyzes the drive mapping model of the dual closedloop four-link knee joint.Secondly,an exoskeleton dual closed-loop controller composed of a position inner loop and a force outer loop is designed.The inner loop of the controller adopts the PID control algorithm,and the outer loop adopts the adaptive admittance control algorithm based on human-robot interaction force(HRI).The controller can adaptively adjust the admittance parameters according to the HRI to respond to dynamic changes in the mechanical and physical parameters of the human-robot system,thereby improving control compliance and the wearing comfort of the exoskeleton system.Finally,we built a joint simulation experiment platform based on SolidWorks/Simulink to conduct virtual prototype simulation experiments and recruited volunteers to wear rehabilitation exoskeletons to conduct related control experiments.Experimental results show that the designed physiologically functional bionic exoskeleton and adaptive admittance controller can significantly improve the accuracy of human-robotic joint motion tracking,effectively reducing human-machine interaction forces and improving the comfort and safety of the wearer.This paper proposes a dual-closed loop four-link knee joint exoskeleton and a variable admittance control method based on HRI,which provides a new method for the design and control of exoskeletons with high compatibility.展开更多
A real-time adaptive roles allocation method based on reinforcement learning is proposed to improve humanrobot cooperation performance for a curtain wall installation task.This method breaks the traditional idea that ...A real-time adaptive roles allocation method based on reinforcement learning is proposed to improve humanrobot cooperation performance for a curtain wall installation task.This method breaks the traditional idea that the robot is regarded as the follower or only adjusts the leader and the follower in cooperation.In this paper,a self-learning method is proposed which can dynamically adapt and continuously adjust the initiative weight of the robot according to the change of the task.Firstly,the physical human-robot cooperation model,including the role factor is built.Then,a reinforcement learningmodel that can adjust the role factor in real time is established,and a reward and actionmodel is designed.The role factor can be adjusted continuously according to the comprehensive performance of the human-robot interaction force and the robot’s Jerk during the repeated installation.Finally,the roles adjustment rule established above continuously improves the comprehensive performance.Experiments of the dynamic roles allocation and the effect of the performance weighting coefficient on the result have been verified.The results show that the proposed method can realize the role adaptation and achieve the dual optimization goal of reducing the sum of the cooperator force and the robot’s Jerk.展开更多
This paper presents an innovative investigation on prototyping a digital twin(DT)as the platform for human-robot interactive welding and welder behavior analysis.This humanrobot interaction(HRI)working style helps to ...This paper presents an innovative investigation on prototyping a digital twin(DT)as the platform for human-robot interactive welding and welder behavior analysis.This humanrobot interaction(HRI)working style helps to enhance human users'operational productivity and comfort;while data-driven welder behavior analysis benefits to further novice welder training.This HRI system includes three modules:1)a human user who demonstrates the welding operations offsite with her/his operations recorded by the motion-tracked handles;2)a robot that executes the demonstrated welding operations to complete the physical welding tasks onsite;3)a DT system that is developed based on virtual reality(VR)as a digital replica of the physical human-robot interactive welding environment.The DT system bridges a human user and robot through a bi-directional information flow:a)transmitting demonstrated welding operations in VR to the robot in the physical environment;b)displaying the physical welding scenes to human users in VR.Compared to existing DT systems reported in the literatures,the developed one provides better capability in engaging human users in interacting with welding scenes,through an augmented VR.To verify the effectiveness,six welders,skilled with certain manual welding training and unskilled without any training,tested the system by completing the same welding job;three skilled welders produce satisfied welded workpieces,while the other three unskilled do not.A data-driven approach as a combination of fast Fourier transform(FFT),principal component analysis(PCA),and support vector machine(SVM)is developed to analyze their behaviors.Given an operation sequence,i.e.,motion speed sequence of the welding torch,frequency features are firstly extracted by FFT and then reduced in dimension through PCA,which are finally routed into SVM for classification.The trained model demonstrates a 94.44%classification accuracy in the testing dataset.The successful pattern recognition in skilled welder operations should benefit to accelerate novice welder training.展开更多
A facial expression emotion recognition based human-robot interaction(FEER-HRI) system is proposed, for which a four-layer system framework is designed. The FEERHRI system enables the robots not only to recognize huma...A facial expression emotion recognition based human-robot interaction(FEER-HRI) system is proposed, for which a four-layer system framework is designed. The FEERHRI system enables the robots not only to recognize human emotions, but also to generate facial expression for adapting to human emotions. A facial emotion recognition method based on2D-Gabor, uniform local binary pattern(LBP) operator, and multiclass extreme learning machine(ELM) classifier is presented,which is applied to real-time facial expression recognition for robots. Facial expressions of robots are represented by simple cartoon symbols and displayed by a LED screen equipped in the robots, which can be easily understood by human. Four scenarios,i.e., guiding, entertainment, home service and scene simulation are performed in the human-robot interaction experiment, in which smooth communication is realized by facial expression recognition of humans and facial expression generation of robots within 2 seconds. As a few prospective applications, the FEERHRI system can be applied in home service, smart home, safe driving, and so on.展开更多
This paper presents a novel enhanced human-robot interaction system based on model reference adaptive control. The presented method delivers guaranteed stability and task performance and has two control loops. A robot...This paper presents a novel enhanced human-robot interaction system based on model reference adaptive control. The presented method delivers guaranteed stability and task performance and has two control loops. A robot-specific inner loop, which is a neuroadaptive controller, learns the robot dynamics online and makes the robot respond like a prescribed impedance model. This loop uses no task information, including no prescribed trajectory. A task-specific outer loop takes into account the human operator dynamics and adapts the prescribed robot impedance model so that the combined human-robot system has desirable characteristics for task performance. This design is based on model reference adaptive control, but of a nonstandard form. The net result is a controller with both adaptive impedance characteristics and assistive inputs that augment the human operator to provide improved task performance of the human-robot team. Simulations verify the performance of the proposed controller in a repetitive point-to-point motion task. Actual experimental implementations on a PR2 robot further corroborate the effectiveness of the approach.展开更多
In this paper,we present a novel data-driven design method for the human-robot interaction(HRI)system,where a given task is achieved by cooperation between the human and the robot.The presented HRI controller design i...In this paper,we present a novel data-driven design method for the human-robot interaction(HRI)system,where a given task is achieved by cooperation between the human and the robot.The presented HRI controller design is a two-level control design approach consisting of a task-oriented performance optimization design and a plant-oriented impedance controller design.The task-oriented design minimizes the human effort and guarantees the perfect task tracking in the outer-loop,while the plant-oriented achieves the desired impedance from the human to the robot manipulator end-effector in the inner-loop.Data-driven reinforcement learning techniques are used for performance optimization in the outer-loop to assign the optimal impedance parameters.In the inner-loop,a velocity-free filter is designed to avoid the requirement of end-effector velocity measurement.On this basis,an adaptive controller is designed to achieve the desired impedance of the robot manipulator in the task space.The simulation and experiment of a robot manipulator are conducted to verify the efficacy of the presented HRI design framework.展开更多
With the increasing presence of robots in our daily life,there is a strong need and demand for the strategies to acquire a high quality interaction between robots and users by enabling robots to understand users’mood...With the increasing presence of robots in our daily life,there is a strong need and demand for the strategies to acquire a high quality interaction between robots and users by enabling robots to understand users’mood,intention,and other aspects.During human-human interaction,personality traits have an important influence on human behavior,decision,mood,and many others.Therefore,we propose an efficient computational framework to endow the robot with the capability of understanding the user’s personality traits based on the user’s nonverbal communication cues represented by three visual features including the head motion,gaze,and body motion energy,and three vocal features including voice pitch,voice energy,and mel-frequency cepstral coefficient(MFCC).We used the Pepper robot in this study as a communication robot to interact with each participant by asking questions,and meanwhile,the robot extracts the nonverbal features from each participant’s habitual behavior using its on-board sensors.On the other hand,each participant’s personality traits are evaluated with a questionnaire.We then train the ridge regression and linear support vector machine(SVM)classifiers using the nonverbal features and personality trait labels from a questionnaire and evaluate the performance of the classifiers.We have verified the validity of the proposed models that showed promising binary classification performance on recognizing each of the Big Five personality traits of the participants based on individual differences in nonverbal communication cues.展开更多
A more natural way for non-expert users to express their tasks in an open-ended set is to use natural language. In this case,a human-centered intelligent agent/robot is required to be able to understand and generate p...A more natural way for non-expert users to express their tasks in an open-ended set is to use natural language. In this case,a human-centered intelligent agent/robot is required to be able to understand and generate plans for these naturally expressed tasks. For this purpose, it is a good way to enhance intelligent robot's abilities by utilizing open knowledge extracted from the web, instead of hand-coded knowledge. A key challenge of utilizing open knowledge lies in the semantic interpretation of the open knowledge organized in multiple modes, which can be unstructured or semi-structured, before one can use it.Previous approaches used a limited lexicon to employ combinatory categorial grammar(CCG) as the underlying formalism for semantic parsing over sentences. Here, we propose a more effective learning method to interpret semi-structured user instructions. Moreover, we present a new heuristic method to recover missing semantic information from the context of an instruction. Experiments showed that the proposed approach renders significant performance improvement compared to the baseline methods and the recovering method is promising.展开更多
Human-robot interaction(HRI) is fundamental for human-centered robotics, and has been attracting intensive research for more than a decade. The series elastic actuator(SEA) provides inherent compliance, safety and fur...Human-robot interaction(HRI) is fundamental for human-centered robotics, and has been attracting intensive research for more than a decade. The series elastic actuator(SEA) provides inherent compliance, safety and further benefits for HRI, but the introduced elastic element also brings control difficulties. In this paper, we address the stiffness rendering problem for a cable-driven SEA system, to achieve either low stiffness for good transparency or high stiffness bigger than the physical spring constant, and to assess the rendering accuracy with quantified metrics. By taking a velocity-sourced model of the motor, a cascaded velocity-torque-impedance control structure is established. To achieve high fidelity torque control, the 2-DOF(degree of freedom) stabilizing control method together with a compensator has been used to handle the competing requirements on tracking performance, noise and disturbance rejection,and energy optimization in the cable-driven SEA system. The conventional passivity requirement for HRI usually leads to a conservative design of the impedance controller, and the rendered stiffness cannot go higher than the physical spring constant. By adding a phase-lead compensator into the impedance controller,the stiffness rendering capability was augmented with guaranteed relaxed passivity. Extensive simulations and experiments have been performed, and the virtual stiffness has been rendered in the extended range of 0.1 to 2.0 times of the physical spring constant with guaranteed relaxed passivity for physical humanrobot interaction below 5 Hz. Quantified metrics also verified good rendering accuracy.展开更多
With the increasing of the elderly population and the growing hearth care cost, the role of service robots in aiding the disabled and the elderly is becoming important. Many researchers in the world have paid much att...With the increasing of the elderly population and the growing hearth care cost, the role of service robots in aiding the disabled and the elderly is becoming important. Many researchers in the world have paid much attention to heaRthcare robots and rehabilitation robots. To get natural and harmonious communication between the user and a service robot, the information perception/feedback ability, and interaction ability for service robots become more important in many key issues.展开更多
This paper proposes a novel approach for physical human-robot interactions(pHRI), where a robot provides guidance forces to a user based on the user performance. This framework tunes the forces in regards to behavior ...This paper proposes a novel approach for physical human-robot interactions(pHRI), where a robot provides guidance forces to a user based on the user performance. This framework tunes the forces in regards to behavior of each user in coping with different tasks, where lower performance results in higher intervention from the robot. This personalized physical human-robot interaction(p2HRI) method incorporates adaptive modeling of the interaction between the human and the robot as well as learning from demonstration(LfD) techniques to adapt to the users' performance. This approach is based on model predictive control where the system optimizes the rendered forces by predicting the performance of the user. Moreover, continuous learning of the user behavior is added so that the models and personalized considerations are updated based on the change of user performance over time. Applying this framework to a field such as haptic guidance for skill improvement, allows a more personalized learning experience where the interaction between the robot as the intelligent tutor and the student as the user,is better adjusted based on the skill level of the individual and their gradual improvement. The results suggest that the precision of the model of the interaction is improved using this proposed method,and the addition of the considered personalized factors to a more adaptive strategy for rendering of guidance forces.展开更多
Human–robot(HR)collaboration(HRC)is an emerging research field because of the complementary advantages of humans and robots.An HRC framework for robotic assembly based on impedance control is proposed in this paper.I...Human–robot(HR)collaboration(HRC)is an emerging research field because of the complementary advantages of humans and robots.An HRC framework for robotic assembly based on impedance control is proposed in this paper.In the HRC framework,the human is the decision maker,the robot acts as the executor,while the assembly environment provides constraints.The robot is the main executor to perform the assembly action,which has the position control,drag and drop,positive impedance control,and negative impedance control modes.To reveal the characteristics of the HRC framework,the switch condition map of different control modes and the stability analysis of the HR coupled system are discussed.In the end,HRC assembly experiments are conducted,where the HRC assembly task can be accomplished when the assembling tolerance is 0.08 mm or with the interference fit.Experiments show that the HRC assembly has the complementary advantages of humans and robots and is efficient in finishing complex assembly tasks.展开更多
In current small batch and customized production mode,the products change rapidly and the personal demand increases sharply.Human-robot cooperation combining the advantages of human and robot is an effective way to so...In current small batch and customized production mode,the products change rapidly and the personal demand increases sharply.Human-robot cooperation combining the advantages of human and robot is an effective way to solve the complex assembly.However,the poor reusability of historical assembly knowledge reduces the adaptability of assembly system to different tasks.For cross-domain strategy transfer,we propose a human-robot cooperative assembly(HRCA)framework which consists of three main modules:expression of HRCA strategy,transferring of HRCA strategy,and adaptive planning of motion path.Based on the analysis of subject capability and component properties,the HRCA strategy suitable for specific tasks is designed.Then the reinforcement learning is established to optimize the parameters of target encoder for feature extraction.After classification and segmentation,the actor-critic model is built to realize the adaptive path planning with progressive neural network.Finally,the proposed framework is verified to adapt to the multi-variety environment,for example,power lithium batteries.展开更多
In order to help the operator perform the human-robot collaboration task and optimize the task performance,an adaptive control method based on optimal admittance parameters is proposed.The overall control structure wi...In order to help the operator perform the human-robot collaboration task and optimize the task performance,an adaptive control method based on optimal admittance parameters is proposed.The overall control structure with the inner loop and outer loop is first established.The tasks of the inner loop and outer loop are robot control and task optimization,respectively.An inner-loop robot controller integrated with barrier Lyapunov function and radial basis function neural networks is then proposed,which makes the robot with unknown dynamics securely behave like a prescribed robot admittance model sensed by the operator.Subsequently,the optimal parameters of the robot admittance model are obtained in the outer loop to minimize the task tracking error and interaction force.The optimization problem of the robot admittance model is transformed into a linear quadratic regulator problem by constructing the human-robot collaboration system model.The model includes the unknown dynamics of the operator and the task performance details.To relax the requirement of the system model,the integral reinforcement learning is employed to solve the linear quadratic regulator problem.Besides,an auxiliary force is designed to help the operator complete the specific task better.Compared with the traditional control scheme,the security performance and interaction performance of the human-robot collaboration system are improved.The effectiveness of the proposed method is verified through two numerical simulations.In addition,a practical human-robot collaboration experiment is carried out to demonstrate the performance of the proposed method.展开更多
For safety reasons,in the automated dispensing medicines process,robots and humans cooperate to accomplish the task of drug sorting and distribution.In this dynamic unstructured environment,such as a humanrobot collab...For safety reasons,in the automated dispensing medicines process,robots and humans cooperate to accomplish the task of drug sorting and distribution.In this dynamic unstructured environment,such as a humanrobot collaboration scenario,the safety of human,robot,and equipment in the environment is paramount.In this work,a practical and effective robot motion planning method is proposed for dynamic unstructured environments.To figure out the problems of blind zones of single depth sensor and dynamic obstacle avoidance,we first propose a method for establishing offline mapping and online fusion of multi-sensor depth images and 3D grids of the robot workspace,which is used to determine the occupation states of the 3D grids occluded by robots and obstacles and to conduct real-time estimation of the minimum distance between the robot and obstacles.Then,based on the reactive control method,the attractive and repulsive forces are calculated and transformed into robot joint velocities to avoid obstacles in real time.Finally,the robot’s dynamic obstacle avoidance ability is evaluated on an experimental platform with a UR5 robot and two KinectV2 RGB-D sensors,and the effectiveness of the proposed method is verified.展开更多
The unmanned aircraft vehicles industry is in the ascendant while traditional interaction ways for an unmanned aerial vehicle(UAV)are not intuitive enough.It is difficult for a beginner to control a UAV,therefore natu...The unmanned aircraft vehicles industry is in the ascendant while traditional interaction ways for an unmanned aerial vehicle(UAV)are not intuitive enough.It is difficult for a beginner to control a UAV,therefore natural interaction methods are preferred.This paper presents a novel interactive control method for a UAV through operator's gesture,and explores the natural interaction method for the UAV.The proposed system uses the leap motion controller as an input device acquiring the gesture position and orientation data.It is found that the proposed human-robot interface can track the movement of the operator with satisfactory accuracy.The biggest advantage of the proposed method is its capability to control the UAV by just one hand instead of a joystick.A series of experiments verified the feasibility of the proposed human-robot interface.The results demonstrate that non-professional operators can easily operate a remote UAV by just using this system.展开更多
Navigation is an essential skill for robots.It becomes a cumbersome task for the robot in a human-populated environment,and Industry 5.0 is an emerging trend that focuses on the interaction between humans and robots.R...Navigation is an essential skill for robots.It becomes a cumbersome task for the robot in a human-populated environment,and Industry 5.0 is an emerging trend that focuses on the interaction between humans and robots.Robot behavior in a social setting is the key to human acceptance while ensuring human comfort and safety.With the advancement in robotics technology,the true use cases of robots in the tourism and hospitality industry are expanding in number.There are very few experimental studies focusing on how people perceive the navigation behavior of a delivery robot.A robotic platform named“PI”has been designed,which incorporates proximity and vision sensors.The robot utilizes a real-time object recognition algorithm based on the You Only Look Once(YOLO)algorithm to detect objects and humans during navigation.This study is aimed towards evaluating human experience,for which we conducted a study among 36 participants to explore the perceived social presence,role,and perception of a delivery robot exhibiting different behavior conditions while navigating in a hotel corridor.The participants’responses were collected and compared for different behavior conditions demonstrated by the robot and results show that humans prefer an assistant role of a robot enabled with audio and visual aids exhibiting social behavior.Further,this study can be useful for developers to gain insight into the expected behavior of a delivery robot.展开更多
基金Supported by National Science Foundation under Grant No.2222881.
文摘The integration of Human-Robot Collaboration(HRC)into Virtual Reality(VR)technology is transforming industries by enhancing workforce skills,improving safety,and optimizing operational processes and efficiency through realistic simulations of industry-specific scenarios.Despite the growing adoption of VR integrated with HRC,comprehensive reviews of current research in HRC-VR within the construction and manufacturing fields are lacking.This review examines the latest advances in designing and implementing HRC using VR technology in these industries.The aim is to address the application domains of HRC-VR,types of robots used,VR setups,and software solutions used.To achieve this,a systematic literature review using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses methodology was conducted on the Web of Science and Google Scholar databases,analyzing 383 articles and selecting 53 papers that met the established selection criteria.The findings emphasize a significant focus on enhancing human-robot interaction with a trend toward using immersive VR experiences and interactive 3D content creation tools.However,the integration of HRC with VR,especially in the dynamic construction environment,presents unique challenges and opportunities for future research,including developing more realistic simulations and adaptable robot systems.This paper offers insights for researchers,practitioners,educators,industry professionals,and policymakers interested in leveraging the integration of HRC with VR in construction and manufacturing industries.
文摘Despite the gradual transformation of traditional manufacturing by the Human-Robot Collaboration Assembly(HRCA),challenges remain in the robot’s ability to understand and predict human assembly intentions.This study aims to enhance the robot’s comprehension and prediction capabilities of operator assembly intentions by capturing and analyzing operator behavior and movements.We propose a video feature extraction method based on the Temporal Shift Module Network(TSM-ResNet50)to extract spatiotemporal features from assembly videos and differentiate various assembly actions using feature differences between video frames.Furthermore,we construct an action recognition and segmentation model based on the Refined-Multi-Scale Temporal Convolutional Network(Refined-MS-TCN)to identify assembly action intervals and accurately acquire action categories.Experiments on our self-built reducer assembly action dataset demonstrate that our network can classify assembly actions frame by frame,achieving an accuracy rate of 83%.Additionally,we develop a HiddenMarkovModel(HMM)integrated with assembly task constraints to predict operator assembly intentions based on the probability transition matrix and assembly task constraints.The experimental results show that our method for predicting operator assembly intentions can achieve an accuracy of 90.6%,which is a 13.3%improvement over the HMM without task constraints.
基金Supported by National Natural Science Foundation of China(Grant Nos.U23A20338,62103131 and 62203149)Hebei Provincial Natural Science Foundation(Grant No.E2022202171).
文摘The wearable exoskeleton system is a typical strongly coupled human-robotic system.Human-robotic is the environment for each other.The two support each other and compete with each other.Achieving high human-robotic compatibility is the most critical technology for wearable systems.Full structural compatibility can improve the intrinsic safety of the exoskeleton,and precise intention understanding and motion control can improve the comfort of the exoskeleton.This paper first designs a physiologically functional bionic lower limb exoskeleton based on the study of bone and joint functional anatomy and analyzes the drive mapping model of the dual closedloop four-link knee joint.Secondly,an exoskeleton dual closed-loop controller composed of a position inner loop and a force outer loop is designed.The inner loop of the controller adopts the PID control algorithm,and the outer loop adopts the adaptive admittance control algorithm based on human-robot interaction force(HRI).The controller can adaptively adjust the admittance parameters according to the HRI to respond to dynamic changes in the mechanical and physical parameters of the human-robot system,thereby improving control compliance and the wearing comfort of the exoskeleton system.Finally,we built a joint simulation experiment platform based on SolidWorks/Simulink to conduct virtual prototype simulation experiments and recruited volunteers to wear rehabilitation exoskeletons to conduct related control experiments.Experimental results show that the designed physiologically functional bionic exoskeleton and adaptive admittance controller can significantly improve the accuracy of human-robotic joint motion tracking,effectively reducing human-machine interaction forces and improving the comfort and safety of the wearer.This paper proposes a dual-closed loop four-link knee joint exoskeleton and a variable admittance control method based on HRI,which provides a new method for the design and control of exoskeletons with high compatibility.
基金The research has been generously supported by Tianjin Education Commission Scientific Research Program(2020KJ056),ChinaTianjin Science and Technology Planning Project(22YDTPJC00970),China.The authors would like to express their sincere appreciation for all support provided.
文摘A real-time adaptive roles allocation method based on reinforcement learning is proposed to improve humanrobot cooperation performance for a curtain wall installation task.This method breaks the traditional idea that the robot is regarded as the follower or only adjusts the leader and the follower in cooperation.In this paper,a self-learning method is proposed which can dynamically adapt and continuously adjust the initiative weight of the robot according to the change of the task.Firstly,the physical human-robot cooperation model,including the role factor is built.Then,a reinforcement learningmodel that can adjust the role factor in real time is established,and a reward and actionmodel is designed.The role factor can be adjusted continuously according to the comprehensive performance of the human-robot interaction force and the robot’s Jerk during the repeated installation.Finally,the roles adjustment rule established above continuously improves the comprehensive performance.Experiments of the dynamic roles allocation and the effect of the performance weighting coefficient on the result have been verified.The results show that the proposed method can realize the role adaptation and achieve the dual optimization goal of reducing the sum of the cooperator force and the robot’s Jerk.
文摘This paper presents an innovative investigation on prototyping a digital twin(DT)as the platform for human-robot interactive welding and welder behavior analysis.This humanrobot interaction(HRI)working style helps to enhance human users'operational productivity and comfort;while data-driven welder behavior analysis benefits to further novice welder training.This HRI system includes three modules:1)a human user who demonstrates the welding operations offsite with her/his operations recorded by the motion-tracked handles;2)a robot that executes the demonstrated welding operations to complete the physical welding tasks onsite;3)a DT system that is developed based on virtual reality(VR)as a digital replica of the physical human-robot interactive welding environment.The DT system bridges a human user and robot through a bi-directional information flow:a)transmitting demonstrated welding operations in VR to the robot in the physical environment;b)displaying the physical welding scenes to human users in VR.Compared to existing DT systems reported in the literatures,the developed one provides better capability in engaging human users in interacting with welding scenes,through an augmented VR.To verify the effectiveness,six welders,skilled with certain manual welding training and unskilled without any training,tested the system by completing the same welding job;three skilled welders produce satisfied welded workpieces,while the other three unskilled do not.A data-driven approach as a combination of fast Fourier transform(FFT),principal component analysis(PCA),and support vector machine(SVM)is developed to analyze their behaviors.Given an operation sequence,i.e.,motion speed sequence of the welding torch,frequency features are firstly extracted by FFT and then reduced in dimension through PCA,which are finally routed into SVM for classification.The trained model demonstrates a 94.44%classification accuracy in the testing dataset.The successful pattern recognition in skilled welder operations should benefit to accelerate novice welder training.
基金supported by the National Natural Science Foundation of China(61403422,61273102)the Hubei Provincial Natural Science Foundation of China(2015CFA010)+1 种基金the Ⅲ Project(B17040)the Fundamental Research Funds for National University,China University of Geosciences(Wuhan)
文摘A facial expression emotion recognition based human-robot interaction(FEER-HRI) system is proposed, for which a four-layer system framework is designed. The FEERHRI system enables the robots not only to recognize human emotions, but also to generate facial expression for adapting to human emotions. A facial emotion recognition method based on2D-Gabor, uniform local binary pattern(LBP) operator, and multiclass extreme learning machine(ELM) classifier is presented,which is applied to real-time facial expression recognition for robots. Facial expressions of robots are represented by simple cartoon symbols and displayed by a LED screen equipped in the robots, which can be easily understood by human. Four scenarios,i.e., guiding, entertainment, home service and scene simulation are performed in the human-robot interaction experiment, in which smooth communication is realized by facial expression recognition of humans and facial expression generation of robots within 2 seconds. As a few prospective applications, the FEERHRI system can be applied in home service, smart home, safe driving, and so on.
基金The work was supported by the National Science Foundation，the Office of Naval Research grant，the AFOSR (Air Force Office of Scientific Research) EOARD (European Office of Aerospace Research and Development) grant，the U.S. Army Research Office grant
文摘This paper presents a novel enhanced human-robot interaction system based on model reference adaptive control. The presented method delivers guaranteed stability and task performance and has two control loops. A robot-specific inner loop, which is a neuroadaptive controller, learns the robot dynamics online and makes the robot respond like a prescribed impedance model. This loop uses no task information, including no prescribed trajectory. A task-specific outer loop takes into account the human operator dynamics and adapts the prescribed robot impedance model so that the combined human-robot system has desirable characteristics for task performance. This design is based on model reference adaptive control, but of a nonstandard form. The net result is a controller with both adaptive impedance characteristics and assistive inputs that augment the human operator to provide improved task performance of the human-robot team. Simulations verify the performance of the proposed controller in a repetitive point-to-point motion task. Actual experimental implementations on a PR2 robot further corroborate the effectiveness of the approach.
基金This work was supported in part by the National Natural Science Foundation of China(61903028)the Youth Innovation Promotion Association,Chinese Academy of Sciences(2020137)+1 种基金the Lifelong Learning Machines Program from DARPA/Microsystems Technology Officethe Army Research Laboratory(W911NF-18-2-0260).
文摘In this paper,we present a novel data-driven design method for the human-robot interaction(HRI)system,where a given task is achieved by cooperation between the human and the robot.The presented HRI controller design is a two-level control design approach consisting of a task-oriented performance optimization design and a plant-oriented impedance controller design.The task-oriented design minimizes the human effort and guarantees the perfect task tracking in the outer-loop,while the plant-oriented achieves the desired impedance from the human to the robot manipulator end-effector in the inner-loop.Data-driven reinforcement learning techniques are used for performance optimization in the outer-loop to assign the optimal impedance parameters.In the inner-loop,a velocity-free filter is designed to avoid the requirement of end-effector velocity measurement.On this basis,an adaptive controller is designed to achieve the desired impedance of the robot manipulator in the task space.The simulation and experiment of a robot manipulator are conducted to verify the efficacy of the presented HRI design framework.
基金supported by the EU-Japan coordinated R&D project on“Culture Aware Robots and Environmental Sensor Systems for Elderly Support,”commissioned by the Ministry of Internal Affairs and Communications of Japan and EC Horizon 2020 Research and Innovation Programme(737858)financial supports from the Air Force Office of Scientific Research(AFOSR-AOARD/FA2386-19-1-4015)。
文摘With the increasing presence of robots in our daily life,there is a strong need and demand for the strategies to acquire a high quality interaction between robots and users by enabling robots to understand users’mood,intention,and other aspects.During human-human interaction,personality traits have an important influence on human behavior,decision,mood,and many others.Therefore,we propose an efficient computational framework to endow the robot with the capability of understanding the user’s personality traits based on the user’s nonverbal communication cues represented by three visual features including the head motion,gaze,and body motion energy,and three vocal features including voice pitch,voice energy,and mel-frequency cepstral coefficient(MFCC).We used the Pepper robot in this study as a communication robot to interact with each participant by asking questions,and meanwhile,the robot extracts the nonverbal features from each participant’s habitual behavior using its on-board sensors.On the other hand,each participant’s personality traits are evaluated with a questionnaire.We then train the ridge regression and linear support vector machine(SVM)classifiers using the nonverbal features and personality trait labels from a questionnaire and evaluate the performance of the classifiers.We have verified the validity of the proposed models that showed promising binary classification performance on recognizing each of the Big Five personality traits of the participants based on individual differences in nonverbal communication cues.
基金supported by the National Natural Science Foundation of China(61175057)the USTC Key-Direction Research Fund(WK0110000028)
文摘A more natural way for non-expert users to express their tasks in an open-ended set is to use natural language. In this case,a human-centered intelligent agent/robot is required to be able to understand and generate plans for these naturally expressed tasks. For this purpose, it is a good way to enhance intelligent robot's abilities by utilizing open knowledge extracted from the web, instead of hand-coded knowledge. A key challenge of utilizing open knowledge lies in the semantic interpretation of the open knowledge organized in multiple modes, which can be unstructured or semi-structured, before one can use it.Previous approaches used a limited lexicon to employ combinatory categorial grammar(CCG) as the underlying formalism for semantic parsing over sentences. Here, we propose a more effective learning method to interpret semi-structured user instructions. Moreover, we present a new heuristic method to recover missing semantic information from the context of an instruction. Experiments showed that the proposed approach renders significant performance improvement compared to the baseline methods and the recovering method is promising.
基金supported by the National Natural Science Foundation of China(61403215)the National Natural Science Foundation of Tianjin(13JCYBJC36600)the Fundamental Research Funds for the Central Universities
文摘Human-robot interaction(HRI) is fundamental for human-centered robotics, and has been attracting intensive research for more than a decade. The series elastic actuator(SEA) provides inherent compliance, safety and further benefits for HRI, but the introduced elastic element also brings control difficulties. In this paper, we address the stiffness rendering problem for a cable-driven SEA system, to achieve either low stiffness for good transparency or high stiffness bigger than the physical spring constant, and to assess the rendering accuracy with quantified metrics. By taking a velocity-sourced model of the motor, a cascaded velocity-torque-impedance control structure is established. To achieve high fidelity torque control, the 2-DOF(degree of freedom) stabilizing control method together with a compensator has been used to handle the competing requirements on tracking performance, noise and disturbance rejection,and energy optimization in the cable-driven SEA system. The conventional passivity requirement for HRI usually leads to a conservative design of the impedance controller, and the rendered stiffness cannot go higher than the physical spring constant. By adding a phase-lead compensator into the impedance controller,the stiffness rendering capability was augmented with guaranteed relaxed passivity. Extensive simulations and experiments have been performed, and the virtual stiffness has been rendered in the extended range of 0.1 to 2.0 times of the physical spring constant with guaranteed relaxed passivity for physical humanrobot interaction below 5 Hz. Quantified metrics also verified good rendering accuracy.
文摘With the increasing of the elderly population and the growing hearth care cost, the role of service robots in aiding the disabled and the elderly is becoming important. Many researchers in the world have paid much attention to heaRthcare robots and rehabilitation robots. To get natural and harmonious communication between the user and a service robot, the information perception/feedback ability, and interaction ability for service robots become more important in many key issues.
文摘This paper proposes a novel approach for physical human-robot interactions(pHRI), where a robot provides guidance forces to a user based on the user performance. This framework tunes the forces in regards to behavior of each user in coping with different tasks, where lower performance results in higher intervention from the robot. This personalized physical human-robot interaction(p2HRI) method incorporates adaptive modeling of the interaction between the human and the robot as well as learning from demonstration(LfD) techniques to adapt to the users' performance. This approach is based on model predictive control where the system optimizes the rendered forces by predicting the performance of the user. Moreover, continuous learning of the user behavior is added so that the models and personalized considerations are updated based on the change of user performance over time. Applying this framework to a field such as haptic guidance for skill improvement, allows a more personalized learning experience where the interaction between the robot as the intelligent tutor and the student as the user,is better adjusted based on the skill level of the individual and their gradual improvement. The results suggest that the precision of the model of the interaction is improved using this proposed method,and the addition of the considered personalized factors to a more adaptive strategy for rendering of guidance forces.
基金supported in part by the National Natural Science Foundation of China(62293514,52275020,and 91948301)。
文摘Human–robot(HR)collaboration(HRC)is an emerging research field because of the complementary advantages of humans and robots.An HRC framework for robotic assembly based on impedance control is proposed in this paper.In the HRC framework,the human is the decision maker,the robot acts as the executor,while the assembly environment provides constraints.The robot is the main executor to perform the assembly action,which has the position control,drag and drop,positive impedance control,and negative impedance control modes.To reveal the characteristics of the HRC framework,the switch condition map of different control modes and the stability analysis of the HR coupled system are discussed.In the end,HRC assembly experiments are conducted,where the HRC assembly task can be accomplished when the assembling tolerance is 0.08 mm or with the interference fit.Experiments show that the HRC assembly has the complementary advantages of humans and robots and is efficient in finishing complex assembly tasks.
基金the National Key Research and Development Program of China(No.2019YFB1706300)the National Natural Science Foundation of China(No.52075094)。
文摘In current small batch and customized production mode,the products change rapidly and the personal demand increases sharply.Human-robot cooperation combining the advantages of human and robot is an effective way to solve the complex assembly.However,the poor reusability of historical assembly knowledge reduces the adaptability of assembly system to different tasks.For cross-domain strategy transfer,we propose a human-robot cooperative assembly(HRCA)framework which consists of three main modules:expression of HRCA strategy,transferring of HRCA strategy,and adaptive planning of motion path.Based on the analysis of subject capability and component properties,the HRCA strategy suitable for specific tasks is designed.Then the reinforcement learning is established to optimize the parameters of target encoder for feature extraction.After classification and segmentation,the actor-critic model is built to realize the adaptive path planning with progressive neural network.Finally,the proposed framework is verified to adapt to the multi-variety environment,for example,power lithium batteries.
基金the National Key R&D Program of China(No.2018YFB1308400)the Natural Science Foundation of Zhejiang Province(No.LY21F030018)。
文摘In order to help the operator perform the human-robot collaboration task and optimize the task performance,an adaptive control method based on optimal admittance parameters is proposed.The overall control structure with the inner loop and outer loop is first established.The tasks of the inner loop and outer loop are robot control and task optimization,respectively.An inner-loop robot controller integrated with barrier Lyapunov function and radial basis function neural networks is then proposed,which makes the robot with unknown dynamics securely behave like a prescribed robot admittance model sensed by the operator.Subsequently,the optimal parameters of the robot admittance model are obtained in the outer loop to minimize the task tracking error and interaction force.The optimization problem of the robot admittance model is transformed into a linear quadratic regulator problem by constructing the human-robot collaboration system model.The model includes the unknown dynamics of the operator and the task performance details.To relax the requirement of the system model,the integral reinforcement learning is employed to solve the linear quadratic regulator problem.Besides,an auxiliary force is designed to help the operator complete the specific task better.Compared with the traditional control scheme,the security performance and interaction performance of the human-robot collaboration system are improved.The effectiveness of the proposed method is verified through two numerical simulations.In addition,a practical human-robot collaboration experiment is carried out to demonstrate the performance of the proposed method.
基金the Interdisciplinary Program of Shanghai Jiao Tong University(No.YG2019QNA25)。
文摘For safety reasons,in the automated dispensing medicines process,robots and humans cooperate to accomplish the task of drug sorting and distribution.In this dynamic unstructured environment,such as a humanrobot collaboration scenario,the safety of human,robot,and equipment in the environment is paramount.In this work,a practical and effective robot motion planning method is proposed for dynamic unstructured environments.To figure out the problems of blind zones of single depth sensor and dynamic obstacle avoidance,we first propose a method for establishing offline mapping and online fusion of multi-sensor depth images and 3D grids of the robot workspace,which is used to determine the occupation states of the 3D grids occluded by robots and obstacles and to conduct real-time estimation of the minimum distance between the robot and obstacles.Then,based on the reactive control method,the attractive and repulsive forces are calculated and transformed into robot joint velocities to avoid obstacles in real time.Finally,the robot’s dynamic obstacle avoidance ability is evaluated on an experimental platform with a UR5 robot and two KinectV2 RGB-D sensors,and the effectiveness of the proposed method is verified.
基金Supported by the National Natural Science Foundation of China(61602182)Science and Technology Planning Project of Guangzhou(201604046029)+4 种基金the Guangdong Natural Science Funds for Distinguished Young Scholar(2017A030306015)Science and Technology Planning Project of Guangdong Province(2017B010116001)Pearl River S&T Nova Program of Guangzhou(201710010059)Guangdong Special Projects(2016TQ03X824)the Fundamental Research Funds for the Central Universities(2017JQ009)
文摘The unmanned aircraft vehicles industry is in the ascendant while traditional interaction ways for an unmanned aerial vehicle(UAV)are not intuitive enough.It is difficult for a beginner to control a UAV,therefore natural interaction methods are preferred.This paper presents a novel interactive control method for a UAV through operator's gesture,and explores the natural interaction method for the UAV.The proposed system uses the leap motion controller as an input device acquiring the gesture position and orientation data.It is found that the proposed human-robot interface can track the movement of the operator with satisfactory accuracy.The biggest advantage of the proposed method is its capability to control the UAV by just one hand instead of a joystick.A series of experiments verified the feasibility of the proposed human-robot interface.The results demonstrate that non-professional operators can easily operate a remote UAV by just using this system.
基金supported by Taif University Researchers Supporting Projects(TURSP).Under number(TURSP-2020/211),Taif University,Taif,Saudi Arabia.
文摘Navigation is an essential skill for robots.It becomes a cumbersome task for the robot in a human-populated environment,and Industry 5.0 is an emerging trend that focuses on the interaction between humans and robots.Robot behavior in a social setting is the key to human acceptance while ensuring human comfort and safety.With the advancement in robotics technology,the true use cases of robots in the tourism and hospitality industry are expanding in number.There are very few experimental studies focusing on how people perceive the navigation behavior of a delivery robot.A robotic platform named“PI”has been designed,which incorporates proximity and vision sensors.The robot utilizes a real-time object recognition algorithm based on the You Only Look Once(YOLO)algorithm to detect objects and humans during navigation.This study is aimed towards evaluating human experience,for which we conducted a study among 36 participants to explore the perceived social presence,role,and perception of a delivery robot exhibiting different behavior conditions while navigating in a hotel corridor.The participants’responses were collected and compared for different behavior conditions demonstrated by the robot and results show that humans prefer an assistant role of a robot enabled with audio and visual aids exhibiting social behavior.Further,this study can be useful for developers to gain insight into the expected behavior of a delivery robot.
