Human Interaction Recognition(HIR)was one of the challenging issues in computer vision research due to the involvement of multiple individuals and their mutual interactions within video frames generated from their mov...Human Interaction Recognition(HIR)was one of the challenging issues in computer vision research due to the involvement of multiple individuals and their mutual interactions within video frames generated from their movements.HIR requires more sophisticated analysis than Human Action Recognition(HAR)since HAR focuses solely on individual activities like walking or running,while HIR involves the interactions between people.This research aims to develop a robust system for recognizing five common human interactions,such as hugging,kicking,pushing,pointing,and no interaction,from video sequences using multiple cameras.In this study,a hybrid Deep Learning(DL)and Machine Learning(ML)model was employed to improve classification accuracy and generalizability.The dataset was collected in an indoor environment with four-channel cameras capturing the five types of interactions among 13 participants.The data was processed using a DL model with a fine-tuned ResNet(Residual Networks)architecture based on 2D Convolutional Neural Network(CNN)layers for feature extraction.Subsequently,machine learning models were trained and utilized for interaction classification using six commonly used ML algorithms,including SVM,KNN,RF,DT,NB,and XGBoost.The results demonstrate a high accuracy of 95.45%in classifying human interactions.The hybrid approach enabled effective learning,resulting in highly accurate performance across different interaction types.Future work will explore more complex scenarios involving multiple individuals based on the application of this architecture.展开更多
Speech recognition rate will deteriorate greatly in human-machine interaction when the speaker's speech mixes with a bystander's voice. This paper proposes a time-frequency approach for Blind Source Seperation...Speech recognition rate will deteriorate greatly in human-machine interaction when the speaker's speech mixes with a bystander's voice. This paper proposes a time-frequency approach for Blind Source Seperation (BSS) for intelligent Human-Machine Interaction(HMI). Main idea of the algorithm is to simultaneously diagonalize the correlation matrix of the pre-whitened signals at different time delays for every frequency bins in time-frequency domain. The prososed method has two merits: (1) fast convergence speed; (2) high signal to interference ratio of the separated signals. Numerical evaluations are used to compare the performance of the proposed algorithm with two other deconvolution algorithms. An efficient algorithm to resolve permutation ambiguity is also proposed in this paper. The algorithm proposed saves more than 10% of computational time with properly selected parameters and achieves good performances for both simulated convolutive mixtures and real room recorded speeches.展开更多
Many human-machine collaborative support scheduling systems are used to aid human decision making by providing several optimal scheduling algorithms that do not take operator's attention into consideration.However...Many human-machine collaborative support scheduling systems are used to aid human decision making by providing several optimal scheduling algorithms that do not take operator's attention into consideration.However, the current systems should take advantage of the operator's attention to obtain the optimal solution.In this paper, we innovatively propose a human-machine collaborative support scheduling system of intelligence information from multi-UAVs based on eye-tracker. Firstly, the target recognition algorithm is applied to the images from the multiple unmanned aerial vehicles(multi-UAVs) to recognize the targets in the images. Then,the support system utilizes the eye tracker to gain the eye-gaze points which are intended to obtain the focused targets in the images. Finally, the heuristic scheduling algorithms take both the attributes of targets and the operator's attention into consideration to obtain the sequence of the images. As the processing time of the images collected by the multi-UAVs is uncertain, however the upper bounds and lower bounds of the processing time are known before. So the processing time of the images is modeled by the interval processing time. The objective of the scheduling problem is to minimize mean weighted completion time. This paper proposes some new polynomial time heuristic scheduling algorithms which firstly schedule the images including the focused targets. We conduct the scheduling experiments under six different distributions. The results indicate that the proposed algorithm is not sensitive to the different distributions of the processing time and has a negligible computational time. The absolute error of the best performing heuristic solution is only about 1%. Then, we incorporate the best performing heuristic algorithm into the human-machine collaborative support systems to verify the performance of the system.展开更多
A complete characterization of the behavior in human-robot interactions(HRI) includes both: the behavioral dynamics and the control laws that characterize how the behavior is regulated with the perception data. In thi...A complete characterization of the behavior in human-robot interactions(HRI) includes both: the behavioral dynamics and the control laws that characterize how the behavior is regulated with the perception data. In this way, this work proposes a leader-follower coordinate control based on an impedance control that allows to establish a dynamic relation between social forces and motion error. For this, a scheme is presented to identify the impedance based on fictitious social forces, which are described by distance-based potential fields.As part of the validation procedure, we present an experimental comparison to select the better of two different fictitious force structures. The criteria are determined by two qualities: least impedance errors during the validation procedure and least parameter variance during the recursive estimation procedure.Finally, with the best fictitious force and its identified impedance,an impedance control is designed for a mobile robot Pioneer 3AT,which is programmed to follow a human in a structured scenario.According to results, and under the hypothesis that moving like humans will be acceptable by humans, it is believed that the proposed control improves the social acceptance of the robot for this kind of interaction.展开更多
Substantial challenges remain in developing fiber devices to achieve uniform and customizable photochromic lighting effects using lightweight hardware.A recent publication in Light Science&Application,spearheaded ...Substantial challenges remain in developing fiber devices to achieve uniform and customizable photochromic lighting effects using lightweight hardware.A recent publication in Light Science&Application,spearheaded by Prof.Yan-Qing Lu and Prof.Guangming Tao presents a methodical approach to surmount the limitations in photochromic fibers.They integrated controllable photochromic fibers into various wearable devices,providing a promising path for future exploration and advancement in the field of human–machine interaction.展开更多
为了满足固沙机功能集成化、智能化方向发展需求,设计了一种智能草沙障固沙机控制系统,采用主从式控制系统结构,以双Arduino Mega 2560为核心主控器,使用模块化对控制系统的硬件和软件进行设计;纵向铺设控制模块、底盘升降控制模块和播...为了满足固沙机功能集成化、智能化方向发展需求,设计了一种智能草沙障固沙机控制系统,采用主从式控制系统结构,以双Arduino Mega 2560为核心主控器,使用模块化对控制系统的硬件和软件进行设计;纵向铺设控制模块、底盘升降控制模块和播种控制模块采用增量式PID控制算法;横向铺设控制模块采用位置式PID控制算法,并结合YOLOv3实现草沙障的检测追踪;以触摸屏和基于蓝牙的固沙机控制Android APP为人机交互界面。控制系统试验结果表明:基于PID算法检测追踪方式与直插式铺设方式相比,采用检测追踪方式固沙机的横向犁沙宽度减小了78.7%,整车振动减小了42.6%,草沙障铺设和播种测试功能均达到样机设计预期,为智能化固沙机研究提供了一种切实可行的设计方案。展开更多
基金supported by the National Research Foundation of Korea(NRF)grant funded by the Korea government(MSIT)(No.RS-2023-00218176)and the Soonchunhyang University Research Fund.
文摘Human Interaction Recognition(HIR)was one of the challenging issues in computer vision research due to the involvement of multiple individuals and their mutual interactions within video frames generated from their movements.HIR requires more sophisticated analysis than Human Action Recognition(HAR)since HAR focuses solely on individual activities like walking or running,while HIR involves the interactions between people.This research aims to develop a robust system for recognizing five common human interactions,such as hugging,kicking,pushing,pointing,and no interaction,from video sequences using multiple cameras.In this study,a hybrid Deep Learning(DL)and Machine Learning(ML)model was employed to improve classification accuracy and generalizability.The dataset was collected in an indoor environment with four-channel cameras capturing the five types of interactions among 13 participants.The data was processed using a DL model with a fine-tuned ResNet(Residual Networks)architecture based on 2D Convolutional Neural Network(CNN)layers for feature extraction.Subsequently,machine learning models were trained and utilized for interaction classification using six commonly used ML algorithms,including SVM,KNN,RF,DT,NB,and XGBoost.The results demonstrate a high accuracy of 95.45%in classifying human interactions.The hybrid approach enabled effective learning,resulting in highly accurate performance across different interaction types.Future work will explore more complex scenarios involving multiple individuals based on the application of this architecture.
文摘Speech recognition rate will deteriorate greatly in human-machine interaction when the speaker's speech mixes with a bystander's voice. This paper proposes a time-frequency approach for Blind Source Seperation (BSS) for intelligent Human-Machine Interaction(HMI). Main idea of the algorithm is to simultaneously diagonalize the correlation matrix of the pre-whitened signals at different time delays for every frequency bins in time-frequency domain. The prososed method has two merits: (1) fast convergence speed; (2) high signal to interference ratio of the separated signals. Numerical evaluations are used to compare the performance of the proposed algorithm with two other deconvolution algorithms. An efficient algorithm to resolve permutation ambiguity is also proposed in this paper. The algorithm proposed saves more than 10% of computational time with properly selected parameters and achieves good performances for both simulated convolutive mixtures and real room recorded speeches.
基金the National Natural Science Foundation of China(No.61403410)
文摘Many human-machine collaborative support scheduling systems are used to aid human decision making by providing several optimal scheduling algorithms that do not take operator's attention into consideration.However, the current systems should take advantage of the operator's attention to obtain the optimal solution.In this paper, we innovatively propose a human-machine collaborative support scheduling system of intelligence information from multi-UAVs based on eye-tracker. Firstly, the target recognition algorithm is applied to the images from the multiple unmanned aerial vehicles(multi-UAVs) to recognize the targets in the images. Then,the support system utilizes the eye tracker to gain the eye-gaze points which are intended to obtain the focused targets in the images. Finally, the heuristic scheduling algorithms take both the attributes of targets and the operator's attention into consideration to obtain the sequence of the images. As the processing time of the images collected by the multi-UAVs is uncertain, however the upper bounds and lower bounds of the processing time are known before. So the processing time of the images is modeled by the interval processing time. The objective of the scheduling problem is to minimize mean weighted completion time. This paper proposes some new polynomial time heuristic scheduling algorithms which firstly schedule the images including the focused targets. We conduct the scheduling experiments under six different distributions. The results indicate that the proposed algorithm is not sensitive to the different distributions of the processing time and has a negligible computational time. The absolute error of the best performing heuristic solution is only about 1%. Then, we incorporate the best performing heuristic algorithm into the human-machine collaborative support systems to verify the performance of the system.
文摘A complete characterization of the behavior in human-robot interactions(HRI) includes both: the behavioral dynamics and the control laws that characterize how the behavior is regulated with the perception data. In this way, this work proposes a leader-follower coordinate control based on an impedance control that allows to establish a dynamic relation between social forces and motion error. For this, a scheme is presented to identify the impedance based on fictitious social forces, which are described by distance-based potential fields.As part of the validation procedure, we present an experimental comparison to select the better of two different fictitious force structures. The criteria are determined by two qualities: least impedance errors during the validation procedure and least parameter variance during the recursive estimation procedure.Finally, with the best fictitious force and its identified impedance,an impedance control is designed for a mobile robot Pioneer 3AT,which is programmed to follow a human in a structured scenario.According to results, and under the hypothesis that moving like humans will be acceptable by humans, it is believed that the proposed control improves the social acceptance of the robot for this kind of interaction.
文摘Substantial challenges remain in developing fiber devices to achieve uniform and customizable photochromic lighting effects using lightweight hardware.A recent publication in Light Science&Application,spearheaded by Prof.Yan-Qing Lu and Prof.Guangming Tao presents a methodical approach to surmount the limitations in photochromic fibers.They integrated controllable photochromic fibers into various wearable devices,providing a promising path for future exploration and advancement in the field of human–machine interaction.
文摘为了满足固沙机功能集成化、智能化方向发展需求,设计了一种智能草沙障固沙机控制系统,采用主从式控制系统结构,以双Arduino Mega 2560为核心主控器,使用模块化对控制系统的硬件和软件进行设计;纵向铺设控制模块、底盘升降控制模块和播种控制模块采用增量式PID控制算法;横向铺设控制模块采用位置式PID控制算法,并结合YOLOv3实现草沙障的检测追踪;以触摸屏和基于蓝牙的固沙机控制Android APP为人机交互界面。控制系统试验结果表明:基于PID算法检测追踪方式与直插式铺设方式相比,采用检测追踪方式固沙机的横向犁沙宽度减小了78.7%,整车振动减小了42.6%,草沙障铺设和播种测试功能均达到样机设计预期,为智能化固沙机研究提供了一种切实可行的设计方案。
