Hand gestures are a natural way for human-robot interaction.Vision based dynamic hand gesture recognition has become a hot research topic due to its various applications.This paper presents a novel deep learning netwo...Hand gestures are a natural way for human-robot interaction.Vision based dynamic hand gesture recognition has become a hot research topic due to its various applications.This paper presents a novel deep learning network for hand gesture recognition.The network integrates several well-proved modules together to learn both short-term and long-term features from video inputs and meanwhile avoid intensive computation.To learn short-term features,each video input is segmented into a fixed number of frame groups.A frame is randomly selected from each group and represented as an RGB image as well as an optical flow snapshot.These two entities are fused and fed into a convolutional neural network(Conv Net)for feature extraction.The Conv Nets for all groups share parameters.To learn longterm features,outputs from all Conv Nets are fed into a long short-term memory(LSTM)network,by which a final classification result is predicted.The new model has been tested with two popular hand gesture datasets,namely the Jester dataset and Nvidia dataset.Comparing with other models,our model produced very competitive results.The robustness of the new model has also been proved with an augmented dataset with enhanced diversity of hand gestures.展开更多
Effective and accurate action recognition is essential to the intelligent breeding of the Jinnan cattle.However,there are still several challenges in the current Jinnan cattle action recognition.Traditional methods ar...Effective and accurate action recognition is essential to the intelligent breeding of the Jinnan cattle.However,there are still several challenges in the current Jinnan cattle action recognition.Traditional methods are based on manual characteristics and low recognition accuracy.This study is aimed at the efficient and accurate development of Jinnan cattle action recognition methods to overcome existing problems and support intelligent breeding.The acquired data from the previous methods contain a lot of noise,which will cause individual cattle to have excessive behaviors due to unsuitability.Concerning the high labor costs,low efficiency,and low model accuracy of the above approaches,this study developed a bottleneck attention-enhanced two-stream(BATS)Jinnan cattle action recognition method.It primarily comprises a Spatial Stream Subnetwork,a Temporal Stream Subnetwork,and a Bottleneck Attention Module.It can capture the spatial-channel dependencies in RGB and optical flow two branches respectively,so as to extract richer and more robust features.Finally,the decision of the two branches can be fused to gain improved cattle action recognition performance.Compared with the traditional methods,the model proposed in this study has achieved state-of-the-art recognition performance,and the accuracy of motion recognition was 96.53%,which was 4.60%higher than other models.This method significantly improves the efficiency and accuracy of behavior recognition and provides an important research foundation and direction for the development of higher-level behavior analysis models in the future development of smart animal husbandry.展开更多
With the intensifying aging of the population,the phenomenon of the elderly living alone is also increasing.Therefore,using modern internet of things technology to monitor the daily behavior of the elderly in indoors ...With the intensifying aging of the population,the phenomenon of the elderly living alone is also increasing.Therefore,using modern internet of things technology to monitor the daily behavior of the elderly in indoors is a meaningful study.Video-based action recognition tasks are easily affected by object occlusion and weak ambient light,resulting in poor recognition performance.Therefore,this paper proposes an indoor human behavior recognition method based on wireless fidelity(Wi-Fi)perception and video feature fusion by utilizing the ability of Wi-Fi signals to carry environmental information during the propagation process.This paper uses the public WiFi-based activity recognition dataset(WIAR)containing Wi-Fi channel state information and essential action videos,and then extracts video feature vectors and Wi-Fi signal feature vectors in the datasets through the two-stream convolutional neural network and standard statistical algorithms,respectively.Then the two sets of feature vectors are fused,and finally,the action classification and recognition are performed by the support vector machine(SVM).The experiments in this paper contrast experiments between the two-stream network model and the methods in this paper under three different environments.And the accuracy of action recognition after adding Wi-Fi signal feature fusion is improved by 10%on average.展开更多
文摘Hand gestures are a natural way for human-robot interaction.Vision based dynamic hand gesture recognition has become a hot research topic due to its various applications.This paper presents a novel deep learning network for hand gesture recognition.The network integrates several well-proved modules together to learn both short-term and long-term features from video inputs and meanwhile avoid intensive computation.To learn short-term features,each video input is segmented into a fixed number of frame groups.A frame is randomly selected from each group and represented as an RGB image as well as an optical flow snapshot.These two entities are fused and fed into a convolutional neural network(Conv Net)for feature extraction.The Conv Nets for all groups share parameters.To learn longterm features,outputs from all Conv Nets are fed into a long short-term memory(LSTM)network,by which a final classification result is predicted.The new model has been tested with two popular hand gesture datasets,namely the Jester dataset and Nvidia dataset.Comparing with other models,our model produced very competitive results.The robustness of the new model has also been proved with an augmented dataset with enhanced diversity of hand gestures.
基金supported by the Shanxi Province Basic Research Program(Grant No.202203021212444)Shanxi Agricultural University Science and Technology Innovation Enhancement Project(Grant No.CXGC2023045)+3 种基金Shanxi Province Higher Education Teaching Reform and Innovation Project(Grant No.J20220274)Shanxi Postgraduate Education and Teaching Reform Project Fund(2022YJJG094)Shanxi Agricultural University Doctoral Research Start-up Project(Grant No.2021BQ88)Shanxi Agricultural University Academic Restoration Research Project(2020xshf38).
文摘Effective and accurate action recognition is essential to the intelligent breeding of the Jinnan cattle.However,there are still several challenges in the current Jinnan cattle action recognition.Traditional methods are based on manual characteristics and low recognition accuracy.This study is aimed at the efficient and accurate development of Jinnan cattle action recognition methods to overcome existing problems and support intelligent breeding.The acquired data from the previous methods contain a lot of noise,which will cause individual cattle to have excessive behaviors due to unsuitability.Concerning the high labor costs,low efficiency,and low model accuracy of the above approaches,this study developed a bottleneck attention-enhanced two-stream(BATS)Jinnan cattle action recognition method.It primarily comprises a Spatial Stream Subnetwork,a Temporal Stream Subnetwork,and a Bottleneck Attention Module.It can capture the spatial-channel dependencies in RGB and optical flow two branches respectively,so as to extract richer and more robust features.Finally,the decision of the two branches can be fused to gain improved cattle action recognition performance.Compared with the traditional methods,the model proposed in this study has achieved state-of-the-art recognition performance,and the accuracy of motion recognition was 96.53%,which was 4.60%higher than other models.This method significantly improves the efficiency and accuracy of behavior recognition and provides an important research foundation and direction for the development of higher-level behavior analysis models in the future development of smart animal husbandry.
基金supported by the National Natural Science Foundation of China(No.62006135)the Natural Science Foundation of Shandong Province(No.ZR2020QF116)。
文摘With the intensifying aging of the population,the phenomenon of the elderly living alone is also increasing.Therefore,using modern internet of things technology to monitor the daily behavior of the elderly in indoors is a meaningful study.Video-based action recognition tasks are easily affected by object occlusion and weak ambient light,resulting in poor recognition performance.Therefore,this paper proposes an indoor human behavior recognition method based on wireless fidelity(Wi-Fi)perception and video feature fusion by utilizing the ability of Wi-Fi signals to carry environmental information during the propagation process.This paper uses the public WiFi-based activity recognition dataset(WIAR)containing Wi-Fi channel state information and essential action videos,and then extracts video feature vectors and Wi-Fi signal feature vectors in the datasets through the two-stream convolutional neural network and standard statistical algorithms,respectively.Then the two sets of feature vectors are fused,and finally,the action classification and recognition are performed by the support vector machine(SVM).The experiments in this paper contrast experiments between the two-stream network model and the methods in this paper under three different environments.And the accuracy of action recognition after adding Wi-Fi signal feature fusion is improved by 10%on average.
