This paper proposes a method to recognize human-object interactions by modeling context between human actions and interacted objects.Human-object interaction recognition is a challenging task due to severe occlusion b...This paper proposes a method to recognize human-object interactions by modeling context between human actions and interacted objects.Human-object interaction recognition is a challenging task due to severe occlusion between human and objects during the interacting process.Since that human actions and interacted objects provide strong context information,i.e.some actions are usually related to some specific objects,the accuracy of recognition is significantly improved for both of them.Through the proposed method,both global and local temporal features from skeleton sequences are extracted to model human actions.In the meantime,kernel features are utilized to describe interacted objects.Finally,all possible solutions from actions and objects are optimized by modeling the context between them.The results of experiments demonstrate the effectiveness of our method.展开更多
Human-object interaction(HOIs)detection is a new branch of visual relationship detection,which plays an important role in the field of image understanding.Because of the complexity and diversity of image content,the d...Human-object interaction(HOIs)detection is a new branch of visual relationship detection,which plays an important role in the field of image understanding.Because of the complexity and diversity of image content,the detection of HOIs is still an onerous challenge.Unlike most of the current works for HOIs detection which only rely on the pairwise information of a human and an object,we propose a graph-based HOIs detection method that models context and global structure information.Firstly,to better utilize the relations between humans and objects,the detected humans and objects are regarded as nodes to construct a fully connected undirected graph,and the graph is pruned to obtain an HOI graph that only preserving the edges connecting human and object nodes.Then,in order to obtain more robust features of human and object nodes,two different attention-based feature extraction networks are proposed,which model global and local contexts respectively.Finally,the graph attention network is introduced to pass messages between different nodes in the HOI graph iteratively,and detect the potential HOIs.Experiments on V-COCO and HICO-DET datasets verify the effectiveness of the proposed method,and show that it is superior to many existing methods.展开更多
Human object interaction(HOI)recognition plays an important role in the designing of surveillance and monitoring systems for healthcare,sports,education,and public areas.It involves localizing the human and object tar...Human object interaction(HOI)recognition plays an important role in the designing of surveillance and monitoring systems for healthcare,sports,education,and public areas.It involves localizing the human and object targets and then identifying the interactions between them.However,it is a challenging task that highly depends on the extraction of robust and distinctive features from the targets and the use of fast and efficient classifiers.Hence,the proposed system offers an automated body-parts-based solution for HOI recognition.This system uses RGB(red,green,blue)images as input and segments the desired parts of the images through a segmentation technique based on the watershed algorithm.Furthermore,a convex hullbased approach for extracting key body parts has also been introduced.After identifying the key body parts,two types of features are extracted.Moreover,the entire feature vector is reduced using a dimensionality reduction technique called t-SNE(t-distributed stochastic neighbor embedding).Finally,a multinomial logistic regression classifier is utilized for identifying class labels.A large publicly available dataset,MPII(Max Planck Institute Informatics)Human Pose,has been used for system evaluation.The results prove the validity of the proposed system as it achieved 87.5%class recognition accuracy.展开更多
文摘This paper proposes a method to recognize human-object interactions by modeling context between human actions and interacted objects.Human-object interaction recognition is a challenging task due to severe occlusion between human and objects during the interacting process.Since that human actions and interacted objects provide strong context information,i.e.some actions are usually related to some specific objects,the accuracy of recognition is significantly improved for both of them.Through the proposed method,both global and local temporal features from skeleton sequences are extracted to model human actions.In the meantime,kernel features are utilized to describe interacted objects.Finally,all possible solutions from actions and objects are optimized by modeling the context between them.The results of experiments demonstrate the effectiveness of our method.
基金Project(51678075)supported by the National Natural Science Foundation of ChinaProject(2017GK2271)supported by the Hunan Provincial Science and Technology Department,China。
文摘Human-object interaction(HOIs)detection is a new branch of visual relationship detection,which plays an important role in the field of image understanding.Because of the complexity and diversity of image content,the detection of HOIs is still an onerous challenge.Unlike most of the current works for HOIs detection which only rely on the pairwise information of a human and an object,we propose a graph-based HOIs detection method that models context and global structure information.Firstly,to better utilize the relations between humans and objects,the detected humans and objects are regarded as nodes to construct a fully connected undirected graph,and the graph is pruned to obtain an HOI graph that only preserving the edges connecting human and object nodes.Then,in order to obtain more robust features of human and object nodes,two different attention-based feature extraction networks are proposed,which model global and local contexts respectively.Finally,the graph attention network is introduced to pass messages between different nodes in the HOI graph iteratively,and detect the potential HOIs.Experiments on V-COCO and HICO-DET datasets verify the effectiveness of the proposed method,and show that it is superior to many existing methods.
基金This research was supported by a grant(2021R1F1A1063634)of the Basic Science Research Program through the National Research Foundation(NRF)funded by the Ministry of Education,Republic of Korea.
文摘Human object interaction(HOI)recognition plays an important role in the designing of surveillance and monitoring systems for healthcare,sports,education,and public areas.It involves localizing the human and object targets and then identifying the interactions between them.However,it is a challenging task that highly depends on the extraction of robust and distinctive features from the targets and the use of fast and efficient classifiers.Hence,the proposed system offers an automated body-parts-based solution for HOI recognition.This system uses RGB(red,green,blue)images as input and segments the desired parts of the images through a segmentation technique based on the watershed algorithm.Furthermore,a convex hullbased approach for extracting key body parts has also been introduced.After identifying the key body parts,two types of features are extracted.Moreover,the entire feature vector is reduced using a dimensionality reduction technique called t-SNE(t-distributed stochastic neighbor embedding).Finally,a multinomial logistic regression classifier is utilized for identifying class labels.A large publicly available dataset,MPII(Max Planck Institute Informatics)Human Pose,has been used for system evaluation.The results prove the validity of the proposed system as it achieved 87.5%class recognition accuracy.
