Recognizing human interactions in RGB videos is a critical task in computer vision,with applications in video surveillance.Existing deep learning-based architectures have achieved strong results,but are computationall...Recognizing human interactions in RGB videos is a critical task in computer vision,with applications in video surveillance.Existing deep learning-based architectures have achieved strong results,but are computationally intensive,sensitive to video resolution changes and often fail in crowded scenes.We propose a novel hybrid system that is computationally efficient,robust to degraded video quality and able to filter out irrelevant individuals,making it suitable for real-life use.The system leverages multi-modal handcrafted features for interaction representation and a deep learning classifier for capturing complex dependencies.Using Mask R-CNN and YOLO11-Pose,we extract grayscale silhouettes and keypoint coordinates of interacting individuals,while filtering out irrelevant individuals using a proposed algorithm.From these,we extract silhouette-based features(local ternary pattern and histogram of optical flow)and keypoint-based features(distances,angles and velocities)that capture distinct spatial and temporal information.A Bidirectional Long Short-Term Memory network(BiLSTM)then classifies the interactions.Extensive experiments on the UT Interaction,SBU Kinect Interaction and the ISR-UOL 3D social activity datasets demonstrate that our system achieves competitive accuracy.They also validate the effectiveness of the chosen features and classifier,along with the proposed system’s computational efficiency and robustness to occlusion.展开更多
A new method for interaction recognition based on sparse representation of feature covariance matrices was presented.Firstly,the dense trajectories(DT)extracted from the video were clustered into different groups to e...A new method for interaction recognition based on sparse representation of feature covariance matrices was presented.Firstly,the dense trajectories(DT)extracted from the video were clustered into different groups to eliminate the irrelevant trajectories,which could greatly reduce the noise influence on feature extraction.Then,the trajectory tunnels were characterized by means of feature covariance matrices.In this way,the discriminative descriptors could be extracted,which was also an effective solution to the problem that the description of the feature second-order statistics is insufficient.After that,an over-complete dictionary was learned with the descriptors and all the descriptors were encoded using sparse coding(SC).Classification was achieved using multiple instance learning(MIL),which was more suitable for complex environments.The proposed method was tested and evaluated on the WEB Interaction dataset and the UT interaction dataset.The experimental results demonstrated the superior efficiency.展开更多
Human Interaction Recognition(HIR)was one of the challenging issues in computer vision research due to the involvement of multiple individuals and their mutual interactions within video frames generated from their mov...Human Interaction Recognition(HIR)was one of the challenging issues in computer vision research due to the involvement of multiple individuals and their mutual interactions within video frames generated from their movements.HIR requires more sophisticated analysis than Human Action Recognition(HAR)since HAR focuses solely on individual activities like walking or running,while HIR involves the interactions between people.This research aims to develop a robust system for recognizing five common human interactions,such as hugging,kicking,pushing,pointing,and no interaction,from video sequences using multiple cameras.In this study,a hybrid Deep Learning(DL)and Machine Learning(ML)model was employed to improve classification accuracy and generalizability.The dataset was collected in an indoor environment with four-channel cameras capturing the five types of interactions among 13 participants.The data was processed using a DL model with a fine-tuned ResNet(Residual Networks)architecture based on 2D Convolutional Neural Network(CNN)layers for feature extraction.Subsequently,machine learning models were trained and utilized for interaction classification using six commonly used ML algorithms,including SVM,KNN,RF,DT,NB,and XGBoost.The results demonstrate a high accuracy of 95.45%in classifying human interactions.The hybrid approach enabled effective learning,resulting in highly accurate performance across different interaction types.Future work will explore more complex scenarios involving multiple individuals based on the application of this architecture.展开更多
We introduce an innovative approach to address a significant challenge in interaction recognition,specificallythe capture of correlation features between different interaction body parts.These features are oftenoverlo...We introduce an innovative approach to address a significant challenge in interaction recognition,specificallythe capture of correlation features between different interaction body parts.These features are oftenoverlooked by traditional graph convolution networks commonly used in interaction recognition tasks.Oursolution,the Merge-and-Split Graph Convolutional Network,takes a unique perspective,treating interactionrecognition as a global problem.It leverages a Merge-and-Split Graph structure to effectively capturedependencies between interaction body parts.To extract the essential interaction features,we introducethe Merge-and-Split Graph Convolution module,which seamlessly combines the Merge-and-Split Graphwith Graph Convolutional Networks.This fusion enables the extraction of rich semantic information betweenadjacent joint points.In addition,we introduce a Short-term Dependence module designed to extract jointand motion characteristics specific to each type of interaction.Furthermore,to extract correlation featuresbetween different hierarchical sets,we present the Hierarchical Guided Attention Module.This module playsa crucial role in highlighting the relevant hierarchical sets that contain essential interaction information.The effectiveness of our proposed model is demonstrated by achieving state-of-the-art performance on 2widely recognized datasets,namely,the NTU60 and NTU120 interaction datasets.Our model’s efficacy isrigorously validated through extensive experiments,and we have made the code available for the researchcommunity at https://github.com/wanghq05/MS-GCN/.展开更多
Human interaction recognition is an essential task in video surveillance.The current works on human interaction recognition mainly focus on the scenarios only containing the close-contact interactive subjects without ...Human interaction recognition is an essential task in video surveillance.The current works on human interaction recognition mainly focus on the scenarios only containing the close-contact interactive subjects without other people.In this paper,we handle more practical but more challenging scenarios where interactive subjects are contactless and other subjects not involved in the interactions of interest are also present in the scene.To address this problem,we propose an Interactive Relation Embedding Network(IRE-Net)to simultaneously identify the subjects involved in the interaction and recognize their interaction category.As a new problem,we also build a new dataset with annotations and metrics for performance evaluation.Experimental results on this datasesthow significant improvements of the proposed method when compared with current methodsdeveloped for human interaction recognition and group activity recognition.展开更多
Molecularly imprinted polymer, exhibiting considerable enantioselectivity for L-mandelic acid, was prepared using metal coordination-chelation interaction. By evaluating the recognition characteristics in the chromato...Molecularly imprinted polymer, exhibiting considerable enantioselectivity for L-mandelic acid, was prepared using metal coordination-chelation interaction. By evaluating the recognition characteristics in the chromatographic mode, the recognition interactions were proposed: specific and nonspecific metal coordination-chelation interaction and hydrophobic interaction were responsible for substrate binding on metal-complexing imprinted polymer; while the selective recognition only came from specific metal coordination-chelation interaction and specific hydrophobic interaction.展开更多
The recognition interaction of Rhodamine B(RB) with DNA was studied in a Britton-Robinson (B-R) buffer solution with pH=7.5 at a glassy carbon electrode by electrochemical techniques. RB shows an irreversible oxidatio...The recognition interaction of Rhodamine B(RB) with DNA was studied in a Britton-Robinson (B-R) buffer solution with pH=7.5 at a glassy carbon electrode by electrochemical techniques. RB shows an irreversible oxidation peak at +0.92 V(vs. SCE). After the addition of DNA in the RB solution, the peak current of RB decreased apparently without the shift of the peak potential. The electrochemical parameters such as the charge transfer coefficient α and the electrode reaction rate constant k s of the interaction system were carefully studied. The parameters did not change before and after the addition of DNA, which indicated that an electrochemical non-active complex had been formed, so the concentration of RB in the solution decreased and the peak current decreased correspondingly. The binding ratio of RB to DNA was 2∶1 with a binding constant of 2.66×10 9.展开更多
The recognition interaction of rhodamine B (RB) with DNA was studied in pH 7.5 Britton-Robinson (B-R) buffer solution by electrochemical techniques. An irreversible oxidation peak at glassy carbon electrode was obtain...The recognition interaction of rhodamine B (RB) with DNA was studied in pH 7.5 Britton-Robinson (B-R) buffer solution by electrochemical techniques. An irreversible oxidation peak at glassy carbon electrode was obtained at +0.92V (vs. SCE). After the addition of DNA into the RB solution, the peak current of RB decreased apparently without the shift of peak potential. The electrochemical parameters such as the charge transfer coefficient a and the electrode reaction standard rate constant ks of RB in the absence and presence of DNA were determined, which did not change, indicating that a non-electroactive complex was formed, so the concentration of RB in the solution decreased and the peak current decreased correspondingly.展开更多
Thermoresponsive biotinylated dendronized copolymers carrying dendritic oligoethylene glycol（OEG）pendants were prepared via free radical polymerization,and their protein recognitions based on biotin-avidin interacti...Thermoresponsive biotinylated dendronized copolymers carrying dendritic oligoethylene glycol（OEG）pendants were prepared via free radical polymerization,and their protein recognitions based on biotin-avidin interaction investigated.Both first（PG1） and second generation（PG2） dendronized copolymers were designed to examine possible thickness effects on the interaction between biotin and avidin.Inherited from the outstanding thermoresponsive properties from OEG dendrons,these biotinylated cylindrical copolymers show characteristic thermoresponsive behavior which provides an envelope to capture avidin through switching temperatures above or below their phase transition temperatures（T_（cp）s）.Thus,the recognition of polymer-supported biotin with avidin was investigated with UV/vis spectroscopy and dynamic laser light scattering.In contrast to the case for PG1,the increased thickness for copolymer PG2 hinders partially and inhibits the recognition of biotin moieties with avidin either below or above its T_（cp）.This demonstrates the significant architecture effects from dendronized polymers on the biotin moieties to shift onto periphery of the collapsed aggregates,which should be a prerequisite for protein recognition.These kinds of novel thermoresponsive copolymers may pave a way for the interesting biological applications in areas such as reversible activity control of enzyme or proteins,and for controlled delivery of drugs or genes.展开更多
Human-human interaction recognition is crucial in computer vision fields like surveillance,human-computer interaction,and social robotics.It enhances systems’ability to interpret and respond to human behavior precise...Human-human interaction recognition is crucial in computer vision fields like surveillance,human-computer interaction,and social robotics.It enhances systems’ability to interpret and respond to human behavior precisely.This research focuses on recognizing human interaction behaviors using a static image,which is challenging due to the complexity of diverse actions.The overall purpose of this study is to develop a robust and accurate system for human interaction recognition.This research presents a novel image-based human interaction recognition method using a Hidden Markov Model(HMM).The technique employs hue,saturation,and intensity(HSI)color transformation to enhance colors in video frames,making them more vibrant and visually appealing,especially in low-contrast or washed-out scenes.Gaussian filters reduce noise and smooth imperfections followed by silhouette extraction using a statistical method.Feature extraction uses the features from Accelerated Segment Test(FAST),Oriented FAST,and Rotated BRIEF(ORB)techniques.The application of Quadratic Discriminant Analysis(QDA)for feature fusion and discrimination enables high-dimensional data to be effectively analyzed,thus further enhancing the classification process.It ensures that the final features loaded into the HMM classifier accurately represent the relevant human activities.The impressive accuracy rates of 93%and 94.6%achieved in the BIT-Interaction and UT-Interaction datasets respectively,highlight the success and reliability of the proposed technique.The proposed approach addresses challenges in various domains by focusing on frame improvement,silhouette and feature extraction,feature fusion,and HMM classification.This enhances data quality,accuracy,adaptability,reliability,and reduction of errors.展开更多
Anion-π interactions as a new member of supramolecular weak interactions are still in the young stage,but they already attract considerable attentions. Now the concerns are shifting from recognition to construction o...Anion-π interactions as a new member of supramolecular weak interactions are still in the young stage,but they already attract considerable attentions. Now the concerns are shifting from recognition to construction of functional systems. In this review, the anion-π functional systems especially anion-πcatalysis and self-assembly were highlighted and summarized together with several solid and recent examples of host-vip recognition. These applications suggest that the great potential of these new interactions.展开更多
Based on light field reconstruction and motion recognition technique, a penetrable interactive floating 3D display system is proposed. The system consists of a high-frame-rate projector, a flat directional diffusing s...Based on light field reconstruction and motion recognition technique, a penetrable interactive floating 3D display system is proposed. The system consists of a high-frame-rate projector, a flat directional diffusing screen, a high-speed data transmission module, and a Kinect somatosensory device. The floating occlusioncorrect 3D image could rotate around some axis at different speeds according to user＇s hand motion. Eight motion directions and speed are detected accurately, and the prototype system operates efficiently with a recognition accuracy of 90% on average.展开更多
基金supported and funded by Princess Nourah bint Abdulrahman University Researchers Supporting Project number(PNURSP2025R410),Princess Nourah bint Abdulrahman University,Riyadh,Saudi Arabia.
文摘Recognizing human interactions in RGB videos is a critical task in computer vision,with applications in video surveillance.Existing deep learning-based architectures have achieved strong results,but are computationally intensive,sensitive to video resolution changes and often fail in crowded scenes.We propose a novel hybrid system that is computationally efficient,robust to degraded video quality and able to filter out irrelevant individuals,making it suitable for real-life use.The system leverages multi-modal handcrafted features for interaction representation and a deep learning classifier for capturing complex dependencies.Using Mask R-CNN and YOLO11-Pose,we extract grayscale silhouettes and keypoint coordinates of interacting individuals,while filtering out irrelevant individuals using a proposed algorithm.From these,we extract silhouette-based features(local ternary pattern and histogram of optical flow)and keypoint-based features(distances,angles and velocities)that capture distinct spatial and temporal information.A Bidirectional Long Short-Term Memory network(BiLSTM)then classifies the interactions.Extensive experiments on the UT Interaction,SBU Kinect Interaction and the ISR-UOL 3D social activity datasets demonstrate that our system achieves competitive accuracy.They also validate the effectiveness of the chosen features and classifier,along with the proposed system’s computational efficiency and robustness to occlusion.
基金Project(51678075) supported by the National Natural Science Foundation of ChinaProject(2017GK2271) supported by the Science and Technology Project of Hunan Province,China
文摘A new method for interaction recognition based on sparse representation of feature covariance matrices was presented.Firstly,the dense trajectories(DT)extracted from the video were clustered into different groups to eliminate the irrelevant trajectories,which could greatly reduce the noise influence on feature extraction.Then,the trajectory tunnels were characterized by means of feature covariance matrices.In this way,the discriminative descriptors could be extracted,which was also an effective solution to the problem that the description of the feature second-order statistics is insufficient.After that,an over-complete dictionary was learned with the descriptors and all the descriptors were encoded using sparse coding(SC).Classification was achieved using multiple instance learning(MIL),which was more suitable for complex environments.The proposed method was tested and evaluated on the WEB Interaction dataset and the UT interaction dataset.The experimental results demonstrated the superior efficiency.
基金supported by the National Research Foundation of Korea(NRF)grant funded by the Korea government(MSIT)(No.RS-2023-00218176)and the Soonchunhyang University Research Fund.
文摘Human Interaction Recognition(HIR)was one of the challenging issues in computer vision research due to the involvement of multiple individuals and their mutual interactions within video frames generated from their movements.HIR requires more sophisticated analysis than Human Action Recognition(HAR)since HAR focuses solely on individual activities like walking or running,while HIR involves the interactions between people.This research aims to develop a robust system for recognizing five common human interactions,such as hugging,kicking,pushing,pointing,and no interaction,from video sequences using multiple cameras.In this study,a hybrid Deep Learning(DL)and Machine Learning(ML)model was employed to improve classification accuracy and generalizability.The dataset was collected in an indoor environment with four-channel cameras capturing the five types of interactions among 13 participants.The data was processed using a DL model with a fine-tuned ResNet(Residual Networks)architecture based on 2D Convolutional Neural Network(CNN)layers for feature extraction.Subsequently,machine learning models were trained and utilized for interaction classification using six commonly used ML algorithms,including SVM,KNN,RF,DT,NB,and XGBoost.The results demonstrate a high accuracy of 95.45%in classifying human interactions.The hybrid approach enabled effective learning,resulting in highly accurate performance across different interaction types.Future work will explore more complex scenarios involving multiple individuals based on the application of this architecture.
基金funding from the NationalNatural Science Foundation of China under Grant.No.62073004support from the Shenzhen Fundamental ResearchProgram under Grants.No.GXWD20201231165807007-20200807164903001 and JCYJ20200109140410340.
文摘We introduce an innovative approach to address a significant challenge in interaction recognition,specificallythe capture of correlation features between different interaction body parts.These features are oftenoverlooked by traditional graph convolution networks commonly used in interaction recognition tasks.Oursolution,the Merge-and-Split Graph Convolutional Network,takes a unique perspective,treating interactionrecognition as a global problem.It leverages a Merge-and-Split Graph structure to effectively capturedependencies between interaction body parts.To extract the essential interaction features,we introducethe Merge-and-Split Graph Convolution module,which seamlessly combines the Merge-and-Split Graphwith Graph Convolutional Networks.This fusion enables the extraction of rich semantic information betweenadjacent joint points.In addition,we introduce a Short-term Dependence module designed to extract jointand motion characteristics specific to each type of interaction.Furthermore,to extract correlation featuresbetween different hierarchical sets,we present the Hierarchical Guided Attention Module.This module playsa crucial role in highlighting the relevant hierarchical sets that contain essential interaction information.The effectiveness of our proposed model is demonstrated by achieving state-of-the-art performance on 2widely recognized datasets,namely,the NTU60 and NTU120 interaction datasets.Our model’s efficacy isrigorously validated through extensive experiments,and we have made the code available for the researchcommunity at https://github.com/wanghq05/MS-GCN/.
基金This work was supported by the National Natural Science Foundation of China(NSFC)(Grant Nos.62072334,U1803264).
文摘Human interaction recognition is an essential task in video surveillance.The current works on human interaction recognition mainly focus on the scenarios only containing the close-contact interactive subjects without other people.In this paper,we handle more practical but more challenging scenarios where interactive subjects are contactless and other subjects not involved in the interactions of interest are also present in the scene.To address this problem,we propose an Interactive Relation Embedding Network(IRE-Net)to simultaneously identify the subjects involved in the interaction and recognize their interaction category.As a new problem,we also build a new dataset with annotations and metrics for performance evaluation.Experimental results on this datasesthow significant improvements of the proposed method when compared with current methodsdeveloped for human interaction recognition and group activity recognition.
文摘Molecularly imprinted polymer, exhibiting considerable enantioselectivity for L-mandelic acid, was prepared using metal coordination-chelation interaction. By evaluating the recognition characteristics in the chromatographic mode, the recognition interactions were proposed: specific and nonspecific metal coordination-chelation interaction and hydrophobic interaction were responsible for substrate binding on metal-complexing imprinted polymer; while the selective recognition only came from specific metal coordination-chelation interaction and specific hydrophobic interaction.
文摘The recognition interaction of Rhodamine B(RB) with DNA was studied in a Britton-Robinson (B-R) buffer solution with pH=7.5 at a glassy carbon electrode by electrochemical techniques. RB shows an irreversible oxidation peak at +0.92 V(vs. SCE). After the addition of DNA in the RB solution, the peak current of RB decreased apparently without the shift of the peak potential. The electrochemical parameters such as the charge transfer coefficient α and the electrode reaction rate constant k s of the interaction system were carefully studied. The parameters did not change before and after the addition of DNA, which indicated that an electrochemical non-active complex had been formed, so the concentration of RB in the solution decreased and the peak current decreased correspondingly. The binding ratio of RB to DNA was 2∶1 with a binding constant of 2.66×10 9.
基金The work was supported by the National Natural Science Foundation of China(Grant No 20375020).
文摘The recognition interaction of rhodamine B (RB) with DNA was studied in pH 7.5 Britton-Robinson (B-R) buffer solution by electrochemical techniques. An irreversible oxidation peak at glassy carbon electrode was obtained at +0.92V (vs. SCE). After the addition of DNA into the RB solution, the peak current of RB decreased apparently without the shift of peak potential. The electrochemical parameters such as the charge transfer coefficient a and the electrode reaction standard rate constant ks of RB in the absence and presence of DNA were determined, which did not change, indicating that a non-electroactive complex was formed, so the concentration of RB in the solution decreased and the peak current decreased correspondingly.
基金the National Natural Science Foundation of China(Nos.21374058,21474060 and 21574078)the Ph.D. Programs Foundation of Ministry of Education of China(No 201331081100166)the Shanghai Rising-Star Program(No.16QA1401800)
文摘Thermoresponsive biotinylated dendronized copolymers carrying dendritic oligoethylene glycol（OEG）pendants were prepared via free radical polymerization,and their protein recognitions based on biotin-avidin interaction investigated.Both first（PG1） and second generation（PG2） dendronized copolymers were designed to examine possible thickness effects on the interaction between biotin and avidin.Inherited from the outstanding thermoresponsive properties from OEG dendrons,these biotinylated cylindrical copolymers show characteristic thermoresponsive behavior which provides an envelope to capture avidin through switching temperatures above or below their phase transition temperatures（T_（cp）s）.Thus,the recognition of polymer-supported biotin with avidin was investigated with UV/vis spectroscopy and dynamic laser light scattering.In contrast to the case for PG1,the increased thickness for copolymer PG2 hinders partially and inhibits the recognition of biotin moieties with avidin either below or above its T_（cp）.This demonstrates the significant architecture effects from dendronized polymers on the biotin moieties to shift onto periphery of the collapsed aggregates,which should be a prerequisite for protein recognition.These kinds of novel thermoresponsive copolymers may pave a way for the interesting biological applications in areas such as reversible activity control of enzyme or proteins,and for controlled delivery of drugs or genes.
基金funding this work under the Research Group Funding Program Grant Code(NU/RG/SERC/12/6)supported via funding from Prince Satam bin Abdulaziz University Project Number(PSAU/2023/R/1444)+1 种基金Princess Nourah bint Abdulrahman University Researchers Supporting Project Number(PNURSP2023R348)Princess Nourah bint Abdulrahman University,Riyadh,Saudi Arabia,and this work was also supported by the Ministry of Science and ICT(MSIT),South Korea,through the ICT Creative Consilience Program supervised by the Institute for Information and Communications Technology Planning and Evaluation(IITP)under Grant IITP-2023-2020-0-01821.
文摘Human-human interaction recognition is crucial in computer vision fields like surveillance,human-computer interaction,and social robotics.It enhances systems’ability to interpret and respond to human behavior precisely.This research focuses on recognizing human interaction behaviors using a static image,which is challenging due to the complexity of diverse actions.The overall purpose of this study is to develop a robust and accurate system for human interaction recognition.This research presents a novel image-based human interaction recognition method using a Hidden Markov Model(HMM).The technique employs hue,saturation,and intensity(HSI)color transformation to enhance colors in video frames,making them more vibrant and visually appealing,especially in low-contrast or washed-out scenes.Gaussian filters reduce noise and smooth imperfections followed by silhouette extraction using a statistical method.Feature extraction uses the features from Accelerated Segment Test(FAST),Oriented FAST,and Rotated BRIEF(ORB)techniques.The application of Quadratic Discriminant Analysis(QDA)for feature fusion and discrimination enables high-dimensional data to be effectively analyzed,thus further enhancing the classification process.It ensures that the final features loaded into the HMM classifier accurately represent the relevant human activities.The impressive accuracy rates of 93%and 94.6%achieved in the BIT-Interaction and UT-Interaction datasets respectively,highlight the success and reliability of the proposed technique.The proposed approach addresses challenges in various domains by focusing on frame improvement,silhouette and feature extraction,feature fusion,and HMM classification.This enhances data quality,accuracy,adaptability,reliability,and reduction of errors.
基金supported by the National Natural Science Foundation of China(No. 21604046)the National Young Thousand Talents ProgramShandong Provincial Natural Science Foundation, China (No. ZR2016XJ004)
文摘Anion-π interactions as a new member of supramolecular weak interactions are still in the young stage,but they already attract considerable attentions. Now the concerns are shifting from recognition to construction of functional systems. In this review, the anion-π functional systems especially anion-πcatalysis and self-assembly were highlighted and summarized together with several solid and recent examples of host-vip recognition. These applications suggest that the great potential of these new interactions.
基金supported by the National Basic Research Program of China(973 Program)(No.2013CB328806)the National High Technology Research and Development Program of China(863 Program)(No.2012AA011902)+1 种基金the National Natural Science Foundation of China(No.61177015)the Research Funds for the Central Universities of China(No.2012XZZX013)
文摘Based on light field reconstruction and motion recognition technique, a penetrable interactive floating 3D display system is proposed. The system consists of a high-frame-rate projector, a flat directional diffusing screen, a high-speed data transmission module, and a Kinect somatosensory device. The floating occlusioncorrect 3D image could rotate around some axis at different speeds according to user＇s hand motion. Eight motion directions and speed are detected accurately, and the prototype system operates efficiently with a recognition accuracy of 90% on average.
