Micro-Expression Recognition(MER)is a challenging task as the subtle changes occur over different action regions of a face.Changes in facial action regions are formed as Action Units(AUs),and AUs in micro-expressions ...Micro-Expression Recognition(MER)is a challenging task as the subtle changes occur over different action regions of a face.Changes in facial action regions are formed as Action Units(AUs),and AUs in micro-expressions can be seen as the actors in cooperative group activities.In this paper,we propose a novel deep neural network model for objective class-based MER,which simultaneously detects AUs and aggregates AU-level features into micro-expression-level representation through Graph Convolutional Networks(GCN).Specifically,we propose two new strategies in our AU detection module for more effective AU feature learning:the attention mechanism and the balanced detection loss function.With these two strategies,features are learned for all the AUs in a unified model,eliminating the error-prune landmark detection process and tedious separate training for each AU.Moreover,our model incorporates a tailored objective class-based AU knowledge-graph,which facilitates the GCN to aggregate the AU-level features into a micro-expression-level feature representation.Extensive experiments on two tasks in MEGC 2018 show that our approach outperforms the current state-of-the-art methods in MER.Additionally,we also report our single model-based micro-expression AU detection results.展开更多
微表情检测旨在视频中定位幅度微弱、时间短暂的表情区间。其难点在于有效提取面部区域间的动态关联特征和多尺度时序特征,进而精准捕捉面部各区域微小动作之间的关联。针对这些问题,提出了一种融合自适应图注意力和多尺度可变空洞卷积...微表情检测旨在视频中定位幅度微弱、时间短暂的表情区间。其难点在于有效提取面部区域间的动态关联特征和多尺度时序特征,进而精准捕捉面部各区域微小动作之间的关联。针对这些问题,提出了一种融合自适应图注意力和多尺度可变空洞卷积的微表情检测网络(AG-DDNet)。通过引入参数可学习矩阵来实现键值对的特征变换,通过计算面部区域特征向量间的相似度得到动态邻接矩阵,并结合图注意力机制计算区域间权重系数,实现特征的动态融合;采用了多尺度可变空洞卷积模块,通过自适应池化与卷积组合的预测器生成动态感受野,从而实现多尺度的特征提取;引入基于Fisher信息矩阵的自然梯度优化机制,通过Fisher Adam优化器有效捕捉参数空间的几何结构信息,实现学习率的精确自适应调整,从而显著增强了模型对微表情和宏表情的协同检测能力。在微表情检测任务中,该算法与同类代表性算法相比,在CAS(ME)2数据集和SAMM Long Videos数据集上的性能分别提升了54.20%和20.11%。与最新算法相比,两个数据集上的提升幅度分别为38.43%和6.81%,有效证明了该方法在长视频微表情检测任务上的优越性能。展开更多
Spectral energy distribution of surface EMG signal is often used but difficultly and effectively control artificial limb, because the spectral energy distribution changes in the process of limb actions. In this paper,...Spectral energy distribution of surface EMG signal is often used but difficultly and effectively control artificial limb, because the spectral energy distribution changes in the process of limb actions. In this paper, the general characteristics of surface EMG signal patterns were firstly characterized by spectral energy change. 13 healthy subjects were instructed to execute forearm supination （FS） and forearm pronation （FP） with their right foreanns when their forearm muscles were ＂fatigue＂ or ＂relaxed＂. All surface EMG signals were recorded from their right forearm flexor during their right forearm actions. Two sets of surface EMG signals were segmented from every surface EMG signal appropriately at preparing stage and acting stage. Relative wavelet packet energy （symbolized by pnp and pna respectively at preparing stage and acting stage, n denotes the nth frequency band） of surface EMG signal firstly was calculated and then, the difference （Pn = Pna-Pnp） were gained. The results showed that Pn from some frequency bands can effectively characterize the general characteristics of surface EMG signal patterns. Compared with Pn in other frequency bands, P4, the spectral energy change from 93.75 to 125 Hz, was more appropriately regarded as the features.展开更多
Human action recognition has become one of the most active research topics in human-computer interaction and artificial intelligence, and has attracted much attention. Here, we employ a low-cost optical sensor Kinect ...Human action recognition has become one of the most active research topics in human-computer interaction and artificial intelligence, and has attracted much attention. Here, we employ a low-cost optical sensor Kinect to capture the action information of the human skeleton. We then propose a two-level hierarchical human action recognition model with self-selection classifiers via skeleton data. Especially different optimal classifiers are selected by probability voting mechanism and 10 times 10-fold cross validation at different coarse grained levels. Extensive simulations on a well-known open dataset and results demonstrate that our proposed method is efficient in human action recognition, achieving 94.19%the average recognition rate and 95.61% the best rate.展开更多
1Introduction Facial action analysis(FAA)focuses on detecting facial movements,particularly facial action units(AUs).FAA tasks,including AU detection,AU intensity estimation,and pain estimation,are crucial for underst...1Introduction Facial action analysis(FAA)focuses on detecting facial movements,particularly facial action units(AUs).FAA tasks,including AU detection,AU intensity estimation,and pain estimation,are crucial for understanding emotions and physical conditions.A major challenge in FAA is the insufficiency of labeled data,hindering the performance and generalization of FAA models.展开更多
基金supported by the Science and Technology Development Fund of Macao(No.0035/2023/ITP1)the National Natural Science Foundation of China(Nos.U1836220 and 61672267)+2 种基金the Postgraduate Research&Practice Innovation Program of Jiangsu Province(No.KYCX19_1616)the Qing Lan Talent Program of Jiangsu ProvinceJiangsu Province Key Research and Development Plan(Industry Foresight and Key Core Technology)-Competitive Project(No.BE2020036).
文摘Micro-Expression Recognition(MER)is a challenging task as the subtle changes occur over different action regions of a face.Changes in facial action regions are formed as Action Units(AUs),and AUs in micro-expressions can be seen as the actors in cooperative group activities.In this paper,we propose a novel deep neural network model for objective class-based MER,which simultaneously detects AUs and aggregates AU-level features into micro-expression-level representation through Graph Convolutional Networks(GCN).Specifically,we propose two new strategies in our AU detection module for more effective AU feature learning:the attention mechanism and the balanced detection loss function.With these two strategies,features are learned for all the AUs in a unified model,eliminating the error-prune landmark detection process and tedious separate training for each AU.Moreover,our model incorporates a tailored objective class-based AU knowledge-graph,which facilitates the GCN to aggregate the AU-level features into a micro-expression-level feature representation.Extensive experiments on two tasks in MEGC 2018 show that our approach outperforms the current state-of-the-art methods in MER.Additionally,we also report our single model-based micro-expression AU detection results.
文摘微表情检测旨在视频中定位幅度微弱、时间短暂的表情区间。其难点在于有效提取面部区域间的动态关联特征和多尺度时序特征,进而精准捕捉面部各区域微小动作之间的关联。针对这些问题,提出了一种融合自适应图注意力和多尺度可变空洞卷积的微表情检测网络(AG-DDNet)。通过引入参数可学习矩阵来实现键值对的特征变换,通过计算面部区域特征向量间的相似度得到动态邻接矩阵,并结合图注意力机制计算区域间权重系数,实现特征的动态融合;采用了多尺度可变空洞卷积模块,通过自适应池化与卷积组合的预测器生成动态感受野,从而实现多尺度的特征提取;引入基于Fisher信息矩阵的自然梯度优化机制,通过Fisher Adam优化器有效捕捉参数空间的几何结构信息,实现学习率的精确自适应调整,从而显著增强了模型对微表情和宏表情的协同检测能力。在微表情检测任务中,该算法与同类代表性算法相比,在CAS(ME)2数据集和SAMM Long Videos数据集上的性能分别提升了54.20%和20.11%。与最新算法相比,两个数据集上的提升幅度分别为38.43%和6.81%,有效证明了该方法在长视频微表情检测任务上的优越性能。
基金China 973 Project,Grant number:2005CB724303Yunnan Education Department Project,Grant number:03Y3081
文摘Spectral energy distribution of surface EMG signal is often used but difficultly and effectively control artificial limb, because the spectral energy distribution changes in the process of limb actions. In this paper, the general characteristics of surface EMG signal patterns were firstly characterized by spectral energy change. 13 healthy subjects were instructed to execute forearm supination （FS） and forearm pronation （FP） with their right foreanns when their forearm muscles were ＂fatigue＂ or ＂relaxed＂. All surface EMG signals were recorded from their right forearm flexor during their right forearm actions. Two sets of surface EMG signals were segmented from every surface EMG signal appropriately at preparing stage and acting stage. Relative wavelet packet energy （symbolized by pnp and pna respectively at preparing stage and acting stage, n denotes the nth frequency band） of surface EMG signal firstly was calculated and then, the difference （Pn = Pna-Pnp） were gained. The results showed that Pn from some frequency bands can effectively characterize the general characteristics of surface EMG signal patterns. Compared with Pn in other frequency bands, P4, the spectral energy change from 93.75 to 125 Hz, was more appropriately regarded as the features.
基金Supported by the National Nature Science Foundation of China under Grant Nos.11475003,61603003,and 11471093the Key Project of Cultivation of Leading Talents in Universities of Anhui Province under Grant No.gxfxZD2016174+2 种基金Funds of Integration of Cloud Computing and Big DataInnovation of Science and Technology of Ministry of Education of China under Grant No.2017A09116Anhui Provincial Department of Education Outstanding Top-Notch Talent-Funded Project under Grant No.gxbjZD26
文摘Human action recognition has become one of the most active research topics in human-computer interaction and artificial intelligence, and has attracted much attention. Here, we employ a low-cost optical sensor Kinect to capture the action information of the human skeleton. We then propose a two-level hierarchical human action recognition model with self-selection classifiers via skeleton data. Especially different optimal classifiers are selected by probability voting mechanism and 10 times 10-fold cross validation at different coarse grained levels. Extensive simulations on a well-known open dataset and results demonstrate that our proposed method is efficient in human action recognition, achieving 94.19%the average recognition rate and 95.61% the best rate.
基金supported by the National Natural Science Foundation of China(Grant Nos.U2336213,62176248).
文摘1Introduction Facial action analysis(FAA)focuses on detecting facial movements,particularly facial action units(AUs).FAA tasks,including AU detection,AU intensity estimation,and pain estimation,are crucial for understanding emotions and physical conditions.A major challenge in FAA is the insufficiency of labeled data,hindering the performance and generalization of FAA models.
