As an essential field of multimedia and computer vision,3D shape recognition has attracted much research attention in recent years.Multiview-based approaches have demonstrated their superiority in generating effective...As an essential field of multimedia and computer vision,3D shape recognition has attracted much research attention in recent years.Multiview-based approaches have demonstrated their superiority in generating effective 3D shape representations.Typical methods usually extract the multiview global features and aggregate them together to generate 3D shape descriptors.However,there exist two disadvantages:First,the mainstream methods ignore the comprehensive exploration of local information in each view.Second,many approaches roughly aggregate multiview features by adding or concatenating them together.The information loss for some discriminative characteristics limits the representation effectiveness.To address these problems,a novel architecture named region-based joint attention network(RJAN)was proposed.Specifically,the authors first design a hierarchical local information exploration module for view descriptor extraction.The region-to-region and channel-to-channel relationships from different granularities can be comprehensively explored and utilised to provide more discriminative characteristics for view feature learning.Subsequently,a novel relation-aware view aggregation module is designed to aggregate the multiview features for shape descriptor generation,considering the view-to-view relationships.Extensive experiments were conducted on three public databases:ModelNet40,ModelNet10,and ShapeNetCore55.RJAN achieves state-of-the-art performance in the tasks of 3D shape classification and 3D shape retrieval,which demonstrates the effectiveness of RJAN.The code has been released on https://github.com/slurrpp/RJAN.展开更多
We present a novel method for scale-invariant 3D face recognition by integrating computer-generated holography with the Mellin transform.This approach leverages the scale-invariance property of the Mellin transform to...We present a novel method for scale-invariant 3D face recognition by integrating computer-generated holography with the Mellin transform.This approach leverages the scale-invariance property of the Mellin transform to address challenges related to variations in 3D facial sizes during recognition.By applying the Mellin transform to computer-generated holograms and performing correlation between them,which,to the best of our knowledge,is being done for the first time,we have developed a robust recognition framework capable of managing significant scale variations without compromising recognition accuracy.Digital holograms of 3D faces are generated from a face database,and the Mellin transform is employed to enable robust recognition across scale factors ranging from 0.4 to 2.0.Within this range,the method achieves 100%recognition accuracy,as confirmed by both simulation-based and hybrid optical/digital experimental validations.Numerical calculations demonstrate that our method significantly enhances the accuracy and reliability of 3D face recognition,as evidenced by the sharp correlation peaks and higher peak-to-noise ratio(PNR)values than that of using conventional holograms without the Mellin transform.Additionally,the hybrid optical/digital joint transform correlation hardware further validates the method's effectiveness,demonstrating its capability to accurately identify and distinguish 3D faces at various scales.This work provides a promising solution for advanced biometric systems,especially for those which require 3D scale-invariant recognition.展开更多
The 3D face recognition attracts more and more attention because of its insensitivity to the variance of illumination and pose.There are many crucial problems to be solved in this topic,such as 3D face representation ...The 3D face recognition attracts more and more attention because of its insensitivity to the variance of illumination and pose.There are many crucial problems to be solved in this topic,such as 3D face representation and effective multi-feature fusion.In this paper,a novel 3D face recognition algorithm is proposed and its performance is demonstrated on BJUT-3D face database.This algorithm chooses face surface property and the principle component of relative relation matrix as the face representation features.The similarity metric measure for each feature is defined.A feature fusion strategy is proposed.It is a linear weighted strategy based on Fisher linear discriminant analysis.Finally,the presented algorithm is tested on the BJUT-3D face database.It is concluded that the performance of the algorithm and fusion strategy is satisfying.展开更多
Expression, occlusion, and pose variations are three main challenges for 3D face recognition. A novel method is presented to address 3D face recognition using scale-invariant feature transform(SIFT) features on 3D mes...Expression, occlusion, and pose variations are three main challenges for 3D face recognition. A novel method is presented to address 3D face recognition using scale-invariant feature transform(SIFT) features on 3D meshes. After preprocessing, shape index extrema on the 3D facial surface are selected as keypoints in the difference scale space and the unstable keypoints are removed after two screening steps. Then, a local coordinate system for each keypoint is established by principal component analysis(PCA).Next, two local geometric features are extracted around each keypoint through the local coordinate system. Additionally, the features are augmented by the symmetrization according to the approximate left-right symmetry in human face. The proposed method is evaluated on the Bosphorus, BU-3DFE, and Gavab databases, respectively. Good results are achieved on these three datasets. As a result, the proposed method proves robust to facial expression variations, partial external occlusions and large pose changes.展开更多
In order to find better simplicity measurements for 3D object recognition, a new set of local regularities is developed and tested in a stepwise 3D reconstruction method, including localized minimizing standard deviat...In order to find better simplicity measurements for 3D object recognition, a new set of local regularities is developed and tested in a stepwise 3D reconstruction method, including localized minimizing standard deviation of angles（L-MSDA）, localized minimizing standard deviation of segment magnitudes（L-MSDSM）, localized minimum standard deviation of areas of child faces （L-MSDAF）, localized minimum sum of segment magnitudes of common edges （L-MSSM）, and localized minimum sum of areas of child face （L-MSAF）. Based on their effectiveness measurements in terms of form and size distortions, it is found that when two local regularities： L-MSDA and L-MSDSM are combined together, they can produce better performance. In addition, the best weightings for them to work together are identified as 10% for L-MSDSM and 90% for L-MSDA. The test results show that the combined usage of L-MSDA and L-MSDSM with identified weightings has a potential to be applied in other optimization based 3D recognition methods to improve their efficacy and robustness.展开更多
Airborne LIDAR can flexibly obtain point cloud data with three-dimensional structural information,which can improve its effectiveness of automatic target recognition in the complex environment.Compared with 2D informa...Airborne LIDAR can flexibly obtain point cloud data with three-dimensional structural information,which can improve its effectiveness of automatic target recognition in the complex environment.Compared with 2D information,3D information performs better in separating objects and background.However,an aircraft platform can have a negative influence on LIDAR obtained data because of various flight attitudes,flight heights and atmospheric disturbances.A structure of global feature based 3D automatic target recognition method for airborne LIDAR is proposed,which is composed of offline phase and online phase.The performance of four global feature descriptors is compared.Considering the summed volume region(SVR) discrepancy in real objects,SVR selection is added into the pre-processing operations to eliminate mismatching clusters compared with the interested target.Highly reliable simulated data are obtained under various sensor’s altitudes,detection distances and atmospheric disturbances.The final experiments results show that the added step increases the recognition rate by above 2.4% and decreases the execution time by about 33%.展开更多
This paper proposes a new set of 3D rotation scaling and translation invariants of 3D radially shifted Legendre moments. We aim to develop two kinds of transformed shifted Legendre moments： a 3D substituted radial sh...This paper proposes a new set of 3D rotation scaling and translation invariants of 3D radially shifted Legendre moments. We aim to develop two kinds of transformed shifted Legendre moments： a 3D substituted radial shifted Legendre moments （3DSRSLMs） and a 3D weighted radial one （3DWRSLMs）. Both are centered on two types of polynomials. In the first case, a new 3D ra- dial complex moment is proposed. In the second case, new 3D substituted/weighted radial shifted Legendremoments （3DSRSLMs/3DWRSLMs） are introduced using a spherical representation of volumetric image. 3D invariants as derived from the sug- gested 3D radial shifted Legendre moments will appear in the third case. To confirm the proposed approach, we have resolved three is- sues. To confirm the proposed approach, we have resolved three issues： rotation, scaling and translation invariants. The result of experi- ments shows that the 3DSRSLMs and 3DWRSLMs have done better than the 3D radial complex moments with and without noise. Sim- ultaneously, the reconstruction converges rapidly to the original image using 3D radial 3DSRSLMs and 3DWRSLMs, and the test of 3D images are clearly recognized from a set of images that are available in Princeton shape benchmark （PSB） database for 3D image.展开更多
Virtual reality(VR)is an emerging communication means and creates extensive opportunities in interacting scenarios such as remote collaboration and metaverse.Human-machine interfaces(HMIs)play important roles in VR as...Virtual reality(VR)is an emerging communication means and creates extensive opportunities in interacting scenarios such as remote collaboration and metaverse.Human-machine interfaces(HMIs)play important roles in VR as they provide interaction platforms between users and virtual environments.However,traditional VR HMIs based on handheld devices or keyboards cannot recognize diverse three-dimensional(3D)gestures,which results in limited freedom of VR interactions.Here,we report a noncontact VR HMI enabled by an electret-nanofiber-based triboelectric sensor(ETS),which is fabricated by the electrospun polylactic acid/thermoplastic polyurethane(PLA/TPU)electret nanofiber film.The nanofiber structure of PLA/TPU electret enhanced the charge retention ability of triboelectric sensor and thus significantly improved its signal strength and stability.Integrated with a deep learning-based multilayer perceptron neural network,the ETS realizes the recognition of 18 different types of 3D gestures with a high average accuracy of 97.3%.An intelligent noncontact VR interactive system based on the ETS is further developed,which is used to manipulate game characters for performing different actions by 3D gestures.Compared with traditional VR HMIs,the proposed VR HMI based on PLA/TPU electret nanofiber film can detect various 3D gestures and offers a superior interaction freedom.This work for the first time introduces the triboelectric 3D gesture recognition method to the VR HMIs,and could make the interaction between human and virtual environments become more efficient and fascinating.展开更多
In the past ten years,research on face recognition has shifted to using 3D facial surfaces,as 3D geometric information provides more discriminative features.This comprehensive survey reviews 3D face recognition techni...In the past ten years,research on face recognition has shifted to using 3D facial surfaces,as 3D geometric information provides more discriminative features.This comprehensive survey reviews 3D face recognition techniques developed in the past decade,both conventional methods and deep learning methods.These methods are evaluated with detailed descriptions of selected representative works.Their advantages and disadvantages are summarized in terms of accuracy,complexity,and robustness to facial variations(expression,pose,occlusion,etc.).A review of 3D face databases is also provided,and a discussion of future research challenges and directions of the topic.展开更多
Based on light field reconstruction and motion recognition technique, a penetrable interactive floating 3D display system is proposed. The system consists of a high-frame-rate projector, a flat directional diffusing s...Based on light field reconstruction and motion recognition technique, a penetrable interactive floating 3D display system is proposed. The system consists of a high-frame-rate projector, a flat directional diffusing screen, a high-speed data transmission module, and a Kinect somatosensory device. The floating occlusioncorrect 3D image could rotate around some axis at different speeds according to user＇s hand motion. Eight motion directions and speed are detected accurately, and the prototype system operates efficiently with a recognition accuracy of 90% on average.展开更多
A 3D face recognition approach which uses principal axes registration(PAR)and three face representation features from the re-sampling depth image:Eigenfaces,Fisherfaces and Zernike moments is presented.The approach ad...A 3D face recognition approach which uses principal axes registration(PAR)and three face representation features from the re-sampling depth image:Eigenfaces,Fisherfaces and Zernike moments is presented.The approach addresses the issue of 3D face registration instantly achieved by PAR.Because each facial feature has its own advantages,limitations and scope of use,different features will complement each other.Thus the fusing features can learn more expressive characterizations than a single feature.The support vector machine(SVM)is applied for classification.In this method,based on the complementarity between different features,weighted decision-level fusion makes the recognition system have certain fault tolerance.Experimental results show that the proposed approach achieves superior performance with the rank-1 recognition rate of 98.36%for GavabDB database.展开更多
The research of emotion recognition based on electroencephalogram(EEG)signals often ignores the related information between the brain electrode channels and the contextual emotional information existing in EEG signals...The research of emotion recognition based on electroencephalogram(EEG)signals often ignores the related information between the brain electrode channels and the contextual emotional information existing in EEG signals,which may contain important characteristics related to emotional states.Aiming at the above defects,a spatiotemporal emotion recognition method based on a 3-dimensional(3 D)time-frequency domain feature matrix was proposed.Specifically,the extracted time-frequency domain EEG features are first expressed as a 3 D matrix format according to the actual position of the cerebral cortex.Then,the input 3 D matrix is processed successively by multivariate convolutional neural network(MVCNN)and long short-term memory(LSTM)to classify the emotional state.Spatiotemporal emotion recognition method is evaluated on the DEAP data set,and achieved accuracy of 87.58%and 88.50%on arousal and valence dimensions respectively in binary classification tasks,as well as obtained accuracy of 84.58%in four class classification tasks.The experimental results show that 3 D matrix representation can represent emotional information more reasonably than two-dimensional(2 D).In addition,MVCNN and LSTM can utilize the spatial information of the electrode channels and the temporal context information of the EEG signal respectively.展开更多
In this paper, illumination-affine invariant methods are presented based onaffine moment normalization techniques, Zernike moments, and multiband correlation functions. Themethods are suitable for the illumination inv...In this paper, illumination-affine invariant methods are presented based onaffine moment normalization techniques, Zernike moments, and multiband correlation functions. Themethods are suitable for the illumination invariant recognition of 3D color texture. Complex valuedmoments (i.e., Zernike moments) and affine moment normalization are used in the derivation ofillumination affine invariants where the real valued affine moment invariants fail to provide affineinvariants that are independent of illumination changes. Three different moment normalizationmethods have been used, two of which are based on affine moment normalization technique and thethird is based on reducing the affine transformation to a Euclidian transform. It is shown that fora change of illumination and orientation, the affinely normalized Zernike moment matrices arerelated by a linear transform. Experimental results are obtained in two tests: the first is usedwith textures of outdoor scenes while the second is performed on the well-known CUReT texturedatabase. Both tests show high recognition efficiency of the proposed recognition methods.展开更多
基金the National Key Research and Development Program of China,Grant/Award Number:2020YFB1711704the National Natural Science Foundation of China,Grant/Award Number:62272337。
文摘As an essential field of multimedia and computer vision,3D shape recognition has attracted much research attention in recent years.Multiview-based approaches have demonstrated their superiority in generating effective 3D shape representations.Typical methods usually extract the multiview global features and aggregate them together to generate 3D shape descriptors.However,there exist two disadvantages:First,the mainstream methods ignore the comprehensive exploration of local information in each view.Second,many approaches roughly aggregate multiview features by adding or concatenating them together.The information loss for some discriminative characteristics limits the representation effectiveness.To address these problems,a novel architecture named region-based joint attention network(RJAN)was proposed.Specifically,the authors first design a hierarchical local information exploration module for view descriptor extraction.The region-to-region and channel-to-channel relationships from different granularities can be comprehensively explored and utilised to provide more discriminative characteristics for view feature learning.Subsequently,a novel relation-aware view aggregation module is designed to aggregate the multiview features for shape descriptor generation,considering the view-to-view relationships.Extensive experiments were conducted on three public databases:ModelNet40,ModelNet10,and ShapeNetCore55.RJAN achieves state-of-the-art performance in the tasks of 3D shape classification and 3D shape retrieval,which demonstrates the effectiveness of RJAN.The code has been released on https://github.com/slurrpp/RJAN.
基金financial supports from the National Natural Science Foundation of China(Grant No.6227511362405124).
文摘We present a novel method for scale-invariant 3D face recognition by integrating computer-generated holography with the Mellin transform.This approach leverages the scale-invariance property of the Mellin transform to address challenges related to variations in 3D facial sizes during recognition.By applying the Mellin transform to computer-generated holograms and performing correlation between them,which,to the best of our knowledge,is being done for the first time,we have developed a robust recognition framework capable of managing significant scale variations without compromising recognition accuracy.Digital holograms of 3D faces are generated from a face database,and the Mellin transform is employed to enable robust recognition across scale factors ranging from 0.4 to 2.0.Within this range,the method achieves 100%recognition accuracy,as confirmed by both simulation-based and hybrid optical/digital experimental validations.Numerical calculations demonstrate that our method significantly enhances the accuracy and reliability of 3D face recognition,as evidenced by the sharp correlation peaks and higher peak-to-noise ratio(PNR)values than that of using conventional holograms without the Mellin transform.Additionally,the hybrid optical/digital joint transform correlation hardware further validates the method's effectiveness,demonstrating its capability to accurately identify and distinguish 3D faces at various scales.This work provides a promising solution for advanced biometric systems,especially for those which require 3D scale-invariant recognition.
基金Supported by National Natural Science Foundation of China(60533030)Beijing Natural Science Foundation(4061001)
文摘The 3D face recognition attracts more and more attention because of its insensitivity to the variance of illumination and pose.There are many crucial problems to be solved in this topic,such as 3D face representation and effective multi-feature fusion.In this paper,a novel 3D face recognition algorithm is proposed and its performance is demonstrated on BJUT-3D face database.This algorithm chooses face surface property and the principle component of relative relation matrix as the face representation features.The similarity metric measure for each feature is defined.A feature fusion strategy is proposed.It is a linear weighted strategy based on Fisher linear discriminant analysis.Finally,the presented algorithm is tested on the BJUT-3D face database.It is concluded that the performance of the algorithm and fusion strategy is satisfying.
基金Project(XDA06020300)supported by the"Strategic Priority Research Program"of the Chinese Academy of SciencesProject(12511501700)supported by the Research on the Key Technology of Internet of Things for Urban Community Safety Based on Video Sensor networks
文摘Expression, occlusion, and pose variations are three main challenges for 3D face recognition. A novel method is presented to address 3D face recognition using scale-invariant feature transform(SIFT) features on 3D meshes. After preprocessing, shape index extrema on the 3D facial surface are selected as keypoints in the difference scale space and the unstable keypoints are removed after two screening steps. Then, a local coordinate system for each keypoint is established by principal component analysis(PCA).Next, two local geometric features are extracted around each keypoint through the local coordinate system. Additionally, the features are augmented by the symmetrization according to the approximate left-right symmetry in human face. The proposed method is evaluated on the Bosphorus, BU-3DFE, and Gavab databases, respectively. Good results are achieved on these three datasets. As a result, the proposed method proves robust to facial expression variations, partial external occlusions and large pose changes.
文摘In order to find better simplicity measurements for 3D object recognition, a new set of local regularities is developed and tested in a stepwise 3D reconstruction method, including localized minimizing standard deviation of angles（L-MSDA）, localized minimizing standard deviation of segment magnitudes（L-MSDSM）, localized minimum standard deviation of areas of child faces （L-MSDAF）, localized minimum sum of segment magnitudes of common edges （L-MSSM）, and localized minimum sum of areas of child face （L-MSAF）. Based on their effectiveness measurements in terms of form and size distortions, it is found that when two local regularities： L-MSDA and L-MSDSM are combined together, they can produce better performance. In addition, the best weightings for them to work together are identified as 10% for L-MSDSM and 90% for L-MSDA. The test results show that the combined usage of L-MSDA and L-MSDSM with identified weightings has a potential to be applied in other optimization based 3D recognition methods to improve their efficacy and robustness.
基金This research was supported by National Natural Science Foundation of China(No.61271353,61871389)Major Funding Projects of National University of Defense Technology(No.ZK18-01-02)Foundation of State Key Laboratory of Pulsed Power Laser Technology(No.SKL2018ZR09).
文摘Airborne LIDAR can flexibly obtain point cloud data with three-dimensional structural information,which can improve its effectiveness of automatic target recognition in the complex environment.Compared with 2D information,3D information performs better in separating objects and background.However,an aircraft platform can have a negative influence on LIDAR obtained data because of various flight attitudes,flight heights and atmospheric disturbances.A structure of global feature based 3D automatic target recognition method for airborne LIDAR is proposed,which is composed of offline phase and online phase.The performance of four global feature descriptors is compared.Considering the summed volume region(SVR) discrepancy in real objects,SVR selection is added into the pre-processing operations to eliminate mismatching clusters compared with the interested target.Highly reliable simulated data are obtained under various sensor’s altitudes,detection distances and atmospheric disturbances.The final experiments results show that the added step increases the recognition rate by above 2.4% and decreases the execution time by about 33%.
文摘This paper proposes a new set of 3D rotation scaling and translation invariants of 3D radially shifted Legendre moments. We aim to develop two kinds of transformed shifted Legendre moments： a 3D substituted radial shifted Legendre moments （3DSRSLMs） and a 3D weighted radial one （3DWRSLMs）. Both are centered on two types of polynomials. In the first case, a new 3D ra- dial complex moment is proposed. In the second case, new 3D substituted/weighted radial shifted Legendremoments （3DSRSLMs/3DWRSLMs） are introduced using a spherical representation of volumetric image. 3D invariants as derived from the sug- gested 3D radial shifted Legendre moments will appear in the third case. To confirm the proposed approach, we have resolved three is- sues. To confirm the proposed approach, we have resolved three issues： rotation, scaling and translation invariants. The result of experi- ments shows that the 3DSRSLMs and 3DWRSLMs have done better than the 3D radial complex moments with and without noise. Sim- ultaneously, the reconstruction converges rapidly to the original image using 3D radial 3DSRSLMs and 3DWRSLMs, and the test of 3D images are clearly recognized from a set of images that are available in Princeton shape benchmark （PSB） database for 3D image.
基金supported by the National Natural Science Foundation of China(No.52303112)the Henan Province Science and Technology Research and Development Program Joint Fund Advantageous Discipline Cultivation Project(No.232301420033)+1 种基金the China Postdoctoral Science Foundation(Nos.2022TQ0281 and 2023M733213)the Key R&D and Promotion Special(Scientific Problem Tackling)Project of Henan Province(No.242102231014).
文摘Virtual reality(VR)is an emerging communication means and creates extensive opportunities in interacting scenarios such as remote collaboration and metaverse.Human-machine interfaces(HMIs)play important roles in VR as they provide interaction platforms between users and virtual environments.However,traditional VR HMIs based on handheld devices or keyboards cannot recognize diverse three-dimensional(3D)gestures,which results in limited freedom of VR interactions.Here,we report a noncontact VR HMI enabled by an electret-nanofiber-based triboelectric sensor(ETS),which is fabricated by the electrospun polylactic acid/thermoplastic polyurethane(PLA/TPU)electret nanofiber film.The nanofiber structure of PLA/TPU electret enhanced the charge retention ability of triboelectric sensor and thus significantly improved its signal strength and stability.Integrated with a deep learning-based multilayer perceptron neural network,the ETS realizes the recognition of 18 different types of 3D gestures with a high average accuracy of 97.3%.An intelligent noncontact VR interactive system based on the ETS is further developed,which is used to manipulate game characters for performing different actions by 3D gestures.Compared with traditional VR HMIs,the proposed VR HMI based on PLA/TPU electret nanofiber film can detect various 3D gestures and offers a superior interaction freedom.This work for the first time introduces the triboelectric 3D gesture recognition method to the VR HMIs,and could make the interaction between human and virtual environments become more efficient and fascinating.
文摘In the past ten years,research on face recognition has shifted to using 3D facial surfaces,as 3D geometric information provides more discriminative features.This comprehensive survey reviews 3D face recognition techniques developed in the past decade,both conventional methods and deep learning methods.These methods are evaluated with detailed descriptions of selected representative works.Their advantages and disadvantages are summarized in terms of accuracy,complexity,and robustness to facial variations(expression,pose,occlusion,etc.).A review of 3D face databases is also provided,and a discussion of future research challenges and directions of the topic.
基金supported by the National Basic Research Program of China(973 Program)(No.2013CB328806)the National High Technology Research and Development Program of China(863 Program)(No.2012AA011902)+1 种基金the National Natural Science Foundation of China(No.61177015)the Research Funds for the Central Universities of China(No.2012XZZX013)
文摘Based on light field reconstruction and motion recognition technique, a penetrable interactive floating 3D display system is proposed. The system consists of a high-frame-rate projector, a flat directional diffusing screen, a high-speed data transmission module, and a Kinect somatosensory device. The floating occlusioncorrect 3D image could rotate around some axis at different speeds according to user＇s hand motion. Eight motion directions and speed are detected accurately, and the prototype system operates efficiently with a recognition accuracy of 90% on average.
基金The authors would like to acknowledge the use of the GavabDB face database in this paper due to Moreno and Sanchez.This work was supported in part by the National Natural Science Foundation of China(Grant No.60872145)the National High Technology Research and Development Program of China(No.2009AA01Z315)the Cultivation Fund of the Key Scientific and Technical Innovation Project,Ministry of Education of China(No.708085).
文摘A 3D face recognition approach which uses principal axes registration(PAR)and three face representation features from the re-sampling depth image:Eigenfaces,Fisherfaces and Zernike moments is presented.The approach addresses the issue of 3D face registration instantly achieved by PAR.Because each facial feature has its own advantages,limitations and scope of use,different features will complement each other.Thus the fusing features can learn more expressive characterizations than a single feature.The support vector machine(SVM)is applied for classification.In this method,based on the complementarity between different features,weighted decision-level fusion makes the recognition system have certain fault tolerance.Experimental results show that the proposed approach achieves superior performance with the rank-1 recognition rate of 98.36%for GavabDB database.
基金supported by the National Natural Science Foundation of China(61872126)the Key Scientific Research Project Plan of Colleges and Universities in Henan Province(19A520004)。
文摘The research of emotion recognition based on electroencephalogram(EEG)signals often ignores the related information between the brain electrode channels and the contextual emotional information existing in EEG signals,which may contain important characteristics related to emotional states.Aiming at the above defects,a spatiotemporal emotion recognition method based on a 3-dimensional(3 D)time-frequency domain feature matrix was proposed.Specifically,the extracted time-frequency domain EEG features are first expressed as a 3 D matrix format according to the actual position of the cerebral cortex.Then,the input 3 D matrix is processed successively by multivariate convolutional neural network(MVCNN)and long short-term memory(LSTM)to classify the emotional state.Spatiotemporal emotion recognition method is evaluated on the DEAP data set,and achieved accuracy of 87.58%and 88.50%on arousal and valence dimensions respectively in binary classification tasks,as well as obtained accuracy of 84.58%in four class classification tasks.The experimental results show that 3 D matrix representation can represent emotional information more reasonably than two-dimensional(2 D).In addition,MVCNN and LSTM can utilize the spatial information of the electrode channels and the temporal context information of the EEG signal respectively.
基金Sino-French Program of Advanced Research under，上海市科委资助项目
文摘In this paper, illumination-affine invariant methods are presented based onaffine moment normalization techniques, Zernike moments, and multiband correlation functions. Themethods are suitable for the illumination invariant recognition of 3D color texture. Complex valuedmoments (i.e., Zernike moments) and affine moment normalization are used in the derivation ofillumination affine invariants where the real valued affine moment invariants fail to provide affineinvariants that are independent of illumination changes. Three different moment normalizationmethods have been used, two of which are based on affine moment normalization technique and thethird is based on reducing the affine transformation to a Euclidian transform. It is shown that fora change of illumination and orientation, the affinely normalized Zernike moment matrices arerelated by a linear transform. Experimental results are obtained in two tests: the first is usedwith textures of outdoor scenes while the second is performed on the well-known CUReT texturedatabase. Both tests show high recognition efficiency of the proposed recognition methods.
