Nonnegativity has been shown to be a powerful principle in linear matrix decompositions,leading to sparse component matrices in feature analysis and data compression.The classical method is Lee and Seung’s Nonnegativ...Nonnegativity has been shown to be a powerful principle in linear matrix decompositions,leading to sparse component matrices in feature analysis and data compression.The classical method is Lee and Seung’s Nonnegative Matrix Factorization.A standard way to form learning rules is by multiplicative updates,maintaining nonnegativity.Here,a generic principle is presented for forming multiplicative update rules,which integrate an orthonormality constraint into nonnegative learning.The principle,called Orthogonal Nonnegative Learning(ONL),is rigorously derived from the Lagrangian technique.As examples,the proposed method is applied for transforming Nonnegative Matrix Factorization(NMF)and its variant,Projective Nonnegative Matrix Factorization(PNMF),into their orthogonal versions.In general,it is well-known that orthogonal nonnegative learning can give very useful approximative solutions for problems involving non-vectorial data,for example,binary solutions.Combinatorial optimization is replaced by continuous-space gradient optimization which is often computationally lighter.It is shown how the multiplicative updates rules obtained by using the proposed ONL principle can find a nonnegative and highly orthogonal matrix for an approximated graph partitioning problem.The empirical results on various graphs indicate that our nonnegative learning algorithms not only outperform those without the orthogonality condition,but also surpass other existing partitioning approaches.展开更多
With the increasing complexity of industrial processes, the high-dimensional industrial data exhibit a strong nonlinearity, bringing considerable challenges to the fault diagnosis of industrial processes. To efficient...With the increasing complexity of industrial processes, the high-dimensional industrial data exhibit a strong nonlinearity, bringing considerable challenges to the fault diagnosis of industrial processes. To efficiently extract deep meaningful features that are crucial for fault diagnosis, a sparse Gaussian feature extractor(SGFE) is designed to learn a nonlinear mapping that projects the raw data into the feature space with the fault label dimension. The feature space is described by the one-hot encoding of the fault category label as an orthogonal basis. In this way, the deep sparse Gaussian features related to fault categories can be gradually learned from the raw data by SGFE. In the feature space,the sparse Gaussian(SG) loss function is designed to constrain the distribution of features to multiple sparse multivariate Gaussian distributions. The sparse Gaussian features are linearly separable in the feature space, which is conducive to improving the accuracy of the downstream fault classification task. The feasibility and practical utility of the proposed SGFE are verified by the handwritten digits MNIST benchmark and Tennessee-Eastman(TE) benchmark process,respectively.展开更多
Signal decomposition and multiscale signal analysis provide many useful tools for timefrequency analysis.We proposed a random feature method for analyzing time-series data by constructing a sparse approximation to the...Signal decomposition and multiscale signal analysis provide many useful tools for timefrequency analysis.We proposed a random feature method for analyzing time-series data by constructing a sparse approximation to the spectrogram.The randomization is both in the time window locations and the frequency sampling,which lowers the overall sampling and computational cost.The sparsification of the spectrogram leads to a sharp separation between time-frequency clusters which makes it easier to identify intrinsic modes,and thus leads to a new data-driven mode decomposition.The applications include signal representation,outlier removal,and mode decomposition.On benchmark tests,we show that our approach outperforms other state-of-the-art decomposition methods.展开更多
Link prediction of combat networks is of significant military value for precisely identifying the vital infrastructure of the enemy target and optimizing the operational plan of our side.Due to the profound uncertaint...Link prediction of combat networks is of significant military value for precisely identifying the vital infrastructure of the enemy target and optimizing the operational plan of our side.Due to the profound uncertainty in the battleground circumstances, the acquired topological information of the opponent combat network always presents sparse characteristics. To solve this problem, a novel approach named network embedding based combat network link prediction(NECLP) is put forward to predict missing links of sparse combat networks. First,node embedding techniques are presented to preserve as much information of the combat network as possible using a low-dimensional space. Then, we put forward a solution algorithm to predict links between combat networks based on node embedding similarity. Last, massive experiments are carried out on a real-world combat network case to verify the validity and practicality of the proposed NECLP. This paper compares six baseline methods, and experimental results show that the NECLP has outstanding performance and substantially outperforms the baseline methods.展开更多
Purpose-The real-time generation of feature descriptors for object recognition is a challenging problem.In this research,the purpose of this paper is to provide a hardware friendly framework to generate sparse feature...Purpose-The real-time generation of feature descriptors for object recognition is a challenging problem.In this research,the purpose of this paper is to provide a hardware friendly framework to generate sparse features that can be useful for key feature point selection,feature extraction,and descriptor construction.The inspiration is drawn from feature formation processes of the human brain,taking into account the sparse,modular,and hierarchical processing of visual information.Design/methodology/approach-A sparse set of neurons referred as active neurons determines the feature points necessary for high-level vision applications such as object recognition.A psycho-physical mechanism of human low-level vision relates edge detection to noticeable local spatial stimuli,representing this set of active neurons.A cognitive memory cell array-based implementation of low-level vision is proposed.Applications of memory cell in edge detection are used for realizing human vision inspired feature selection and leading to feature vector construction for high-level vision applications.Findings-True parallel architecture and faster response of cognitive circuits avoid time costly and redundant feature extraction steps.Validation of proposed feature vector toward high-level computer vision applications is demonstrated using standard object recognition databases.The comparison against existing state-of-the-art object recognition features and methods shows an accuracy of 97,95,69 percent for Columbia Object Image Library-100,ALOI,and PASCAL VOC 2007 databases indicating an increase from benchmark methods by 5,3 and 10 percent,respectively.Originality/value-A hardware friendly low-level sparse edge feature processing system isproposed for recognizing objects.The edge features are developed based on threshold logic of neurons,and the sparse selection of the features applies a modular and hierarchical processing inspired from the human neural system.展开更多
文摘Nonnegativity has been shown to be a powerful principle in linear matrix decompositions,leading to sparse component matrices in feature analysis and data compression.The classical method is Lee and Seung’s Nonnegative Matrix Factorization.A standard way to form learning rules is by multiplicative updates,maintaining nonnegativity.Here,a generic principle is presented for forming multiplicative update rules,which integrate an orthonormality constraint into nonnegative learning.The principle,called Orthogonal Nonnegative Learning(ONL),is rigorously derived from the Lagrangian technique.As examples,the proposed method is applied for transforming Nonnegative Matrix Factorization(NMF)and its variant,Projective Nonnegative Matrix Factorization(PNMF),into their orthogonal versions.In general,it is well-known that orthogonal nonnegative learning can give very useful approximative solutions for problems involving non-vectorial data,for example,binary solutions.Combinatorial optimization is replaced by continuous-space gradient optimization which is often computationally lighter.It is shown how the multiplicative updates rules obtained by using the proposed ONL principle can find a nonnegative and highly orthogonal matrix for an approximated graph partitioning problem.The empirical results on various graphs indicate that our nonnegative learning algorithms not only outperform those without the orthogonality condition,but also surpass other existing partitioning approaches.
基金Projects(62125306, 62133003) supported by the National Natural Science Foundation of ChinaProject(TPL2019C03) supported by the Open Fund of Science and Technology on Thermal Energy and Power Laboratory,ChinaProject supported by the Fundamental Research Funds for the Central Universities(Zhejiang University NGICS Platform),China。
文摘With the increasing complexity of industrial processes, the high-dimensional industrial data exhibit a strong nonlinearity, bringing considerable challenges to the fault diagnosis of industrial processes. To efficiently extract deep meaningful features that are crucial for fault diagnosis, a sparse Gaussian feature extractor(SGFE) is designed to learn a nonlinear mapping that projects the raw data into the feature space with the fault label dimension. The feature space is described by the one-hot encoding of the fault category label as an orthogonal basis. In this way, the deep sparse Gaussian features related to fault categories can be gradually learned from the raw data by SGFE. In the feature space,the sparse Gaussian(SG) loss function is designed to constrain the distribution of features to multiple sparse multivariate Gaussian distributions. The sparse Gaussian features are linearly separable in the feature space, which is conducive to improving the accuracy of the downstream fault classification task. The feasibility and practical utility of the proposed SGFE are verified by the handwritten digits MNIST benchmark and Tennessee-Eastman(TE) benchmark process,respectively.
基金supported in part by the NSERC RGPIN 50503-10842supported in part by the AFOSR MURI FA9550-21-1-0084the NSF DMS-1752116.
文摘Signal decomposition and multiscale signal analysis provide many useful tools for timefrequency analysis.We proposed a random feature method for analyzing time-series data by constructing a sparse approximation to the spectrogram.The randomization is both in the time window locations and the frequency sampling,which lowers the overall sampling and computational cost.The sparsification of the spectrogram leads to a sharp separation between time-frequency clusters which makes it easier to identify intrinsic modes,and thus leads to a new data-driven mode decomposition.The applications include signal representation,outlier removal,and mode decomposition.On benchmark tests,we show that our approach outperforms other state-of-the-art decomposition methods.
基金supported by the National Natural Science Foundation of China (7190121271971213)。
文摘Link prediction of combat networks is of significant military value for precisely identifying the vital infrastructure of the enemy target and optimizing the operational plan of our side.Due to the profound uncertainty in the battleground circumstances, the acquired topological information of the opponent combat network always presents sparse characteristics. To solve this problem, a novel approach named network embedding based combat network link prediction(NECLP) is put forward to predict missing links of sparse combat networks. First,node embedding techniques are presented to preserve as much information of the combat network as possible using a low-dimensional space. Then, we put forward a solution algorithm to predict links between combat networks based on node embedding similarity. Last, massive experiments are carried out on a real-world combat network case to verify the validity and practicality of the proposed NECLP. This paper compares six baseline methods, and experimental results show that the NECLP has outstanding performance and substantially outperforms the baseline methods.
文摘Purpose-The real-time generation of feature descriptors for object recognition is a challenging problem.In this research,the purpose of this paper is to provide a hardware friendly framework to generate sparse features that can be useful for key feature point selection,feature extraction,and descriptor construction.The inspiration is drawn from feature formation processes of the human brain,taking into account the sparse,modular,and hierarchical processing of visual information.Design/methodology/approach-A sparse set of neurons referred as active neurons determines the feature points necessary for high-level vision applications such as object recognition.A psycho-physical mechanism of human low-level vision relates edge detection to noticeable local spatial stimuli,representing this set of active neurons.A cognitive memory cell array-based implementation of low-level vision is proposed.Applications of memory cell in edge detection are used for realizing human vision inspired feature selection and leading to feature vector construction for high-level vision applications.Findings-True parallel architecture and faster response of cognitive circuits avoid time costly and redundant feature extraction steps.Validation of proposed feature vector toward high-level computer vision applications is demonstrated using standard object recognition databases.The comparison against existing state-of-the-art object recognition features and methods shows an accuracy of 97,95,69 percent for Columbia Object Image Library-100,ALOI,and PASCAL VOC 2007 databases indicating an increase from benchmark methods by 5,3 and 10 percent,respectively.Originality/value-A hardware friendly low-level sparse edge feature processing system isproposed for recognizing objects.The edge features are developed based on threshold logic of neurons,and the sparse selection of the features applies a modular and hierarchical processing inspired from the human neural system.
