Deep neural networks(DNNs)are effective in solving both forward and inverse problems for nonlinear partial differential equations(PDEs).However,conventional DNNs are not effective in handling problems such as delay di...Deep neural networks(DNNs)are effective in solving both forward and inverse problems for nonlinear partial differential equations(PDEs).However,conventional DNNs are not effective in handling problems such as delay differential equations(DDEs)and delay integrodifferential equations(DIDEs)with constant delays,primarily due to their low regularity at delayinduced breaking points.In this paper,a DNN method that combines multi-task learning(MTL)which is proposed to solve both the forward and inverse problems of DIDEs.The core idea of this approach is to divide the original equation into multiple tasks based on the delay,using auxiliary outputs to represent the integral terms,followed by the use of MTL to seamlessly incorporate the properties at the breaking points into the loss function.Furthermore,given the increased training dificulty associated with multiple tasks and outputs,we employ a sequential training scheme to reduce training complexity and provide reference solutions for subsequent tasks.This approach significantly enhances the approximation accuracy of solving DIDEs with DNNs,as demonstrated by comparisons with traditional DNN methods.We validate the effectiveness of this method through several numerical experiments,test various parameter sharing structures in MTL and compare the testing results of these structures.Finally,this method is implemented to solve the inverse problem of nonlinear DIDE and the results show that the unknown parameters of DIDE can be discovered with sparse or noisy data.展开更多
Recent studies for computer vision and deep learning-based,post-earthquake inspections on RC structures mainly perform well for specific tasks,while the trained models must be fine-tuned and re-trained when facing new...Recent studies for computer vision and deep learning-based,post-earthquake inspections on RC structures mainly perform well for specific tasks,while the trained models must be fine-tuned and re-trained when facing new tasks and datasets,which is inevitably time-consuming.This study proposes a multi-task learning approach that simultaneously accomplishes the semantic segmentation of seven-type structural components,three-type seismic damage,and four-type deterioration states.The proposed method contains a CNN-based encoder-decoder backbone subnetwork with skip-connection modules and a multi-head,task-specific recognition subnetwork.The backbone subnetwork is designed to extract multi-level features of post-earthquake RC structures.The multi-head,task-specific recognition subnetwork consists of three individual self-attention pipelines,each of which utilizes extracted multi-level features from the backbone network as a mutual guidance for the individual segmentation task.A synthetical loss function is designed with real-time adaptive coefficients to balance multi-task losses and focus on the most unstably fluctuating one.Ablation experiments and comparative studies are further conducted to demonstrate their effectiveness and necessity.The results show that the proposed method can simultaneously recognize different structural components,seismic damage,and deterioration states,and that the overall performance of the three-task learning models gains general improvement when compared to all single-task and dual-task models.展开更多
In this paper, we proposed a new semi-supervised multi-manifold learning method, called semi- supervised sparse multi-manifold embedding (S3MME), for dimensionality reduction of hyperspectral image data. S3MME exploit...In this paper, we proposed a new semi-supervised multi-manifold learning method, called semi- supervised sparse multi-manifold embedding (S3MME), for dimensionality reduction of hyperspectral image data. S3MME exploits both the labeled and unlabeled data to adaptively find neighbors of each sample from the same manifold by using an optimization program based on sparse representation, and naturally gives relative importance to the labeled ones through a graph-based methodology. Then it tries to extract discriminative features on each manifold such that the data points in the same manifold become closer. The effectiveness of the proposed multi-manifold learning algorithm is demonstrated and compared through experiments on a real hyperspectral images.展开更多
Classic sparse representation, as one of prevalent feature learning methods, is successfully applied for different computer vision tasks. However it has some intrinsic defects in object detection. Firstly, how to lear...Classic sparse representation, as one of prevalent feature learning methods, is successfully applied for different computer vision tasks. However it has some intrinsic defects in object detection. Firstly, how to learn a discriminative dictionary for object detection is a hard problem. Secondly, it is usually very time-consuming to learn dictionary based features in a traditional exhaustive search manner like sliding window. In this paper, we propose a novel feature learning framework for object detection with the structure sparsity constraint and classification error minimization constraint to learn a discriminative dictionary. For improving the efficiency, we just learn sparse representation coefficients from object candidate regions and feed them to a kernelized SVM classifier. Experiments on INRIA Person Dataset and Pascal VOC 2007 challenge dataset clearly demonstrate the effectiveness of the proposed approach compared with two state-of-the-art baselines.展开更多
High-dimensional data arising from diverse scientific research fields and industrial development have led to increased interest in sparse learning due to model parsimony and computational advantage. With the assumptio...High-dimensional data arising from diverse scientific research fields and industrial development have led to increased interest in sparse learning due to model parsimony and computational advantage. With the assumption of sparsity, many computational problems can be handled efficiently in practice. Structured sparse learning encodes the structural information of the variables and has been quite successful in numerous research fields. With various types of structures discovered, sorts of structured regularizations have been proposed. These regularizations have greatly improved the efficacy of sparse learning algorithms through the use of specific structural information. In this article, we present a systematic review of structured sparse learning including ideas, formulations, algorithms, and applications. We present these algorithms in the unified framework of minimizing the sum of loss and penalty functions, summarize publicly accessible software implementations, and compare the computational complexity of typical optimization methods to solve structured sparse learning problems. In experiments, we present applications in unsupervised learning, for structured signal recovery and hierarchical image reconstruction, and in supervised learning in the context of a novel graph-guided logistic regression.展开更多
Biological domain has been blessed with more and more data from biotechnologies as well as data integration tools.In the renaissance of machine learning and artificial intelligence,there is so much promise of data-dri...Biological domain has been blessed with more and more data from biotechnologies as well as data integration tools.In the renaissance of machine learning and artificial intelligence,there is so much promise of data-driven biological knowledge discovery.However,it is not straight forward due to the complexity of the domain knowledge hidden in the data.At any level,be it atoms,molecules,cells or organisms,there are rich interdependencies among biological components.Machine learning approaches in this domain usually involves analyzing interdependency structures encoded in graphs and related formalisms.In this report,we review our work in developing new Machine Learning methods for these applications with improved performances in comparison with state-of-the-art methods.We show how the networks among biological components can be used to predict properties.展开更多
针对K-奇异值分解(sigular value decomposition,SVD)算法存在的问题,结合结构聚类和字典学习,提出了一种基于非局部正则化稀疏表示的图像去噪算法。首先,利用非局部去噪的思想将结构相似的图像块聚类,每一类图像块单独进行字典学习,增...针对K-奇异值分解(sigular value decomposition,SVD)算法存在的问题,结合结构聚类和字典学习,提出了一种基于非局部正则化稀疏表示的图像去噪算法。首先,利用非局部去噪的思想将结构相似的图像块聚类,每一类图像块单独进行字典学习,增强了字典的自适应性;其次,利用稀疏K-SVD替代传统的K-SVD进行类内字典学习,改善了字典的结构性;最后,引入稀疏系数误差正则项来修正稀疏系数以进一步改善图像的重构效果。实验结果表明,与传统的K-SVD算法相比,该算法能够有效地保持图像的结构信息,并且提升了去噪效果,同时,在不降低图像结构相似度的基础上,峰值信噪比很接近甚至部分好于目前先进的去噪算法。展开更多
文摘Deep neural networks(DNNs)are effective in solving both forward and inverse problems for nonlinear partial differential equations(PDEs).However,conventional DNNs are not effective in handling problems such as delay differential equations(DDEs)and delay integrodifferential equations(DIDEs)with constant delays,primarily due to their low regularity at delayinduced breaking points.In this paper,a DNN method that combines multi-task learning(MTL)which is proposed to solve both the forward and inverse problems of DIDEs.The core idea of this approach is to divide the original equation into multiple tasks based on the delay,using auxiliary outputs to represent the integral terms,followed by the use of MTL to seamlessly incorporate the properties at the breaking points into the loss function.Furthermore,given the increased training dificulty associated with multiple tasks and outputs,we employ a sequential training scheme to reduce training complexity and provide reference solutions for subsequent tasks.This approach significantly enhances the approximation accuracy of solving DIDEs with DNNs,as demonstrated by comparisons with traditional DNN methods.We validate the effectiveness of this method through several numerical experiments,test various parameter sharing structures in MTL and compare the testing results of these structures.Finally,this method is implemented to solve the inverse problem of nonlinear DIDE and the results show that the unknown parameters of DIDE can be discovered with sparse or noisy data.
基金National Key R&D Program of China under Grant No.2019YFC1511005the National Natural Science Foundation of China under Grant Nos.51921006,52192661 and 52008138+2 种基金the China Postdoctoral Science Foundation under Grant Nos.BX20190102 and 2019M661286the Heilongjiang Natural Science Foundation under Grant No.LH2022E070the Heilongjiang Province Postdoctoral Science Foundation under Grant Nos.LBH-TZ2016 and LBH-Z19064。
文摘Recent studies for computer vision and deep learning-based,post-earthquake inspections on RC structures mainly perform well for specific tasks,while the trained models must be fine-tuned and re-trained when facing new tasks and datasets,which is inevitably time-consuming.This study proposes a multi-task learning approach that simultaneously accomplishes the semantic segmentation of seven-type structural components,three-type seismic damage,and four-type deterioration states.The proposed method contains a CNN-based encoder-decoder backbone subnetwork with skip-connection modules and a multi-head,task-specific recognition subnetwork.The backbone subnetwork is designed to extract multi-level features of post-earthquake RC structures.The multi-head,task-specific recognition subnetwork consists of three individual self-attention pipelines,each of which utilizes extracted multi-level features from the backbone network as a mutual guidance for the individual segmentation task.A synthetical loss function is designed with real-time adaptive coefficients to balance multi-task losses and focus on the most unstably fluctuating one.Ablation experiments and comparative studies are further conducted to demonstrate their effectiveness and necessity.The results show that the proposed method can simultaneously recognize different structural components,seismic damage,and deterioration states,and that the overall performance of the three-task learning models gains general improvement when compared to all single-task and dual-task models.
文摘In this paper, we proposed a new semi-supervised multi-manifold learning method, called semi- supervised sparse multi-manifold embedding (S3MME), for dimensionality reduction of hyperspectral image data. S3MME exploits both the labeled and unlabeled data to adaptively find neighbors of each sample from the same manifold by using an optimization program based on sparse representation, and naturally gives relative importance to the labeled ones through a graph-based methodology. Then it tries to extract discriminative features on each manifold such that the data points in the same manifold become closer. The effectiveness of the proposed multi-manifold learning algorithm is demonstrated and compared through experiments on a real hyperspectral images.
基金Supported by the National Natural Science Foundation of China(61231015,61170023)National High Technology Research and Development Program of China(863 Program,2015AA016306)+3 种基金Internet of Things Development Funding Project of Ministry of Industry in 2013(No.25)Technology Research Program of Ministry of Public Security(2014JSYJA016)Major Science and Technology Innovation Plan of Hubei Province(2013AAA020)the Natural Science Foundation of Hubei Province(2014CFB712)
文摘Classic sparse representation, as one of prevalent feature learning methods, is successfully applied for different computer vision tasks. However it has some intrinsic defects in object detection. Firstly, how to learn a discriminative dictionary for object detection is a hard problem. Secondly, it is usually very time-consuming to learn dictionary based features in a traditional exhaustive search manner like sliding window. In this paper, we propose a novel feature learning framework for object detection with the structure sparsity constraint and classification error minimization constraint to learn a discriminative dictionary. For improving the efficiency, we just learn sparse representation coefficients from object candidate regions and feed them to a kernelized SVM classifier. Experiments on INRIA Person Dataset and Pascal VOC 2007 challenge dataset clearly demonstrate the effectiveness of the proposed approach compared with two state-of-the-art baselines.
基金Project supported by the National Natural Science Foundation of China （No. 61303264）
文摘High-dimensional data arising from diverse scientific research fields and industrial development have led to increased interest in sparse learning due to model parsimony and computational advantage. With the assumption of sparsity, many computational problems can be handled efficiently in practice. Structured sparse learning encodes the structural information of the variables and has been quite successful in numerous research fields. With various types of structures discovered, sorts of structured regularizations have been proposed. These regularizations have greatly improved the efficacy of sparse learning algorithms through the use of specific structural information. In this article, we present a systematic review of structured sparse learning including ideas, formulations, algorithms, and applications. We present these algorithms in the unified framework of minimizing the sum of loss and penalty functions, summarize publicly accessible software implementations, and compare the computational complexity of typical optimization methods to solve structured sparse learning problems. In experiments, we present applications in unsupervised learning, for structured signal recovery and hierarchical image reconstruction, and in supervised learning in the context of a novel graph-guided logistic regression.
基金The work is partially supported by Japan MEXT Kakenhi 18K11434 and Vingroup In-novation Foundation(VINIF)project code VINIF.2019.DA18.
文摘Biological domain has been blessed with more and more data from biotechnologies as well as data integration tools.In the renaissance of machine learning and artificial intelligence,there is so much promise of data-driven biological knowledge discovery.However,it is not straight forward due to the complexity of the domain knowledge hidden in the data.At any level,be it atoms,molecules,cells or organisms,there are rich interdependencies among biological components.Machine learning approaches in this domain usually involves analyzing interdependency structures encoded in graphs and related formalisms.In this report,we review our work in developing new Machine Learning methods for these applications with improved performances in comparison with state-of-the-art methods.We show how the networks among biological components can be used to predict properties.
文摘针对K-奇异值分解(sigular value decomposition,SVD)算法存在的问题,结合结构聚类和字典学习,提出了一种基于非局部正则化稀疏表示的图像去噪算法。首先,利用非局部去噪的思想将结构相似的图像块聚类,每一类图像块单独进行字典学习,增强了字典的自适应性;其次,利用稀疏K-SVD替代传统的K-SVD进行类内字典学习,改善了字典的结构性;最后,引入稀疏系数误差正则项来修正稀疏系数以进一步改善图像的重构效果。实验结果表明,与传统的K-SVD算法相比,该算法能够有效地保持图像的结构信息,并且提升了去噪效果,同时,在不降低图像结构相似度的基础上,峰值信噪比很接近甚至部分好于目前先进的去噪算法。
