Recent years have witnessed growing interests in solving partial differential equations by deep neural networks,especially in the high-dimensional case.Unlike classical numerical methods,such as finite difference meth...Recent years have witnessed growing interests in solving partial differential equations by deep neural networks,especially in the high-dimensional case.Unlike classical numerical methods,such as finite difference method and finite element method,the enforcement of boundary conditions in deep neural networks is highly nontrivial.One general strategy is to use the penalty method.In the work,we conduct a comparison study for elliptic problems with four different boundary conditions,i.e.,Dirichlet,Neumann,Robin,and periodic boundary conditions,using two representative methods:deep Galerkin method and deep Ritz method.In the former,the PDE residual is minimized in the least-squares sense while the corresponding variational problem is minimized in the latter.Therefore,it is reasonably expected that deep Galerkin method works better for smooth solutions while deep Ritz method works better for low-regularity solutions.However,by a number of examples,we observe that deep Ritz method can outperform deep Galerkin method with a clear dependence of dimensionality even for smooth solutions and deep Galerkin method can also outperform deep Ritz method for low-regularity solutions.Besides,in some cases,when the boundary condition can be implemented in an exact manner,we find that such a strategy not only provides a better approximate solution but also facilitates the training process.展开更多
We propose a deep learning-based method,the Deep Ritz Method,for numerically solving variational problems,particularly the ones that arise from par-tial differential equations.The Deep Ritz Method is naturally nonline...We propose a deep learning-based method,the Deep Ritz Method,for numerically solving variational problems,particularly the ones that arise from par-tial differential equations.The Deep Ritz Method is naturally nonlinear,naturally adaptive and has the potential to work in rather high dimensions.The framework is quite simple and fits well with the stochastic gradient descent method used in deep learning.We illustrate the method on several problems including some eigenvalue problems.展开更多
Using deep neural networks to solve PDEs has attracted a lot of attentions recently.However,why the deep learning method works is falling far behind its empirical success.In this paper,we provide a rigorous numerical ...Using deep neural networks to solve PDEs has attracted a lot of attentions recently.However,why the deep learning method works is falling far behind its empirical success.In this paper,we provide a rigorous numerical analysis on deep Ritz method(DRM)[47]for second order elliptic equations with Neumann boundary conditions.We establish the first nonasymptotic convergence rate in H^(1)norm for DRM using deep networks with ReLU^(2)activation functions.In addition to providing a theoretical justification of DRM,our study also shed light on how to set the hyperparameter of depth and width to achieve the desired convergence rate in terms of number of training samples.Technically,we derive bound on the approximation error of deep ReLU^(2)network in C^(1)norm and bound on the Rademacher complexity of the non-Lipschitz composition of gradient norm and ReLU^(2)network,both of which are of independent interest.展开更多
We introduce an adaptive sampling method for the Deep Ritz method aimed at solving partial differential equations(PDEs).Two deep neural networks are used.One network is employed to approximate the solution of PDEs,whi...We introduce an adaptive sampling method for the Deep Ritz method aimed at solving partial differential equations(PDEs).Two deep neural networks are used.One network is employed to approximate the solution of PDEs,while the other one is a deep generative model used to generate new collocation points to refine the training set.The adaptive sampling procedure consists of two main steps.The first step is solving the PDEs using the Deep Ritz method by minimizing an associated variational loss discretized by the collocation points in the training set.The second step involves generating a new training set,which is then used in subsequent computations to further improve the accuracy of the current approximate solution.We treat the integrand in the variational loss as an unnormalized probability density function(PDF)and approximate it using a deep generative model called bounded KRnet.The new samples and their associated PDF values are obtained from the bounded KRnet.With these new samples and their associated PDF values,the variational loss can be approximated more accurately by importance sampling.Compared to the original Deep Ritz method,the proposed adaptive method improves the accuracy,especially for problems characterized by low regularity and high dimensionality.We demonstrate the effectiveness of our new method through a series of numerical experiments.展开更多
基金the grants NSFC 11971021National Key R&D Program of China(No.2018YF645B0204404)NSFC 11501399(R.Du)。
文摘Recent years have witnessed growing interests in solving partial differential equations by deep neural networks,especially in the high-dimensional case.Unlike classical numerical methods,such as finite difference method and finite element method,the enforcement of boundary conditions in deep neural networks is highly nontrivial.One general strategy is to use the penalty method.In the work,we conduct a comparison study for elliptic problems with four different boundary conditions,i.e.,Dirichlet,Neumann,Robin,and periodic boundary conditions,using two representative methods:deep Galerkin method and deep Ritz method.In the former,the PDE residual is minimized in the least-squares sense while the corresponding variational problem is minimized in the latter.Therefore,it is reasonably expected that deep Galerkin method works better for smooth solutions while deep Ritz method works better for low-regularity solutions.However,by a number of examples,we observe that deep Ritz method can outperform deep Galerkin method with a clear dependence of dimensionality even for smooth solutions and deep Galerkin method can also outperform deep Ritz method for low-regularity solutions.Besides,in some cases,when the boundary condition can be implemented in an exact manner,we find that such a strategy not only provides a better approximate solution but also facilitates the training process.
基金supported in part by the National Key Basic Research Program of China 2015CB856000Major Program of NNSFC under Grant 91130005,DOE Grant DE-SC0009248ONR Grant N00014-13-1-0338.
文摘We propose a deep learning-based method,the Deep Ritz Method,for numerically solving variational problems,particularly the ones that arise from par-tial differential equations.The Deep Ritz Method is naturally nonlinear,naturally adaptive and has the potential to work in rather high dimensions.The framework is quite simple and fits well with the stochastic gradient descent method used in deep learning.We illustrate the method on several problems including some eigenvalue problems.
基金supported by the National Key Research and Development Program of China(No.2020YFA0714200)the Science and Technology Major Project of Hubei Province under Grant 2021AAA010+2 种基金the National Science Foundation of China(Nos.12125103,12071362,11871474,11871385)the Natural Science Foundation of Hubei Province(No.2019CFA007)by the research fund of KLATASDSMOE.
文摘Using deep neural networks to solve PDEs has attracted a lot of attentions recently.However,why the deep learning method works is falling far behind its empirical success.In this paper,we provide a rigorous numerical analysis on deep Ritz method(DRM)[47]for second order elliptic equations with Neumann boundary conditions.We establish the first nonasymptotic convergence rate in H^(1)norm for DRM using deep networks with ReLU^(2)activation functions.In addition to providing a theoretical justification of DRM,our study also shed light on how to set the hyperparameter of depth and width to achieve the desired convergence rate in terms of number of training samples.Technically,we derive bound on the approximation error of deep ReLU^(2)network in C^(1)norm and bound on the Rademacher complexity of the non-Lipschitz composition of gradient norm and ReLU^(2)network,both of which are of independent interest.
文摘We introduce an adaptive sampling method for the Deep Ritz method aimed at solving partial differential equations(PDEs).Two deep neural networks are used.One network is employed to approximate the solution of PDEs,while the other one is a deep generative model used to generate new collocation points to refine the training set.The adaptive sampling procedure consists of two main steps.The first step is solving the PDEs using the Deep Ritz method by minimizing an associated variational loss discretized by the collocation points in the training set.The second step involves generating a new training set,which is then used in subsequent computations to further improve the accuracy of the current approximate solution.We treat the integrand in the variational loss as an unnormalized probability density function(PDF)and approximate it using a deep generative model called bounded KRnet.The new samples and their associated PDF values are obtained from the bounded KRnet.With these new samples and their associated PDF values,the variational loss can be approximated more accurately by importance sampling.Compared to the original Deep Ritz method,the proposed adaptive method improves the accuracy,especially for problems characterized by low regularity and high dimensionality.We demonstrate the effectiveness of our new method through a series of numerical experiments.
