Colloquium




Abstract
 
Neural network-based techniques for solving differential equations can be traced at least to the 1990s. The remarkable success of deep learning in the last decade has stimulated a significant amount of efforts in the development of deep neural network (DNN) based PDE solvers. DNN-based PDE solvers have witnessed a robust and dramatic growth in the past few years, with several successful ones emerging.

Can neural network-based methods out-compete traditional numerical methods in computational performance for solving PDEs? This is a question hanging in the air ever since the early studies of neural networks for differential equations and intriguing both computational mathematicians and machine learning practitioners. Here by "out-compete" we mean that one method achieves a better accuracy under the same computational budget/cost or incurs a lower computational cost to achieve the same accuracy. While their computational performance is promising, the existing DNN-based PDE solvers suffer from several limitations, which make them numerically less than satisfactory and computationally uncompetitive. The most prominent include the limited accuracy, a general lack of convergence with a certain convergence rate, and extremely high computational cost (very long time to train). Due to these limitations, these solvers seem to fall short, at least in their current state, and cannot compete with traditional numerical methods, except perhaps for certain problems such as high-dimensional problems.

In this talk we discuss a neural network-based method (termed local extreme learning machines, or locELM) for solving linear and nonlinear PDEs that exhibits a disparate computational performance from the above DNN-based PDE solvers and in a sense overcomes the above limitations. This method combines the ideas of extreme learning machines, domain decomposition and local neural networks. The field solution on each sub-domain is represented by a local feed-forward neural network, and \(C^k\) continuity is imposed on the sub-domain boundaries. Each local neural network consists of a small number of hidden layers, whose coefficients are pre-set to random values and fixed through the computation, and the trainable parameters consist of the output-layer coefficients. The overall neural network is trained by a linear or nonlinear least squares computation, not by the back-propagation (or gradient descent) type algorithms.

The presented method exhibits a clear sense of convergence with respect to the degrees of freedom in the system. For smooth solutions its numerical errors decrease exponentially as the number of training parameters or the number of training data points increases, much like the traditional spectral or spectral element type methods. We compare the current locELM method with the state-of-the-art DNN-based PDE solvers, such as the physics-informed neural network (PINN) and the deep Galerkin method (DGM), and with the classical and high-order finite element methods (FEM). The numerical errors and network training time of locELM are considerably smaller, typically by orders of magnitude, than those of PINN and DGM. We show evidence that the current method far outperforms the classical 2nd-order FEM. The computational performance of the presented method is comparable to that of the high-order FEM for smaller problem sizes, and for larger problem sizes it markedly outperforms the high-order FEM. A number of numerical benchmarks will be presented to demonstrate these points.



For future talks or to be added to the mailing list: www.math.uh.edu/colloquium