A Novel Method for Solving Ordinary Differential Equations with Artificial Neural Networks

Roseline N.  Okereke; Olaniyi S.  Maliki; Ben I.  Oruh

doi:10.4236/am.2021.1210059

Roseline N. Okereke, Olaniyi S. Maliki, Ben I. Oruh

Abstract: This research work investigates the use of Artificial Neural Network (ANN) based on models for solving first and second order linear constant coefficient ordinary differential equations with initial conditions. In particular, we employ a feed-forward Multilayer Perceptron Neural Network (MLPNN), but bypass the standard back-propagation algorithm for updating the intrinsic weights. A trial solution of the differential equation is written as a sum of two parts. The first part satisfies the initial or boundary conditions and contains no adjustable parameters. The second part involves a feed-forward neural network to be trained to satisfy the differential equation. Numerous works have appeared in recent times regarding the solution of differential equations using ANN, however majority of these employed a single hidden layer perceptron model, incorporating a back-propagation algorithm for weight updation. For the homogeneous case, we assume a solution in exponential form and compute a polynomial approximation using statistical regression. From here we pick the unknown coefficients as the weights from input layer to hidden layer of the associated neural network trial solution. To get the weights from hidden layer to the output layer, we form algebraic equations incorporating the default sign of the differential equations. We then apply the Gaussian Radial Basis function (GRBF) approximation model to achieve our objective. The weights obtained in this manner need not be adjusted. We proceed to develop a Neural Network algorithm using MathCAD software, which enables us to slightly adjust the intrinsic biases. We compare the convergence and the accuracy of our results with analytic solutions, as well as well-known numerical methods and obtain satisfactory results for our example ODE problems.

Keywords: Ordinary Differential Equations, Multilayer Perceptron Neural Networks, Gaussian Radial Basis Function, Network Training, MathCAD (Computer Aided Design) 14, IBM-SPSS (Statistical Package for Social Science) 23

Cite this paper: Okereke, R. , Maliki, O. and Oruh, B. (2021) A Novel Method for Solving Ordinary Differential Equations with Artificial Neural Networks. Applied Mathematics, 12, 900-918. doi: 10.4236/am.2021.1210059.

References

[1] McCulloch, W.S. and Pitts W. (1943) A Logical Calculus of the Ideas Immanent in Nervous Activity. The Bulletin of Mathematical Biophysics, 5, 115-133.
https://doi.org/10.1007/BF02478259

[2] Neumann, J.V. (1951) The General and Logical Theory of Automata. Wiley, New York.

[3] Graupe, D. (2007) Principles of Artificial Neural Networks. Vol. 6, 2nd Edition, World Scientific Publishing Co. Pte. Ltd., Singapore.

[4] Rumelhart, D.E. and McClelland, J.L. (1986) Parallel Distributed Processing, Explorations in the Microstructure of Cognition I and II. MIT Press, Cambridge.
https://doi.org/10.7551/mitpress/5236.001.0001

[5] Werbos, P.J. (1974) Beyond Recognition, New Tools for Prediction and Analysis in the Behavioural Sciences. Ph.D. Thesis, Harvard University, Cambridge.

[6] Lagaris, I.E., Likas, A.C. and Fotiadis, D.I. (1997) Artificial Neural Network for Solving Ordinary and Partial Differential Equations. arXiv: physics/9705023v1.

[7] Cybenco, G. (1989) Approximation by Superposition of a Sigmoidal Function. Mathematics of Control, Signals and Systems, 2, 303-314.
https://doi.org/10.1007/BF02551274

[8] Hornic, K., Stinchcombe, M. and White, H. (1989) Multilayer Feedforward Networks Are Universal Approximators. Neural Networks, 2, 359-366.
https://doi.org/10.1016/0893-6080(89)90020-8

[9] Lee, H. and Kang, I.S. (1990) Neural Algorithms for Solving Differential Equations. Journal of Computational Physics, 91, 110-131.
https://doi.org/10.1016/0021-9991(90)90007-N

[10] Majidzadeh, K. (2011) Inverse Problem with Respect to Domain and Artificial Neural Network Algorithm for the Solution. Mathematical Problems in Engineering, 2011, Article ID: 145608, 16 p.
https://doi.org/10.1155/2011/145608

[11] Chen, R.T.Q., Rubanova, Y., Bettencourt, J. and Duvenaud, D. (2018) Neural Ordinary Differential Equations. arXiv: 1806.07366v1.

[12] Okereke, R.N. (2019) A New Perspective to the Solution of Ordinary Differential Equations using Artificial Neural Networks. Ph.D Dissertation, Mathematics Department, Michael Okpara University of Agriculture, Umudike.

[13] Mall, S. and Chakraverty, S. (2013) Comparison of Artificial Neural Network Architecture in Solving Ordinary Differential Equations. Advances in Artificial Neural Systems, 2013, Article ID: 181895.
https://doi.org/10.1155/2013/181895

[14] IBM (2015) IBM SPSS Statistics 23
http://www.ibm.com

[15] PTC (Parametric Technology Corporation) (2007) Mathcad Version 14.
http://communications@ptc.com

[16] Otadi, M. and Mosleh, M. (2011) Numerical Solution of Quadratic Riccati Differential Equations by Neural Network. Mathematical Sciences, 5, 249-257.