Department of

August 2018 September 2018 October 2018 Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa 1 2 3 4 1 1 2 3 4 5 6 5 6 7 8 9 10 11 2 3 4 5 6 7 8 7 8 9 10 11 12 13 12 13 14 15 16 17 18 9 10 11 12 13 14 15 14 15 16 17 18 19 20 19 20 21 22 23 24 25 16 17 18 19 20 21 22 21 22 23 24 25 26 27 26 27 28 29 30 31 23 24 25 26 27 28 29 28 29 30 31 30

Tuesday, September 25, 2018

**Abstract:** Neural network models in machine learning have revolutionized fields such as image, text, and speech recognition. There's also growing interest in using neural networks for applications in science, engineering, medicine, and finance. Despite their immense success in practice, there is limited mathematical understanding of neural networks. We mathematically study neural networks in the asymptotic regime of simultaneously (A) large network sizes and (B) large numbers of stochastic gradient descent training iterations. We rigorously prove that the neural network satisfies a Law of Large Numbers (LLN) and a Central Limit Theorem (CLT). The LLN is the solution of a nonlinear partial differential equation while the CLT satisfies a stochastic partial differential equation.