Universal Approximation Theorem
Published:
The most important application of neural networks is in machine learning, where neural networks are “trained” to approximate a function. Thus, a fundamental question for neural networks is whether they can approximate reasonable functions to an arbitrary degree of accuracy. This depends on the activation function \(\sigma\) and is the subject of many papers, including the paper studied for this project. This project was mentored by Zach Wagner and supported by UCSB’s Directed Reading Program. We won the People’s Choice Award luckily. You can access our presented poster via this link if you are interested.
Leshno et al. proved in their paper “Multilayer Feedforward Networks With a Nonpolynomial Activation Function Can Approximate Any Function” that, under modest assumptions, a broad class of activation functions are suitable for building neural networks to approximate continuous functions. We studied this paper to understand the mathematics underlying the result.