Universal Approximation Theorem

I went through the universal approximation theorem which basically suggests that a deep neural network can be used to represent any function.

For example, let's say you have some 2-dimensional data and you plot it on a 2-D plane. After plotting the data you realized that there is no way you can divide the data in a linear format. So, you decide to go with some high order function.

Now your job is to find some approximation of the real function which can divide the data into well-defined boundaries.

What you can do is to use two sigmoid functions together and combine them together with weights 1 and -1 respectively to get the small part of the original function.

And if we keep on combining these small resulting functions together( after combining two sigmoid functions together), we will reach a state where we will be able to correctly approximate the given function.

I am going to come up with a post on this. Subscribe to know when I do, https://ranvir.xyz/blog/subscribe