Regularization methods and gradient check

In this section, we will motivate the reason why dropout is regarded as a regularization technique. We will consider some simple neural networks where a better understanding of dropout is available and infer this interpretation to other deep networks. In addition to this, we will describe “batch normalization” to enhance training speed in deep networks. Finally, we will present gradient checking as a technique to validate backpropagation gradients. This method is important to verify manual implementations of the optimization procedure, or novel functions not present in the framework’s library.

Class: 

Data Science 2: Advanced Topics in Data Science

Attach Files: