From the course: Deep Learning with Python: Optimizing Deep Learning Models

Applying L2 regularization to a deep learning model - Python Tutorial

From the course: Deep Learning with Python: Optimizing Deep Learning Models

Applying L2 regularization to a deep learning model

“

- [Instructor] In this video, you will learn how to apply L2 Regularization, also known as Ridge Regularization, to a deep learning model in order to reduce overfitting, I'll be writing a code in the 02_04e file. You can follow along by completing the empty code cells in the 02_04b file. Make sure to run the previously written code to import and pre-process the data as well as to build and train the baseline model. I've already done so. Looking at the results of the baseline model, we can see that there is a clear indicator of overfitting. We see the divergence in the training and validation loss metrics, which is visible in the training and validation loss curves that we see here. To help minimize overfitting in this baseline model, let's apply L2 Regularization. L2 Regularization adds a penalty proportional to the squared values of the weights. This discourages large weights, helping the model generalize better. To apply L2 Regularization to the baseline model, we set the kernel_regularizer argument within each hidden layer of the network. So in this example, we set the regularization parameter to 0.001. Before we begin, we import L2 from tensorflow.keras.regularizers. Then we define our model. Within each Dense layer, we specify the kernel_regularizer as L2 and give it a parameter value. So let's go ahead and run our code. Once we're done defining our model, we now need to compile it, which is what we do in this next step by calling the compile method of our model. After that, we now apply the model to our training data. So we call the fit method to do so. We specify the number of epochs as 15, so it's going to go through 15 training iterations. We specify the batch_size is 128 and the validation_split as 0.1. This means that the model each time will use 90% of the training data for training and 10% for validation. So let's give this some time to finish all 15 iterations of 15 epochs. And then we're going to go ahead and plot the train and validation loss metrics to see what the impact of L2 Regularization has been on our model. Okay, now we can go ahead and plot the validation and training loss metrics. And this time we see that the model's training and validation loss metrics tend to decrease at a similar rate throughout the training process. This is a strong indication that L2 Regularization is effectively reducing overfitting by discouraging the model from relying too heavily on large weight values. Great work. You now know how to use L2 Regularization to reduce overfitting in a deep learning model in Python.

Contents