From the course: Deep Learning with Python: Optimizing Deep Learning Models

Applying L1 regularization to a deep learning model - Python Tutorial

From the course: Deep Learning with Python: Optimizing Deep Learning Models

Applying L1 regularization to a deep learning model

“

- [Instructor] In this video, you will learn how to apply L1 Regularization, also known as Lasso regularization, to a deep learning model in order to reduce overfitting, I will be running the code in the 02_03e file. You can follow along by completing the empty code cells in the 02_03b file. Make sure to run the previously written code to import and pre-process the data as well as to build and train the baseline model. I've already done so. So we can see the result from the previous model. A clear indicator of overfitting is a divergence in the training and validation loss metrics, which is visible in the training curves above. L1 Regularization adds a penalty proportional to the absolute values of the weight during training. This encourages sparsity, meaning the model learns to rely only on the most important features. To apply L1 Regularization to the baseline model we created above, we set the kernel_regularizer argument within each hidden layer of the network to L1. In parentheses, we pass in 0.001. This means that the regularization parameter is set to 0.001. So to do this, we begin by importing l1 from tensorflow.keras.regularizers. Then we define our model. So in our model, within each of the dense layers, the hidden layers, we specify the kernel_regularizer argument and we specify the regularization parameter. So let's go ahead and run our code. Next we compile the regularized model. And then we train the regularized model against our training data. So we're going to let the model here train for 15 epochs against the training data. We set the batch_size as 128 and the validation_split at 0.1. Feel free to modify these variables and these values in your own environment to see what the impact is. Once training is complete, we can now plot the training and validation loss metrics. So let's take a look at that. This time we see that the two metrics reduce in value at a similar rate as training continues, which is quite different from what we saw before. This indicates that L1 Regularization is effectively helping the model generalize better by encouraging sparsity in the learned weights. By penalizing the absolute values of the weights, L1 Regularization pushes many weights towards zero, effectively simplifying the model and reducing the risk of overfitting to retraining data. Good job. You now know how to use L1 Regularization to reduce overfitting in a deep learning model in Python.

Contents