Training a deep learning model using callbacks - Python Tutorial

- [Instructor] In this video, you will learn how to use callbacks to apply early stopping and linear rate scheduling to a deep learning model in Python. I'll be running the code in the 05_07e file. You can follow along by completing the empty code cells in the 05_07b file. Know that this is the third in a three-video sequence that teaches you how to apply batch normalization, gradient clipping, early stopping, and learning rate scheduling to a deep learning model. If you have not done so, watch the previous course videos on how to apply batch normalization to a deep learning model and how to apply gradients clipping to a deep learning model. Those videos provide a detailed explanation of the prior code. Before we begin, let's run the code we created in those videos to get our environment up to speed. So as we've done in the past, the first thing we want to do is select our kernel. You can say Python environments in 3.10. Then I'm going to click on my current code cell and I'm going to say run previous code cells. I'm scroll up a little bit so I can monitor progress. Alright, so we're done there. In Keras, a callback is an object that can perform custom actions at specific points during the training process, such as at the end of each epoch or batch. One of the most commonly used callbacks is EarlyStopping, which monitors a particular metric, such as validation loss, and stops training if that metric does not improve after a specified number of epochs, which is known as patience. This helps prevent overfitting by halting training when the model has reached its optimal performance on the validation set. So to implement EarlyStopping, we're going to import early_stopping using keras.callbacks. So we're now going to specify the early stopping specifications. We're going to call early_stopping function. We're going to specify the metric to monitor, which is validation loss, specify the patience, which we learned about earlier, and we're going to say restore_best_weights is equal to true, which means that whenever we realize that the weights that we had before were better than the ones that we currently have, we should restore those ones, okay? So let's go ahead and run this to specify the early stopping specifications. Another commonly used callback in Keras is ReduceLROnPlateau. This implements learning rate scheduling. With this callback, the learning rate is automatically reduced when a monitored metric has stopped improving. So this is something that we have talked about in the previous video. So to implement learning rate scheduling here using ReduceLROnPlateau, we're going to go ahead and import ReduceLROnPlateau from keras.callbacks. Then we specify the requirements or the specifications for it. Want to monitor validation loss. Want to say the factor that we want to reduce the learning rate by is 0.1 every time. The patience duration is two, and the minimum learning rate that we want to have is 0.0001. Okay? So let's go ahead and specify that here. All right, so once we've specified both of these callbacks, early stopping and learning rate scheduling, using the ReduceLROnPlateau, we can now use these callbacks to train our model. So as defining our callbacks, we can combine them into a list and pass them to the callbacks argument in the fit method. So we're going to go ahead and create a callbacks list, my_callbacks. Create a list of those early stopping and learning rate scheduling. We're going to now fit our model, so we specify model.fit, the training data, the labels, number of epochs, the splits, batch size, and then in callbacks we specify the callback list that we just made. So let's go ahead and use that here. And so let's allow our model to train. So we're going to, we specified 20 epochs, so watch for that carefully here. So let's give you some time here to work through its process. The model is still training, and we notice that the training process has stopped at 11. Okay? So that's interesting. Even though we specified 20 epochs within the fit method up here, only 11 epochs were executed. This is because of early stopping. This is exactly what we're expecting to happen here. So early stopping detected that the validation loss had not improved for the configured patience period of three. So it halted the training process before reaching epoch 20. So now let's plot the training and validation loss metrics to get a better sense of how our model performed. So we see that what was going on here is that our model was improving. So we see a drop in the loss, and after a while, it starts to plateau, so that's kind of why it eventually stopped. So having added batch normalization, gradient clipping, early stopping, and learning rate scheduling to our training process, we have introduced multiple methods that help stabilize and optimize our deep learning model's behavior, right? And so each of these techniques address different potential pitfalls in model training, from exploding gradients to overfitting, and together, they all yield a model that converges more reliably and generalizes better on unseen data. So, congrats. If you've been following along through all three videos, it means that you now know how to apply batch normalization, gradient clipping, early stopping, and learning rate scheduling to a deep learning model in Python using Keras.