adporn.net Applying batch normalization to a deep learning model - Python Video Tutorial | LinkedIn Learning, formerly Lynda.com

LinkedIn and 3rd parties use essential and non-essential cookies to provide, secure, analyze and improve our Services, and to show you relevant ads (including professional and job ads) on and off LinkedIn. Learn more in our Cookie Policy.

Select Accept to consent or Reject to decline non-essential cookies for this use. You can update your choices at any time in your settings.

Start free trial Sign in

From the course: Deep Learning with Python: Optimizing Deep Learning Models

Applying batch normalization to a deep learning model - Python Tutorial

From the course: Deep Learning with Python: Optimizing Deep Learning Models

Applying batch normalization to a deep learning model

“

- [Instructor] In this video, you'll learn how to apply batch normalization to a deep learning model. I'll be writing the code in the 05_02e file. You can follow along by completing the empty code cells in the 05_02b file. Note that this is the first in the three-video sequence that teaches you how to apply batch normalization, gradient clipping, early stopping, and learning rate scheduling to a deep learning model. Let's get started by running the previously written code to import and pre-process the data. So the first thing I need to do here is actually select the kernel for my environment. So Python Environments. I'm going to say Python 3.10. Now I'm going to click on my next code cell, and I'm going to say Run previous. Okay, so this is going to go ahead and run the code above to import and pre-process the data. Okay, so that is done. So our model consists of an input layer with 784 nodes, two hidden layers with 512 and 128 nodes respectively, and an output layer with 10 nodes. Between each of the hidden layers, we will normalize the outputs of one layer before feeding them into the next. This is what we know as batch normalization. Batch normalization can stabilize the training of a deep learning model and help it converge faster. To apply batch normalization to a model, we simply include a BatchNormalization layer to the model. So here, we're going to import the Input, Dense, and BatchNormalization layers using keras.layers. So as we define our model, we specify keras.Sequential to initialize the model. Specify the Input layer. The first Dense layer, which is the first hidden layer. 512 nodes with an activation function of relu. And between that and the next Dense layer, we specify a BatchNormalization layer. That's it. We do the same thing again between the second Dense layer and the output layer. So let's go ahead and run that to initialize our model. And so, yes, this error that we have here, or this one that we have here is actually benign, so nothing to worry about. All it's saying is that our environment is not set up to use the GPU. We're not going to be using the GPU for these examples, so we should be good to go. Okay? So good job. We have successfully applied batch normalization to a deep learning model in Python. That is it. Next, we discuss what gradient clipping is, and then we walk through how to use it in Python.

Contents