Create a basic GAN

- So this is the part of the course where we'll actually start looking at some code. So there's tens of thousands of tutorials online about GANs and about 90% of them use the same data set that we are using. The difference here though is that I'm going to go line by line through the code and explain what's happening. I'll also give a few prompts and a few opportunities to improve the code in order to generate better images. Let's get started. Welcome to GitHub Code Spaces. What I'll do here is I'll go through line by line how the generative adversarial network code comes together how it reflects the architecture that we spoke about earlier, and how it's able to actually train and learn from the models. So first what I'll do is I'll just go through all the different steps. So this is a notebook on GitHub code spaces. So if you have access to the repo you should be able to open GAN.ipynb and you should be able to run this code in the same way that I'm running it here. In the first cell at the top, essentially what we're doing here is just importing a number of supporting libraries in Python specifically the Torch Libraries. And PyTorch that will help manage all of the array calculations, will help manage the gradients and it will help us put together the neural networks for the generator and the discriminator. The next cell here is essentially setting up a random seed so it's taking a random number and setting it as the seed. This seed will allow that anyone who runs this notebook they will get the same results at the end if they use the same seed. So it's a good idea to set this up at the start. In this example we actually won't be using the computer's GPU for training. You may have a GPU compatible computer. In this case, we're just running it on cpu. So this just as a bit of a check to see if a GPU is available. We actually hard set the CPU here but you can change this to cudnn and then your device number. Then we set up some variables for the training. We set up the width of the latent space that the generator samples from. We then set the shape of the generator and discriminator matrices. We then set the number of colors, so in this case we're using black and white images, so it's just one but if we were using color, it should be three. Then we set the batch size, which is the number of images per pass that will process, and the number of CPU workers that we'll use to create the data set. Now, fortunately with Torch Vision and PyTorch we have support to get some open source data sets and we're able to get one of the original data sets from the original paper and that's the handwritten digits MNIST data set. With the function MNIST, we're able to download and import that entire data set and also apply some transforms. The transforms that we'll apply will be setting the shape converting it to a tensor or an array in which we can use for the neural network, and also normalizing the images. Then we create a data loader which again is provided by PyTorch which allows us to efficiently query that data set get samples, and provide that for the training. In the next function, what we do is we set up a way to initialize the weights of the neural network because what they need to be given some type of value before we can start training. And depending on the type of layer in the neural network there's a different function, which is included here. Okay, so now we've got to the point where we can actually define what are the models in the network so what is the actual generator and the discriminator. And they follow the same type of pattern. With a generator class, we'll have an init function and we'll have a forward function. The init function will set the main architecture of the neural network, and in this case we're using the same as the tutorial on the PyTorch website. It's creating a convolutional layer, a batch norm function and then a ReLU activation function and it's doing that four times. You can change this architecture to whatever you see fit. You can use a number of different combinations. The thing to be mindful about though is making sure that the data shape going in and then going out is always kept consistent. The next function is the forward pass. So that's essentially pushing the data through the network or multiplying it through all the different layers. Once we have that definition, we create an instance of it. We initialize the weights of the network and then we just print out the layers. Now the discriminator definition is essentially the same but obviously the details are different of the network. It'll have initiation layer, where it will create the actual object and the layers of the network, and then it will have a forward function. Then we'll create an instance of the discriminator, initialize the weights and then just print out the layers just to show. The next step is to create a loss function and we're able to use that from the neural network model of PyTorch We create a fixed noise amount that we use for the sampling and then we'll create a real label and a fake label designation of binary one and zero. We'll then create two optimizer objects which we'll able to get again from PyTorch and what they will do is they'll update the weights of the model and also control the changes over time. So in essence, these are the different steps in order to create the objects that we need to be able to start training the generative adversarial network, we have the data set, we have the transform data loader. We have the generative model, the discriminator model. We have a loss function, and we have optimizers. Now we'll look at how do we put this all together.