Challenges and shortcomings of GPT-3

- [Instructor] GPT-3 is not without its problems. GPT-3 was trained on data that is biased. Now, human language and text reflects our bias, and you and I are in many sense privileged as we have access to computers, we can easily publish our thoughts online, whether that's through a blog post or on a site like Reddit. And part of this data that GPT-3 was trained on was data that was deemed interesting on Reddit based on up votes from other users. Unfortunately, this means that biases and dominant worldviews then make it into training data and are encoded in large language models. Let's see if there are examples of bias in the model. So I'm going to head over to the OpenAI GPT-3 model. So this is the playground. Now, I'm not going to cherry pick any results. I'm going to give them to you exactly as I see them. So if we take our first sentence, So after a long day's work at the hospital, the nurse was tired because, and I select submit. And you can see it continues with she had to work a double shift. Now you can see that the nurse is female. This is demonstrating gender bias. Now, if we try another sentence this time using the doctor as the profession, and we go ahead and select submit. And you can see that GPT-3 switches things around and the doctor is male. This is again showing gender bias. Nurses are almost always female, and doctors are almost always male. Now, if we do the same exercise and we have another sentence, we ask the receptionist for directions to our room and select submit. So it looks like the receptionist is female. Again, this is an example of gender bias. Now, if we change the sentence, after long meeting with the board, the company president was tired because, and this is another example of gender bias, because there's no indication that the company president is ever female. And just one last one. After spending the entire morning staring at the screen, the programmer stepped away for lunch because he was hungry. Now, I've tried all these examples hundreds of times each and the genders are very rarely switched. I've never seen a female programmer or a male nurse. At least in the examples we've looked at, lower skilled and lower paid jobs are more readily linked to women, and higher skilled and higher paid ones more readily linked to men. And certain professions are more directly linked to men rather than women. So are there more female nurses than male nurses? Absolutely. Are there more receptionists that are female than male? Yes, again, so what's the problem? Most people now apply for jobs online, and in many cases where resumes are filtered by AI systems, these are downstream tasks from large language models. So where the model has a strong association between gender and certain professions, this means there's a bias where there are more men for certain types of employments. So for example, you don't want a resume to be preferred only because it's clear that the applicant is a man rather than a woman. There are other examples of gender bias in GPT, and we haven't carried out an in depth study. But it's clear that we need a human in the loop to check the output for some of the downstream tasks. So where does this bias come from? Well, it's clearly from the data the models were trained on. This includes Reddit and Common Crawl, amongst others. Now in early 2022, OpenAI tried to address some of these challenges by creating a new model called InstructGPT. They hired a team to help label and assess whether the response from the model was in line with the intent of the prompt. So let's say the model was given a prompt, such as writer story about a wise frog. If the response was in line with the same, then the model received a more favorable score. If instead, the response was off topic and used violent language and was biased in content, then the model received a negative score. They created a reward function to get the model to pick the one that the labelers would prefer. The more it got right, the more it was rewarded. One of the other major concerns with GPT-3 and large language models is their environmental impact. A carbon emission study of large language models was conducted by Google and Berkeley in 2021 and found that training GPT-3 would've resulted in energy consumption of almost 1,300 megawatt hours and a release of 550 tons of CO2. We've looked at two of these shortcomings of GPT-3: bias and environmental impact. It's not surprising that some of the large language models that follow GPT-3 tried to optimize the model and address these challenges. And we'll take a look at some of them next.