From the course: Hands-On AI: Building LLM-Powered Apps

Large language model capabilities - Python Tutorial

From the course: Hands-On AI: Building LLM-Powered Apps

Large language model capabilities

“

- [Instructor] Now we have the background on language models and how we tokenize text to communicate with them. Let's put a definition on large language models. So how large is large? The current standard of large is more than 1 billion parameters, and the model has read more than 1 billion tokens of text compared to linear regressions, which usually have only a few parameters, this is a significant increase in size. So what happened was that when we scale out, compute and data to trend language models on enormous amount of data to predict the next tokens, reading the whole internet to predict the next token and only predicting the next token. We surprisingly found that large language models start to exhibit emergent abilities. This was discovered in 2022, and we found that we can now prompt it to do arithmetics. We can ask it to do machine translation. We can even ask it to provide some reasoning capabilities. All this are very amazing to observe. And in summary, large language models can now be prompted to do many, many tasks. Next up, we will get our hands dirty and start building the first part of our chat with PDF application.

Contents