From the course: Applied AI: Getting Started with Hugging Face Transformers
Unlock the full course today
Join today to access over 24,600 courses taught by industry experts.
The decoder
From the course: Applied AI: Getting Started with Hugging Face Transformers
The decoder
- [Instructor] Having discussed the encoder stack in the previous video, let's see how its output is processed with the decoder. The decoder stack will use the encoder's hidden states to iteratively generate a sequence of output tokens, one at a time. The specific form of the output depends upon the use case. A classification use case will produce a final label, while a translation use case may produce another set of tokens. Like the encoder, the decoder will also have several decoder layers. Let's start by discussing a single decoder layer. For the first decoder layer, the input is the previous output of the decoder stack. How do we obtain the previous output? Let's discuss that later in the video. For now, consider that the previous output is itself a set of tokens. This then goes through positional encoding and embedding matrix creation, similar to the encoder They are then combined and sent to the decoder layer.…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.