From the course: Advanced RAG Applications with Vector Databases
Demo: Metadata
- [Instructor] This section is about storing metadata with our chunk data. We'll start from the exact same place we left off in the last video. There's only a small change to make here. In link chain, metadata is stored within the document object. Last time we stored the chunks in the page content parameter. To store metadata, all we need to do is add a metadata parameter. Metadata is stored as a dictionary. You can define the metadata that you want to store. One of the most common pieces of metadata to store is the title of the document. For this example, we store this under doc title. We'll also store the chunk number. This tells us where in the document the chunk was taken from. Other than adding the metadata parameter to the last co snippet, there's another change to make here as well. We enumerate through the list instead of just looping through it. This is so we can access the chunk number and know where in the document the chunk was taken from. Let's take a look at our page content and metadata. See here that with this file text, we now have the chunk number as well as the title of the document that it came from.
Contents
-
-
-
Introduction to preprocessing for RAG4m 57s
-
Chunking considerations5m 12s
-
Chunking examples4m 32s
-
Introduction to embeddings9m 50s
-
Embedding examples2m 57s
-
Metadata3m 12s
-
Demo: Chunking2m 32s
-
Demo: Metadata1m 23s
-
Demo: Embed and store2m
-
Demo: Querying1m 8s
-
Demo: Adding the LLM2m 1s
-
Challenge: Cite your document sources47s
-
Solution: Cite your document sources59s
-
Challenge: Change the chunk size44s
-
Solution: Change the chunk size55s
-
-
-
-