From the course: Hands-On AI: RAG using LlamaIndex

LlamaIndex evaluation - LlamaIndex Tutorial

From the course: Hands-On AI: RAG using LlamaIndex

LlamaIndex evaluation

“

- [Instructor] I concluded the last section by talking to you about Ensemble Retrievals and Ensemble Query engines, and how you can use these to kind of evaluate responses, and kind of get a sense of how your RAG system is performing. Although we didn't cover it in this course, I do want to point you to a place where you can go and learn more about Evaluation within the Llamaindex framework. So, if you go to docs.llamaindex.ai, and you go to Advanced Topics, scroll down to Evaluation, you'll see that Llamaindex actually has a module for Evaluation. I didn't cover it in this course because like I mentioned before, this would be another two to three hour course in itself. But in it, you can do Component-Wise Evaluation, and you can evaluate the Query Engine without using Retrieval. You can do an End-to-End Evaluation, where you evaluate the entire system using a number of metrics. And so I touched on some of these metrics at a high level, when I talked about RAG evaluation, but if you'd like, you can go deeper, and get a full kind of understanding of what each one of these metrics is like. At the end of the day, each one of these metrics, for the most part, is just using calls to a language model. You can even go a bit deeper into Evaluation. You can ask yourself whether you should do End-to End or Component-Wise, where do you start? And do a real deep-dive into Evaluation. So, if you're wondering where to go next after learning about these paradigms and the techniques that fall into these paradigms, the next step would be Evaluation. And so Llamaindex does have a pretty robust set of documentation that you can look at and sort out how to use a Llamaindex for evaluation.

Contents