From the course: Hands-On AI: Building LLM-Powered Apps

Prompt playground for LLM apps - Python Tutorial

From the course: Hands-On AI: Building LLM-Powered Apps

Prompt playground for LLM apps

“

- [Instructor] In this video, we will go through how to operate a prompt playground that's built-in in Chainlit. The prompt playground will greatly accelerate the prompt iteration process and allow us to see the results in real time. Now let's start the application and upload a simple PDF file, and then let's start asking questions. The question we had the problem with is what is the operating margin? So that's going to ask this question and it says the operating margin is not provided in the given information. From here, we can see that it took one step so let's click on the downward pointing arrow to expand this. And it says, took two steps. Let's expand that again, and here is the retriever doing its work, retrieving the documents, and then it goes to StuffDocumentsChain and it took one step and it caused the MChain, it took one step. And the result is the operating margin is not provided. So now here is a little bug icon. So if we click on this, we can inspect everything in the prompt playground. So click on this and it will navigate to Chainlit's prompt playground. Here we can see the exact prompt that is sent to the OpenAI API, and if we scroll all the way back down, it's a long one, but if we scroll all the way back down, we can see the assistant replying with the message. And this is the exact responses that we get. On the right hand side, we can choose to use different LLM providers, so let's stick with ChatOpenAI, and we can choose different models to use. Right here we have GPT-3-5-turbo, turbo 16K, GPT-4, and different temperature and different max token sessions, and a whole bunch of knobs that we can use to tune. As a reminder, temperature, the higher value will make the output more random and the lower value will make it more focused. And if we click on submit, it is going to run the whole chain again. So this is the prompt playground and we can use this to accelerate our prompt engineering efforts and attempts. And just a reminder, sometimes some of the models might not be available to you depending on your OpenAI subscription. So with this, in the next lab, we will use the prompt playground extensively to fix the deficiencies in our chat with PDF application.

Contents