From the course: MLOps Essentials: Model Deployment and Monitoring

Unlock the full course today

Join today to access over 24,600 courses taught by industry experts.

Managing concept drift

Managing concept drift

- We will discuss the techniques and best practices for managing concept drift in this video. How do we measure concept drift? It is actually pretty straightforward. What inputs do we need? We only need the predicted labels and true labels in production. For example, if we are predicting if a website user will eventually buy a product we need the prediction and if the user actually made the purchase later in that session or in a week we need to collect this information for many predictions over a period of time, say a week or a month. Then we can compute the overall accuracy of the predictions and see how much it deviates from the baseline accuracy found during model training. Based on these values, we can review and take action. The biggest challenge in measuring concept drift is that true labels are not available all the time to verify if the prediction is accurate. In such cases, we may have to collect explicit feedback…

Contents