From the course: MLOps Essentials: Model Deployment and Monitoring
Join today to access over 24,600 courses taught by industry experts.
Planning for infrastructure
From the course: MLOps Essentials: Model Deployment and Monitoring
Planning for infrastructure
- [Instructor] Infrastructure plays a vital part in delivering a stable and resilient serving for machine learning. Let's explore various elements and best practices for infrastructure in this video. There are two types of infrastructure elements that participate in serving: compute resources, and tools and technologies. Having enough compute resources are essential for maintaining operational performance goals like latency and concurrency. First, we need CPUs and GPUs to run the models with expected latency. GPUs are important for ML and need to be managed efficiently. Then comes memory to cache models and other data. Accelerators are available today for improving the performance of the models. Storage also becomes essential with the amount of data being captured and archived. Networking should be fast and reliable. Finally, resiliency features like standbys and backups are also needed for all the infrastructure…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.
