From the course: Introduction to Modern Data Engineering with Snowflake

Unlock this course with a free trial

Join today to access over 24,700 courses taught by industry experts.

Recap and best practices for pipeline orchestration

Recap and best practices for pipeline orchestration - Snowflake Tutorial

From the course: Introduction to Modern Data Engineering with Snowflake

Recap and best practices for pipeline orchestration

In this module, you learned how to level up your data pipelines by adding automation to them. Let's quickly go over what we covered. Tasks are the key ingredient for adding automation to your pipeline. You can create and manage tasks yourself or have Snowflake do the heavy lifting for you with serverless tasks. You also saw how they can be used to automate just about anything, from simple SQL queries to complex logic within a stored procedure. You learned that tasks can be chained together to form task graphs, also known as DAGs. These are key to coordinating tasks that depend on each other and keep moving parts of your pipeline in the correct sync. We've come so far. Join me in the next video to close out and recap all that you learned.

Contents