From the course: Introduction to Modern Data Engineering with Snowflake

Unlock this course with a free trial

Join today to access over 24,400 courses taught by industry experts.

What is data ingestion?

What is data ingestion? - Snowflake Tutorial

From the course: Introduction to Modern Data Engineering with Snowflake

Start my 1-month free trial Buy for my team

What is data ingestion?

“

"80% of the battle is going to be getting all of the data from all of the different sources into one platform." These are wise words from my colleague, Jeremiah Hansen. And I think they're just the right way to kick off our exploration of the first phase of our data engineering framework, data ingestion. In the context of building data pipelines, ingestion refers to the gathering, collecting, or loading of raw data, often into a central platform. In this course, Snowflake will be that central platform. Out of the three phases in our ITD data engineering framework, I find ingestion to be the most interesting, mainly because the approaches to ingesting data can vary so widely. And the reason these approaches vary so much is because there are some pretty big challenges associated with ingesting data like scale, meaning how much data will need to be ingested; frequency: at what rate does the data need to be ingested? Is your use case satisfied with, say, a daily ingestion routine? Well…

Contents