From the course: Advanced Microsoft Fabric Implementation and Governance

Unlock the full course today

Join today to access over 24,500 courses taught by industry experts.

Monitoring Spark jobs

Monitoring Spark jobs

- [Instructor] Fabric Spark monitoring can be done two different ways. Earlier, we saw how to monitor within the Spark hub, and Fabric also includes monitoring at the Spark level to show an even greater level of insight. Let's take a look inside of Fabric to see how we can investigate our Spark jobs. Let's take a look at monitoring inside of Synapse Data Engineering, and take a look at one of our notebooks. Well, as a matter of fact, let's just filter it so that all that we see are notebooks. We see that the activity names all have numbers after them, indicating that they have been run. So let's take a look at one. Let's look at Data Exploration. Now we can see the individual steps that were taken from each job. This you can also see, which we looked at earlier at the cell level. But if I select any one of these items, it takes me to the Spark details, so that we can see exactly how to look at it in Spark. We can take a look at the event timeline. Looking at the different executors…

Contents