Scheduled triggers are useful when you want to run a pipeline regularly at a specific time. The default values for a scheduled trigger are to run every 15 minutes, which equates to 96 executions per day. One point to mention when running jobs with short intervals has to do with overlapping executions. You need to make sure that the actions being taken for a given run can be completed before the next scheduled runtime. If there is overlap, meaning you have two instances of the same pipeline running simultaneously, then data can be corrupted, so be sure to make the duration long enough to complete in all scenarios. Another option is to run the pipeline hourly, with the frequency defined by the number in the first text box that defaulted to 15. If the value is 15, it means the job will run every 15 hours, which is a little less than twice per day. A scenario for this would be when you expect to have new data arriving on or around a 15‐hour interval that needs to be ingested, transformed, and made available for consumption.

A daily scheduled trigger provides the option to run the job every day at a given time. The default time is the 1 set in the Start Date text box on the New Trigger window. As shown in Figure 6.33, you can change the schedule execution time from the default value.

FIGURE 6.33 Azure Synapse Analytics daily scheduled trigger

This is achieved by expanding the Advanced Recurrence options group and adding values for hours and minutes. When you select the weekly schedule, you are given the option to select the days you want the job to run. You can select every day of the week or just a few days, as shown in Figure 6.34.

FIGURE 6.34 Azure Synapse Analytics weekly scheduled trigger

The configuration shown in Figure 6.34 will run twice per day: once at 06:15 and again at 15:15 on Monday, Wednesday, and Friday. The last option for configuring a scheduled trigger is monthly. As shown in Figure 6.35, you can select which days of the month the pipeline should run. In this scenario the job will run at 06:15 and 15:15 on the 1st, 11th, 21st, and the 22nd day of the month.

FIGURE 6.35 Azure Synapse Analytics monthly scheduled trigger

When you select the Week Days radio button, you are given two options: the iteration and the week day. The iteration values are First, Second, Third, Fourth, Fifth, and Last. The week day values are Sunday through Saturday. This means you can trigger the pipeline the first Monday of every month, the last Tuesday, or the third Friday, for example. You can also add multiple configurations, meaning you can trigger the pipeline to run on the fourth Wednesday and the second Saturday—almost any combination you need can be achieved.

The last scenario to cover regarding scheduled triggers is the many‐to‐many relationship they can have with pipelines. This means that a single schedule can trigger multiple pipelines, and pipelines can be triggered by multiple scheduled triggers. Figure 6.36 shows a way to achieve this.

FIGURE 6.36 Azure Synapse Analytics many‐to‐many scheduled trigger

An Execute Pipeline activity is located within the General group. As in all pipelines, there can be many activities, and, in this case, all the activities are pipelines themselves. When this pipeline is triggered, it will run two pipelines in parallel. The Execute Pipeline activity named Execute Pipeline TE has a dependency on two parent pipelines and will run only after both are complete. However, Execute Pipeline TransformSessionFrequencyToMedian is dependent only on a single parent pipeline and will run after it is completed. To create a scheduled trigger, complete Exercise 6.7.