After the Azure Batch pool has an allocation state of Steady, you might receive an AccountCoreQuotaReached error. If you experience this, check the Quota blade, in the Settings section, for your Azure Batch account, which displays your allocated limit. If the text box for Total Dedicated vCPUs is 0 and/or a 0 is to the left of all listed VMs Series, then click the Request Quota Increase button. If the values are zeros, then you have no allocated quota and therefore will not be able to run the batch job.
The code run in the brainjammer‐batch.exe is available on GitHub in the Chapter06/Ch06Ex01 folder. The code is C# and is in the Program.cs file. The first action the code takes is to capture the command‐line arguments. The arguments you configured in the run.bat file provide the storage account name and container where the brainjammer brain wave JSON files are stored and where the output is written to once processed. The current year, month, day, and hour are dynamically appended to the end of the provided outputLocation value and will create the directory on the storage account, if it does not already exist. The accountKey is accessible on the Access Keys blade for the Azure storage account you configured in step 1 of Exercise 6.1. Click the Show Keys button to copy and paste the access key into the run.bat file. This is not the most secure means for granting access to your storage account, but until you learn more about managed identities and service accounts in Chapter 8, “Keeping Data Safe and Secure,” this is the method used. It is secure, and the connection uses a secure protocol (HTTPS); it is just not as secure as the other approaches just mentioned.
The program then loads all the brainjammer brain wave JSON formatted sessions in the provided inputLocation argument. In this case the value is SessionJson/ClassicalMusic/POW, which analyzes all ClassicalMusic sessions. The code calculates the median value for each frequency for the entire session. Most sessions contain approximately 1,000 readings. The following is an example of the output. The output is the median frequency reading value for a single given brainjammer brain wave ClassicalMusic session. The values are expected to fall into the ranges that were identified through the EDA performed in Chapter 5, “Transform, Manage, and Prepare Data,” and provided in Table 5.2.
To configure and trigger the Azure Batch job from Azure Synapse Analytics, perform Exercise 6.2. Note that both a job and task are created when the Batch Service activity is run, which is why you did not create them in Exercise 6.1. You will experience this as you work through Exercise 6.2.
