In this lab you will create visualizations of IoT data using Azure Stream Analytics and a dashboard using the Azure Web App service.

Table of Contents

• Table of Contents
• Using Stream Analytics to Process and Route IoT Data
  • Create an Event Hub
    • Configure Access Policy
  • Create the Stream Analytics Job
    • Define the Input Data Stream
    • Define Output Data Streams
    • Write the Query
  • Create Azure Web App that Shows Event Hub Data
• Conclusion

In this lab you will build a data pipeline that captures the data coming into IoT Hub, processes it with Azure Stream Analytics, then routes it to downstream services. The first service you’ll build will consume the data and present it through an Azure Web App where it is rendered as a real-time graph.

Using Stream Analytics to Process and Route IoT Data

Azure Stream Analytics is a service that does real-time data processing in the cloud. You will create a new Stream Analytics job and define the input data stream as the data coming from your IoT Hub. Next you will define an output data stream that sends data to an Event Hub. Then, you will write a SQL-like query that collects data coming in on the input stream and routes it to the output stream.

The output of the Stream Analytics job is an Event Hub that is created during this lab. An Azure Web App is being configured to listen for the data coming to the Event Hub and routes it to a dashboard UI that can be accessed with Web browsers.

Create an Event Hub

First, you need to create an Event Hub to queue the data coming out of the Azure Stream Analytics Job. Open a new browser tab and navigate to the https://manage.windowsazure.com. Click on the NEW icon in the lower-left corner.

1. Select + > APP SERVICES > SERVICE BUS > EVENT HUB > QUICK CREATE and enter the following:
   • EVENT HUB NAME: You can use anything that is a valid name here, such as thinglabs-eventhub-[yourname, initials, etc]
   • REGION: If you created your IoT Hub in East US, select East US 2. Select the same region you created your IoT Hub in.
   • SUBSCRIPTION: Select the subscription you’ve created your resources in.
   • NAMESPACE: You can use anything that is a valid name here, such as thinglabs-eh-[yourname, initials, etc]

Configure Access Policy

1. After the Event Hub is created, you need to create a shared access policy which the Web App can use to listen for the data. To do so, click on the View Connection String or Connection Information button and configure a policy like in the image below (remember to save):

1. Then, click again the View Connection String or Connection Information button and take a note of the connection string visible on the screen:

Create the Stream Analytics Job

Next, you can create the Stream Analytics Job by opening a new browser tab and navigate to https://manage.windowsazure.com. Login if necessary. Click on the NEW icon in the lower-left corner.

1. Select DATA SERVICES > STREAM ANALYTICS > QUICK CREATE and enter the following:
   • JOB NAME: You can use anything you’d like here…like iotlab or something similar so you can identify it easily later.
   • REGION: If you created your IoT Hub in East US, select East US 2. Select the same region you created your IoT Hub in.
   • REGIONAL MONITORING STORAGE ACCOUNT: Select or create a storage account.

   

2. Click CREATE STREAM ANALYTICS JOB. It will take a few minutes for the Steam Analytics job to get created and become available.

When the job indicates that it is created, click into it to create the data streams and query. Once you are in the Stream Analytics job you will need to define the job input, query, and output.

Define the Input Data Stream

The data will come in as a data stream from the Event Hub that was automatically created when you created the Azure IoT Hub.

1. Click on the INPUTS header.

   

2. Click on ADD AN INPUT.
3. Select Data stream and click on the forward arrow in the lower-right.
4. Select IoT Hub and click on the forward arrow in the lower-right.
5. Complete the form as follows:
   • INPUT ALIAS - DeviceInputStream
   • SUBSCRIPTION - choose your subscription
   • CHOOSE AN IOT HUB - choose the IoT Hub you created earlier
   • IOT HUB SHARED ACCESS POLICY NAME - leave this as the default, which should be iothubowner
   • IOT HUB CONSUMER GROUP - enter the default consumer group name: $Default
6. Click on the forward arrow in the lower-right.
7. On the Serialization settings form, leave the defaults (Event Serialization Format:JSON and Encoding:UTF8) click on the checkmark in the lower-right.

After a few seconds, a new input will be listed.

Define Output Data Streams

Before defining the query that will select data from the input and send it to the outputs you need to define the outputs. For this lab you will output the results of the query to an Azure Web App dashboard you’ll create soon.

1. Click on the OUTPUTS header.

   

2. Click on ADD AN OUTPUT.

3. Select Event Hub and click on the forward arrow in the lower-right.

   

4. Configure the Event Hub

   

   1. Give it a unique name
   2. Select “Use Event Hub from Current Subscription”
   3. From the drop-down, select your previously created Event Hub
   4. Click the right arrow to go to the “Serialization Settings”
   5. Accept the defaults (JSON, UTF8, Line Separated)
5. Click checkmark on the lower-right to configure the Event Hub

Write the Query

In the query, you want to select data from the input stream and put it into the output stream. With data like temperature, you can do interesting things like applying operations on the data as you query it. For this example, you will write a query that selects from the input stream and sends the output stream the minimum, maximum and average temperature values across all devices, and enables you to group the data by either location or device ID. Using a TumblingWindow you will send data to the output stream in rolling increments of 5-seconds.

1. Click on the QUERY header.

   

2. Write the following query:

WITH ProcessedData as (
    SELECT
        MAX(celsius) MaxTemperature,
        MIN(celsius) MinTemperature,
        AVG(celsius) AvgTemperature,
        location,
        deviceId,
        System.Timestamp AS Timestamp
    FROM
        [DeviceInputStream]
    GROUP BY
        TumblingWindow (second, 5), deviceId, location
)

-- Make sure this matches your Event Hub output name from above,
-- If you've forgotten it you can go back and get it in another browser tab
SELECT * INTO [ThingLabsEHOutput] FROM ProcessedData

1. Click SAVE in the lower middle of the screen.
2. Once the query is saved, click START to start the Stream Analytics job.

If your app from the previous lab isn’t still running, go ahead and start it up. It will take a few minutes for the Stream Analytics job to get started and to start sending data to the output. Remember, the TumblingWindow is set to 5-seconds, so data will only update every 5 seconds.

Create Azure Web App that Shows Event Hub Data

You need to create a Web App to visualize the data coming out of the Event Hub. Click on the NEW icon in the lower-left corner.

1. Select + > COMPUTE > WEB APP > QUICK CREATE
   • URL: You can use anything that is a valid name here, such as thinglabs-eventhub-[yourname, initials, etc]
   • APP SERVICE PLAN: Select “Create a new App Service Plan”.
   • REGION: If you created your IoT Hub in East US, select East US 2. Select the same region you created your IoT Hub in.

   

2. Setup deployment from github:
   • On the web app configuration dashboard, click “Set up deployment from source control.” (on the lower righthand side of the page)

   

   • Select “External repositiory” from the dialog

   (Note: You’d think this should be “Github repository”, but we’re avoiding having you fork and maintain your own version fo the code for today. If you want to modify it later, you can fork the repository and modify it. Then you would have to update your deployment source to “Github Repository” which finds your repositories.)

   

   • Navigate to the ThingLabs Web App repository, copy the git repo url

   

   • Paste it into the dialog on the configuration (the External Repository Page)

   

   • Click checkmark on the lower-right to setup your deployment from the ThingLabs Github Repository
   • Click on “Configure” to customize settings for your the Web App.
     • Turn Web Sockets on

     

     • Add an App Setting (Key: THINGLABS_EVENTHUB_CONNSTRING Value: Connection String from your EventHub)

     

     • Use the connection string from your Event Hub

     

     • Save the changes
   • Restart the Web Application
3. Browse to your site
   • It will take a few minutes for data to flow, but you should start to see a graph render as data is recieved.

   

Conclusion

In this lab you learned how to create an Azure Stream Analytics job to query data coming in to Azure IoT Hub, process it and send it to Event Hub and a Web App.

If you want to see in detail how the data is routed from the Event Hub to a UI seen in a Web browser, go ahead and check out the sources from the ThingLabs Web App repository.