The Internet of Things on AWS – Official Blog

Introducing AWS IoT SiteWise Edge

Note: SiteWise Edge is now generally available. Please see this blog post to get started with SiteWise Edge.

From assembling automobiles to producing pharmaceuticals, successful industrial operations are fruits of continuous efforts to minimize equipment downtime, maximize product quality, and ensure personnel safety. Industrial IoT promises capabilities that can help manufacturers unlock insights quickly from a growing volume of sensor data from their equipment. Armed with these insights, operators can respond to equipment issues, product quality issues, and safety hazards as soon as they occur. In this post we will walk through AWS IoT SiteWise Edge, a new feature that enables customers to collect and process equipment data on-premises for low latency applications that must continue to work even if connection to the cloud is unavailable. With AWS IoT SiteWise Edge, developers can skip building time-series data acquisition and edge processing infrastructure to focus on building local applications that offer insights and support quick decision making on the factory floor.

A lap around AWS IoT SiteWise Edge features

AWS IoT SiteWise Edge brings features of AWS IoT SiteWise in the cloud to the customer’s premises. Specifically, you can now use asset models defined in the cloud service to process data in the SiteWise gateway locally, and visualize equipment data using local SiteWise Monitor dashboards served from the SiteWise gateway. You can also read data directly from the gateway using the same GET APIs that you use to read data from AWS IoT SiteWise in the cloud. AWS IoT SiteWise Edge software can be installed on 3rd party Linux industrial computers, AWS Snow Family devices, and AWS Outposts. AWS IoT SiteWise Edge provides infrastructure for key functions of an IoT edge application bearing in mind the application lifecycle from development to support:

Data collection from disparate sources

Industrial IoT applications, ranging from monitoring dashboards to anomaly detection, utilize data from multiple disparate equipment and sensors. Typically this data is already logged in a historian database (a local time-series database). AWS IoT SiteWise Edge includes collectors that can securely read multiple time-series data streams from equipment and historian databases using OPC-UA, EtherNet/IP, and Modbus protocols. AWS IoT SiteWise Edge normalizes these data streams to a common format before storing them in an internal local time-series database for additional processing or routing them to AWS IoT SiteWise.

Data organization and contextualization

Time-series data collected from historians and equipment needs to be labelled with contextual information to be understood by developers and be used in applications. For example, a temperature data stream may be represented by a non-descriptive alphanumeric ID such as “X012232.” AWS IoT SiteWise allows you to create asset models for each type of equipment that describe its data streams or “measurements.” For example, you can describe an Evaporator asset that has Temperature, State, and Pressure as measurements. You can then map non-descriptive data stream IDs to each of these descriptive labels. You can create asset models for each type of equipment and then organize them in a hierarchy to represent production lines or entire factories. You can use APIs to query data streams using descriptive labels consistently across your applications. With AWS IoT SiteWise Edge you can configure the same asset models to be available locally so you can query labelled data directly from the gateway. Additionally, AWS IoT SiteWise Edge automatically syncs any changes made to asset models with all your gateways ensuring your applications are always using the current definition of assets.

Data transformation and metrics processing

Often data streams have to be processed further into metrics that indicate the health and performance of industrial operations. You can specify formulas in asset models to compute metrics over fixed time intervals from labelled data streams (e.g. Average Temperature over 1 min). Asset models also support transformation of time-series data points for use in the metric calculations. For example, you can configure a model to transform temperature values in Fahrenheit to Celsius or equipment state values into binary true or false values based on a criteria and then compute an average temperature metric reported in Celsius. AWS IoT SiteWise Edge automatically computes transforms and metrics defined in asset models configured for the edge. If the gateway collects a data stream that has an associated asset model, it processes the data stream according to the formulas available in the asset model, stores the result locally, and also forwards the result to the cloud for additional processing and longer term storage. Local applications can read processed data using the same GET APIs available in the cloud directly from the gateway. You can use this feature to filter data to only values of interest, aggregate values over time, such as computing averages, or total time spent in a given state to compute sophisticated metrics such as Overall Equipment Effectiveness (OEE).

Data visualization

Besides reading data into custom applications through GET APIs, you can also use AWS IoT SiteWise Monitor to create monitoring dashboards without writing any code (or SQL queries) and access them locally. You can load the dashboards in a web browser running on a computer that can reach the gateway over the local factory network. AWS IoT SiteWise Edge serves the dashboard directly from the gateway and automatically syncs any changes made to the dashboards in the cloud to all your gateways.

Gateway management

Supporting on-premises gateways across multiple sites can be challenging. AWS IoT SiteWise helps you remotely configure and centrally manage gateways through the AWS Management Console. AWS IoT SiteWise regularly collects metrics on gateway health and connectivity with industrial data sources in Amazon CloudWatch and shows them on the gateway section of the console. Now, AWS IoT SiteWise Edge provides the “OpsHub for AWS IoT SiteWise” application for monitoring and troubleshooting gateways entirely locally. The OpsHub for AWS IoT SiteWise application can be installed on any Windows PC. The application connect directly to your gateway over the local network. The application gives you access to device health metrics (e.g. memory, CPU, cloud connectivity), status of edge software (e.g. uptime of dashboard applications), and recent data collected from equipment. You can use the application to debug disruptions in the flow of your data and restart software on the gateway to fix issues.

Getting started

Pre-requisites

At a minimum, AWS IoT SiteWise Edge requires an industrial computer running Linux with a x86 64 bit quad-core processor, 16GB RAM, and 256GB in disk space. In preview, we recommend you use dedicated hardware for use with AWS IoT SiteWise Edge. In preview, AWS IoT SiteWise Edge software runs on AWS IoT Greengrass v1 (v1.10.2 recommended) installed on Ubuntu Server 18.04+ LTS OS for x86. You also have to install Java8, Python 3.7, and Docker software packages. Additionally, ensure that ports 443, 8443, and 8883 are externally accessible on your device.

To install AWS IoT Greengrass, please follow Quick Start: Greengrass device setup.

Create an AWS IoT SiteWise gateway

Once you have installed the pre-requisite software on your device, you can setup the gateway from the AWS IoT SiteWise Console. AWS IoT SiteWise Edge bundles capabilities in “packs” that are installed on the gateway. The Data Processing pack enables computation of metrics and transforms using asset models and visualizing them with AWS IoT SiteWise Monitor dashboards at the edge. It also enables REST APIs on the gateway that you can use to query data for your local applications. The Data Collection pack supports collecting data from common industrial data sources and transferring it to the AWS Cloud. You can collect data from OPC-UA servers, Modbus servers, and over EtherNet/IP. You can transfer the collected data to Amazon S3, AWS IoT SiteWise, AWS IoT Core, Amazon Timestream, and Amazon Kinesis.

To setup a new gateway:

  1. Open the AWS IoT SiteWise console.
  2. Navigate to the Gateways section from the Edge menu item and Add a new gateway.
  3. Select the AWS IoT Greengrass group associated with your gateway device. The console will install AWS IoT SiteWise software as AWS IoT Greengrass Connectors in this group.
  4. Enable the Data Processing pack option and click Create Gateway. The Data Collection pack is required and installed by default when you create the gateway.

Create SiteWise gateway

Once the gateway is created, you configure its data sources from the gateway detail page. You can configure OPC-UA, Modbus, and EtherNet/IP data sources. Please see AWS IoT SiteWise data source configuration documentation to learn more.

Update AWS IoT Greengrass service roles

AWS IoT SiteWise Edge deploys software in Docker containers on AWS IoT Greengrass core. The container images are hosted on Amazon Elastic Container Registry (ECR) and deployed by the service to the gateway device. To enable your device to access those images, you need to grant AWS IoT SiteWise permissions to Greengrass core running on the device.

To grant permissions to the role:

  1. Navigate to the IAM console and search for the Greengrass_ServiceRole (or any alternative role you may have associated with the Greengrass group you are using for your gateway). The Greengrass_ServiceRole role is typically created automatically when you create a group in the AWS IoT Greengrass console.
  2. Attach an inline policy to this role.

Policy definition (JSON):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "ecr:GetDownloadUrlForLayer",
                "ecr:BatchGetImage"
            ],
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": "ecr:GetAuthorizationToken",
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor2",
            "Effect": "Allow",
            "Action": "iotsitewise:*",
            "Resource": "*"
        }
    ]
}

Deploy AWS IoT Greengrass Connectors

Once you have configured your sources, navigate to the AWS IoT Greengrass v1 (classic) console to deploy the connectors to your device.

To do this:

  1. Navigate to Groups in the AWS IoT Greengrass v1 (classic) console
  2. Select the group you picked for use with your gateway in the previous step, and navigate to the Connectors page .
  3. You should see two connectors with “SiteWise” in the title in your group. Choose Actions and then choose Deploy. AWS IoT Greengrass will start the deployment.

The deployment can take up to 5 minutes to complete. When completed, the deployment status will change to “succeeded.”

Configure Asset Models

Once the Data Processing pack is enabled and all connectors are successfully deployed to the gateway, it can begin processing data using asset models.

To configure an asset model for the edge:

  1. Navigate to the Build section in the console and find the asset model you wish to configure.
  2. Open the Configure for Edge setting on the top left side of the console. You can select from one of three options presented:
    • No Edge configuration (default) – This options sends all data points to the cloud. Metrics and transforms are computed in the cloud. Select this option for properties you want to use in remote applications e.g. cross-site monitoring dashboards and analytics.
    • Compute all properties at the edge – This options configures all transforms and metrics to compute in the gateway and only send the final results to AWS IoT SiteWise. Depending on the specifics of the asset model, you may want to understand how much compute resources your metrics and transforms may require. Select this option if you want to all data for the asset to be available for querying locally to minimize latency and ensure the data is available regardless of cloud connectivity.
    • Custom configuration – You can choose which asset model properties you want to compute at the edge. This option allows you to incrementally move compute to the edge. For example, you could initially only choose the properties required by local applications to be computed at the edge. This option is also handy for filtering data streams. You can define filters on measurements as transforms computed at the edge and keep those measurements at the edge. For this configuration, the gateway will only send the filtered values for those measurements to the cloud.

Asset Model Edge Configuration

In preview, you can process up to 150 transactions per second on a quad-core 16GB device. A transaction can be a measurement data point collected, a transform computed, or a metrics computed. For example, you can process data for 10 assets with 5 measurements (1 data point per second), 5 transforms, and 5 metrics (10 x [5+5+5]). An AWS IoT SiteWise gateway fetches all  instances of the asset model from the service and processes data for measurements it is able to collect. All you need to do is configure the asset models themselves and keep the above load guidance in mind.

Set up gateway credentials

Before you can access data or dashboards from the gateway, you need to create Sigv4 credentials on the gateway. Setting these credentials is mandatory and ensures any unauthorized clients on the network cannot gain access to the gateway. The credentials are created with a user defined 12 character password, which is also used by the OpsHub for AWS IoT SiteWise application and local SiteWise Monitor dashboards to securely retrieve data from the gateway.

To create the gateway credentials:

  1. ssh into your SiteWise gateway
  2. Execute sudo /sitewise_edge/credentials/create-credentials.sh
  3. The terminal will display the message below to prompt you to enter your desired password. Choose a 12 character password.
gateway-host % ./create-credentials.sh 

Password has following minimum requirements:
1. Must be at least 12 characters long.
2. Contain mixture of both uppercase and lowercase letters.
3. Contain a mixture of letters and numbers.
4. Include of at least one special character, e.g., ! @ # ? ].
5. Characters < or > are prohibited as both can cause problems in Web browsers.

Enter Password: 
Confirm Password: 
Enter a date (yyyy-mm-dd) [leave empty for never]: 2022-12-31

--------------------------------------------------------------------------------
Security Reminder:

Security is a shared responsibility between AWS and you. When you use
AWS IoT SiteWise Edge, you are also responsible for securing your devices, local
network connection, and private keys. Encrypt and secure your gateway, so your
industrial data is secure as it moves through the gateway. If your gateway has
a hardware security module, you can configure AWS IoT Greengrass to secure your
gateway. For more information, see Hardware security integration in the
AWS IoT Greengrass Developer Guide. Otherwise, consult the documentation for
your operating system to learn how to encrypt and secure your file system.
--------------------------------------------------------------------------------


Your API access credentials expiring on (2022-12-31) are:
export AWS_ACCESS_KEY_ID="EDGE3DZTKQUF5IAIWUVS"
export AWS_SECRET_ACCESS_KEY="e6174a0fbd3d2a42072e0beb61de42280fc6656722533e0a0a16bf4f7e57c968"
export AWS_REGION="Edge"

Finally, acquire the gateway certificate. You will need this certificate to call APIs and use the OpsHub for AWS IoT SiteWise application. To acquire the certificate follow these steps:

  1. SSH into the gateway.
  2. Copy the certificate file (servercert.pem) from /sitewise_edge/certificates/servercert.pem to the client machine Note: you have to be root and sudo cp /sitewise_edge/certificates/servercert.pem
  3. scp the file to your local machine.

Monitor your gateway

Once your gateway is in operation you can monitor it locally and the AWS IoT SiteWise console. You can use the OpsHub for AWS IoT SiteWise application locally to check the state and performance of your gateway in real-time. This data can help you debug gateways and also understand if you need to upgrade the device hardware to support your workload well.

OpsHub for AWS IoT SiteWise

The application provides the following data and options to help with debugging and management:

  • List of software components running on the gateway
  • Uptime status of software services (e.g. are the dashboard services running or down?)
  • Device performance metrics (e.g. disk usage, CPU usage, memory usage)
  • Cloud connectivity status
  • Latest data points collected and computed for asset properties (to confirm that data is processing as expected)
  • Option to restart the gateway software. Restarting or power cycling the gateway restarts all software services and fetches the latest asset model definitions from the cloud service

To use the OpsHub for AWS IoT SiteWise application download and install it from here. To sign in, follow these steps:

  1. Enter password created during gateway setup.
  2. Select the certificate obtained from the gateway.
  3. Enter the IP address of the gateway on your local factory network. Make sure both port 8443 and 443 are open and externally accessible on the gateway.

Consuming data in local applications

Once you have setup your gateway and configured asset models for edge, you can access data directly from the gateway using GET APIs in the AWS IoT SiteWise SDK. You can also use the OpsHub for AWS IoT SiteWise application to browse a list of AWS IoT SiteWise Monitor portals available on the gateway and launch them in a web browser.

Querying data from the gateway

You can use AWS IoT SiteWise to get data from the gateway. You can use the AWS CLI or Postman app to test the APIs to confirm the gateway is processing data as expected. A generic http client like Postman would require a service name and region to work. Use service name as “iotsitewise” and region as “Edge.”

To get data from the gateway, you will need to configure your AWS IoT SiteWise SDK:

  1. Configure the SDK to use the access-key-id and secret-access-key provided during the gateway setup step above. Here is an example on how to do this in node.js.
  2. Register your gateway certificate with your SDK. Here is an example to do this in node.js.

If the gateway is processing the data, the results of an GetAssetProperty() call should include computed values for transforms and metrics. You can also see the latest value computed using the Asset Models service in OpsHub for AWS IoT SiteWise application.

Accessing AWS IoT SiteWise Monitor portals

You can see a list of AWS IoT SiteWise Monitor portals in the OpsHub for AWS IoT SiteWise application. Once connected, navigate to SiteWise Monitor Portals item in the services menu to see a list of portals available on the gateway. You can launch the SiteWise Monitor portal in your web browser. You can bookmark this link or share it with your colleagues. The dashboards can be accessed through these links from a web browser on any machine on the same network as the gateway.

Sample Wind Farm Monitoring Dashboard

Note: you may have to accept self signed certificates from both https 443 and 8443 ports to be able to access the dashboards in your web browser.

Conclusion

With AWS IoT SiteWise Edge you can skip building complex data processing infrastructure, and can jump right into developing compelling applications for process engineers and technicians on the factory floor. In building AWS IoT SiteWise, we take a number of common use cases into account. Customers tell us it is often helpful to start the Industrial IoT journey by enabling a few “hero” capabilities. These capabilities can demonstrate the benefits of Industrial IoT to stakeholders and discover additional opportunities to improve operations on the factory floor. Here are few ideas that you can tackle with the service today to quickly assess and demonstrate value of Industrial IoT to your organization.

Manage efficiency of your factory: Minimizing waste in end to end processes is the primary objective of lean manufacturing. With AWS IoT SiteWise, you compute metrics such as Overall Equipment Effectiveness (OEE), Mean Time Between Failures (MTBF), and Mean Time to Resolution (MTTR) from equipment state, quality count, and throughput data collected. These metrics help you can track the progress of your lean manufacturing initiatives and find room for improvement. Now with AWS IoT SiteWise Edge, you can compute these metrics entirely in the gateway for local monitoring dashboards that must continue to work even if the connection of the factory to the cloud is interrupted, ensuring that factory staff are able to identify and root-case every bottleneck as soon as it arises. You can send the computed metrics to the cloud from all your production operations to compare your efforts across multiple sites.

Improve product quality with real-time analytics: Catching product defects during the manufacturing process quickly can help reduce waste arising from units that must be rejected due to quality issues. By combining data from equipment, secondary sensors, enterprise resource planning (ERP) systems, and manufacturing execution systems (MES) data sources you can develop systems to catch defect causing conditions in real-time. With AWS IoT SiteWise Edge, you can read data from the gateway as it is collected and further process through machine learning models in real time to identify anomalies which you can then use to trigger alerts for staff on the factory floor. Learn how Terex developed an Industrial IoT solution with AWS IoT SiteWise to identify inconsistent and improper pretreatment parameters in near real-time of their painting process for Genie lifts.

Test products rigorously, yet cost-effectively: Often automotive, electronics, and aerospace products need to be acceptance tested before they are shipped to the customer. For example, a heavy duty truck engine may be run at full-throttle to ensure readiness for the road. Such tests can generate thousands of data points per second from multiple sensors embedded in the product and the testing equipment. While this data needs to be processed in real-time for the test operators, not all of it may be needed longer term. For example, it might be sufficient to store a single data point per second for each of the sensors. With AWS IoT SiteWise Edge, you can process data locally for real-time dashboards and store just the results in the cloud to optimize your bandwidth and storage costs.

To get started, head over to https://aws.amazon.com/iot-sitewise/. We look forward to feedback on our setup experience, data processing capabilities, and gateway monitoring tools.

About the author 

Usman is a Principal Product Manager on the AWS IoT team working at the intersection of edge computing and industrial software. He recently re-joined AWS from Astra, where he led efforts to build software and IT infrastructure for defining, manufacturing, testing, and launching orbital class rockets. Prior to Astra, Usman launched and grew AWS IoT SiteWise in preview. He is passionate about using Day 1 approaches to tackle product challenges in manufacturing, robotics, and aerospace.