Use AWS IoT Device Defender and Splunk to monitor the security posture of your IoT application

With the growing adoption of Internet of Things (IoT) applications in regulated industries, such as healthcare, hardening IoT security devices has become a requirement. In addition to ensuring that backend systems are resilient, organizations increasingly invest effort to secure devices outside the traditional enterprise perimeter with zero trust principles. For example, fleet operators for connected medical devices need to ensure that the product does not exhibit anomalous behavior and function as designed. When a device’s security posture is compromised, it’s vital that these events are efficiently identified, analyzed, and managed by a centralized security team to safeguard the delivery of end-to-end patient care.

AWS IoT Device Defender, a fully managed cloud service, continuously monitors IoT fleets to detect any abnormal device behavior, trigger security alerts, and provide built-in mitigation actions. This service can audit device-related resources against AWS IoT security best practices, and evaluate device-side and cloud-side metrics in near real-time against a predefined threshold. You can then receive alerts when AWS IoT Device Defender detects deviations. AWS IoT Device Defender also has a feature called ML Detect that monitors metrics in near real-time, and applies machine learning (ML) algorithms to detect anomalies, and to raise alerts.

AWS Partners, such as Splunk, provide security information and event management (SIEM) solutions that enable organizations to detect and respond to incidents in near real-time. A security solution that integrates AWS IoT Device Defender with the Splunk Platform can enhance your organization’s security posture by delivering data-driven cyber security to end-to-end IoT applications.

In this blog, we illustrate how you can use AWS IoT Device Defender, Amazon Data Firehose, and the Splunk Platform to ingest security-related metrics from IoT devices into a centralized SIEM. We also discuss how you can configure the security system to quickly identify risks and systematically measure their impact.

Solution overview

This is a fully serverless solution consisting of AWS IoT Core, AWS IoT Device Defender, Amazon Data Firehose, and the Splunk Platform.

Figure 1: Solution architecture

The solution’s primary audience:

• IoT application developers are responsible to develop and release new features. Their objective is to maximize their time writing robust code that delivers business value. While security is paramount, they don’t want to spend time writing custom code that extracts, processes, and transmits metrics that are relevant for security professionals to analyze system operations.
• Security operations center (SOC) analysts are responsible to identify and react to security threats, and safeguard business operations. They use centralized SIEM tooling to monitor and gather intelligence on near real-time risks. They also enact manual and automated processes to strengthen the organization’s security posture.

How this solution works

1. The IoT application is built using the AWS IoT Device Client so that supported device-side metrics are sent automatically. The SDK publishes these metrics to AWS IoT Core Message Queueing Telemetry Transport (MQTT) topics reserved for use by AWS IoT Device Defender. Supported device-side metrics include established TCP connections count, listening TCP ports, destination IP addresses, and the number of outbound packets.
2. AWS IoT Device Defender processes device-side metrics alongside cloud-side metrics. Supported cloud-side metrics include number of authorization failures, source IP address, connection attempts, message size, messages sent, messages received, disconnects, and disconnect duration. Cloud-side metrics are generated regardless of the presence of device-side metrics.
3. The security profile of AWS IoT Device Defender’s detect feature is configured to publish the metrics to a user-defined MQTT topic. You can use this feature to configure rules and actions in AWS IoT Core to process and forward the metrics to other event consumers.
4. AWS IoT Core rules and actions are then configured on the MQTT topic to send the metrics to an Amazon Data Firehose delivery stream. In this design, Firehose provides a scalable data streaming pipeline that is capable of batching, buffering, and transforming payloads.
5. AWS IoT Device Defender’s audit feature can send audit findings to an Amazon Simple Notification Service (Amazon SNS) topic. The Amazon Data Firehose delivery stream subscribes to the Amazon SNS topic and receives the audit reports in its stream. Supported audit checks include monitoring overly permissive roles, shared device certificates, and conflicting MQTT client IDs.
6. The solution then uses an AWS Lambda function within the streaming pipeline to transform the source records into a format that the SIEM solution can digest. This example adds a unique sourcetype key to the payload and restructures it under an event key. This makes the events easier to index and identify when searching through Splunk’s Search Processing Language (SPL). Lambda provides flexibility to modify the data structure to align with downstream consumer requirements. For example, the Lambda function could further enrich the data by pulling device ownership information from a configuration management database (CMDB).
7. Amazon Data Firehose sends events to supported destinations. Both device-side and client-side metrics, as well as audit findings, are ingested into the SIEM solution via the Amazon Data Firehose delivery stream.
8. SIEM solutions, such as Splunk, support log ingestion from a large number of sources, including other AWS services, cloud workloads, and on-premises workloads. This holistic data aggregation allows the SOC to have full visibility into the organizational security posture – not just the silos where they have access.
9. SOC analysts can use the array of features available in an overarching SIEM solution. For example, if you use the Splunk Platform, you can use Enterprise Security and Security Orchestration, Automation and Response (SOAR) to explore, analyze, and react to incoming data.  You can use dashboards to visualize device-side and cloud-side metrics alongside other logs. You can use queries to aggregate, enrich, and search through the metrics. You can also automate responses using playbooks. For example, if a network port is unintentionally left open, you can detect if a device’s security posture has been weakened. If it has, you can assess the risk to the wider environment.

Deploying the solution

An AWS Serverless Application Model (SAM) template is available to deploy all AWS resources required by this solution, including the Python code used by the Lambda function. This template can be found in the aws-iot-device-defender-and-splunk GitHub repository.

Refer to the README file for required prerequisites, deployment steps, and ways to test the solution.

AWS IoT Device Defender configurations

Once the solution is deployed, AWS IoT Device Defender configurations facilitate the metrics and audit reports publishing to Firehose.

Metrics

Navigate to the AWS IoT Console. Expand Detect in the Navigation pane and the choose Security profiles. Notice there is a security profile for you. The Additional metrics to retain tab contains a list of preconfigured metrics.

Figure 2: Viewing additional metrics to retain

From the Exported metrics tab, you will also see that these metrics are exported to a predetermined MQTT topic.

Figure 3: Viewing exported metrics

Audits

Navigate to the Settings page under Audit. The solution has enabled all audit checks and the results are published to a designated SNS topic.

Figure 4: Viewing audit settings

Analyzing the events

Once the security data is ingested into the SIEM solution, the SOC analyst works to understand and assess the risks presented within their environments. In this example, we use the Splunk Processing Language (SPL) to perform this analysis.

Metrics

Once the solution generates data, navigate to the Search & Reporting Splunk App in the Splunk console, and use the following SPL query:

index="<YOUR INDEX>" sourcetype="<YOUR SPLUNK SOURCE TYPE>"

The search returns all cloud and client-side metrics generated by AWS IoT Device Defender and to prove that the data is ingested into the chosen index.

Now write a new SPL query to monitor the aws:num-listening-tcp-ports value over time, by device. Use the following query:

index="<YOUR INDEX>" sourcetype="<YOUR SPLUNK SOURCE TYPE>" | spath name | search name="aws:num-listening-tcp-ports"
| chart max(value.count) as tcp_count over _time by thing

This query demonstrates that the total count of open TCP ports has changed on a single device, which warrants a deeper investigation by a security analyst.

Figure 5: Displaying total number of open TCP ports

Using the name of the device exhibiting suspicious behavior, run another SPL query to determine which ports may be open.

index="<YOUR INDEX>" sourcetype="<YOUR SPLUNK SOURCE TYPE>" | where thing="<YOUR THING NAME>"
| spath name
| search name="aws:listening-tcp-ports"
| spath value.ports{} output=open-ports
| mvexpand open-ports
| chart count(open-ports) over _time by open-ports

Figure 6: Displaying open TCP ports on device

The security analyst can now further interrogate other data points, such as aws:all-packets-out or aws:all-bytes-out, to see if there may be other data exfiltration indicators. These data points can be assessed alongside data from other devices (such as network switches, routers, and workstations) to provide a complete picture of what might have happened to this device and the level of risk posed to the organization.

Audits

Audits can be scheduled or run immediately. In the AWS IoT Core console, navigate to Audit, then Results, and choose Create. Select Available checks and select Run audit now (once), under Set schedule, and choose Create.

The security analyst can track the status of the historical audit reports over time using SPL similar to the following:

index="<YOUR INDEX>" sourcetype="<YOUR SPLUNK SOURCE TYPE>" | where isnotnull(checkName)

Figure 7: Displaying audit reports

Conclusion

This post demonstrated how AWS IoT Device Defender’s export metrics and audit features, together with Amazon Data Firehose and Splunk’s platform can be used to ingest security data from IoT devices at scale. By using SIEM solutions, such as the Splunk Platform, SOC analysts can assess the risk to the business from deployed IoT devices, and make informed decisions on how to best safeguard business continuity. To learn more about how AWS IoT Device Defender can be used to manage the security of your IoT fleet, see AWS IoT Device Defender.

Author bio