Building an IoT solution to securely transmit MQTT messages under private networks

To comply with different industrial regulations, private networks can help IoT service providers deploy a solution that offers critical data privacy and security encapsulation. This blog post discusses a use case from Internet of Things (IoT) service providers that use AWS services to enhance their private networks for MQTT message transmission and safeguard data transmission.

We will discuss how to build an IoT solution on AWS that can implement a full-chain, MQTT transmission sequence between IoT devices and AWS cloud services under a private network framework.

Introduction

Private networks enable you to maintain greater control over industrial IoT (IIoT) data. This offers you a compliance path to satisfy regulatory requirements regarding data privacy and security. By confining IIoT traffic within a private network infrastructure, you can demonstrate adherence to industry-specific regulations and standards, and reduce your risks. Private networks offer heightened security compared to public alternatives, and minimize the threat of unauthorized access, data breaches, and cyber-attacks. By isolating IIoT data within the private network infrastructure, sensitive industrial data remains shielded from external threats. By integrating private network-based MQTT channels into your IoT architecture on AWS, you can securely transmit crucial MQTT messages across devices and multiple AWS accounts while building industry-specific compliant solutions.

Solution overview

The architecture in Figure 1 shows how you can use AWS services to build an end-to-end private channel on AWS. This channel can transmit MQTT data between the IoT devices deployed in an on-premises data center and AWS IoT Core, and then between AWS IoT Core and a data consumers’ applications across AWS accounts. Multiple AWS accounts are included in this architecture since many IoT service providers prefer a multiple-account strategy on AWS.

Figure 1: The architecture of an end-to-end private MQTT channel.

The following describes the architecture of an end-to-end private MQTT channel from an IoT device perspective,

AWS Direct Connect establishes a secure and dedicated private connection between the on-premises network and the Amazon Virtual Private Cloud (this is the device VPC in the network account on AWS). The network-related resources are placed into the device VPC. (Note: The virtual private network (VPN) connection provided over the Internet is not considered in this scenario because the on-premises network is not allowed to access the Internet.)
Amazon Route 53 Resolver, located in the network account, enables Domain Name Service (DNS) resolution between the on-premises network and the Amazon VPC. The DNS queries from the on-premises private network are forwarded to the inbound endpoint and allows DNS queries to resolve within the device VPC.
AWS Transit Gateway uses attachments to connect the device VPC in the network account and the “gatekeeper VPC” in the AWS IoT account. The Transit Gateway allows traffic destined for the gatekeeper VPC to be routed from the device VPC through the Transit Gateway. Amazon VPC interface endpoints and the customer’s AWS Lambda functions reside in the gatekeeper VPC.
AWS Resource Access Manager shares the Transit Gateway in the network account with the AWS IoT account. After sharing, a Transit Gateway attachment can be created and tied to the device VPC, and communicate back to the Transit Gateway.
Amazon Route 53 private hosted zone, in the AWS IoT account, adds DNS records and provides private DNS names for the Amazon VPC interface endpoints. These endpoints include the AWS IoT Core data plane, AWS IoT Core credential provider, and AWS IoT Greengrass control plane. The hosted zone is associated with both the device VPC and the gatekeeper VPC to ensure that the private DNS names can be used by the IoT devices and the customer applications to access the endpoints.
AWS PrivateLink powers the connection between the gatekeeper VPC and AWS IoT Core without the use of an internet gateway. Amazon VPC interacts with endpoints that reside in the gatekeeper VPC and communicates with AWS IoT Core through the private connection.

The telemetry data from IoT devices is ingested through MQTT topics, into AWS IoT Core, and then to the end-to-end private data channel, which are powered by:

Direct Connect for the traffic between the on-premises network and the device VPC in the AWS network.
Transit Gateway for the traffic between the device VPC in the network account and the gatekeeper VPC in the AWS IoT account.
PrivateLink for the traffic between the gatekeeper VPC and AWS IoT Core.

Next, let’s discuss the Figure 1 architecture from the telemetry data consumer perspective,

Route 53 private hosted zone, in the legacy account or third-party account, adds DNS records and provides the private DNS names to the Amazon VPC interface endpoints.
The legacy and third-party applications run on Amazon Elastic Compute Cloud (Amazon EC2) instances within the VPC private subnets and in their own AWS account. The hosted zone associated with the VPC ensures that the application MQTT clients can use the private DNS names to access the interface endpoints.
PrivateLink powered service, known as an endpoint service, is created to share the Amazon VPC interface endpoints, in the gatekeeper VPC, to the legacy AWS account and third-party AWS account. The traffic from the endpoint service securely flows through the endpoints into the legacy and third-party applications.
The endpoint service requires either a Network Load Balancer (NLB) or a Gateway Load Balancer. In this solution, we use a NLB. The load balancer receives the traffic from the endpoint service and routes it to the Amazon VPC interface endpoints in the gatekeeper VPC.
The telemetry data consumers, residing in the legacy and third-party AWS accounts, ingests the data from the AWS IoT Core MQTT topics. This data is then transmitted in the end-to-end private data channel that is powered by PrivateLink and built on the endpoints.

Private DNS names for endpoints

Private DNS names are essential so that the MQTT clients can resolve the AWS IoT Core endpoint DNS name to the associated private IP addresses. The private DNS names are A records pointing to the endpoints in Route 53 private hosted zone associated with those VPCs. The following steps walk through how the private DNS name for the device VPC is created.

The Route 53 private hosted zone (privateiotchannel-ats.iot.us-west-2.amazonaws.com) is created in the AWS IoT account and the associated gatekeeper VPC. You can use the following command to associate the device VPC in the network account to the hosted zone. You need this command because you can’t use the Route 53 console to make the association across AWS accounts. You can also accomplish this by using AWS CloudFormation and AWS CDK.

% aws route53 create-vpc-association-authorization --hosted-zone-id Z076213424HT3P2H8VAU8 --vpc VPCRegion-us-west-2, VPCId=vpc-0c002950575f4ac06
{
  "HostedZoneId": "Z076213424HT3P2H8VAU8", 
  "VPC": {
    "VPCRegion": "us-west-2", 
    "VPCId": "vpc-0c002950575f4ac06"
  }
}

Figure 2: AWS CLI command to associate an Amazon VPC to a private hosted zone across accounts.

The A record, created in the hosted zone, points to the DNS name for the AWS IoT Core data endpoint. The DNS query, from the on-premises network, is forwarded to the device VPC through Direct Connect. The A record then resolves the query to the endpoint’s IP address. MQTT traffic from the on-premises network is routed to the gatekeeper VPC. This occurs after the Transit Gateway connects the device VPC to the gatekeeper VPC across accounts, and finally to the endpoint IP.

Figure 3: The A record for Amazon VPC endpoint defined in private hosted zone.

AWS IoT data interface endpoint

To ensure MQTT data traverses within private networks, an Amazon VPC interface endpoint is created in the gatekeeper VPC’s private subnets. The endpoint has one IP address in each subnet. In this case the endpoint has two private IP addresses (see Figure 4). MQTT traffic enters the endpoint and is routed to the private IP addresses.

Figure 4: Amazon VPC Interface endpoint for AWS IoT data in private subnets.

To reinforce AWS IoT Core to accept MQTT traffic only over private networks, you can attach the policy in Figure 5 to AWS IoT things. Access to AWS IoT Core is rejected if the access point is not the endpoint vpce-0fb5376e25d0e53d6.

{
  "Version": "2012-10-17", 
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "iot: Connect"
      ],
      "Resource": [
        "arn:aws:iot:us-west-2:123456789012: client/${iot: Connection. Thing. ThingName}"
      ],
      "Condition": {
        "StringEquals": {
          "aws: SourceVpce": "vpce-0fb5376e25d0e53d6"
        }
      }
    },
    ......
  ]
}

Figure 5: The IoT policy for only accepting connection through Amazon VPC endpoint.

In 2023, we launched support to create private AWS IoT Core Credential Provider endpoints in Amazon VPC and AWS IoT Greengrass control plane operations. In addition to enabling telemetry data transmission through private networks between the on-premises network and AWS IoT Core within the architecture, you can perform the following operations using private networks,

Operations requiring AWS IoT Core credential provider; for example, device fleet provisioning.
Operations requiring AWS IoT Greengrass control plane; for example, AWS IoT Greengrass component deployment.

Endpoint service and NLB configuration

You can share an interface only with the data consumers. (You can use this option instead of using Transit Gateway, or linking your VPC with the help of Amazon VPC peering, to set up a connection in the network layer.) By using PrivateLink endpoint service, the IoT service provider can avoid complex network configurations used to restrict the data consumers from accessing the endpoints for AWS IoT Core credential provider and AWS IoT Greengrass control plane.

In Figure 6, the endpoint service is established within the AWS IoT account and associated to the load balancer. This configuration enables the endpoint service to distribute MQTT data to the load balancer. Based on the unique service name provided by AWS for the endpoint service, both applications in the legacy and third-party accounts can establish private connections to the endpoint service by creating an interface VPC endpoint.

Figure 6: Endpoint service to share endpoint for AWS IoT Core data across AWS accounts.

The load balancer in Figure 7 extends across two private subnets that are within the gatekeeper VPC. This load balancer uses the two private IP addresses designated for the AWS IoT data endpoint in Figure 4. Through this configuration, the load balancer facilitates data distribution to AWS IoT Core through the endpoint.

Figure 7: Connecting endpoint service to endpoint for AWS IoT Core data.

To consume the data, it is important to create Amazon VPC interface endpoints for the legacy applications and third-party applications. Then point the endpoints to the PrivateLink endpoint service and set up the private DNS names for the endpoints in the legacy account and the third-party account. After that, the applications can use the private DNS names in their MQTT clients to access AWS IoT Core MQTT topics through private networks.

Summary

By leveraging the private network architecture introduced in this post, you can implement private network-based MQTT channels for IIoT data transmission within your IoT platforms. You can also safeguard against potential revenue loss from IIoT data pollution, cultivate reliability and low latency of data transmission, and enhance your IoT platform’s security posture. Beyond risk mitigation, adopting the private network architecture helps to maintain data privacy and comply with regulations such as, General Data Protection Regulation (GDPR), Health Insurance Portability and Accountability Act (HIPAA), or Payment Card Industry Data Security Standard (PCI DSS).

We look forward to seeing how you enable private networks for MQTT to reinforce the data security of your IoT platforms built on AWS. Get started with AWS IoT by going to AWS Management Console.

About the author

Shi Yin is a senior IoT consultant with AWS Professional Services and is based in California. Shi has worked with many enterprise customers to leverage AWS IoT services to build IoT solutions and platforms; for example, Smart Homes, Smart Warehouses, Connected Vehicles, Commercial IoT, and Industrial IoT.

The Internet of Things on AWS – Official Blog