Update: June 28, 2024 – Corrections were made to Figure 5 and the subsequent packet walkthrough. 

AWS Cloud WAN is a managed wide-area networking (WAN) service that you can use to build and operate wide area networks that connect your data centers and branch offices, as well as your Amazon Virtual Private Cloud (Amazon VPC) VPCs. You can use network policies to centrally configure and automate network management and security tasks, and get a complete view of your global network. Since the launch, customers increasingly recognize AWS Cloud WAN for its ability to create a global network and facilitate connectivity among resources. However, with this emphasis on connectivity comes the need for security inspection functions when using AWS native or third-party firewalls. Customers are increasingly seeking ways to inspect and safeguard their networks, prompting them to look for guidance about designing secure global networks using AWS Cloud WAN. To help customers learn, we have already published several blog posts about how to architect a secured global network using AWS Cloud WAN.

Challenges with the traditional security inspection approach

While the existing inspection architecture patterns offer many benefits, customers have faced challenges, especially when implementing at scale with AWS Cloud WAN:

• Because inspection segments are being shared, the VPC route using Classless Inter-Domain Routing (CIDRs) are dynamically propagated with respective VPC attachments as the next hop into the inspection segment route table. Thus, to route traffic to the respective firewall, it was necessary to establish a static route in the AWS Cloud WAN’s inspection segment route table for every newly added VPC.
• Multi-AWS Region geocentric inspection architecture complexities as covered in Achieve optimal routing with AWS Cloud WAN for multi-Region networks.
• With one global inspection segment, there wasn’t a way to force traffic to route through firewalls in the respective Region with the existing solution. To meet the requirement of inspecting the traffic in the respective AWS Region and to maintain symmetric routing, the customer had to create a unique global shared inspection segment for each Region or edge location.
• Each unique shared inspection segment is counted against the segments per core network AWS Cloud WAN segment quota.

In this blog post, we will look at AWS Cloud WAN service insertion (service insertion), a new capability that you can use to insert AWS and third-party networking and security services on AWS Cloud WAN using a central policy document, thereby providing increased operational simplicity. Using this feature, you can easily steer VPC-to-VPC, internet egress and hybrid environment traffic through network or security appliances by defining simple policy statements or using the AWS Management Console. This feature also supports policy-based traffic steering to AWS services such as AWS Network Firewall and Gateway Load Balancer that are deployed in inspection VPCs for east-west and north-south security inspection, allowing seamless integration of your security infrastructure with rest of your AWS Cloud WAN deployment.

Benefits of service insertion

Customers deploy centralized security architectures using AWS Cloud WAN in order to consolidate security resources, reduce their management burden and save security infrastructure costs. Security inspection functions can include protecting both east-west (VPC-to-VPC), north-south (internet egress) or hybrid environments.

• Simplified routing for network services insertion: Customers often need inter-VPC or VPC-to-internet or on-premises traffic to be routed through network appliances such as network firewalls or load balancers. With AWS Cloud WAN service insertion, customers can easily steer network traffic of interest to network or security appliances deployed in VPCs without having to create and manage complex routing configurations or use third-party automation tools.
• Ease of deploying multi-Region security inspection: Most AWS Cloud WAN customers deploy multi-Region networks to support Region expansion or disaster recovery use-cases. The service insertion capability simplifies multi-Region deployments and allows you to steer both intra-Region and inter-Region traffic through your security infrastructure without needing complex multi-Region network configurations.

Key concepts

The blog post assumes you’re familiar with the following AWS Cloud WAN components: global network, core network, core network policy, attachments, core network edge and network segment.

Besides the concepts mentioned above; in order to grasp the architectures outlined in this blog post, it is important to understand a new component of AWS Cloud WAN service insertion feature called network functions groups (NFG).

Network functions groups

An NFG, under the hood is a single segment consisting of a collection of core network attachments that point to specialized network or security functions such as NATs, firewalls (AWS Network Firewall or Gateway Load Balancer, or third-party services), intrusion detection and prevention systems and data loss prevention that are deployed as part of the global AWS Cloud WAN network.

NFG allows you to automatically steer same-segment or cross-segment traffic using network functions deployed in VPCs or on-premises networks attached to AWS Cloud WAN. You can specify an NFG that contains a set of core network attachments (VPC, Site-to-Site (VPN), connect, or transit gateway route table) where your network functions reside. You can then specify a segment or segment pairs for which traffic needs to be redirected to the network functions. AWS Cloud WAN then automatically re-directs network traffic between the segments through the specified core network attachments for the respective NFG. This redirection works for both same Region (intra-Region) and cross Region (inter-Region) traffic on the core network. You can use the AWS Cloud WAN Core network policy document to specify the main configuration items as follows:

1. An NFG that represents one or more network functions.
2. Core network attachments that are part of the NFG.
3. Segments with cross-segment or same-segment traffic that must be steered using the network functions (attachments in scenario 2).

It is also important to understand the key differences between Cloud WAN network segments and NFG.

To steer the traffic, NFG introduce the following segment actions:

• ‘send-via’: Use to steer traffic between workloads in the same segments or different segments through attachments in the NFG using next-hop override.
  • ‘single hop’: Traffic traverses a single intermediate attachment, using the deterministically preferred source or destination Region. You can set a list of Regions to use, as well as setting a preferred Region to use as a priority.
  • ‘dual-hop’: Traffic traverses inserted attachments in both the source and destination core network edges. For this option, the inspection attachment should be located in all Regions for each service insertion-enabled segment.
• ‘send-to’: use to steer egress traffic from workloads associated with segments to Internet or hybrid through attachments in the NFG using the default route (0/0, ::/0).

See the AWS Cloud WAN documentation to learn more about these concepts.

Figure 1: Service insertion NFG

Architectures

Here, we will focus on the following three scenarios:

1. Intra-Region, inter-segment inspection
2. Inter-Region, inter-segment dual inspection
3. Inter-Region, inter-segment single inspection

Scenario 1 – Intra-Region, inter-segment inspection

The diagram that follows (Figure 2) show scenario 1, where we might need to allow attachments in different segments in the same Region to communicate with each other, but only through an inspection appliance. For example, when your development team must push an engineering change into production. Prod VPC (10.1.0.0/16) and Dev VPC (10.11.0.0/16) are in the same Region (Region 1) but associated to the Prod and Dev segments respectively. Inspection VPC 1 (100.64.1.0/24), is also in the same Region.

Figure 2: Intra-Region, inter-segment inspection

Let’s do a walkthrough of the steps required to insert an NFG into the traffic path of Prod VPC 1 to Dev VPC 1 within the same Region (intra-Region):

Step 1: Using the sample policy shown below, you create an InspectionNFG network functions group:

  "network-function-groups": [
    {
      "name": "InspectionNFG",
      "require-attachment-acceptance": false
    }
  ],

Step 2: Using the sample attachment policy shown below, you then associate the Inspection VPC 1 attachment to the InspectionNFG network functions group:

  "attachment-policies": [
    {
      "rule-number": 100,
      "condition-logic": "or",
      "conditions": [
        {
          "type": "tag-exists",
          "key": "stage"
        }
      ],
      "action": {
        "association-method": "tag",
        "tag-value-of-key": "stage"
      }
    }
  ]

Step 3: Before configuring segment-actions to insert network function aka Inspection VPC 1 in the traffic path, you must make sure the Inspection VPC 1 attachment is created and associated with the InspectionNFG. You can create an attachment using sample AWS CLI shown below:

aws networkmanager create-vpc-attachment \
--core-network-id "<core-network-id>" \
--vpc-arn "<vpc-arn>" \
--subnet-arns "<subnet1-arn>" "<subnet2-arn>" \
--tags Key=stage,Value=InspectionNFG

Step 4: Now that you have created and associated Inspection VPC 1 attachment with Inspection NFG, use the sample segment-actions shown below to insert the InspectionNFG network functions group in inter-segment traffic:

  "segment-actions": [
    {
      "action": "send-via",
      "segment": "prod",
      "mode": "single-hop",
      "when-sent-to": {
        "segments": "*"
      },
      "via": {
        "network-function-groups": [
          "InspectionNFG"
        ]
      }
    }
  ],

Note that under “when-sent-to“, “segments”: “*” mean all segments – that is, any traffic sent between Prod and any other segments that you might in your setup (for example, Dev, Test, Stage, QA and so on). A “*” will also provide intra-segment service-insertion for isolated segments.

After you have inserted the Inspection VPC 1 network function in the traffic path, all inter-segment traffic between Prod and Dev segment (or any other segments that exist in the core network) will be automatically steered using core network attachments that have tag key stage and tag value InspectionNFG through the firewall in Inspection VPC 1 (Figure 3). AWS Cloud WAN will:

• Propagate routes from all workload segments into the InspectionNFG route-table with next-hops redirected to the appropriate workload attachments. Routes propagated in the same Region will have their next hop re-directed to the local network function group attachment. Note: In case of inter-Region routing, described in the subsequent scenarios, routes from a different Region will have their next hop redirected to the NFG attachment in the preferred Region which can be user-configured or system-default.
• For all segment pairs specified in the policy, propagate routes to the corresponding workload segment route-tables with a next hop as the local NFG attachment.

Figure 3: Intra-Region, inter-segment inspection traffic flow

Packet walkthrough:

The following steps describe a packet walkthrough when a resource in Prod VPC 1 communicates with a resource in Dev VPC 1 in the same Region:

• (A) When a resource in Prod VPC 1 starts a connection to a resource in Dev VPC 1, it does a Prod VPC 1 route table lookup. The packet matches the default route entry with the core network Amazon Resource Name (ARN) as the target and the packet gets routed to the core network.
• (B) When the packet arrives at the core network, because Prod VPC 1 is associated with the Prod segment, it does a Prod segment route table lookup with Region 1 as the edge location. The packet matches the Dev VPC 1 CIDR entry with the Inspection VPC 1 attachment as the target and the packet is routed to Inspection VPC 1.
• (C) After the firewall in Inspection VPC 1 has inspected the traffic and allowed it based on the configured security policy, the packet is routed back to the core network.
• (D) When the packet arrives at the core network, because Inspection VPC 1 is associated with the InspectionNFG, it does an InspectionNFG route table lookup with Region 1 as the edge location. The packet matches the Dev VPC 2 CIDR entry with Dev VPC 1 attachment as the target and the packet gets routed to Dev VPC 1.

Return traffic follows the same path in the opposite direction.

Scenario 2 – Inter-Region, inter-segment, dual inspection

The diagram that follows (Figure 4) show scenario 2. Scenario 2 is like scenario 1, except the Prod and Dev VPCs are located in different Regions. Prod VPC 1 (10.1.0.0/16) is located in Region 1 and associated with Prod segment and Dev VPC 2 (172.25.0.0/16) is located in Region 2 and associated with Dev segment.

Apart from the need to inspect traffic between attachments in the different segments, you might also need to treat each AWS Region as a separate protected environment. To achieve this, each Region has its own Inspection VPCs, Inspection VPC 1 (100.64.1.0/24) for Region 1 and Inspection VPC 2 (100.64.2.0/24) for Region 2. Like Scenario 1 (steps 1-3), we create an InspectionNFG network functions group and associate both the Inspection VPCs to the InspectionNFG.

Figure 4: Inter-Region, inter-segment, dual inspection

Using the segment-actions mode dual-hop shown below, you then insert the InspectionNFG NFG in the inter-Region, inter-segment traffic path. This enables to steering traffic to Inspection VPCs in each Region:

  "segment-actions": [
    {
      "action": "send-via",
      "segment": "prod",
      "mode": "dual-hop",
      "when-sent-to": {
        "segments": "*"
      },
      "via": {
        "network-function-groups": [
          "InspectionNFG"
        ]
      }
    }
  ],

After you have inserted the Inspection VPC 1 and Inspection VPC 2 network functions in the traffic path, all cross-segment traffic between the Prod and Dev segment (or any other segments that exist in the core network) will be automatically steered using core network attachments that have the tag key stage and tag value InspectionNFG through the firewall in Inspection VPC 1 and Inspection VPC 2, respectively. We show this in Figure 5 that follows.

Figure 5: Inter-Region, inter-segment, dual inspection, traffic flow

Packet walkthrough:

The following steps describe a packet walkthrough when a resource in Prod VPC 1 communicates with a resource in Dev VPC 2:

• (A) When a resource in Prod VPC 1 starts a connection to a resource in Dev VPC 2, it does a Prod VPC 1 route table lookup. The packet matches the default route entry with the core network ARN as the target and the packet gets routed to the core network.
• (B) When the packet arrives at the core network, because Prod VPC 1 is associated with the Prod segment, it does a Prod segment route table lookup with Region 1 as the edge location. The packet matches the Dev VPC 2 CIDR entry with the Inspection VPC 1 attachment as the target, and the packet is routed to Inspection VPC 1 in Region 1.
• (C) After the firewall in Inspection VPC 1 has inspected the traffic and allowed it based on the configured security policy, the packet is routed back to the core network.
• (D) When the packet arrives at the core network, because Inspection VPC 1 is associated with the InspectionNFG, it does an InspectionNFG route table lookup with Region 1 as the edge location. The packet matches the Dev VPC 2 CIDR with the Inspection VPC 2 attachment as the target, and the packet is routed to Inspection VPC 2 in Region 2.
• (E) After the firewall in Inspection VPC 2 has inspected the traffic and allowed it based on the configured security policy, the packet is routed back to the core network.
• (F) When the packet arrives at the core network, because Inspection VPC 2 is associated with the InspectionNFG, it does an InspectionNFG route table lookup with Region 2 as the edge location. The packet matches the Dev VPC 2 CIDR entry with Dev VPC 2 attachment as the target, and the packet is routed to Dev VPC 2.

Return traffic follows the same path in the opposite direction.

Scenario 3 – Inter-Region, inter-segment, single inspection

For a Region pair, service insertion allows you to select a Region for inter-segment traffic inspection. We default to the ordered list, to select the Region that’s used for the inter-Region, inter-segment traffic inspection. Using the segment-actions mode single-hop shown below, you then insert the InspectionNFG NFG in the inter-Region, inter-segment traffic path:

  "segment-actions": [
    {
      "action": "send-via",
      "segment": "prod",
      "mode": "single-hop",
      "when-sent-to": {
        "segments": "*"
      },
      "via": {
        "network-function-groups": [
          "InspectionNFG"
        ]
      }
    }
  ],

As shown in Figure 6, for the pair of Region 1 and Region 2, where Region 1 is us-east-1 and Region 2 is us-west-2, as per the ordered list,  Region 1 (us-east-1) is selected as the default Region for inspection. Thus, all the inter-Region traffic between Prod and Dev VPCs is routed through Inspection VPC 1 in Region 1 for inspection.

Figure 6: Inter-Region, inter-segment, single inspection, traffic flow

Packet walkthrough:

The following steps describe a packet walkthrough when a resource in Prod VPC 1 communicates with a resource in Dev VPC 2 using single inspection:

• The steps (A) through (C) remain the same as mentioned in Scenario 2, except that
• (D) When the packet arrives at the core network, because Inspection VPC 1 is associated with the InspectionNFG, it does an InspectionNFG Route Table lookup with Region 2 as the edge location. The packet matches the Dev VPC 2 CIDR entry asking to perform a InspectionNFG route table lookup with Region 2 as the edge location.

• (E) We do a InspectionNFG Route Table lookup with Region 2 as the edge location. The packet matches the Dev VPC 2 CIDR entry with Dev VPC 2 attachment as the target versus Inspection VPC 2 as the target. And the packet gets routed to Dev VPC 2.

Return traffic follows the same path in the opposite direction.

Considerations

• NFG is global in the sense that you can add attachments to it from any of the AWS Regions belonging to the core network. For example, if your core network operates across three AWS Regions (such as us-east-1, eu-west-1 and ap-west-1) you can add inspection or firewall VPCs from any of those three Regions in a single NFG.
• NFG consumes one global segment out of the total AWS Cloud WAN segments per core network quota.
• All attachment types are supported (VPC, VPN, connect, transit gateway) for both workload segments and NFGs.
• At the time of writing this blog post, dynamic routes advertised through dynamic attachment (VPN, connect, transit gateway peering) to a NFG are not installed in the NFG route table. Also, customers are required to advertise the default route (0/0, ::/0) over the dynamic attachment to the NFG for service insertion to work. See the AWS Cloud WAN documentation for more details.
• Multiple NFGs per core network are supported, however a single NFG per insertion within a segment-pair is allowed. You can associate an attachment to either a segment or an NFG but not both. Typically, workload attachments (aka workload VPCs) are associated with a segment and an inspection VPC (hosting firewalls, intrusion detection systems, and intrusion prevention system appliances) would be associated with an NFG.
• You can configure multiple NFGs in the same core network. However, you cannot insert multiple NFGs with the same segment or segment pair.
• If you need to inspect traffic between resources attached to the same segment, enable isolated mode for the respective workload segments. This setting ensures there is no direct connectivity between attachments associated with the segment bypassing the network functions that need insertion.
• AWS Cloud WAN appliance mode is orthogonal to service insertion and is required if you want to ensure stateful inspection through your security infrastructure in the VPC. You need to enable appliance mode on your inspection VPC attachments to ensure traffic in both directions for a particular network flow are steered to the same Availability Zone and as a result to the same security appliance for purposes of stateful inspection.
• AWS Cloud WAN supports both IPv4 and IPv6.
• There are no additional charges for using service insertion other than the regular AWS Cloud WAN charges.

Industry leading Partners

We had the pleasure to be working with leading industry partners at the launch of AWS Cloud WAN service insertion. We want to highlight the great things our partners have been doing and saying:

Conclusion

In this blog post, we discussed service insertion on AWS Cloud WAN, a new capability that lets you seamlessly insert AWS and third-party networking and security services on AWS Cloud WAN using a central policy document. Using this feature, you can easily steer VPC-to-VPC or VPC-to-on-premises traffic through network or security appliances by defining simple policy statements or using the console. We also covered the intra-Region and inter-Region inspection architectures using this new capability. For more information, see AWS Cloud WAN documentation.