New – Application Load Balancer Simplifies Deployment with Weighted Target Groups

One of the benefits of cloud computing is the possibility to create infrastructure programmatically and to tear it down when it is no longer needed. This allows to radically change the way developers deploy their applications. When developers used to deploy applications on premises, they had to reuse existing infrastructure for new versions of their applications. In the cloud, developers create new infrastructure for new versions of their applications. They keep the previous version running in parallel for awhile before to tear it down. This technique is called blue/green deployments. It allows to progressively switch traffic between two versions of your apps, to monitor business and operational metrics on the new version, and to switch traffic back to the previous version in case anything goes wrong.

To adopt blue/green deployments, AWS customers are adopting two strategies. The first strategy consists of creating a second application stack, including a second load balancer. Developers use some kind of weighted routing technique, such as DNS, to direct part of the traffic to each stack. The second strategy consists of replacing infrastructure behind the load balancer. Both strategies can cause delays in moving traffic between versions, depending on DNS TTL and caching on client machines. They can cause additional costs to run the extra load balancer, and potential delays to warm up the extra load balancer.

A target group tells a load balancer where to direct traffic to : EC2 instances, fixed IP addresses; or AWS Lambda functions, amongst others. When creating a load balancer, you create one or more listeners and configure listener rules to direct the traffic to one target group.

Today, we are announcing weighted target groups for application load balancers. It allows developers to control how to distribute traffic to multiple versions of their application.

Multiple, Weighted Target Groups
You can now add more than one target group to the forward action of a listener rule, and specify a weight for each group. For example, when you define a rule having two target groups with weights of 8 and 2, the load balancer will route 80% of the traffic to the first target group and 20% to the other.

To experiment with weighted target groups today, you can use this CDK code. It creates two auto scaling groups with EC2 instances and an Elastic Load Balancer in front of them. It also deploys a sample web app on the instances. The blue version of the web app is deployed to the blue instance and the green version of the web app is deployed to the green instance. The infrastructure looks like this:

You can git clone the CDK project and type npm run build && cdk bootstrap && cdk deploy to deploy the above infrastructure. To show you how to configure the load balancer, the CDK code creates the auto scaling, the load balancer and a generic target group. Let’s manually finish the configuration and create two weighted target groups, one for each version of the application.

First, I navigate to the EC2 console, select Target Groups and click the Create Target Group button. I create a target group called green. Be sure to select the correct Amazon Virtual Private Cloud (Amazon VPC), the one created by the CDK script has a name starting with “AlbWtgStack...“, then click Create.

I repeat the operation to create a blue target group. My Target Groups console looks like this:

Next, I change the two auto scaling groups to point them to the blue and green target groups. In the AWS Management Console, I click Auto Scaling Groups, select one of the two auto scaling groups and I pay attention to the name (it contains either ‘green’ or ‘blue’), I click Actions then Edit.

In the Edit details screen, I remove the target group that has been created by the CDK script and add the target group matching the name of the auto scaling group (green or blue). I click Save at the bottom of the screen and I repeat the operation for the other auto scaling group.

Next, I change the listener rule to add these two target groups, each having their own weight. In the EC2 console, I select Load Balancers on the left side, then I search for the load balancer created by the CDK code (the name starts with “alb”). I click Listeners, then View / edit rules:

There is one rule created by the CDK script. I modify it by clicking the edit icon on the top, then again the edit icon on the left of the rule. I delete the Foward to rule by clicking the trash can icon.

Then I click “+ Add Action” to add two Forward to rules, each having a target group, (blue and green) weighted with 50 and 50.

Finally, click Update on the right side. I am now ready to test the weighted load balancing.

I point my browser to the DNS name of the load balancer. I see either the green or the blue version of the web app. I force my browser to reload the page and I observe the load balancer in action, sending 50% of the requests to the green application and 50% to the blue application. Some browsers might cache the page and not reflect the weight I defined. Safari and Chrome are less aggressive than Firefox at this exercise.

Now, in the AWS Management Console, I change the weights to 80 and 20 and continue to refresh my browser. I observe that the blue version is displayed 8 times out of 10, on average.

I can also adjust the weight from the ALB ModifyListener API, the AWS Command Line Interface (AWS CLI) or with AWS CloudFormation.

For example, I use the AWS Command Line Interface (AWS CLI) like this:

aws elbv2 modify-listener    \
     --listener-arn "<listener arn>" \
     --default-actions        \
        '[{
          "Type": "forward",
          "Order": 1,
          "ForwardConfig": {
             "TargetGroups": [
               { "TargetGroupArn": "<target group 1 arn>",
                 "Weight": 80 },
               { "TargetGroupArn": "<target group 2 arn>",
                 "Weight": 20 }
             ]
          }
         }]'

Or I use AWS CloudFormation with this JSON extract:

"ListenerRule1": {
      "Type": "AWS::ElasticLoadBalancingV2::ListenerRule",
      "Properties": {
        "Actions": [{
          "Type": "forward",
          "ForwardConfig": {
            "TargetGroups": [{
              "TargetGroupArn": { "Ref": "TargetGroup1" },
              "Weight": 1
            }, {
              "TargetGroupArn": { "Ref": "TargetGroup2" },
              "Weight": 1
            }]
          }
        }],
        "Conditions": [{
          "Field": "path-pattern",
          "Values": ["foo"]
        }],
        "ListenerArn": { "Ref": "Listener" },
        "Priority": 1
      }
    }

If you are using an external service or tool to manage your load balancer, you may need to wait till the provider updates their APIs to support weighted routing configuration on Application load balancer.

Other uses
In addition to blue/green deployments, AWS customers can use weighted target groups for two other use cases: cloud migration, or migration between different AWS compute resources.

When you migrate an on-premises application to the cloud, you may want to do it progressively, with a period where the application is running both on the on-premises data center and in the cloud. Eventually, when you have verified that the cloud version performs satisfactorily, you may completely deprecate the on-premises application.

Similarly, when you migrate a workload from EC2 instances to Docker containers running on AWS Fargate for example, you can easily bring up your new application stack on a new target group and gradually move the traffic by changing the target group weights, with no downtime for end users. With Application Load Balancer supporting a variety of AWS resources like EC2, Containers (Amazon Elastic Container Service (Amazon ECS), Amazon Elastic Kubernetes Service (Amazon EKS), AWS Fargate), AWS Lambda functions and IP addresses as targets, you can choose to move traffic between any of these.

Target Group Stickiness
There are situations when you want the clients to experience the same version of the application for a specified duration. Or you want clients currently using the app to not switch to the newly deployed (green) version during their session. For these use cases, we also introduce target group stickiness. When target group stickiness is enabled, the requests from a client are all sent to the same target group for the specified time duration. At the expiry of the duration, the requests are distributed to a target group according to the weight. ALB issues a cookie to maintain target group stickiness.

Note that target group stickiness is different from the already existing target stickiness (also known as Sticky Sessions). Sticky Sessions makes sure that the requests from a client are always sticking to a particular target within a target group. Target group stickiness only ensures the requests are sent to a particular target group. Sticky sessions can be used in conjunction with the target group level stickiness.

To add or configure target group stickiness from the AWS Command Line Interface (AWS CLI), you use the TargetGroupStickinessConfig parameter, like below:

aws elbv2 modify-listener \
    --listener-arn "<listener arn" \
    --default-actions \
    '[{
       "Type": "forward",
       "Order": 1,
       "ForwardConfig": {
          "TargetGroups": [
             {"TargetGroupArn": "<target group 1 arn>", "Weight": 20}, \
             {"TargetGroupArn": "<target group 2 arn>", "Weight": 80}, \
          ],
          "TargetGroupStickinessConfig": {
             "Enabled": true,
             "DurationSeconds": 2000
          }
       }
   }]'

Availability
Application Load Balancer supports up to 5 target groups per listener’s rules, each having their weight. You can adjust the weights as many times as you need, up to the API threshold limit. There might be a slight delay before the actual traffic weight is updated.

Weighted target group is available in all AWS Regions today. There is no additional cost to use weighted target group on Application Load Balancer.

PS: do not forget to delete the example infrastructure created for this blog post and stop accruing AWS charges. As we manually modified an infrastructure created by the CDK, a simple cdk destroy will immediately return. Connect to the AWS CloudFormation console instead and delete the AlbWtgStack. You also need to manually delete the blue and green target groups in the EC2 console.