Limiting requests to a web application using a Gatekeeper Solution

Many types of organizations encounter DDOS attacks daily. DDoS, or Distributed Denial of Service, is an attack pattern that generates fake traffic to overwhelm organisation’s properties (websites, APIs, applications, etc.) and make it unavailable to legitimate users. They majority of these attacks involve a huge burst of fake traffic that exhausts the application’s resources (Memory and CPU). Nevertheless, there are instances of DDOS attacks that occur at an unusually low rate. In such cases, the attacker targets services that are built to serve low levels of traffic. The attacker distributes traffic into smaller chunks of requests but spreads it to a larger number of fake requesters. The request rate is small enough to evade throttling mechanisms implemented by firewalls but sufficient to make application unavailable. To tackle these low-rate attacks, we built a utility that functions as a gatekeeper for your application.

Why do we need rate limits?

This section highlights the necessity of implementing a customized rate limit system that is not met by standard Web Application Firewalls (WAF). Our customers from domains like EdTech’s or Educational Technology platforms, receive a predictable volume of requests on their pages. While the request rate varies per API, it remains relatively low. Attackers typically target these APIs to disrupt the service. Despite the low rate of requests, attacks exhibit anomalous behaviour, making them easily detectable.

The issues with these attacks are:

• They last for a very short window of time.
• The attack patterns change frequently.

The options available for preventing these attacks are:

1. Deploy WAF on entry points for overall traffic:
   1. This will work if the sum of all of the requests breach WAF minimum throttles.
   2. It will make sure that the back end is protected from attacks.
   3. But this may stop genuine users from accessing the website at the time of the attack.
2. Run anomaly detection algorithms on access logs and add detected IPs to a blocklist.
   1. This would be the best option, as we can pinpoint the attacker.
   2. But since the attack duration is low, by the time we detect the anomaly the attack has finished or changed pattern.
3. Adding AWS WAF CAPTCHA:
   1. CAPTCHA helps differentiate humans from attackers and block it.
   2. But some customers don’t like how this adds a step for the end-user.

All of this together is why we built the Gatekeeper Solution described in the rest of this post.

What to expect?

This guide helps you understand how we built the Gatekeeper mechanism and how to deploy it on Amazon CloudFront. It limits requests to your web applications or APIs and prevents the attacks mentioned earlier.

This will help you:

1. Block attacks with ultra-low rate limit restrictions. This protection either applies to all requests, or only to a set of requests that match path patterns (such as protecting certain APIs only).
2. Allow only certain requests for an interval and show static content/response to the rest, so it does not flood the application server with requests. During the flood scenario, we serve the static version of the webpages from Amazon Simple Storage Service (Amazon S3).

What’s the trick?

We have two serverless functions deployed as Lambda@Edge (L@E) functions on CloudFront. It’s a combination of a sliding window and blanket throttle.

The first L@E function helps you protect endpoints with rates that are estimated at the application level. This is done with a single throttle value. For example, login APIs in EdTechs authenticate the user to the platform. We invoke this once per user-device. Hence, RPS or Requests per second is easily estimated at the application level.

Second, L@E helps you impose a sliding window for each IP. We use this for APIs, which need very few requests but can vary for users. For example, entitlement APIs in EdTech are used to check if the user may watch a tutorial. This is invoked only once per video-content, and the number of requests stays fairly low. Adding a sliding window prevents these attacks.

Now, once we detect an attack, instead of returning a 429 or 4XX, we return a valid dummy response. This is to make sure the attacker isn’t aware that their request is being throttled. Even if the attack continues, we serve them a static page from CloudFront.

Architecture

The solution presented in this blog post uses L@E deployed at origin-request on an Amazon CloudFront and manipulates request URL based on session details stored in Amazon DynamoDB, as shown in Figure 1.

Figure 1: Reference architecture for the Gatekeeper solution

Prerequisites

To deploy this Gatekeeper application, you need:

1. AWS Command Line Interface (AWS CLI) installed.
2. AWS credentials with permissions to create the resources like Amazon DynamoDB, AWS Identity and Access Management (IAM) Role, AWS Lambda, etc.
3. AWS Management Console Access.
4. A dummy response (webpage/file), which has to be presented in case of an attack.

Note:

If you don’t have AWS CLI installed, this Getting Started page contains the information that you must know to download and configure the AWS CLI. You must have Python (any version from 2.6.x up to 3.3.x) installed if you are on Linux or OS X. You can install the CLI using easy_install, pip, or from a Windows MSI.

You can set your AWS credentials for the CLI using environment variables or by using a configuration file. If you’re running the CLI on an Amazon Elastic Compute Cloud (Amazon EC2) instance, then you can also use an IAM role.

Steps to deploy the application

Create DynamoDB Table

Use the following AWS CLI command to create a DynamoDB database in our account:

$ aws dynamodb create-table \
    --table-name cf-request-counter-table\
    --attribute-definitions \
        AttributeName=PId,AttributeType=S \
        AttributeName=RequestTimeStamp,AttributeType=N \
    --key-schema \
        AttributeName=PId,KeyType=HASH \
        AttributeName=RequestTimeStamp,KeyType=RANGE \
    --provisioned-throughput \
        ReadCapacityUnits=5,WriteCapacityUnits=5 \
    --table-class STANDARD

Create an IAM Role

You must create a file with the name Edge-Role-Trust-Policy.json with the following content:

{
  "Version": "2012-10-17",
  "Statement": {
    "Effect": "Allow",
    "Principal": {
"Service": [
              "lambda.amazonaws.com",
       "edgelambda.amazonaws.com"
       ]},
    "Action": "sts:AssumeRole"
  }
}

Then, use the following AWS CLI command to create an IAM role in our account:

$ aws iam create-role \
    --role-name EdgeFunctionRole \
    --assume-role-policy-document file://Edge-Role-Trust-Policy.json 
 Note: Copy and save the role ARN to be consumed while creating Lambdas.

Next, attach role policies using the following AWS CLI commands:

$ aws iam attach-role-policy \
    --role-name EdgeFunctionRole \
    --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole    

$ aws iam attach-role-policy \
    --role-name EdgeFunctionRole \
    --policy-arn arn:aws:iam::aws:policy/AWSXrayWriteOnlyAccess

$ aws iam attach-role-policy \
    --role-name EdgeFunctionRole \
    --policy-arn arn:aws:iam::aws:policy/AmazonDynamoDBFullAccess

Create Lambda functions for low throttle rate

Create a Project directory and create a file named index.js with the following code:

'use strict';
const DURATION_FOR_THROTTLE = 30, //Update your value in seconds
REQUEST_COUNT_FOR_THROTTLE = 5,   //Update your value
CUSTOM_REDIRECT_URL = "YOUR_DUMMY_RESPONSE_HTML_URL", //Update your value
DYNAMO_DB_TABLE_NAME = "cf-request-counter-ddbtable"; //Update your value

var AWS = require("aws-sdk");
AWS.config.update({ region: "REGION_OF_DYNAMODB_TABLE" });
var docClient = new AWS.DynamoDB.DocumentClient();

const redirectResponse = {
 status: '302',
 statusDescription: 'Moved Permanently',
 headers: {
  'location': [{
   key: 'Location',
   value: CUSTOM_REDIRECT_URL,
  }],
  'cache-control': [{
   key: 'Cache-Control',
   value: "max-age=0"
  }],
 },
};
var getRequests = async(clientip, duration) => {
 const myPromise = new Promise((resolve, reject) => {
  var params = {
   TableName: DYNAMO_DB_TABLE_NAME,
   KeyConditionExpression: "#pk = :clientip and RequestTimeStamp > :timestamp",
   ExpressionAttributeNames: {
    "#pk": "PId"
   },
   ExpressionAttributeValues: {
    ":clientip": clientip,
    ":timestamp": duration
   }
  };
  docClient.query(params, function(err, data) {
   if (err) {
    console.error("Unable to read item. Error JSON:", JSON.stringify(err, null, 2));
    reject(err);
   }
   else {
    console.log("GetItem succeeded:", JSON.stringify(data, null, 2));
    resolve(data.Count);
   }
  });
 });
 return myPromise;
}
var addnewRequest = async(clientip, current_time) => {
 const myPromise = new Promise((resolve, reject) => {
  var params = {
   TableName: DYNAMO_DB_TABLE_NAME,
   Item: {
    "PId": clientip,
    "RequestTimeStamp": current_time
   }
  };
  console.log("Adding a new item...");
  console.log(params);
  docClient.put(params, function(err, data) {
   if (err) {
    console.error("Unable to add item. Error JSON:", JSON.stringify(err, null, 2));
    reject(err);
   }
   else {
    console.log("Added item:", JSON.stringify(data, null, 2));
    resolve(data);
   }
  });
 });
 return myPromise;
}
exports.lambdaHandler = async(event, context) => {
 console.time("functionstart");
 const request = event.Records[0].cf.request;
 const ip = request.clientIp,
  current_time = new Date().getTime(),
  duration = current_time - (DURATION_FOR_THROTTLE * 1000);
 console.time("PutDDBfunction");
 var new_request = await addnewRequest(ip, current_time);
 console.log(new_request);
 console.timeEnd("PutDDBfunction");
 console.time("getDDBfunction");
 var old_requests_count = await getRequests(ip, duration);
 if (old_requests_count > REQUEST_COUNT_FOR_THROTTLE) {
  return redirectResponse;
 }
 console.timeEnd("getDDBfunction");
 console.timeEnd("functionstart")
 return request;
};

Note: Make sure that you’re replacing the lines of the code which are commented "Update your value" with your parameters.

Now, in the same directory, zip this file into a package using the following command:

$ zip lambdaFunc.zip index.js

Note: Alternatively, you can use any zipping software to create zip.

Use the following AWS CLI command to create a Lambda function in our account:

$ aws lambda create-function \
    --function-name LowThrottleRateFunction \
    --zip-file fileb://lambdaFunc.zip --handler index.handler \ 
    --role "<YOUR_ROLE_ARN>" \
    --runtime nodejs12.x \
    --timeout 3 \
    --memory-size 128 \
    --region us-east-1
  Note: Make sure that you are creating L@E function in us-east-1.

Create Lambda Functions for request throttle

Create a new Project directory and create a file named index.js with the following code:

'use strict';
const DURATION_FOR_THROTTLE = 30,   //Update your value in seconds
    REQUEST_COUNT_FOR_THROTTLE = 5, //Update your value
    CUSTOM_REDIRECT_URL = "YOUR_DUMMY_RESPONSE_HTML_URL", //Update your value
    DYNAMO_DB_TABLE_NAME = "cf-request-counter-ddbtable", //Update your value
    APPLICATION_NAME = "mywebapplication",
    APPLICATION_ID = 100;
var AWS = require("aws-sdk");
AWS.config.update({ region: "REGION_OF_DYNAMODB_TABLE" }); 
var docClient = new AWS.DynamoDB.DocumentClient();
var getRequests = async(applicationname, applicationId) => {
    const myPromise = new Promise((resolve, reject) => {
        var params = {
            TableName: DYNAMO_DB_TABLE_NAME,
            KeyConditionExpression: "#pk = :PID and RequestTimeStamp = :timestamp",
            ExpressionAttributeNames: {
                "#pk": "PId"
            },
            ExpressionAttributeValues: {
                ":PID": applicationname,
                ":timestamp": applicationId
            }
        };
        docClient.query(params, function(err, data) {
            if (err) {
                console.error("Unable to read item. Error JSON:", JSON.stringify(err, null, 2));
                reject(err);
            }
            else {
                console.log("GetItem succeeded:", JSON.stringify(data, null, 2));
                resolve(data.Items[0]);
            }
        });
    });
    return myPromise;
}
var addnewRequest = async(applicationname, applicationId, count, timestamp) => {
    const myPromise = new Promise((resolve, reject) => {
        var params = {
            TableName: DYNAMO_DB_TABLE_NAME,
            Item: {
                "PId": applicationname,
                "RequestTimeStamp": applicationId,
                "RequestCount": count,
                "LastTimeStamp": timestamp
            }
        };
        console.log("Adding a new item...");
        console.log(params);
        docClient.put(params, function(err, data) {
            if (err) {
                console.error("Unable to add item. Error JSON:", JSON.stringify(err, null, 2));
                reject(err);
            }
            else {
                console.log("Added item:", JSON.stringify(data, null, 2));
                resolve(data);
            }
        });
    });
    return myPromise;
}
exports.lambdaHandler = async(event, context) => {
    console.time("functionstart");
    const request = event.Records[0].cf.request;
    const ip = request.clientIp,
        current_time = new Date().getTime(),
        duration = current_time - (DURATION_FOR_THROTTLE * 1000);
    var applicationname = APPLICATION_NAME,
        applicationId = APPLICATION_ID,
        count = 0,
        timestamp = 0;
    console.time("getDDBfunction");
    var old_requests_obj = await getRequests(APPLICATION_NAME, APPLICATION_ID);
    if (old_requests_obj) {
        var old_requests_timestamp = old_requests_obj.LastTimeStamp,
            old_requests_count = old_requests_obj.RequestCount;
        if (duration <= old_requests_timestamp) {
            if (++old_requests_count > REQUEST_COUNT_FOR_THROTTLE) {
                request.uri = CUSTOM_REDIRECT_URL;
                return request;
            }
            count = old_requests_count;
        }
        else {
            count = 1;
        }
    }
    else {
        count = 1;
    }
    timestamp = current_time;
    console.timeEnd("getDDBfunction");
    console.time("PutDDBfunction");
    var new_request = await addnewRequest(applicationname, applicationId, count, timestamp);
    console.log(new_request);
    console.timeEnd("PutDDBfunction");
    console.timeEnd("functionstart")
    return request;
};

Note: Make sure that you’re replacing the lines of the code which are commented "Update your value" with your parameters.

Now, in the same directory, zip this file into a package using the following command:

$ zip lambdaFunc.zip index.js 

Note: Alternatively, you can use any zipping software to create zip.

Use the following AWS CLI command to create a Lambda function in your account:

$ aws lambda create-function \
    --function-name RequestThrottleFunction \
    --zip-file fileb://lambdaFunc.zip --handler index.handler \ 
    --role "<YOUR_ROLE_ARN>" \
    --runtime nodejs12.x \
    --timeout 3 \
    --memory-size 128 \
    --region us-east-1

Note: Make sure that you are creating L@E function in us-east-1.

Update Lambda Function for low throttle rate

Now that we have our Lambda function, “LowThrottleRateFunction,” created, let’s configure CloudFront to run our function and monitor any request that CloudFront receives.

Let’s configure the CloudFront event for your function:

• Sign in to the Console and open the Lambda console.
• In the list of AWS Regions at the top of the page, choose US East (N. Virginia).
• In the list of functions, choose your function: LowThrottleRateFunction.
• Open your function.
• Choose Qualifiers, choose the Versionstab, and choose the numbered version to which you want to add triggers.
• Copy the ARN that appears at the top of the page, for example: arn:aws:lambda:us-east-1:123456789012:function:LowThrottleRateFunction:3 (The number at the end (3 in this example) is the version number of the function.)

Note: You can add event only to a numbered version, not to $LATEST.

• Open the CloudFront console.
• In the list of distributions, choose the ID of your distribution to which you want to add the L@E function.
• Choose the behaviors tab.
• Select the check box for the cache behaviour to which you want to add the L@E function, and then choose Edit.
• At Lambda Function Associations, in the Event Type list, choose origin-request
• Paste the ARN of the Lambda function that you copied earlier.
• Choose Yes, Edit.

Note: The function processes requests only after the CloudFront distribution is deployed.

Update Lambda function for request throttle

Now that we have our Lambda function, “RequestThrottleFunction”, created, let’s configure the CloudFront event to run our function to monitor any request that CloudFront receives.

Let’s configure the CloudFront event for your function:

• Sign in to the Console and open the Lambda console.
• In the list of AWS Regions at the top of the page, choose US East (N. Virginia).
• In the list of functions, choose your function: RequestThrottleFunction.
• Open your function.
• Choose Qualifiers, choose the Versionstab, and choose the numbered version to which you want to add event.
• Copy the ARN that appears at the top of the page, for example: arn:aws:lambda:us-east-1:123456789012:function: RequestThrottleFunction:3 (The number at the end (3 in this example) is the version number of the function.)

Note: You can add events only to a numbered version, not to $LATEST.

• Open the CloudFront console.
• In the list of distributions, choose the ID of your distribution to which you want to add these L@E function.
• Choose the behaviors tab.
• Select the check box for the cache behavior to which you want to add L@E function, and then choose Edit.
• At Lambda Function Associations, in the Event Type list, choose origin-request
• Paste the ARN of the Lambda function that you copied earlier.
• Choose Yes, Edit.

Note: The function processes requests only after CloudFront distribution is deployed.

Test Gatekeeper

To verify the functionality of Gatekeeper, you can invoke the URL on which Gatekeeper’s L@E functions are configured. It is important to ensure that the correct event invokes the function and that the response is valid for your application. The testing should be repeated until the minimum request rate set in the L@E function is reached. This blog post sets the minimum request rate to five requests in 30 seconds. Therefore, the sixth request will cause the display of the configured dummy page. The Gatekeeper uses the parameters “DURATION_FOR_THROTTLE” and “REQUEST_COUNT_FOR_THROTTLE” mentioned in L@E function, to make rate limiting decisions. Thus, we successfully tested and learned how Gatekeeper can protect your endpoints from low-rate attacks.

Cleanup

Use the following AWS CLI commands to delete resources from your account:

$ aws lambda delete-function \
    --function-name LowThrottleRateFunction \
    --region us-east-1

$ aws lambda delete-function \
    --function-name RequestThrottleFunction \
    --region us-east-1

$ aws iam delete-role \
    --role-name EdgeFunctionRole

$ aws dynamodb delete-table \
    --table-name cf-request-counter-table  Note: This step assumes that the names of the resources created are the same as those mentioned in the post. Use the same names provided while creating the resources.

Conclusion

This blog post has provided a comprehensive guide on implementing a custom rate limiting solution, GateKeeper, using Lambda@Edge and DynamoDB. We discussed the importance of rate limiting and the steps involved in configuring the same on L@E functions. By implementing Gatekeeper, you can protect your web application or API against low-rate attacks, ensure reliable access to your resources, and reduce the risk of downtime or performance issues. With AWS Lambda@Edge and DynamoDB, you can easily implement and manage a fast, scalable and cost-effective rate-limiting solution for your organization’s needs.