Protect against bots with AWS WAF Challenge and CAPTCHA actions

Protecting against bot threats requires insights into the client environment beyond what is available through network-level characteristics of a request, such as TCP or HTTP payload signatures. AWS WAF uses CAPTCHA and Challenge actions to undertake a client-side interaction, whether on a mobile device or browser, to understand this client environment before they can be allowed by AWS WAF. Challenge requires the client (browser or mobile device) to solve a problem silently before proceeding. CAPTCHA presents a problem in the user’s browser that requires a human to solve.

These actions complement the Allow, Count, and Block actions by providing additional visibility into the requesting client’s environment beyond what is available in a given network request. This visibility is a useful mechanism to identify and manage bot traffic while minimizing impact on legitimate users.

In this post, we will walk through how Challenge and CAPTCHA actions work and how you can use them to mitigate specific bot threats.

How Challenge and CAPTCHA actions work to identify and manage bots

Challenge and CAPTCHA actions involve a browser or mobile device interaction. This interaction is made up of three stages:

1. A challenge that requires a client to solve a problem silently (Challenge) or human interaction to solve a puzzle (CAPTCHA).
2. A client-side script that generates a fingerprint by interrogating the client environment.
3. AWS WAF validating the solution to the problem and using the fingerprint to generate a token that uniquely identifies a client session. This token is included in future requests to validate whether a request can pass through CAPTCHA or Challenge actions.

This interaction can be handled in two ways:

• AWS WAF Integration SDKs: Integrating your application with AWS WAF allows you to explicitly control how these stages fit in with your application and user experience. This is the approach recommended by AWS.
• Interstitial Interaction: AWS WAF completes the interaction stages by serving a custom response containing an interstitial that delivers the challenge during page load.

We now illustrate how Challenge and CAPTCHA actions interact with end users and clients for both of these cases.

How the Challenge action interacts with clients

The Challenge action runs a silent challenge in the client environment without requiring user interaction and is not intended to have a discernible impact on the user’s experience. The challenge requires a client to complete a computationally expensive task (proof of work). This approach intends to provide legitimate users with a seamless mechanism to validate their environment while increasing the cost for bot operators attempting to engage with your application.

Using the Challenge action without the integration SDK (Interstitial Interaction)

The Challenge action, by default, responds to HTML requests (Accept: text/html) with a JavaScript challenge payload that silently completes the challenge. The user is then automatically redirected back to the original page with a token that permits their request through subsequent rules requiring a Challenge. The workflow in Figure 1 illustrates this interaction.

Figure 1: Sequence diagram of the interstitial challenge interaction between a user, their browser, and AWS WAF protecting a resource

Using the Challenge action with the application integration SDK

If you don’t integrate with the application integration SDK then non-HTML requests that trigger a Challenge action returns an HTTP 202 response that breaks asset and fetch requests. Therefore, we recommend that you integrate with the application integration SDK so that the client can complete a challenge and get the token before submitting a request. The workflow in Figure 2 illustrates this interaction.

Figure 2: Sequence diagram of the challenge SDK interaction between a user, their browser, and AWS WAF protecting a resource

Integrating with the application integration SDK requires that you use at least one Amazon Managed Rule from AWS WAF Bot Control for Targeted Bots or AWS WAF Fraud Control. You can read our post, Use AWS WAF CAPTCHA to protect your application against common bot traffic, and our post, Prevent account creation fraud with AWS WAF Fraud Control – Account Creation Fraud Prevention. Two types of application integration SDK are available:

1. JavaScript SDK: This SDK is suited for browser-based applications, particularly Single Page Applications (SPA) where API endpoints are commonly protected against bots with AWS WAF rules. There are three parts to integrating with the JavaScript SDK that ensures a token is generated and available for each request:
   1. Required: Embed Javascript in the HTML head. This runs the challenge during page load.
   2. Recommended: Update your fetch calls to use the application integration SDK fetch wrapper: AwsWafIntegration.fetch. This ensures you always have a token before making a request. Alternatively, you can set the x-aws-waf-token header by getting the AWS WAF token using AwsWafIntegration.getToken().
   3. Recommended: Configure the token domain if your application makes requests across domains (for example www.example.com making an API request to api.example.com).
2. Mobile SDK: This SDK is available for both Android and iOS. The underlying workflow is similar to the JavaScript SDK as described in AWS Documentation.

Example: Protecting applications from DDoS threats using the Challenge action

Bot operators can target applications by using tens of thousands of bots to launch a small number of requests simultaneously. The total number of concurrent requests across all bots may still overwhelm the application as the requests per IP fall below the AWS WAF’s rate limit rule threshold of 100 requests per minute.

You can minimize the impact of these threats by configuring an AWS WAF rule to require a challenge to be completed before a request can be forwarded to your application. This requires a bot operator to complete proof of work and thus increases the cost for the operator. To setup your own AWS WAF rule that requires a challenge to be completed for all requests for the endpoint /account:

1. Open the AWS WAF console, and then do one of the following:
   1. To create a new web ACL, choose Create web ACL.
   2. To edit an existing web ACL, choose the name of the ACL.
2. On the Rules tab, for the Add Rules dropdown, select Add my own rules and rule group.
3. For the Rule configuration.
   1. Enter a Name for your rule.
   2. Leave the rule as Regular rule.
4. For the Statement configuration provide the following information as shown in Figure 3.
   1. For Inspect, select URI path.
   2. For Match type, select Exactly matches string.
   3. For the String to match, enter /account.
5. For the Action configuration, select Challenge.
   

   Figure 3: AWS WAF console configuration for Challenge action

6. To add the rule to your web ACL, choose Add rule.
7. Configure the rule priority for your web ACL, and choose Save.
8. Optional: integrate your application with the AWS WAF application integration SDK if you are also using an AWS WAF Bot Control or AWS WAF Fraud Control rule group.
   1. Return to the AWS WAF Console.
   2. Select Application Integration.
   3. For the Web ACLs that are enabled for application integration configuration.
      1. Select the region your webACL is deployed in from the drop down box.
      2. Select the Web ACL Name for your webACL.
      3. Expand the JavaScript SDK configuration.
      4. Copy the code to your application.
   4. Configure the SDK in your application as described earlier.

How the CAPTCHA action interacts with clients

The CAPTCHA action requires that a human completes a puzzle before the request can succeed. This puzzle is intended to prove that the user is a human. Figure 4 shows two types of puzzles that users are required to solve.

Figure 4: AWS WAF CAPTCHA problems

Using the CAPTCHA action without integration

If you don’t integrate your application with the CAPTCHA action then a request for HTML (Accept: text/html) that triggers a CAPTCHA action returns a page that presents the CAPTCHA puzzle, and solves a Challenge in the background. After solving a CAPTCHA, AWS WAF issues a token that permits subsequent requests through rules with CAPTCHA actions until the token expires. We discuss tokens in more detail below. The workflow in Figure 5 illustrates the interaction between the user, their browser, and AWS WAF.

Figure 5: Sequence diagram of the interstitial CAPTCHA interaction between a user, their browser, and AWS WAF protecting a resource

Using the CAPTCHA action with CAPTCHA JavaScript API

The CAPTCHA JavaScript API provides the ability for you to display the CAPTCHA puzzle within your frontend web application at any time before a request is sent to a resource protected by a CAPTCHA action. This also completes a Challenge in the background. This is the recommended approach as assets (images, Javascript) and fetch requests outside of text/html receives an HTTP 405 response code that may break your frontend web application and thus user experience. The workflow in Figure 6 illustrates the interaction required for an AWS WAF token to be generated.

We have a post, Use AWS WAF CAPTCHA to protect your application against common bot traffic, which details the process for integrating the CAPTCHA JavaScript API into your application, and a code sample for integrating the CAPTCHA JavaScript SDK into your React Frontend. AWS WAF CAPTCHA is not supported by mobile devices.

Figure 6: Sequence diagram of the CAPTCHA SDK interaction between a user, their browser, and AWS WAF protecting a resource

Example: Selectively requiring a human to complete a CAPTCHA based on IP reputation

IP reputation rules are useful for identifying and blocking IP addresses that are typically associated with bots and other threat traffic. For example, you can use the AWSManagedIPDDoSList rule to block IP traffic that AWS threat intelligence has identified as engaging in DDoS activities. This can be problematic if a legitimate user happens to share the same IP address, perhaps due to another device on their network being under the control of a bot operator. You instead permit legitimate users through this rule by requiring them to solve a CAPTCHA before they can send requests to your application:

1. Open the AWS WAF console, and then do one of the following:
   1. To create a new web ACL, choose Create web ACL.
   2. To edit an existing web ACL, choose the name of the ACL.
2. On the Rules tab, for the Add Rules dropdown, select Add managed rule groups.
3. Add the Amazon IP reputation list rule set to the web ACL. Then, choose Edit to edit the rule configuration.
4. For the Amazon IP reputation list rules.
   1. For AWSManagedIPDDoSList choose CAPTCHA, as shown in Figure 7.
      

      Figure 7: Amazon IP reputation rule configuration with CAPTCHA override

5. To save the configuration, choose Save.
6. To add the Amazon IP reputation list rules rule group to your web ACL, choose Add rules.
7. Configure the rule priority for your web ACL, and choose Save.
8. Optional: Integrate your application with the CAPTCHA JavaScript SDK.

How tokens work

AWS WAF generates a token as a result of both Challenge and CAPTCHA actions. The token itself is encrypted, tamper-proof, and implemented as the cookie aws-waf-token. The token is opaque to users but contains details that are useful for identifying the client session. Some of these details include:

• Timestamp: When the token was generated. By default, a client is immune from re-challenge for 5 minutes. You can find information on setting the immunity time period for your web ACL in AWS Documentation.
• Browser Fingerprint: A generated hash made up of a collection of data points on the client’s browser environment. This includes browser plugins, JavaScript environment, and other characteristics.
• Puzzle Type Solved: Whether a Challenge or CAPTCHA (or both) was completed.
• Token Domain: This is the domain that requests being received from is accepted by AWS WAF. The default is set based on the client hostname, for example www.example.com, and is the domain setting used for the token cookie. This cookie, and thus token, is sent in every request to www.example.com. You can update the token domain in your frontend application and webACL based on the documentation. This is useful for situations where you have a Single Page Application (SPA) running on www.example.com and APIs running on api.example.com – setting the token domain to .example.com means that the browser sends the token/cookie for all sub-domains of example.com. You can find further details on configuring the token domain in our documentation.

How Challenge, CAPTCHA, and tokens are used to manage bot traffic

These actions, and the subsequently generated tokens, have the following characteristics that are useful for identifying and managing bot traffic:

1. Requires fully-fledged clients: Both Challenge and CAPTCHA actions require a client to be able to execute a client-side script. This mitigates simple HTTP-request driven bots that are not equipped to run JavaScript.
2. Increases costs for bot operators: Solving Challenge and CAPTCHA puzzles require bot operators to invest more in building and running sophisticated bots. This acts as another barrier to entry for bots using your application.
3. Uniquely identifies clients: Clients sharing the same IP address, or rotating through IP addresses, can be distinguished by unique tokens meaning bots are unable to easily mimic multiple clients from a single host/IP.
4. Fingerprints client environment: The fingerprint provided in the token can be used to identify browser and mobile device configurations that may indicate signs of bot automation.

Conclusion

In this post, we showed you how Challenge and CAPTCHA actions work with AWS WAF, and how you can use these actions to mitigate specific bot threats. Our documentation for CAPTCHA and Challenge actions best practices provides further guidance on making the most of these actions.

About the authors

David MacDonald

David is a Senior Solutions Architect focused on helping New Zealand startups build secure and scalable solutions. He has spent most of his career building and operating SaaS products that serve a variety of industries.

Jess Izen

Jess Izen is a Senior Software Development Engineer with AWS WAF, building products like Bot Control and Fraud Control. She likes to work on performance sensitive, highly concurrent Rust services as well as distributed systems involving technologies like Kafka and Redis/Valkey. In her free time, she rides bikes and competes in amateur Muay Thai. You can connect with her on LinkedIn.