Simplify managing access to Amazon ElastiCache for Redis clusters with IAM

Amazon ElastiCache for Redis is a fully managed, Redis-compatible, in-memory caching service that provides microsecond speed to support real-time applications. ElastiCache for Redis combines the speed, simplicity, and versatility of open-source Redis with the reliability, scalability, manageability, and security from AWS to power the most demanding real-time applications in media and entertainment, financial services, ecommerce, AdTech, gaming, and healthcare.

We’re excited to share that you can now connect to ElastiCache for Redis clusters using AWS Identity and Access Management (IAM) identities. This capability is available to you at no additional cost when you use ElastiCache for Redis version 7.0 or above with encryption in transit enabled. Managing access to your Redis cluster using a centralized identity store like IAM makes it easy to adhere to security best practices and guidelines. You can now define which IAM identities can access your ElastiCache for Redis clusters with IAM policy statements. It also provides a secure and consistent way for you to manage access across all your AWS resources.

Applications can now generate temporary security credentials that are used to establish their identity and authenticate access to your Redis clusters. This new method of authentication is designed to be used in conjunction with role-based access control (RBAC), where administrators can specify access strings, further defining which commands and keys can be accessed by each ElastiCache user. AWS customers can now federate authentication (SAML, LDAP, OpenID or OAuth) with their own directories via single sign-on (SSO) directly to ElastiCache for Redis.

In this post, we show you how to use your IAM identity to authenticate and access an ElastiCache for Redis cluster.

Benefits of IAM authentication

IAM provides fine-grained access control across several AWS services and resources including ElastiCache for Redis. With IAM, you can specify who can access which services and resources by providing the necessary policies. Access is denied by default and is allowed only when a policy explicitly grants access. With IAM policies, you manage permissions to your workforce and systems to ensure least-privilege permissions.

Previously, you needed to set up authentication for ElastiCache for Redis clusters using Redis user passwords or store the password in AWS Secrets Manager or on a third-party secrets management tool. However, in large organizations that host many applications, passwords can often become out of sync when it comes time to rotate the password. IAM authentication provides a streamlined security posture by allowing access management from a centralized service. With IAM authentication, ElastiCache users can use their IAM identities when connecting to their Redis clusters. Benefits of IAM authentication for ElastiCache users include:

• Centralized management of credentials using IAM, which is consistent with how you manage other AWS services and resources
• Validate temporary security credentials generated by applications to authenticate access to Redis clusters and password rotation on token invalidation
• Designed to work in conjunction with RBAC, where you can further define which commands and keys can be accessed by each ElastiCache user

Solution overview

The following diagram illustrates the solution architecture.

IAM authentication works by generating a short-lived (15 minutes) temporary security credential, which your application provides to the Redis client when connecting to the ElastiCache cluster. This credential is then used to establish your identity and apply the associated policies that determine the resources that can connect to the Redis clusters. To use your IAM identity with ElastiCache for Redis, you need to create an IAM-enabled ElastiCache user via the ElastiCache API, or the AWS Command Line Interface (AWS CLI), or AWS Management Console, and then attach a policy that permits the new elasticache:Connect action. After attaching the necessary policies, add the ElastiCache user to the corresponding user group and replication group. An application then generates a temporary security credential, which is used to establish the IAM identity, map it to an ElastiCache for Redis user, and grant the application access to the ElastiCache cluster. We go through these steps in detail in the rest of the post.

Prerequisites

This feature is available for ElastiCache for Redis clusters (version 7.0 and above) with encryption in transit enabled. To get started, we assume that you have the following infrastructure already set up:

1. An Amazon Virtual Private Cloud (Amazon VPC) with a public and private subnet to host an Amazon Elastic Compute Cloud (Amazon EC2) instance and ElastiCache for Redis cluster.
2. An EC2 instance to connect to ElastiCache for Redis cluster. Note that you can access ElastiCache from only within the same VPC.
3. Security groups that allow access from the Amazon EC2 instance to the ElastiCache for Redis cluster.
4. Have appropriate permissions to modify IAM permissions and policies.
5. Have appropriate permissions to modify and create EC2 instance and ElastiCache for Redis cluster.
6. Have the latest version of the AWS CLI.

Verify that you have an up-to-date AWS CLI version (2.9.0 or above) by running the following command:

$ aws –-version

Set up ElastiCache authentication with IAM

In this section, we walk you through the steps to setup ElastiCache authentication using your IAM identity. The following diagram illustrates the workflow for the rest of the post.

You create an IAM role and then attach a policy to it, which allows the IAM role to connect to the ElastiCache cluster as an ElastiCache user. Then you create an ElastiCache user group and add the default user along with the newly created IAM-enabled ElastiCache user to it, which is then attached to a replication group.

Set up an IAM role

This section explains how to create a policy that allows you to connect to an ElastiCache cluster and associate it with an IAM role. You then attach this role to an Amazon EC2 instance profile with the policy outlined in this section. As a general best practice, you shouldn’t use your root account to test IAM authentication to ElastiCache for Redis clusters because the root account is automatically allowed access to all resources.

1.  Before you start, confirm that your AWS CLI is using your correct account details:

   $ aws sts get-caller-identity

   {
       "Account": "<your-account-id>",
       "UserId": "<your-user-id>",
       "Arn": "<your-arn>",
   }

2. Create a file called trust-policy.json and copy the following configuration, which allows your EC2 instance to assume the new role (see IAM JSON policy elements reference for more information):

   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Effect": "Allow",
               "Action": [
                   "sts:AssumeRole"
               ],
               "Principal": {
                   "Service": [
                       "ec2.amazonaws.com"
                   ]
               }
           }
       ]
   }

3. Create a file called policy.json and copy the following configuration into it:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect" : "Allow",
         "Action" : [
           "elasticache:Connect"
         ],
         "Resource" : [
           "arn:aws:elasticache:us-east-1:<your-account-id>:replicationgroup:ec-cluster-iam-tutorial",
           "arn:aws:elasticache:us-east-1:<your-account-id>:user:iam-test-user-01"
         ]
       }
     ]
   }

   This policy only allows access to the resources used in this post. You can extend it to other resources by adding wildcards to the ARN path (see Using wildcards and paths in ARNs for additional information).

4. Create an IAM role that your EC2 instance assumes while connecting to the ElastiCache cluster (see create-role for more information):

   $ aws iam create-role \
     --role-name "elasticache-iam-auth-app" \
     --assume-role-policy-document file://trust-policy.json

5. Create the IAM policy using the policy.json file that allows access to the ElastiCache cluster (see create-policy for more information):

   $ aws iam create-policy \
     --policy-name "elasticache-iam-policy" \
     --policy-document file://policy.json

6. Attach the IAM policy to the role (see attach-role-policy for more information):

   $ aws iam attach-role-policy \
     --role-name "elasticache-iam-auth-app" \
     --policy-arn "arn:aws:iam::<your-account-id>:policy/elasticache-iam-policy"

7. Verify that the policy is attached:

   $ aws iam list-attached-role-policies \
     --role-name "elasticache-iam-auth-app"

   The output lists the policy that was attached:

   {
       "AttachedPolicies": [
           {
               "PolicyName": "elasticache-iam-policy",
               "PolicyArn": "arn:aws:iam::<your-account-id>:policy/elasticache-iam-policy"
           }
       ]
   }

8. Create an instance profile, which you later attach to the EC2 instance that is used to connect to the ElastiCache cluster (see create-instance-profile for more information):

   $ aws iam create-instance-profile --instance-profile-name ElastiCacheIAMConnect

9. Add the IAM-enabled ElastiCache role to this instance profile (see add-role-to-instance-profile for more information):

   $ aws iam add-role-to-instance-profile \
     --role-name "elasticache-iam-auth-app" \
     --instance-profile-name "ElastiCacheIAMConnect"

10. Now you go to your Amazon EC2 dashboard and use the Modify IAM role option under Actions, Security, choose ElastiCacheIAMConnect, and update the IAM role attached to the EC2 instance.

11. SSH into the EC2 instance, and verify that the instance role has propagated:

    $ aws sts get-caller-identity

    {
        "Account": "<your-account-id>",
        "UserId": "<your-user-id>",
        "Arn": "arn:aws:sts::<your-account-id>:assumed-role/elasticache-iam-auth-app/<your-instance-id>"
    }

    You later use this EC2 instance to connect to your ElastiCache cluster.

Set up an IAM-enabled ElastiCache user

Now that you can authenticate with IAM, lets create the IAM enabled ElastiCache user and also tighten permissions on the built-in default user, ensuring the principle of least privilege.

1. Create a new default user and provide a password (see create-user for more information) with the following code:

   $ aws elasticache create-user \
     --user-id "restricted-default-user" \
     --user-name "default" \
     --engine "redis" \
     --authentication-mode "Type=password,Passwords="<your-password>"" \
     --access-string "off -@all"

   We restrict the permissions for this user by providing off -@all as the access string.

2. Create a new IAM-enabled ElastiCache user:

   $ aws elasticache create-user \
     --user-name "iam-test-user-01" \
     --user-id "iam-test-user-01" \
     --authentication-mode "Type=iam" \
     --engine "redis" \
     --access-string "on ~* +@all"

   You see something similar to the following JSON as the output:

   {
       "UserId": "iam-test-user-01",
       "UserName": "iam-test-user-01",
       "Status": "active",
       "Engine": "redis",
       "MinimumEngineVersion": "7.0",
       "AccessString": "on ~* +@all",
       "UserGroupIds": [],
       "Authentication": {
           "Type": "iam"
       },
       "ARN": "arn:aws:elasticache:us-east-1:<your-account-id>:user:iam-test-user-01"
   }

3. Create a user group and add the new default user restricted-default-user and the IAM enabled ElastiCache user iam-test-user-01 (see create-user-group for more information). Each user group in ElastiCache must contain a default user.

   $ aws elasticache create-user-group \
     --user-group-id "iam-test-ug-01" \
     --engine "redis" \
     --user-ids "restricted-default-user" "iam-test-user-01"

   You see something similar to the following JSON as the output:

   {
       "UserGroupId": "iam-test-ug-01",
       "Status": "creating",
       "Engine": "redis",
       "UserIds": [
           "restricted-default-user",
           "iam-test-user-01"
       ],
       "MinimumEngineVersion": "7.0",
       "ReplicationGroups": [],
       "ARN": "arn:aws:elasticache:us-east-1:<your-account-id>:usergroup:iam-test-user-group-01"
   }

4. Create a replication group (see create-replication-group for more information) using the following command and attach the user group to it. Note that a replication group supports only one user group.

   $ aws elasticache create-replication-group \
     --replication-group-id "ec-cluster-iam-tutorial" \
     --replication-group-description "test for ElastiCache IAM Authentication" \
     --engine "redis" \
     --engine-version "7.0" \
     --cache-node-type "cache.m6g.large" \
     --transit-encryption-enabled \
     --cache-subnet-group "<insert-subnet-group>" \
     --security-group-ids "<insert-security-group>" \
     --tags Key=env,Value=test Key=task,Value=ec-iam-tutorial \
     --user-group-ids "iam-test-ug-01"

If you have a subnet group name and security group ID, then provide their corresponding values in the preceding command. Otherwise, refer to Creating a subnet group and Creating a security group for more information.

Test IAM authentication

For testing, you can connect to the ElastiCache cluster either using redis-cli or the Java-based demo application provided by AWS. The demo application uses a Redis Lettuce client and implements a Redis credentials provider that uses Signature V4 signing process to generate a temporary security credential. Follow the instructions listed in the README to clone and setup the demo application on your EC2 instance. Note that you can access your ElastiCache cluster from only within the same VPC.

Generate a temporary security credential using the demo application

After you set up the demo application on the EC2 instance using the instructions provided in the GitHub repo, generate a token using the following commands:

$ cd elasticache-iam-auth-demo-app/
$ java -cp target/ElastiCacheIAMAuthDemoApp-1.0-SNAPSHOT.jar \
com.amazon.elasticache.IAMAuthTokenGeneratorApp \
--region us-east-1 \
--replication-group-id ec-cluster-iam-tutorial \
--user-id iam-test-user-01

This command uses your EC2 instance role to generate a temporary security credential. You see something similar to the following as the output:

ec-cluster-iam-tutorial/?Action=connect&User=iam-test-user-01&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20221104T231633Z&X-Amz-SignedHeaders=host&X-Amz-Expires=900&X-Amz-Credential=<redacted>us-east-1%2Felasticache%2Faws4_request&X-Amz-Signature=<redacted>

This is the temporary security credential and is valid for 15 minutes from the time it was created.

Option A: Connect to the ElastiCache cluster using a temporary security credential and redis-cli

You can now connect with a redis-cli session, by running the following command from the terminal.

$ redis-cli -h <host> -p 6379 --tls
> AUTH iam-test-user-01 <insert-token>

Option B: Connect to the ElastiCache cluster using the demo application

The demo application uses the default AWS Credentials provider chain to generate a temporary security credential using your AWS caller identity:

$ java -jar target/ElastiCacheIAMAuthDemoApp-1.0-SNAPSHOT.jar \
	--redis-host <elasticache-cluster-configuration-endpoint> \
	--region us-east-1 \
	--replication-group-id ec-cluster-iam-tutorial \
	--user-id iam-test-user-01 \
	--tls

For cluster-mode enabled replication groups, please add the --cluster-mode flag.
You see something similar to the following as the output:

=> Connected clients: 1
Using credentials: iam-test-user-01, ec-cluster-iam-tutorial/?Action=connect&User=iam-test-user-01&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20221104T231633Z&X-Amz-SignedHeaders=host&X-Amz-Expires=900&X-Amz-Credential=<redacted>us-east-1%2Felasticache%2Faws4_request&X-Amz-Signature=<redacted>

The output includes the number of connected clients, the temporary security credential used and its expiry from the time it was created. The demo application also caches and re-uses the credential until it expires (15 minutes from the time it was created). After the credential expires, the token gets invalidated and application automatically creates a new temporary security credential.

Cleanup

To maintain the principle of least privilege and avoid incurring future charges, delete the resources you created as part of this post. Delete the ElastiCache cluster (see deleting a cluster for more information), EC2 instance (delete-instance), IAM policy (delete-policy) and role (delete-role).

Conclusion

ElastiCache for Redis users can now use IAM identities to authenticate and connect to their Redis clusters. This provides a consistent way to manage access to ElastiCache for Redis clusters with how you manage all your AWS resources. Applications can generate temporary security credentials to establish their IAM identity and seamlessly connect to ElastiCache for Redis clusters. The temporary security credential gets invalidated on expiry and automatically rotated by the application. This simplifies the overall administrative overhead when compared to using service-specific credentials and improves the security posture of your application infrastructure.

In this post, you learned how to enable IAM authentication for your ElastiCache for Redis clusters. You created an IAM role with the necessary policies that specifies the principals who can assume the role and connect to the ElastiCache for Redis cluster. You then attached this role to an EC2 instance and used the demo application to generate a temporary security credential. The application then used this credential to connect to the ElastiCache for Redis cluster as an IAM-enabled ElastiCache user instead of using Redis user passwords or a long-term security credential via Secrets Manager. The demo application also demonstrated how to rotate the temporary security credential automatically on invalidation, making it simple to adhere to security best practices.

This new functionality for ElastiCache for Redis provides security enhancements to the service. You can now have a seamless experience on how you connect your applications with a single identity provider across several services. We’d love to hear your feedback and answer any questions about this feature. Let us know your thoughts in the comments section.

About the authors

Maheedhar Gunturu is a Senior Specialist Solutions Architect focussed on In-memory Databases at AWS’s Worldwide Specialist Organization based in Palo Alto, California. He has over 16 years of industry experience working on a variety of Databases and Analytics. His days are spent helping global business leaders rethink their IT strategy and to get the most value out of their data. Outside of work, Mahee loves watching basketball, cricket and football with family and friends.

Andrey Belik is a Senior Product Manager for in-Memory Databases, AWS, based in Dublin, Ireland. Prior to joining AWS, Andrey spend over 20 years building various distributed, cloud systems for Microsoft, DELL, Huawei and MongoDB. In his spare time he enjoys hiking, traveling and making various electronic, robotic and mechanical devices and puzzles.

Christoph Koerner is a Senior Software Engineer in the AWS In-Memory Database services team with a primary focus on compliance and security. He has over 10 years of experience building large-scale data, machine learning and analytics systems for companies like Microsoft, Hubspot and T-Mobile. Christoph has also published and co-authored multiple books in this domain. When not in front of a screen, Christoph loves traveling, music, reading and cooking.