Implement multi-factor authentication based managed file transfer using AWS Transfer Family and AWS Secrets Manager

Highly regulated industries, such as finance, healthcare and government, need to exchange business to business files securely. Common protocols like SFTP provide a broadly-supported, standard method for moving files securely across public networks. For some customers, compliance requirements drive a higher bar for their SFTP authentication. To meet more stringent compliance requirements, aside from controlling network security, these customers often desire a multi-factor authentication (MFA) or two-factor authentication (2FA) mechanism for their managed file transfer (MFT) architecture.

Recently, AWS Transfer Family launched the ability to support multiple methods of authentication. This feature allows customers to choose to authenticate their users with both a standard public/private key exchange in addition to a password. While this new feature allows for two-factor authentication (2FA), customers sometimes require MFA to meet their corporate governance or security requirements. AWS Transfer Family provides a fully managed SFTP service that is backed by Amazon S3 or EFS to power your MFT architectures. AWS Transfer Family supports integration with other AWS services or third-party solutions to implement MFA authentication. Customers can find third-party solutions to implement MFA authentication for AWS Transfer Family using external Identity Provider (IdP) such as Okta and Azure Active Directory. For customers who might prefer to use a less complex solution with an easier deployment, this post will help deploy a quick MFA solution for AWS Transfer Family using AWS native services.

In this post, I will walk through an architecture for allowing MFA authentication for your MFT workloads using secure file transfer protocol (SFTP). MFT deployed with cloud-native services reduces cost and speeds deployment. You will learn how to build managed SFTP with MFA-based authentication using AWS Transfer Family backed by Amazon S3 or EFS. This architecture also uses other AWS services, such as AWS Lambda to handle authentication processing logic, and AWS Secrets Manager to store user authentication details.

Solution overview

We will use Google Authenticator as the MFA for the Transfer Family authentication. Google Authenticator uses time-based one-time password (TOTP; specified in RFC 6238). In order to generate a unique one-time password from Google Authenticator, you need to generate and register an 80-bit secret key into time-based account in Google Authenticator app. The 80-bit secret key will be stored in Secrets Manager along with user name and password for the user’s account. In order for this architecture to function properly, each user name needs a corresponding folder with the same name in the S3 bucket or EFS filesystem that provides the home directory for user’s configuration in AWS Transfer Family.

The following diagram shows the architecture and authentication process with MFA for AWS Transfer Family.

Architecture and authentication process with MFA for AWS Transfer Family using AWS Lambda, AWS Secrets Manager and S3 or EFS as a storage

The authentication steps are as follows:

Open Google Authenticator app and retrieve the unique token generated for the account that was setup in the app.
Log into Transfer Family for SFTP using any SFTP client. Enter the username and also the password concatenated with the unique token generated to log in.
AWS Transfer Family will invoke a Lambda function to complete the authentication with the username and password (combined with the token) as function parameters.
The Lambda function will retrieve credentials from Secrets Manager using account username that user has entered, and will authenticate the password as well as the secret key stored in Secrets Manager
Upon successful authentication, Lambda will return IAM permission which allow Transfer Family to list files from S3 bucket folder or EFS folder that is configured for the user’s account.

Prerequisites

To begin, you will need an AWS account to deploy the solution. To create an account, refer to the documentation on creating an AWS account.
Access to AWS account with permission to run CloudFormation and perform IAM actions for AWS Transfer Family, Lambda, S3 and Secrets Manager.
An S3 bucket or EFS folder with POSIX profile as a storage. To create an S3 bucket, refer to the documentation on creating a bucket. To create an EFS folder with POSIX profile, refer to the documentation on creating EFS file systems and creating an access point.
(Optional) To use least-privilege permission for the CloudFormation to deploy the solution, you will need to create an AWS CloudFormation service role and attach the following IAM policy to the role. Choose this role when setting permission for CloudFormation in Configure stack options.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "S31",
            "Effect": "Allow",
            "Action": [
                "s3:CreateBucket",
                "s3:PutBucketTagging",
                "s3:DeleteBucket",
                "s3:PutBucketPublicAccessBlock"
            ],
            "Resource": "*"
        },
        {
            "Sid": "Lambda1",
            "Effect": "Allow",
            "Action": [
                "lambda:CreateFunction",
                "lambda:TagResource"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "aws:RequestTag/blogSource": "TransferFamilyMFA"
                }
            }
        },
        {
            "Sid": "Lambda2",
            "Effect": "Allow",
            "Action": [
                "lambda:InvokeFunction",
                "lambda:AddPermission",
                "lambda:RemovePermission",
                "lambda:GetFunction",
                "lambda:DeleteFunction",
                "lambda:ListTags",
                "lambda:PutFunctionConcurrency"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "aws:ResourceTag/blogSource": "TransferFamilyMFA"
                }
            }
        },
        {
            "Sid": "IAM1",
            "Effect": "Allow",
            "Action": [
                "iam:CreateRole",
                "iam:TagRole"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "aws:RequestTag/blogSource": "TransferFamilyMFA"
                }
            }
        },
        {
            "Sid": "IAM2",
            "Effect": "Allow",
            "Action": [
                "iam:AttachRolePolicy",
                "iam:DeleteRole",
                "iam:PutRolePolicy",
                "iam:DeleteRolePolicy",
                "iam:DetachRolePolicy",
                "iam:GetRolePolicy",
                "iam:ListRoleTags",
                "iam:GetRole",
                "iam:ListPolicyTags"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "aws:ResourceTag/blogSource": "TransferFamilyMFA"
                }
            }
        },
        {
            "Sid": "IAM3",
            "Effect": "Allow",
            "Action": [
                "iam:PassRole"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "iam:PassedToService": [
                        "lambda.amazonaws.com",
                        "transfer.amazonaws.com"
                    ]
                }
            }
        },
        {
            "Sid": "SecretsManager1",
            "Effect": "Allow",
            "Action": [
                "secretsmanager:CreateSecret",
                "secretsmanager:TagResource"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "aws:RequestTag/blogSource": "TransferFamilyMFA"
                }
            }
        },
        {
            "Sid": "SecretsManager2",
            "Effect": "Allow",
            "Action": [
                "secretsmanager:DeleteSecret"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "aws:ResourceTag/blogSource": "TransferFamilyMFA"
                }
            }
        },
        {
            "Sid": "TransferFamily1",
            "Effect": "Allow",
            "Action": [
                "transfer:CreateServer",
                "transfer:TagResource"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "aws:RequestTag/blogSource": "TransferFamilyMFA"
                }
            }
        },
        {
            "Sid": "TransferFamily2",
            "Effect": "Allow",
            "Action": [
                "transfer:DeleteServer",
                "transfer:DescribeServer",
                "transfer:ListTagsForResource"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "aws:ResourceTag/blogSource": "TransferFamilyMFA"
                }
            }
        },
        {
            "Sid": "TransferFamily3",
            "Effect": "Allow",
            "Action": [
                "ec2:CreateVpcEndpoint",
                "ec2:DeleteVpcEndpoints",
                "ec2:DescribeVpcEndpoints",
                "ec2:DescribeVpcEndpointConnections"
            ],
            "Resource": "*"
        }
    ]
}

Solution setup steps

To setup the solution, you will need to perform following steps:

Set up MFA on your mobile device
Deploy solution in your AWS environment

Set up MFA on your mobile device

Download Google Authenticator app on your mobile. You can download it from here for Android devices and from here for Apple devices.
Generate a base 32 encoding secret string, which is a string with a minimum length of 16 characters consisting only alphabet A to Z and 2 to 7 as specified in RFC 4648, and register it as a time-based key into Google Authenticator app under account’s user name. To do that, open Google Authenticator app on your Mobile, select Add a code or tap on a plus icon on the bottom right of the screen. Select Enter a setup key and under Enter account details page, enter account username, e.g. “demo-user01”, and secret key, e.g. “ABCDEFGHIJ234567”. Make sure Time-based is selected under the dropdown option and then click Add button to complete the setup. Google Authenticator app will generate a unique token for that account that is rotated every 30 seconds.

Deploy solution in your AWS environment

In this section, you will use AWS CloudFormation to deploy the following resources:

A Secrets Manager secret for username, password and MFA secret key.
An Amazon S3 bucket to host Transfer Family’s account folder.
An IAM role with IAM policy allowing a Lambda Function to create an account home folder in S3 bucket.
An AWS Lambda function to create an account home folder in your S3 bucket.
A CloudFormation Custom resource to execute AWS Lambda to create an account home folder in S3 bucket.
An IAM role for all authenticated AWS Transfer Family users.
An IAM role with IAM policy allowing the Lambda Function to access Secrets Manager and CloudManager Logs.
An AWS Lambda function which serves as the custom Identity Provider (IdP).
An AWS IAM role which defines the permission policy for AWS Transfer Family to trigger the Lambda.
An AWS Transfer Family server which references the Lambda function as its custom IdP.

Note: This AWS CloudFormation template was tested in AWS Region ca-central-1 and us-east-1. Deploying this into your AWS account will incur cost. Steps for cleaning up the deployed resources are included later in this post. Be sure to follow these instructions so you do not incur additional cost beyond the time you spend testing this sample solution.

To deploy the CloudFormation, follow these steps:

To use S3 as a storage option, download the CloudFormation template TransferFamily-Lambda-SecretsManager-MFA-S3.yaml. To use EFS as a storage option, download the CloudFormation template TransferFamily-Lambda-SecretsManager-MFA-EFS.yaml.
On the AWS CloudFormation console, choose Create stack.
Select Upload a template file and Choose file. Upload file name link, then choose Next.
On the Specify stack details page, enter stack name Transfer-SFTP-MFA. On the same page, there are several parameters:
1. If you choose S3 as a storage option template, enter an Existing S3 Bucket name that will be used as your user’s storage. If you leave it empty, CloudFormation will create a new S3 bucket.
2. If you choose EFS as a storage option template, enter Existing EFS ARN. For EFS, you will also need to provide POSIX User ID and Group ID.
3. For UserName, enter the username that you registered in Google Authenticator earlier.
4. For Password, enter a password for the user to authenticate.
5. For MFAAuthCode, enter the user’s secret key that you entered in Google Authenticator earlier.
6. For Public or Private, select private if you want to make AWS Transfer Family server only accessible within a VPC. If Private option is selected, you will have to provide networking information specified in the VPC and Network Configuration for Private Deployment section.
On the Configure stack options page, choose Next.
On the Review page, select I acknowledge that AWS CloudFormation might create IAM resources,and choose Create stack. The script typically takes less than 10 minutes to run. The stack state will change to CREATE_COMPLETE once the deployment is complete.
On the Output tab, you will find the following outputs:
1. S3Bucket, which contains the name of S3 bucket that hosts Transfer Family’s account folder if you selected S3 as storage option template.
2. TransferFamilyServer, which contains the SFTP endpoint of created AWS Transfer Family server.
3. SecretsManagerPrefixName, which contains prefix name for subsequent user name that you can create in Secrets Manager.

Note: To create another user account for the Transfer Family server, create an entry in Secrets Manager with prefix name from output SecretsManagerPrefixName and create parameters UserName, Password, MFAAuthCode and POSIX profile (for EFS storage option) for new user account as described in step 4. For S3 as a storage option, go to the S3 bucket selected or created by CloudFormation, and create a new folder for each new user. Be sure the folder name matches the username for the new account. For EFS as a storage option, create a new access point and specify the new user account name as the root directory path. To create access for a POSIX root user to access EFS root folder, you will need to add client root access in the trusted role that provides Transfer Family with IAM permissions. Refer to Using AWS Transfer Family to access EFS for more detail.

Testing your AWS Transfer Family server configuration

Open a terminal from your local machine (or command prompt if you are using Windows), or any preferred SFTP client. Connect to your Transfer Family endpoint using the username and SFTP endpoint created previously.

sftp <username>@<endpoint>

When prompted for password, enter the password followed by the temporary token retrieved from the Google Authenticator MFA app for the user.
Once you are authenticated, you can upload files from your local computer to the Transfer Family SFTP server:

put <path-and-filename-of-local-file-to-upload>

To exit SFTP session, type exit in the terminal.

exit

Cleaning up

Before you proceed with deleting the CloudFormation stack, ensure that you have deleted from S3 (or EFS) any created user’s home directory and uploaded files the directory contains. The CloudFormation stack deletion process will fail if there are still any objects inside the created S3 Bucket for SFTP Family Transfer.

To remove resources created by CloudFormation, go to the AWS CloudFormation console and select the previously created stack (Transfer-SFTP-MFA). Click Delete and followed by Delete Stack.

Conclusion

Customers in highly regulated industries are looking for additional ways to improve the security of their MFT workflows. In this post, I walked through how you can setup AWS Transfer Family with AWS Lambda and AWS Secrets Manger to build an MFT solution with MFA. For customers whose governance or compliance standards mandate MFA, this solution will enable you to build a simple MFT architecture with MFA using AWS native services.

AWS Transfer Family is a fully managed service, meaning you also can take full benefit of AWS providing scale and failover to help you achieve agility, elasticity and cost savings. To learn more, read about how to use AWS Transfer Family, AWS Lambda, AWS Secrets Manager, Amazon S3, Amazon EFS, and AWS CloudFormation.

Thanks for reading this blog post! If you have any comments or questions, feel free to leave a comment in comments section.