Deploying bastion hosts in private subnets is a way to provide temporary and limited access to non-production private resources in a virtual private cloud (VPC). Bastion hosts typically sit in public subnets. But in a non-production environment, if you want to allow a group of developers to access an private resource, you might not want to use a bastion host accessible from the internet.
In this post, we explain how to provision scalable and extendable secure bastion hosts in private subnets using AWS Cloud Development Kit (AWS CDK). First, I’ll show you how to configure AWS CDK and clone the GitHub repository I’ve prepared. Second, we demonstrate using AWS CDK to define the target environment and deploy the bastion host stack into a new or existing VPC.
Figure 1 shows the high-level architecture of our process. Only IAM users can access the bastion host in the private subnet of the VPC. They can use the Session Manager plugin for AWS CLI, SSH, or the AWS Systems Manager console.
Figure 1. Accessing a bastion host in the private subnet
Before getting started, make sure that you have the following.
- (Optional) Provide a
Default region name
and Default output format
.
- Clone the GitHub repository I’ve created for you.
$ git clone https://github.com/aws-samples/secure-bastion-cdk
- Navigate to the repository’s root directory.
$ cd secure-bastion-cdk
- Run the following
cdk
command to bootstrap your AWS environment. The cdk
command is the primary tool for interacting with an AWS CDK application.
$ cdk bootstrap aws://{account_id}/{your_selected_region}
Note: Bootstrapping launches resources into your AWS environment that are required by AWS CDK. These include an Amazon Simple Storage Service (Amazon S3) bucket for storing files and IAM roles that grant the necessary permissions to run the deployment.
Step 2: Configure target environments
In this section, we explain how to edit cdk.json
to define the environment to deploy.
In cdk.json
, deploy the bastion host into the private subnet by entering a value for the existingVpcId
parameter. To deploy a new VPC, keep the existingVpcId
parameter blank and specify VPC settings in the vpcConfig
section. In the allowedSecurityGroups
section, enter the IDs of the security groups to which you want the bastion host to connect.
In the following cdk.json
code example, dev
is the name of the environment.
"dev": {
"region": "eu-central-1",
"prefix": "dev",
"existingVpcId": "",
"instances": [
{
"instanceId": "BastionHost",
"instanceType": "t3.medium",
"keyName": "BastionHostKey",
"allowedSecurityGroups": "sg-ps-rds","sg-is-rds1","sg-is-rds2"
}
],
"vpcConfig":{
"cidr": "10.100.0.0/17",
"maxAZs": 3,
"isolatedSubnetCidrMask": 23,
"privateSubnetCidrMask": 20,
"publicSubnetCidrMask": 23,
"ssmPrefix": "/bastion/network"
}
},
The bastion host is configured to access security groups sg-ps-rds
, sg-is-rdsdb1
, and sg-isrds
. Because existingVpcId
is blank, a new VPC is configured using the vpcConfig
parameters.
In the next section, you can define multiple environments in cdk.json
and specify the one you want to deploy using AWS CDK.
Step 3: Stack deployment
Now let’s deploy the stacks.
Note: If you use an existing VPC in cdk.json
, deploy only the second stack.
- Run the following Npm commands.
$ npm install
$ npm test
- If you specified an existing VPC in the
cdk.json
file, skip to step 4. To deploy a new VPC, run the following command. In the example shown, replace environment with the name of an environment from the cdk.json
file. Replace the account_ID
value with the ID of the target AWS account.
$ cdk deploy -c environment="<environment>" -c account="<account_ID>" AwsBastion-NetworkCdkStack --profile bastion-cdk
- When you’re prompted to deploy the changes, choose
y
. If it’s successful, AWS CDK returns the CIDR block and ID of the VPC (Outputs
), and the Amazon Resource Name (ARN) of the stack (Stack ARN
).
- Run the following command to deploy
AwsBastion-Ec2CdkStack
. In the example shown, replace environment with the name of an environment from the cdk.json
file. Replace the account_ID
value with the ID of the target AWS account.
$ cdk deploy -c environment="<environment>" -c account="<account_ID>" AwsBastion-Ec2CdkStack --profile bastion-cdk
- AWS CDK shows the IAM Statement Changes, IAM Policy Changes, and Security Group Changes that are deployed. When prompted, choose
y
to deploy the changes. If it’s successful, AWS CDK returns the stack’s ARN (Stack ARN
).
Step 4: Create an IAM policy and user
In this section, we create an IAM policy that enables an IAM user to use Session Manager. We create a user and attach the policy to it. Then, in the next section, we demonstrate three operations an IAM user can use to connect to it.
- Create an IAM policy named bastion test using the following code. In the code provided, replace
{account_id}
with the ID of the target AWS account. To limit access to secret resources using tags, add a value for {tag_key}
and {tag_value}
.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ssm:StartSession"
],
"Resource": [
"arn:aws:ec2:*:{account_id}:instance/*"
],
"Condition": {
"StringLike": {
"ssm:resourceTag/Name": [
"BastionHost"
]
}
}
},
{
"Effect": "Allow",
"Action": [
"ec2:DescribeInstances",
"ec2:DescribeImages",
"ec2:DescribeTags",
"ec2:DescribeSnapshots"
],
"Resource": [
"*"
]
},
{
"Effect": "Allow",
"Action": [
"ssm:StartSession"
],
"Resource": [
"arn:aws:ssm:*::document/AWS-StartPortForwardingSession",
"arn:aws:ssm:*::document/AWS-StartSSHSession"
]
},
{
"Effect": "Allow",
"Action": [
"ssm:TerminateSession"
],
"Resource": [
"arn:aws:ssm:*:*:session/${aws:username}-*"
]
},
{
"Effect": "Allow",
"Action": [
"secretsmanager:GetSecretValue",
"secretsmanager:DescribeSecret",
"secretsmanager:ListSecretVersionIds",
"secretsmanager:ListSecrets"
],
"Resource": [
"arn:aws:secretsmanager:*:{account_id}:secret:*"
],
"Condition": {
"StringEquals": {
"secretsmanager:ResourceTag/{tag_key}": "{tag_value}"
}
}
}
]
}
- Create an IAM user in your AWS account and attach the bastion test policy to it.
- Configure the AWS profile.
aws configure --profile bastion-test
- The AWS CLI prompts you for four configuration settings. The following example shows sample values. Replace
AWS Access Key ID
and AWS Secret Access Key
with the credentials for the user created in step 2.
AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Default region name [None]: us-west-2
Default output format [None]: json
- (Optional) Provide a
Default region name
and Default output format
.
- Run the following command.
$INSTANCE_ID=$(aws ec2 describe-instances \<br />
--filter "Name=tag:Name,Values= BastionHost" \<br />
--query "Reservations[].Instances[?State.Name =='running'].InstanceId[]" \<br />--output text
--profile bastion-test)
Step 5: Connect to the bastion host
In this section we demonstrate three ways an IAM user can connect to the bastion host.
Operation 1: Connect using AWS Systems Manager Session Manager
Use Session Manager to connect to the bastion host with the following command.
aws ssm start-session --target $INSTANCE_ID --profile bastion-test
Alternatively, use Session Manager to run the following command syntax to connect with port forwarding. Replace remote_port_number
and your_local_port_number
with the target remote and local port numbers.
aws ssm start-session --target $INSTANCE_ID \
--document-name AWS-StartPortForwardingSession \
--parameters '{"portNumber":["{remote_port_number}"],"localPortNumber":["{your_local_port_number}"]}' --profile bastion-test
Operation 2: Connect using SSH
Use this operation to connect to the bastion host using SSH.
- Update the SSH configuration file to allow a proxy command to run Session Manager. From a command prompt, open the SSH configuration file in the Vim editor.
vim ~/.ssh/config
Add the following to the configuration file.
#Add SSH over Session Manager
host i-* mi-*
ProxyCommand sh -c "aws ssm start-session --target %h --document-name AWS-StartSSHSession --parameters 'portNumber=%p'"
- Press
Esc
.
- Enter
:wq
and press Enter
.
- Retrieve the host key pair for the bastion host from AWS Secrets Manager. This is created as part of the
AwsBastion-Ec2CdkStack
stack deployment.
aws secretsmanager get-secret-value \
--secret-id ec2-ssh-key/BastionHostKey/private \
--query SecretString \
--output text --profile bastion-test > bastion-key-pair.pem
- Run the following command.
chmod 400 bastion-key-pair.pem
- Run the following command to open an SSH tunnel. In the syntax shown, replace
local_port
with your local port. Replace host
with the host IP address (such as an Amazon RDS endpoint). Replace remote_port
with the target remote port (such as 3306
for a MySQL server).
ssh -f -N ec2-user@$INSTANCE_ID -L {local_port}:{host}:{remote_port} -i bastion-key-pair.pem
Enter yes when prompted to continue connecting. The $INSTANCE_ID will be added to the list of known hosts.
Operation 3: Connect using the AWS Management Console
A third operation is to connect to the bastion host using Systems Manager in the AWS Management Console.
- Sign into the AWS Systems Manager console as the IAM user that you created in “Step 4: Create an IAM policy and user”.
- In the navigation pane under Node Management, choose Session Manager.
- Choose Start session.
- (Optional) Enter a reason for connecting to the instance in the Reason for session – optional field.
- For Target instances, choose the
BastionHost
instance. This is the instance AwsBastion-Ec2CdkStack
deploys.
- Choose Start session.
- In the Session Manager terminal, you can run bash or PowerShell commands to connect to the private resources configured in
cdk.json
. When you finish, close the terminal to end Session Manager.
Cleanup
To avoid unexpected charges to your account, delete the resources you deployed during the walkthrough.
- If you deployed the VPC stack (
AwsBastion-NetworkCdkStack
), run the following command to delete it.
cdk destroy -c environment="<environment_name>" -c account="<ACCOUNT ID>" AwsBastion-NetworkCdkStack --profile bastion-cdk
Choosey
when prompted to confirm stack deletion. If it’s successful, AWS CDK returns AwsBastion-Ec2CdkStack: destroyed
.
- Delete the bastion host stack (
AwsBastion-NetworkCdkStack
) by running the following command. Replace the environment and account values with the ones used to deploy the stack.
cdk destroy -c environment="<environment_name>" -c account="<account_ID>" AwsBastion-Ec2CdkStack --profile bastion-cdk
Choose y
when prompted to confirm stack deletion. If successful, AWS CDK returns AwsBastion-NetworkCdkStack: destroyed
.
Conclusion
In this blog post, we demonstrated using AWS CDK to deploy a bastion host in the private subnet of a new or existing VPC. By deploying a bastion host in the private subnet, you can provide secure access to private subnet resources in your VPC.
We invite you to adapt the code provided in our GitHub repository. For extra credit, we challenge you to integrate the process I’ve shown with a continuous integration and continuous delivery (CI/CD) pipeline. In this way, you can automatically deploy secure bastion hosts into private subnets across multiple environments. To get started, see Continuous integration and delivery (CI/CD) using CDK Pipelines, then aws-cdk/pipelines module for a construct library. For more about AWS CDK and Amazon RDS, refer to Use AWS CDK to initialize Amazon RDS instances.
Use the comments section to let me know if you have any questions.
About the authors
Ramy Nasreldin
Ramy Nasreldin is a DevOps architect at AWS, based in Sweden. Ramy helps customers design and implement their systems to run on the AWS Cloud. He also preaches best practices by automating everything, from infrastructures to application delivery, to achieve the most resilient and scalable solutions that best serve end users in a sustainable way. In his spare time, he enjoys swimming, playing football, and spending time with his family.
Rolando Santamaria Maso
Rolando is a senior cloud application development consultant at AWS Professional Services, based in Germany. He helps customers migrate and modernize workloads in the AWS Cloud, with a special focus on modern application architectures and development best practices, but he also creates IaC using AWS CDK. Outside work, he maintains open-source projects and enjoys spending time with family and friends.
Prasanna Tuladhar Gitelman
Prasanna is a cloud infrastructure architect at AWS Professional Services, based in Germany. He likes to explore new challenges, be it databases, containers, or cloud infrastructures. Outside work, he likes jogging, hiking, and spending time with his family.