AWS Partner Network (APN) Blog
How to Import and Manage AWS Networking with Terraform and ControlMonkey
By Ioannis Moustakis, Solutions Architect – AWS
By Aharon Twizer, CEO – ControlMonkey
 |
| ControlMonkey |
 |
Two challenges cloud operations teams face today is tracking version control issues and the risk of misconfigurations from manual inputs.
The gold standard to address these challenges is to adopt infrastructure as code (IaC), but many organizations maintain production environments painstakingly created by hand or through a collection of scripts or legacy frameworks. Transitioning manually-created environments to IaC is often a labor-intensive, tedious, and potentially error-prone task.
In this post, we discuss how to transition manually-created environments to an infrastructure as code approach, which can help you reduce the risk of misconfigurations resulting from human errors.
Specifically, we’ll dive deep into how ControlMonkey can assist with transitioning manually-created environments to an IaC approach, integrating seamlessly with GitOps and Terraform by HashiCorp.
ControlMonkey is an AWS Specialization Partner and AWS Marketplace Seller with the DevOps Competency. It’s also a TFOps (Terraform Operations) platform that enables networking and DevOps teams to take a proactive DevOps strategy regarding cloud operations.
Customer Challenges
Many cloud ops teams rely on manual implementation of resources using “ClickOps,” that is, using a graphical user interface (GUI) to manually create a resource, rather than programmatically. However, if you have a need to carefully track changes, similar to examining updates in software versions using tools like Git, the approach described in this post can help provide a process to manage your existing infrastructure programmatically.
The ClickOps approach also leaves organizations without a straightforward means to replicate or audit their environments. As a result, creating an automated disaster recovery strategy and action plan is cumbersome and often overlooked.
Networking components, among others, are some of the resources created manually via the AWS console. Without an IaC setup that allows organizations to develop tested DR plans, there are risks related to downtime, operations, data loss, financial impact, and reputation damage.
But what if you’re already dealing with thousands of resources? How do you import them into Terraform without being entangled in protracted and exhausting projects?
This post demonstrates how you can transition your existing AWS networking resources, currently managed manually, to a Terraform-driven approach, enabling continuous management through GitOps. Our example revolves around the hub-and-spoke topology with a centralized inspection Amazon VPC (second deployment model).
Use Case
The hub-and-spoke architecture on AWS is a networking approach that addresses the complexities of interconnecting multiple virtual private clouds (VPCs) by utilizing a central hub VPC that controls access to numerous isolated spoke VPCs.
The specific setup encompasses three core elements: inspection VPC, AWS Transit Gateway, and workload spoke VPCs. The inspection VPC acts as a centralized inspection point, typically using a layer 3-7 firewall to inspect traffic and apply security policy. AWS Transit Gateway controls traffic routing between the spokes using routing tables.
In Figure 1 below, AWS Transit Gateway uses two routing tables—one for routing traffic from the spoke VPCs to the central “inspection” VPC, and another for routing traffic from the inspection VPC out to the spoke VPCs.
By leveraging multiple routing tables in this hub-and-spoke topology, AWS Transit Gateway ensures communication passes through the central inspection VPC rather than directly between the spoke VPCs.
Further, AWS Network Firewall is deployed as an endpoint service, similar to other network services such as AWS PrivateLink. Your AWS Network Firewall endpoint must be deployed in a dedicated subnet within your VPC, and AWS Network Firewall inspects all traffic that’s routed to the endpoint.
Lastly, the spoke VPCs, which house application workloads, are isolated from each other and can only send and receive traffic via the hub. By default, they access the internet exclusively through the inspection VPC’s Network Access Translation (NAT) gateways.
Figure 1 – AWS hub-and-spoke architecture with an inspection VPC.
In this use case, assume we have a hub-and-spoke architecture with one centralized VPC, and two spoke VPCs that were created manually from the AWS console. The two spoke VPCs contain a dedicated private subnet for Amazon Elastic Compute Cloud (Amazon EC2) instances and another private subnet for VPC endpoints so we can connect securely via AWS Systems Manager.
Now, let’s see how ControlMonkey helps to bring this under Terraform management.
Import to Terraform with ControlMonkey
Prerequisites
- ControlMonkey account.
- AWS account with hub-and-spoke and inspection VPC resources already deployed. You can use this Github repo as a reference.
- Version control system such as GitHub or GitLab.
Steps to Import
To get started with ControlMonkey, you need to connect two data sources:
- AWS account: Begin by linking your AWS account to a cross-account AWS Identity and Access Management (IAM) role. ControlMonkey asks for a read-only cross-account IAM role to discover all resources within your AWS account. It also needs permission to read the Terraform state file (if any) from your dedicated Amazon Simple Storage Service (Amazon S3) bucket.
- Version control system.
You’ll gain visibility into your Terraform coverage within a few minutes. This lets you quickly spot which resources are outside Terraform’s management.
You can check your IaC coverage percentage in ControlMonkey’s dashboard:
Figure 2 – ControlMonkey’s dashboard.
You can also drill down and look for specific resources and their IaC state in the “Resource Finder” view.
Figure 3 – ControlMonkey’s Resource Finder page.
Now, let’s move to the import part. ControlMonkey automatically stacks together resources not managed by Terraform with some contextual relation between them.
ControlMonkey automatically stacks together the AWS Transit Gateway and the related VPCs in the hub-and-spoke example. Under the “Terraform Import” part, you can see a new suggested stack for the AWS Transit Gateway.
Figure 4 – ControlMonkey’s Terraform Import page.
You can see all of the resources under that suggested stack. You can customize the suggested stack resources or build a new custom model with resources as you see fit.
Figure 5 – ControlMonkey’s Terraform Import Stack Customization page.
Now, to import those resources to Terraform, we need two things:
- Terraform code representing those resources.
- Terraform state file that connects between the terraform code and the actual resources running on AWS.
ControlMonkey provides them both. When clicking on “Import resources” on the suggested stack, you get into the import wizard. In this wizard, you can see the Terraform code for those resources. Copy the code and push it to your Git repository.
Next, tell ControlMonkey where you push the code to (you’ll see why in a few sentences). You will create a “ControlMonkey Stack” at the end of the import wizard. Once you finish, ControlMonkey automatically pushes the Terraform state according to the remote backend configuration in your code.
Figure 6 – ControlMonkey’s Terraform Import Wizard page.
Now, you have the Terraform code in your Git repository and the state file in your remote backend. You also have a “ControlMonkey stack” which is the last part of the transition to GitOps and provides the following:
- Terraform CI/CD: Listens to any changes in your Git code (the specific directory you pushed the code to), reacts to any pull request or push events, and performs plan/apply respectively. You can also incorporate proactive policies into that CI/CD process to block any non-compliant changes before applying.
- Drift detection: If someone changes the resources directly from the AWS console and not through the code, the stack will notify you about this and offer a quick fix for the drift.
Now that your hub-and-spoke resources are managed by Terraform, you have a drift detection mechanism in place and a Terraform CI/CD to keep updating your resources in an audited and governed way. Your networking resources have completed the shift from ClickOps to GitOps.
Conclusion
In the ever-changing world of AWS networking, transitioning from ClickOps to GitOps is a game-changer, especially regarding disaster recovery planning (DRP). By importing your existing infrastructure to Terraform with the help of ControlMonkey, you’re streamlining processes and enhancing your DR readiness.
Shifting to GitOps with ControlMonkey empowers you to craft a proactive DevOps strategy. By managing your network as code, you can utilize proactive controls within your Terraform CI/CD pipeline to prevent mistakes and quickly identify discrepancies between the desired state (the code) and the actual state (what’s running in your AWS account).
By leveraging GitOps and tools like ControlMonkey, you optimize networking management, fortify your DR capabilities, and proactively ensure the accuracy of your network configurations.
.
.
ControlMonkey – AWS Partner Spotlight
ControlMonkey is an AWS Specialization Partner and Terraform Operations platform that enables networking and DevOps teams to take a proactive DevOps strategy regarding cloud operations.