How to Add Security and Compliance Auto-Remediation with the Cloud Conformity Platform

By Pranay Sharma, Cloud Application Architect at AWS
By Srijit Mitra, DevOps Consultant at AWS

Automation is becoming more and more critical to organizations needing to manage and govern cloud infrastructure.

Cloud Conformity’s auto-remediation tool helps to alleviate security and compliance concerns by using AWS Lambda to fix any non-compliant resources within your Amazon Web Services (AWS) account.

Cloud Conformity is an assurance and governance tool that continuously monitors one or more AWS services based on AWS Well-Architected best practices. Each rule (best practice) comes with its own violation remediation steps via the AWS Management Console and AWS Command Line Interface (CLI). A full list of these rules can be found on Cloud Conformity’s knowledge base.

Running more than 490 million checks each day, the Cloud Conformity platform is highly customizable and allows you to set rule priorities, communication settings, and reporting preferences. This enables customers to set up rules and Lambda functions to perform custom checks, as well as custom remediations on their fleet at scale.

Cloud Conformity is an AWS Partner Network (APN) Advanced Technology Partner with AWS Competencies in Security and Cloud Management Tools. They were named an APN Technology Partner of the Year for 2019, and many organizations around the world use Cloud Conformity’s platform to implement and maintain best-in-class infrastructure security, compliance, and optimization.

In this post, we will go over the fundamentals of how Cloud Conformity monitors an AWS account and performs auto-remediation when needed. We will provide best practices for onboarding your account with Cloud Conformity, and share guidelines on how to write a custom remediation Lambda function and trigger it from the Cloud Conformity platform.

Auto-Remediation Fundamentals

Cloud Conformity created auto-remediation as an MIT open-source project, which is actively maintained by the team. As such, it is freely accessible to anyone online. As part of the Cloud Conformity Security & Compliance package, the framework comes with 48 out-of-the-box auto-remediation rules. These rules offer self-healing against security vulnerabilities or failures.

To get started, set up your Cloud Conformity account and follow the steps. Once installation is complete, Cloud Conformity will continuously monitor your AWS account via APIs to detect any compliance issues.

As part of the installation, you’ll deploy one Amazon Simple Notification Service (SNS) topic and multiple Lambdas into your AWS account, which will be used to perform remediation actions.

Figure 1 – Cloud Conformity auto-remediation call flow.

As shown in the diagram above, after Cloud Conformity is set up it will continuously monitor your AWS account for any changes. Once a user makes a non-compliant change, the Cloud Conformity engine will publish a message onto an SNS topic, triggering a specific auto-remediation Lambda.

Published messages contain details on the specific violation that occurred within an AWS account, and is used by the Orchestrator Lambda to run a specific remediation function.

Technical Onboarding

AWS and Cloud Conformity recently worked with a major North American telecommunications company to implement Cloud Conformity and auto-remediation across the customer’s entire 25,000-person global business.

This organization leveraged AWS Landing Zone to configure and provision a secure and scalable, multi-account AWS environment in an automated fashion. Following AWS best practices, accounts were set up for shared services, log archive, and security.

Once the initial foundation was complete, the Account Vending Machine was used to create new AWS accounts in Organizational Units (OUs) that were preconfigured with an account security baseline and predefined network.

As part of the security baseline, Cloud Conformity’s auto-remediation functions were deployed into each account. This post focuses specifically on deployment of auto-remediation.

Note that information about integrating Cloud Conformity with AWS accounts more generally can be found in this video.

The diagram in Figure 2 below shows a high-level view of how the deployment of auto-remediation functions was fully automated by setting up a cross-account pipeline with AWS CodePipeline.

GitHub webhooks notify the pipeline on any code change, and a Lambda in the security account triggers the deployments in the target accounts.

Figure 2 – Cross-account deployment design.

An AWS CodeBuild job and Cloud Conformity cross-account AWS Identity and Access Management (IAM) role is deployed in every child account during account creation via the Account Vending Machine.

A Lambda in the security account assumes this role from the child account and triggers the AWS CodeBuild job. The pipeline in the security account orchestrates the execution of the Lambda function.

The screenshot below shows the pipeline, and note that manual approvals are required before deployments to the staging and production accounts.

Figure 3 – AWS CodePipeline cross-account deployment pipeline.

Anyone can use the code in the Github repository where the framework is found. At the time of this writing, there were 48 ready-to-go auto-remediation rules that can be added into AWS environments via the console.

Taking customization further, users can create their own custom rules and Lambda functions that allow the platform, framework, and self-healing features to go even further in supporting their unique needs.

Custom AWS Lambda Functions

Cloud Conformity’s knowledge base provides a consolidated list of the Lambda functions that are included in the continuous assurance checks. AWS assisted the telecommunications customer with mapping its internal security controls to the Cloud Conformity rules and identifying gaps.

We wrote the custom Lambdas to fill in these gaps. Some examples of include preventing public Amazon API Gateway endpoints, enforcing custom Amazon Simple Storage Service (Amazon S3) bucket policies, and preventing SNS topics from being publicly accessible.

Educational workshops were held with the security team during this process to get staff up to speed with the AWS services and their functionality. With the rules identified, we worked backwards with the customer’s InfoSec team to create a test framework. This would generate resources to violate the Cloud Conformity checks in order to trigger the auto-remediation workflow.

When a rule is triggered, the Orchestrator Lambda function receives an SNS event. The message body contains a JSON object identifying the rule number that was triggered as a result of the violation.

The snippet below shows a sample JSON Object received when a rule is violated.

{  
   "id":"ccc:[SCRUBBED]:GD-002:GuardDuty:us-west-2:[SCRUBBED]",
   "organisationId":"[SCRUBBED]",
   "accountId":"[SCRUBBED]",
   "ruleId":"GD-002",
   "ruleTitle":"GuardDuty Findings",
   "link":"https://us-west-2.console.aws.amazon.com/guardduty/home?region=us-west-2#/findings?macros=current%3Dtrue%2Carchived%3Dfalse",
   "service":"GuardDuty",
   "region":"us-west-2",
   "riskLevel":"MEDIUM",
   "categories":[  
      "security"
   ],
   "compliances":[  
      "APRA",
      "MAS",
      "AWAF",
      "NIST4"
   ],

   "message":"Unusual console login seen from principal abc.xyz@[SCRUBBED].com. Login activity using this client application, from the specific location has not been seen before from this principal.",
   "resource":"[SCRUBBED]",
   "descriptorType":"gd-finding",
   "resourceName":"Guard Duty Finding",
   "status":"FAILURE",
....
…
...
"createdDate":1565213713508,
   "createdBy":"SYSTEM",
   "resolutionPageURL":"https://www.cloudconformity.com/conformity-rules/GuardDuty/findings.html#[SCRUBBED]"
}

The most significant fields in the JSON object above are:

• ruleId: The ID of the rule being violated. In this example, it’s GD-002.
• Service: The name of the AWS service, which in this case is Amazon GuardDuty.
• riskLevel: Helps trigger an appropriate remediation, such as email alerts for low, and email and text alerts for medium.
• Categories: Identifies which teams need to be notified.
• Message: A descriptive message of the violation.
• resolutionPageURL: Details on how this violation can be remedied.

The Orchestrator Lambda looks for a function with the name AutoRemediate<ruleId>. In this example, it would be AutoRemediateGD-002. If the function exists, it’s invoked and the message is passed to it. Otherwise, the function exits gracefully with a log message reporting that the Lambda function AutoRemediate<ruleId> does not exist.

AWS provided the customer with 15 additional auto-remediation functions to cover all of the rules they had identified.

Best Practices with Cloud Conformity

Cloud Conformity enables AWS users to maintain best practices through its knowledge base of automated rules, all of which are rooted in the AWS Well-Architected Framework.

For each rule, one of the five pillars of Well-Architected—Security, Reliability, Cost Optimization, Performance Efficiency, Operational Excellence—is shown to clearly indicate which pillar the rule is supporting.

As shown in Figure 4 below, Cloud Conformity’s dashboard provides a quick glance at the Well-Architected compliance score for a customer’s entire AWS infrastructure and individual accounts. You can follow link to view all checks that are currently configured.

Figure 4 – Cloud Conformity shows the compliance status of all accounts.

The auto-remediation tool also ensures businesses stay compliant with industry standards such as HIPAA, PCI-DSS, GDPR, NIST, and CIS. Using a simple filtering option, the platform shows which standard-relevant rules are failing, including those coming under auto-remediation.

The screenshot below shows an example of how you can filter your checks based on specific compliance standards, making it easy to check the status of AWS resources from a particular standard point of view.

Figure 5 – Filter checks based on AWS services or compliance standards.

Cloud Conformity has also released a new Compliance Standards Reporting feature, which has the platform’s rules mapped to the individual controls of the standards. Currently, this feature is enabled for Well-Architected Reviews and NIST 800-43 (Rev.4).

Summary

In this post, we demonstrated how Cloud Conformity can be implemented by the security organization to provide a centralized dashboard to monitor, alert, and automatically remediate security vulnerabilities in their AWS accounts.

We leveraged AWS Landing Zone and AWS CodePipeline to build a pipeline that delivers the auto-remediation functions to all of the child AWS accounts in the enterprise. We then covered how Cloud Conformity’s platform can be extended to meet enterprise security compliance requirements by writing custom auto-remediation functions.

This entire solution is serverless and can scale to any number of accounts while providing a consistent security platform.

.

.

Cloud Conformity – APN Partner Spotlight

Cloud Conformity is an AWS Competency Partner. Its agentless, SaaS monitoring tool provides peace of mind for your AWS infrastructure, delivering automated best practice checks across the five pillars of the AWS Well-Architected Framework.

Contact Cloud Conformity | Solution Overview | AWS Marketplace

*Already worked with Cloud Conformity? Rate this Partner

*To review an APN Partner, you must be an AWS customer that has worked with them directly on a project.