Scale Your Security Vulnerability Testing with Amazon Inspector

My colleague Eric Fitzgerald wrote the guest post below in order to show you how to use an AWS Lambda function to forward Amazon Inspector findings to your ticketing and workflow systems.

— Jeff;

At AWS Re:Invent 2015 we announced Amazon Inspector, our security vulnerability assessment service that helps customers test for security vulnerabilities early and often. Using Amazon Inspector, customers can automate security testing across development, test, and production environments, identifying security vulnerabilities as part of the entire software development, deployment, and operations lifecycle.

Customer feedback on the Amazon Inspector approach to automated security testing has been overwhelming positive. Customers have told us that with Amazon Inspector, they are able to run security assessments more frequently and are catching security vulnerabilities earlier than they have in the past. However, identifying the security vulnerabilities is only half the battle, the vulnerabilities that are found need to be remediated. Many of our customers have started to integrate Amazon Inspector with their workflow and ticketing systems in order to automate and accelerate the remediation workflow for Amazon Inspector findings. We designed Amazon Inspector with this in mind and thought we would share more detail on one method for integrating Amazon Inspector findings with email, workflow, and ticketing systems.

Using AWS Lambda to push Amazon Inspector Findings to a Ticketing System
In this example, we are using an AWS Lambda function to connect Amazon Inspector to systems that can handle incident creation via email. Here’s the chain of events:

Amazon Inspector runs and performs a security assessment. It sends a message to an Amazon Simple Notification Service (Amazon SNS) topic at the end of the run.
The Lambda function is invoked by the SNS message.
The function fetches the findings from the security assessment.
The function formats and emails the findings using another SNS topic.

Along the way, the function creates the destination topic and the email subscription if necessary.

Setting up the Function
You will need to set up the function in the AWS Region where you run your Amazon Inspector assessments. If you run Amazon Inspector in more than one region, you’ll need to repeat the steps for each one. Here are the steps:

Create the SNS topic for Amazon Inspector.
Configure Amazon Inspector to send findings to the newly created topic.
Set up the Lambda function to fetch, format, and email the findings.

Configure an SNS Topic
The first major step is to configure an Amazon SNS topic that Amazon Inspector will notify when there are new findings, and an Amazon SNS topic that will format and send findings as email to other systems.

Navigate to the Amazon SNS Console and create a new Amazon SNS topic. This will be the topic where Amazon Inspector will deliver notifications to. It does not matter what you name the topic.

Next, assign the following policy to the topic. You can do this in the Amazon SNS Console by selecting the topic, clicking on Other topic actions, and selecting Edit topic policy. In the advanced view, replace the existing policy text with this policy:

{
  "Version": "2008-10-17",
  "Id": "inspector-sns-publish-policy",
  "Statement": [
    {
      "Sid": "inspector-sns-publish-statement",
      "Effect": "Allow",
      "Principal": {
        "Service": "inspector.amazonaws.com"
      },
      "Action": "SNS:Publish",
      "Resource": "arn:aws:sns:*"
    }
  ]
}

If you are familiar with AWS Identity and Access Management (IAM) policies, then a security best practice is to change the value of the Resource field of the policy to exactly match the Amazon SNS topic ARN, in order to restrict Amazon Inspector so that it can only publish to this topic.

Configure Amazon Inspector
Navigate to the Amazon Inspector Console, visit the Assessment templates page, and select the assessment template whose findings you want sent to the external system. Expand the row, and you’ll see a section called SNS topics. Click the pencil icon to the left of the Amazon SNS topics section and you’ll be able to pick the Amazon SNS topic you just created from a drop-down list. Once you’ve selected the topic, click on Save.

Set up the Lambda Function
Navigate to the Lambda Console and create a new function using the SNS-message-python blueprint:

Select SNS for the event source and then select the SNS topic that you created in the first step:

To finish configuring the function, click Next. Type a name and description for the function, choose the Python 2.7 runtime, and replace the sample function code with this code:

from __future__ import print_function
import boto3
import json
import datetime

sns = boto3.client('sns')
inspector = boto3.client('inspector')

# SNS topic - will be created if it does not already exist
SNS_TOPIC = "Inspector-Finding-Delivery"

# Destination email - will be subscribed to the SNS topic if not already
DEST_EMAIL_ADDR = "eric@example.com"

# quick function to handle datetime serialization problems
enco = lambda obj: (
    obj.isoformat()
    if isinstance(obj, datetime.datetime)
    or isinstance(obj, datetime.date)
    else None
)

def lambda_handler(event, context):

    # extract the message that Inspector sent via SNS
    message = event['Records'][0]['Sns']['Message']

    # get inspector notification type
    notificationType = json.loads(message)['event']

    # skip everything except report_finding notifications
    if notificationType != "FINDING_REPORTED":
        print('Skipping notification that is not a new finding: ' + notificationType)
        return 1
    
    # extract finding ARN
    findingArn = json.loads(message)['finding']

    # get finding and extract detail
    response = inspector.describe_findings(findingArns = [ findingArn ], locale='EN_US')
    print(response)
    try:
        finding = response['findings'][0]
    except OSError as err:
        print("OS error: {0}".format(err))
    except:
        print("Unexpected error:", sys.exc_info()[0])
        raise
        
    # skip uninteresting findings
    title = finding['title']
    if title == "Unsupported Operating System or Version":
        print('Skipping finding: ', title)
        return 1
        
    if title == "No potential security issues found":
        print('Skipping finding: ', title)
        return 1
    
    # get the information to send via email
    subject = title[:100] # truncate @ 100 chars, SNS subject limit
    messageBody = "Title:\n" + title + "\n\nDescription:\n" + finding['description'] + "\n\nRecommendation:\n" + finding['recommendation']
    
    # un-comment the following line to dump the entire finding as raw json
    # messageBody = json.dumps(finding, default=enco, indent=2)

    # create SNS topic if necessary
    response = sns.create_topic(Name = SNS_TOPIC)
    snsTopicArn = response['TopicArn']

    # check to see if the subscription already exists
    subscribed = False
    response = sns.list_subscriptions_by_topic( TopicArn = snsTopicArn )

    # iterate through subscriptions array in paginated list API call
    while True:
        for subscription in response['Subscriptions']:
            if ( subscription['Endpoint'] == DEST_EMAIL_ADDR ):
                subscribed = True
                break
        
        if 'NextToken' not in response:
            break
        
        response = sns.list_subscriptions_by_topic(
            TopicArn = snsTopicArn,
            NextToken = response['NextToken']
            )
        
    # create subscription if necessary
    if ( subscribed == False ):
        response = sns.subscribe(
            TopicArn = snsTopicArn,
            Protocol = 'email',
            Endpoint = DEST_EMAIL_ADDR
            )

    # publish notification to topic
    response = sns.publish(
        TopicArn = snsTopicArn,
        Message = messageBody,
        Subject = subject
        )

    return 0

Be sure to edit the DEST_EMAIL_ADDR value, and put in the actual email address that is used to send incidents to your incident management system. Optionally, you can change the name of the SNS topic that Amazon Inspector will use to send findings.

Leave the function handler (lambda_function.lambda_handler) as-is, and give the function a name:

Choose *basic execution role from the Role drop-down. After Lambda navigates to a new page, view the policy document, and use this one instead:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "arn:aws:logs:*:*:*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "inspector:DescribeFindings",
                "SNS:CreateTopic",
                "SNS:Subscribe",
                "SNS:ListSubscriptionsByTopic",
                "SNS:Publish"
            ],
            "Resource": "*"
        }
    ]
}

Click the Allow button to create the role and return to AWS Lambda, then leave the advanced settings as-is.

Be sure to click on Enable event source on the Review page:

Click Create function to save the function.

And that’s it!

Ready to Roll
For any assessments where you want the findings sent to another system, just add the first Amazon SNS topic (the one you created with these instructions) to the assessment template, and ensure that new finding reports are selected for publication to that topic.

The first time you run an assessment, Amazon Inspector will notify Lambda that you have new findings, and the Lambda function that you just created will create the SNS topic (if it doesn’t already exist), subscribe the destination email address to the topic (if not already subscribed), and send the findings as email to that address. If Lambda had to subscribe the email address to the topic, then you’ll only get one email requiring you to click a link to confirm that you want to subscribe. After confirmation, Amazon Inspector will deliver findings to that email address.

If you want to connect to Atlassian’s Jira Service Desk, it’s super easy from here on out. In Jira ServiceDesk, navigate to Customer Channels. This will display the email address that can receive email and create new issues. Put that email address into the Lambda function’s Python script and that’s where Inspector will deliver its findings. ServiceDesk will automatically turn them into ServiceDesk issues, and you can manage your workflow there.

Stay Tuned
Thank you for using Amazon Inspector, and look for more from us soon!

— Eric Fitzgerald, Principal Security Engineer

AWS News Blog

Scale Your Security Vulnerability Testing with Amazon Inspector

Resources

Follow