Invoking AWS Lambda functions via Amazon SNS

We released a new feature today for Amazon SNS that enables developers to perform custom message handling or publish messages to other AWS services by subscribing AWS Lambda functions to SNS topics. When a message is published to an SNS topic that has a Lambda function subscribed to it, the Lambda function is invoked with the payload of the published message.  The Lambda function receives this message payload as an input parameter and can use information contained in the payload to manipulate it, publish it to another set of SNS topics, or send the message to other AWS services. Additionally, we have now added delivery status support for Lambda destinations.

In this post, we will show you how to create a sample message history store using SNS, Lambda, and Amazon DynamoDB. To accomplish this, we will first create a DynamoDB table. Once the DynamoDB table is created, we will create a Lambda function, prepare it for invocation, subscribe it to an SNS topic and finally invoke it by publishing a message to the SNS topic via the SNS console or the SNS APIs.

Creating a DynamoDB table

Begin by creating a table in DynamoDB named snslambda with a primary hash key named SnsTopicArn and a primary range key named SnsPublishTime. To query the table with a SNS MessageId, you can optionally add a global secondary index on SnsMessageId. For instructions on how to create a DynamoDB table, refer to our DynamoDB documentation.

Creating the Lambda function

Using the Lambda console create the following Lambda function and name it messageStore:

console.log('Loading event');
var aws = require('aws-sdk');
var ddb = new aws.DynamoDB({params: {TableName: 'snslambda'}});

exports.handler = function(event, context) {
  var SnsMessageId = event.Records[0].Sns.MessageId;
  var SnsPublishTime = event.Records[0].Sns.Timestamp;
  var SnsTopicArn = event.Records[0].Sns.TopicArn;
  var LambdaReceiveTime = new Date().toString();
  var itemParams = {Item: {SnsTopicArn: {S: SnsTopicArn},
  SnsPublishTime: {S: SnsPublishTime}, SnsMessageId: {S: SnsMessageId},
  LambdaReceiveTime: {S: LambdaReceiveTime}  }};
  ddb.putItem(itemParams, function() {
    context.done(null,'');
  });
};

This Lambda function takes the SNS message it receives (see below for the exact SNS message structure) and extracts the MessageId, Timestamp, and TopicArn. It then stores this data into the DynamoDB table that we created prior to this step. Additionally, it stores the current time from the Lambda function’s perspective.

To use DynamoDB from a Lambda function, you also need to add a policy to your IAM lambda_exec_role. Here is an example policy:

{
  "Version": "2012-10-17",
  "Statement":[
    {
      "Sid":"Stmt1428510662000",
      "Effect":"Allow",
      "Action":[
        "dynamodb:GetItem",
        "dynamodb:DeleteItem",
        "dynamodb:PutItem",
        "dynamodb:Scan",
        "dynamodb:Query",
        "dynamodb:UpdateItem",
        "dynamodb:BatchWriteItem",
        "dynamodb:BatchGetItem",
        "dynamodb:DescribeTable"
      ],
      "Resource":["arn:aws:dynamodb:us-east-1:123456789012:table/snslambda"]
    }
  ]
}

Preparing the Lambda function for invocation

Once your Lambda function is created you need to assign a policy document to your Lambda function to allow it to be invoked by SNS. This is done using the AddPermissions API call. The AddPermissions API takes a policy document as an argument. Here is an example policy document for our Lamba function, messageStore and an SNS topic named lambda_topic:

{
  "Statement":[
    {"Condition":
      {"ArnLike":{"AWS:SourceArn":"arn:aws:sns:us-east-1:123456789012:lambda_topic"}},
      "Resource":"arn:aws:lambda:us-east-1:123456789023:function:messageStore",
      "Action":"lambda:invokeFunction",
      "Principal":{"Service":"sns.amazonaws.com"},
      "Sid":"sns invoke","Effect":"Allow"
    }],
  "Id":"default",
  "Version":"2012-10-17"
}

The policy is automatically set for you if you are using the Lambda console. Once the policy has been assigned, the Lambda function is ready to be subscribed to an SNS topic.

Update – May 26, 2020: When using Amazon SNS to deliver messages from opt-in regions to regions which are enabled by default, you must alter the policy created in the AWS Lambda function by replacing the principal sns.amazonaws.com with sns.<opt-in-region>.amazonaws.com. For example, if you want to subscribe an AWS Lambda function in US East (N. Virginia) to an SNS topic in Asia Pacific (Hong Kong), the principal in the AWS Lambda function’s policy must be changed to sns.ap-east-1.amazonaws.com. Opt-in regions include any regions launched after March 20, 2019, which at the time of this update include Asia Pacific (Hong Kong), Middle East (Bahrain), EU (Milano), and Africa (Cape Town). Regions launched prior to March 20, 2019 are enabled by default. Note that cross-region delivery to AWS Lambda from regions that are enabled by default to opt-in regions is not supported. Cross-region forwarding of SNS messages from opt-in regions to other opt-in regions is also not supported.

Subscribing Lambda functions to SNS topics

Step 1:  Using the SNS console, navigate to the Topics page. Identify an existing SNS topic that you would like to subscribe the Lambda function to. If you do not have an existing SNS topic, create one by clicking the Create new topic button, entering details such as Topic name and Display name, and then clicking Create topic. In this example, we will use a SNS topic named lambda_topic. Once the SNS topic has been identified or created, click the ARN of the SNS topic in the Topics page to open the Topic Details page.

You can also create the SNS topic and retrieve its ARN using the SNS APIs. Here is an example:

String topicArn = 
  snsClient.createTopic(new CreateTopicRequest("lambda_topic" /*topic Name*/).getTopicArn();

Step 2:  In the Topic Details page, click the Create subscription button. In the dialog box that appears, select AWS Lambda in the Protocol drop-down list. In the Endpoint field, select the ARN of the Lambda function, and click Create subscription. You will see a new subscription appear in the table of subscriptions associated with your SNS topic in the Topic Details page. This new subscription should have Lambda as the Protocol and the Lambda function ARN as the Endpoint.

The Lambda function can also be subscribed to an SNS topic using the SNS APIs. Here is an example:

String subscriptionArn = 
  snsClient.subscribe(new SubscribeRequest(topicArn,"lambda", 
    "arn:aws:lambda:us-east-1:123456789012:function:messageStore"));

Publishing messages to invoke the Lambda function

Now that the Lambda function has subscribed to an SNS topic, the next step is to publish a message to the SNS topic and observe the invocation of the Lambda function in the Lambda console. Using the SNS console, navigate to the Topic Details page for the SNS topic that has the Lambda function subscribed to it. Click the Publish to topic button, which will redirect you to the Publish a message page. On this page, enter a subject, select a message format, and enter the message payload that you want to invoke the Lambda function. If you select Raw (same message body for all protocols), you can simply enter your message in the Message text area. If you select JSON, you must encode the message in the JSON message format expected by SNS. You can also use the JSON message generator for assistance with converting your message to the expected SNS format.

When you have finished creating your message payload, click the Publish message button. This will publish the message to the SNS topic, and SNS will then attempt to deliver to all subscribers for that SNS topic. In this case, it will attempt to deliver the message to the subscribed Lambda function, thereby invoking it and passing a JSON formatted SNSEvent as an input parameter. Here is an example SNSEvent:

{
  "Records":[
    {
      "EventSource":"aws:sns",
      "EventVersion": "1.0",
      "EventSubscriptionArn": "arn:aws:sns:us-east-1:123456789012:lambda_topic:0b6941c3-f04d-4d3e-a66d-b1df00e1e381",
      "Sns":{
        "Type": "Notification",
        "MessageId":"95df01b4-ee98-5cb9-9903-4c221d41eb5e",
	"TopicArn":"arn:aws:sns:us-east-1:123456789012:lambda_topic",
        "Subject":"TestInvoke",
	"Message":"<message payload>",
        "Timestamp":"2015-04-02T07:36:57.451Z",
	"SignatureVersion":"1",
	"Signature":"r0Dc5YVHuAglGcmZ9Q7SpFb2PuRDFmJNprJlAEEk8CzSq9Btu8U7dxOu++uU",
        "SigningCertUrl":"http://sns.us-east-1.amazonaws.com/SimpleNotificationService-d6d679a1d18e95c2f9ffcf11f4f9e198.pem",
	"UnsubscribeUrl":"http://cloudcast.amazon.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-1:123456789012:example_topic:0b6941c3-f04d-4d3e-a66d-b1df00e1e381",
	"MessageAttributes":{"key":{"Type":"String","Value":"value"}}
      }
    }
  ]
}

Once the Lambda function is invoked and starts executing, you can see the results of your function invocation in the Lambda console. Additionally, you should also see records getting created in your DynamoDB table for every message that gets published to the SNS topic.

You can also publish a message to a SNS topic to invoke a subscribed Lambda function using the SNS API. Here is an example:

snsClient.publish(topicArn, "example_message");

Checking the delivery status of messages sent to Lambda functions

You can use the delivery status feature in SNS to track success and failure rates of deliveries to Lambda functions. For more information on how to use the delivery status feature, see our documentation.

We look forward to hearing from you about this feature. You can provide feedback using the SNS forum, the Lambda forum or our Twitter account.