AWS Compute Blog

Introducing the C++ Lambda Runtime

This post is courtesy of Marco Magdy, AWS Software Development Engineer – AWS SDKs and Tools

Today, AWS Lambda announced the availability of the Runtime API. The Runtime API allows you to write your Lambda functions in any language, provided that you bundle it with your application artifact or as a Lambda layer that your application uses.

As an example of using this API and based on the customer demand, AWS is releasing a reference implementation of a C++ runtime for Lambda. This C++ runtime brings the simplicity and expressiveness of interpreted languages while maintaining the superiority of C++ performance and low memory footprint. These are benefits that align well with the event-driven, function-based, development model of Lambda applications.

Hello World

Start by writing a Hello World Lambda function in C++ using this runtime.

Prerequisites

You need a Linux-based environment (I recommend Amazon Linux. You can find the Amazon Linux AMI that Lambda uses in Lambda Execution Environment and Available Libraries, Note: Amazon Linux 2 is not yet supported for Runtime API), with the following packages installed:

  • A C++11 compiler, either GCC 5.x or later or Clang 3.3 or later. On Amazon Linux, run the following commands: 
    $ yum install gcc64-c++ libcurl-devel
$ export CC=gcc64
$ export CXX=g++64
  • CMake v.3.5 or later. On Amazon Linux, run the following command: 
    $ yum install cmake3
  • Git

Download and compile the runtime

The first step is to download & compile the runtime:

$ cd ~ 
$ git clone https://github.com/awslabs/aws-lambda-cpp.git
$ cd aws-lambda-cpp
$ mkdir build
$ cd build
$ cmake3 .. -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF \
   -DCMAKE_INSTALL_PREFIX=~/out
$ make && make install

This builds and installs the runtime as a static library under the directory ~/out.

Create your C++ function

The next step is to build the Lambda C++ function.

  1. Create a new directory for this project: 
    $ mkdir hello-cpp-world
$ cd hello-cpp-world
  2. In that directory, create a file named main.cpp with the following content: 
    // main.cpp
#include <aws/lambda-runtime/runtime.h>

using namespace aws::lambda_runtime;

invocation_response my_handler(invocation_request const& request)
{
   return invocation_response::success("Hello, World!", "application/json");
}

int main()
{
   run_handler(my_handler);
   return 0;
}
  3. Create a file named CMakeLists.txt in the same directory, with the following content: 
    cmake_minimum_required(VERSION 3.5)
set(CMAKE_CXX_STANDARD 11)
project(hello LANGUAGES CXX)

find_package(aws-lambda-runtime REQUIRED)
add_executable(${PROJECT_NAME} "main.cpp")
target_link_libraries(${PROJECT_NAME} PUBLIC AWS::aws-lambda-runtime)
aws_lambda_package_target(${PROJECT_NAME})
  4. To build this executable, create a build directory and run CMake from there: 
    $ mkdir build
$ cd build
$ cmake3 .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH=~/out
$ make

    This compiles and links the executable in release mode.

  5. To package this executable along with all its dependencies, run the following command: 
    $ make aws-lambda-package-hello

    This creates a zip file in the same directory named after your project, in this case hello.zip.

Create the Lambda function

Using the AWS CLI, you create the Lambda function. First, create a role for the Lambda function to execute under.

  1. Create the following JSON file for the trust policy and name it trust-policy.json. 
    {
 "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": ["lambda.amazonaws.com"]
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
  2. Using the AWS CLI, run the following command: 
    $ aws iam create-role \
--role-name lambda-cpp-demo \
--assume-role-policy-document file://trust-policy.json

    This should output JSON that contains the newly created IAM role information. Make sure to note down the “Arn” value from that JSON. You need it later. The Arn looks like the following:

    “Arn”: “arn:aws:iam::<account_id>:role/lambda-cpp-demo”

  3. Create the Lambda function: 
    $ aws lambda create-function \
--function-name hello-world \
--role <specify the role arn from the previous step> \
--runtime provided \
--timeout 15 \
--memory-size 128 \
--handler hello \
--zip-file fileb://hello.zip
  4. Invoke the function using the AWS CLI:
    <bash>

    $ aws lambda invoke --function-name hello-world --payload '{ }' output.txt

    You should see the following output:

    {
  "StatusCode": 200
}

    A file named output.txt containing the words “Hello, World!” should be in the current directory.

Beyond Hello

OK, well that was exciting, but how about doing something slightly more interesting?

The following example shows you how to download a file from Amazon S3 and do some basic processing of its contents. To interact with AWS, you need the AWS SDK for C++.

Prerequisites

If you don’t have them already, install the following libraries:

  • zlib-devel
  • openssl-devel
  1. Build the AWS SDK for C++: 
    $ cd ~
$ git clone https://github.com/aws/aws-sdk-cpp.git
$ cd aws-sdk-cpp
$ mkdir build
$ cd build
$ cmake3 .. -DBUILD_ONLY=s3 \
 -DBUILD_SHARED_LIBS=OFF \
 -DENABLE_UNITY_BUILD=ON \
 -DCMAKE_BUILD_TYPE=Release \
 -DCMAKE_INSTALL_PREFIX=~/out

$ make && make install

    This builds the S3 SDK as a static library and installs it in ~/out.

  2. Create a directory for the new application’s logic: 
    $ cd ~
$ mkdir cpp-encoder-example
$ cd cpp-encoder-example
  3. Now, create the following main.cpp: 
    // main.cpp
#include <aws/core/Aws.h>
#include <aws/core/utils/logging/LogLevel.h>
#include <aws/core/utils/logging/ConsoleLogSystem.h>
#include <aws/core/utils/logging/LogMacros.h>
#include <aws/core/utils/json/JsonSerializer.h>
#include <aws/core/utils/HashingUtils.h>
#include <aws/core/platform/Environment.h>
#include <aws/core/client/ClientConfiguration.h>
#include <aws/core/auth/AWSCredentialsProvider.h>
#include <aws/s3/S3Client.h>
#include <aws/s3/model/GetObjectRequest.h>
#include <aws/lambda-runtime/runtime.h>
#include <iostream>
#include <memory>

using namespace aws::lambda_runtime;

std::string download_and_encode_file(
    Aws::S3::S3Client const& client,
    Aws::String const& bucket,
    Aws::String const& key,
    Aws::String& encoded_output);

std::string encode(Aws::String const& filename, Aws::String& output);
char const TAG[] = "LAMBDA_ALLOC";

static invocation_response my_handler(invocation_request const& req, Aws::S3::S3Client const& client)
{
    using namespace Aws::Utils::Json;
    JsonValue json(req.payload);
    if (!json.WasParseSuccessful()) {
        return invocation_response::failure("Failed to parse input JSON", "InvalidJSON");
    }

    auto v = json.View();

    if (!v.ValueExists("s3bucket") || !v.ValueExists("s3key") || !v.GetObject("s3bucket").IsString() ||
        !v.GetObject("s3key").IsString()) {
        return invocation_response::failure("Missing input value s3bucket or s3key", "InvalidJSON");
    }

    auto bucket = v.GetString("s3bucket");
    auto key = v.GetString("s3key");

    AWS_LOGSTREAM_INFO(TAG, "Attempting to download file from s3://" << bucket << "/" << key);

    Aws::String base64_encoded_file;
    auto err = download_and_encode_file(client, bucket, key, base64_encoded_file);
    if (!err.empty()) {
        return invocation_response::failure(err, "DownloadFailure");
    }

    return invocation_response::success(base64_encoded_file, "application/base64");
}

std::function<std::shared_ptr<Aws::Utils::Logging::LogSystemInterface>()> GetConsoleLoggerFactory()
{
    return [] {
        return Aws::MakeShared<Aws::Utils::Logging::ConsoleLogSystem>(
            "console_logger", Aws::Utils::Logging::LogLevel::Trace);
    };
}

int main()
{
    using namespace Aws;
    SDKOptions options;
    options.loggingOptions.logLevel = Aws::Utils::Logging::LogLevel::Trace;
    options.loggingOptions.logger_create_fn = GetConsoleLoggerFactory();
    InitAPI(options);
    {
        Client::ClientConfiguration config;
        config.region = Aws::Environment::GetEnv("AWS_REGION");
        config.caFile = "/etc/pki/tls/certs/ca-bundle.crt";

        auto credentialsProvider = Aws::MakeShared<Aws::Auth::EnvironmentAWSCredentialsProvider>(TAG);
        S3::S3Client client(credentialsProvider, config);
        auto handler_fn = [&client](aws::lambda_runtime::invocation_request const& req) {
            return my_handler(req, client);
        };
        run_handler(handler_fn);
    }
    ShutdownAPI(options);
    return 0;
}

std::string encode(Aws::IOStream& stream, Aws::String& output)
{
    Aws::Vector<unsigned char> bits;
    bits.reserve(stream.tellp());
    stream.seekg(0, stream.beg);

    char streamBuffer[1024 * 4];
    while (stream.good()) {
        stream.read(streamBuffer, sizeof(streamBuffer));
        auto bytesRead = stream.gcount();

        if (bytesRead > 0) {
            bits.insert(bits.end(), (unsigned char*)streamBuffer, (unsigned char*)streamBuffer + bytesRead);
        }
    }
    Aws::Utils::ByteBuffer bb(bits.data(), bits.size());
    output = Aws::Utils::HashingUtils::Base64Encode(bb);
    return {};
}

std::string download_and_encode_file(
    Aws::S3::S3Client const& client,
    Aws::String const& bucket,
    Aws::String const& key,
    Aws::String& encoded_output)
{
    using namespace Aws;

    S3::Model::GetObjectRequest request;
    request.WithBucket(bucket).WithKey(key);

    auto outcome = client.GetObject(request);
    if (outcome.IsSuccess()) {
        AWS_LOGSTREAM_INFO(TAG, "Download completed!");
        auto& s = outcome.GetResult().GetBody();
        return encode(s, encoded_output);
    }
    else {
        AWS_LOGSTREAM_ERROR(TAG, "Failed with error: " << outcome.GetError());
        return outcome.GetError().GetMessage();
    }
}

    This Lambda function expects an input payload to contain an S3 bucket and S3 key. It then downloads that resource from S3, encodes it as base64, and sends it back as the response of the Lambda function. This can be useful to display an image in a webpage, for example.

  4. Next, create the following CMakeLists.txt file in the same directory. 
    cmake_minimum_required(VERSION 3.5)
set(CMAKE_CXX_STANDARD 11)
project(encoder LANGUAGES CXX)

find_package(aws-lambda-runtime REQUIRED)
find_package(AWSSDK COMPONENTS s3)

add_executable(${PROJECT_NAME} "main.cpp")
target_link_libraries(${PROJECT_NAME} PUBLIC
                      AWS::aws-lambda-runtime
                       ${AWSSDK_LINK_LIBRARIES})

aws_lambda_package_target(${PROJECT_NAME})
  5. Follow the same build steps as before: 
    $ mkdir build
$ cd build
$ cmake3 .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH=~/out
$ make
$ make aws-lambda-package-encoder

    Notice how the target name for packaging has changed to aws-lambda-package-encoder. The CMake function aws_lambda_package_target() always creates a target based on its input name.

    You should now have a file named “encoder.zip” in your build directory.

  6. Before you create the Lambda function, modify the IAM role that you created earlier to allow it to access S3. 
    $ aws iam attach-role-policy \
--role-name lambda-cpp-demo \
--policy-arn arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess
  7. Using the AWS CLI, create the new Lambda function: 
    $ aws lambda create-function \
--function-name encode-file \
--role <specify the same role arn used in the prior Lambda> \
--runtime provided \
--timeout 15 \
--memory-size 128 \
--handler encoder \
--zip-file fileb://encoder.zip
  8. Using the AWS CLI, run the function. Make sure to use a S3 bucket in the same Region as the Lambda function: 
    $ aws lambda invoke --function-name encode-file --payload '{"s3bucket": "your_bucket_name", "s3key":"your_file_key" }' base64_image.txt

    You can use an online base64 image decoder and paste the contents of the output file to verify that everything is working. In a real-world scenario, you would inject the output of this Lambda function in an HTML img tag, for example.

Conclusion

With the new Lambda Runtime API, a new door of possibilities is open. This C++ runtime enables you to do more with Lambda than you ever could have before.

More in-depth details, along with examples, can be found on the GitHub repository. With it, you can start writing Lambda functions with C++ today. AWS will continue evolving the contents of this repository with additional enhancements and samples. I’m so excited to see what you build using this runtime. I appreciate feedback sent via issues in GitHub.

Happy hacking!