AWS Compute Blog

Migrating AWS Lambda functions from the Go1.x runtime to the custom runtime on Amazon Linux 2

Update Feb 13, 2024: Support for the provided runtime family, as well as 3rd-party container images, was added in the v1.18 release of aws-lambda-go. Functions that have not had their dependencies updated since July 2020 will need to be re-compiled to prevent invocation errors when using the provided.al2 or provided.al2023 runtimes. If you prefer not to rename your executable to bootstrap, you can instead create a shell script called bootstrap and use that to run your executable. Be sure to set the correct permissions on your bootstrap file.

Update Dec 15, 2023: Lambda has since added Amazon Linux 2023 to the provided runtime family. We recommend that customers upgrading from the go1.x runtime use the new provided.al2023 runtime as their first preference. The provided.al2 runtime remains available and will be supported until at least June 30, 2025.

This post is written by Micah Walter, Senior Solutions Architect, Yanko Bolanos, Senior Solutions Architect, and Ramesh Mathikumar, Senior DevOps Consultant.

This blog post describes our plans to improve performance and streamline the user experience for customers writing AWS Lambda functions using Go.

Today, customers using Go with Lambda can either use the go1.x runtime, or use the provided.al2 runtime. Going forward, we plan to deprecate the go1.x runtime in line with the end-of-life of Amazon Linux 1, currently scheduled for December 31, 2023.

Customers using the go1.x runtime should migrate their functions to the provided.al2 runtime to continue to benefit from the latest runtime updates and security patches. Customers who deploy Go functions using container images who are currently using the go1.x base container image should similarly migrate to the provided.al2 base image.

Lambda remains committed to supporting the Go programming language. Using the provided.al2 runtime offers several benefits over the go1.x runtime. First, it supports running Lambda functions on AWS Graviton2 processors, offering up to 34% better price-performance compared to functions running on x86_64 processors. Second, it offers a streamlined implementation with a smaller deployment package and faster function invoke path. Finally, this change aligns Go with other languages that also compile to native code such as Rust or C++, which also run on the provided.al2 runtime.

This migration in most cases does not require any code changes. The only changes relate to how you build your deployment package and configure your function. This blog post outlines the steps required to update your build scripts and tooling to use the provided.al2 runtime for your Go functions.

There is a difference in Lambda billing between the go1.x runtime and the provided.al2 runtime. With the go1.x runtime, Lambda does not bill for time spent during function initialization (cold start), whereas with the provided.al2 runtime Lambda includes function initialization time in the billed function duration. Since Go functions typically initialize very quickly, and since Lambda reduces the number of initializations by re-using function execution environments for multiple function invokes, in practice the difference in your Lambda bill should be very small.

Compiling for the provided.al2 runtime

In order to run a compiled Go application on Lambda, you must compile your code for Linux. While the go1.x runtime allows you to use any executable name, the provided.al2 runtime requires you to use bootstrap as the executable name. On macOS and Linux, here’s the simplest form of the build command:

GOARCH=amd64 GOOS=linux go build -o bootstrap main.go

This build command creates a Go binary file called bootstrap compatible with the x86_64 instruction set for Lambda. To compile for AWS Graviton processors, set GOARCH=arm64 in the preceding command.

The final step is to compress this binary into a ZIP file deployment package, ready to deploy to Lambda:

zip myFunction.zip bootstrap

For users compiling on Windows, Go supports compiling for Linux without using a Linux virtual machine or build container. However, Lambda uses POSIX file permissions, which must be set correctly. Lambda provides a helper tool which builds a deployment package that is valid for Lambda—see the Lambda documentation for details. Existing users of this tool should update to the latest version to make sure their build scripts are up-to-date.

Removing the RPC dependency

The go1.x runtime uses two processes within the Lambda execution environment to route requests to your handler function. The first process, which is included in the runtime, retrieves function invocation requests from the Lambda runtime API, and uses RPC to pass the invoke to the second process. This second process runs the executable which you deploy, and comprises the aws-lambda-go package and your function code. The aws-lambda-go package receives the RPC request and executes your function.

The following runtime architecture diagram for the go1.x runtime shows the runtime client process calling the runtime API to retrieve a function invocation and using RPC to call a separate process containing the function code.

Execution environment

Go functions deployed to the provided.al2 runtime use a simpler, single-process architecture. When building the Go executable, you include the same aws-lambda-go package as before. However, in this case the aws-lambda-go package acts as the runtime client, retrieving invocation requests from the runtime API directly.

The following runtime architecture diagram shows a Go function running on the provided.al2 runtime. A single process retrieves the function invocation from the runtime API and executes the function code.

Go running on provided.al2

Removing the additional process and RPC hop streamlines the function execution path, resulting in faster invokes. You can also remove the RPC component from the aws-lambda-go package, giving a smaller binary size and faster code loading during cold starts. To remove the RPC dependency, add the lambda.norpc tag to your build command:

GOARCH=amd64 GOOS=linux go build -tags lambda.norpc -o bootstrap main.go

Creating a new Lambda function

Once your deployment package is ready, you can create a new Lambda function using the provided.al2 runtime using the Lambda console:

Creating in the Lambda console

Migrating existing functions

If you have existing Lambda functions that use the go1.x runtime, you can migrate these functions by following these steps:

  1. Recompile your binary using the preceding commands, making sure to name your binary bootstrap.
  2. If you are using the same instruction set architecture, open the runtime settings and switch the runtime to “Provide your own bootstrap on Amazon Linux 2”.
  3. Upload the new version of your binary as a zip file.
    Edit runtime settings

Note: The handler value is not used by the provided.al2 runtime, nor the aws-lambda-go library, and may be set to any value. We recommend the setting the value to bootstrap to help with migrating between go1.x and provided.al2.

To switch instruction set architecture to Graviton (arm64), save your changes, and then re-open the runtimes settings to make the architecture change.

Migrating Go1.x Lambda container images

Lambda allows you to run your Go code as a container image. Customers that are using the go1.x base image for Lambda containers must migrate to the provided.al2 base image. The Lambda documentation includes instructions on how to build and deploy Go functions using the provided.al2 base image.

The following Dockerfile uses a two-stage build to avoid unnecessary layers and files in your final image. The first stage of the process builds the application. This stage installs Go, downloads the dependencies for the code, and compiles the binary. The second stage copies the executable to a new container without the dependencies of the build process.

  1. Create a Dockerfile in your project directory: 
    FROM public.ecr.aws/lambda/provided:al2 as build
# install compiler
RUN yum install -y golang
RUN go env -w GOPROXY=direct
# cache dependencies
ADD go.mod go.sum ./
RUN go mod download
# build
ADD . .
RUN go build -tags lambda.norpc -o /main
# copy artifacts to a clean image
FROM public.ecr.aws/lambda/provided:al2
COPY --from=build /main /main
ENTRYPOINT [ "/main" ]
  2. Build your Docker image with the Docker build command: 
    docker build -t hello-world .
  3. Authenticate the Docker CLI to your Amazon ECR registry: 
    aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 123456789012.dkr.ecr.us-east-1.amazonaws.com
  4. Tag and push your image to the Amazon ECR registry: 
    docker tag hello-world:latest 123456789012.dkr.ecr.us-east-1.amazonaws.com/hello-world:latest

docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/hello-world:latest

You can now create or update your Go Lambda function to use the new container image.

Changes to tooling

To migrate your functions from the go1.x runtime to the provided.al2 runtime, you must make configuration changes to your build scripts or CI/CD configurations. Here are some common examples.

Makefiles and build scripts

If you use Makefiles or custom build scripts to build Go functions, you must modify to ensure the executable file is named bootstrap when deploying to the provided.al2 runtime.

Here is an example Makefile which compiles the main.go file into an executable called bootstrap in the bin folder. It also creates a zip file, which you can deploy to Lambda using the console or via the AWS CLI.

GOARCH=arm64 GOOS=linux go build -tags lambda.norpc -o ./bin/bootstrap
(cd bin && zip -FS bootstrap.zip bootstrap)

CloudFormation

If you deploy your Lambda functions using AWS CloudFormation templates, change the Handler and Runtime settings under Properties:

Resources:
  Function:
    Type: AWS::Serverless::Function
    Properties:
      Handler: bootstrap
      Runtime: provided.al2
      ... # Other required properties

AWS Serverless Application Model

If you use the AWS Serverless Application Model (AWS SAM) to build and deploy your Go functions, make the same changes to the Handler and Runtime settings as for CloudFormation. You must also add the BuildMethod:

Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function
    Metadata:
      BuildMethod: go1.x 
    Properties:
      CodeUri: hello-world/ # folder where your main program resides
      Handler: bootstrap
      Runtime: provided.al2
      Architectures:
        - x86_64

Cloud Development Kit (CDK)

If you use the AWS Cloud Development Kit (AWS CDK), you can compile your Go executable and place it under a folder in your project. Next, specify the location by using awslambda.Code_FromAsset, and AWS CDK packages the binary into a zip file and uploads it.

// Go CDK
awslambda.NewFunction(stack, jsii.String("HelloHandler"), &awslambda.FunctionProps{
    Code:         awslambda.Code_FromAsset(jsii.String("lambda"), nil), //folder where bootstrap executable is located
    Runtime:      awslambda.Runtime_PROVIDED_AL2(),
    Handler:      jsii.String("bootstrap"), // Handler named bootstrap
    Architecture: awslambda.Architecture_ARM_64(),
})

Taking this further, AWS CDK can perform build commands as part of your AWS CDK build process by using the native AWS CDK bundling functionality. With the bundling parameter, AWS CDK can perform steps before staging the files in the cloud assembly. Instead of placing the binary file, place the Go code in a folder and use the Bundling option to compile the code in a Docker container.

This example uses the golang:1.20.1 Docker image. After compilation, AWS CDK creates a zip file with the binary and creates the Lambda function:

// Go CDK
awslambda.NewFunction(stack, jsii.String("HelloHandler"), &awslambda.FunctionProps{
        Code: awslambda.Code_FromAsset(jsii.String("go-lambda"), &awss3assets.AssetOptions{
                Bundling: &awscdk.BundlingOptions{
                        Image: awscdk.DockerImage_FromRegistry(jsii.String("golang:1.20.1")),
                        Command: &[]*string{
                                jsii.String("bash"),
                                jsii.String("-c"),
                                jsii.String("GOCACHE=/tmp go mod tidy && GOCACHE=/tmp GOARCH=arm64 GOOS=linux go build -tags lambda.norpc -o /asset-output/bootstrap"),
                        },
                },
        }),
        Runtime:      awslambda.Runtime_PROVIDED_AL2(),
        Handler:      jsii.String("bootstrap"),
        Architecture: awslambda.Architecture_ARM_64(),
})

If, after upgrading, you receive a Phase: init Status: timeout error, upgrade the aws-lambda-go and aws-sdk-go packages.

Conclusion

Lambda is deprecating the go1.x runtime in line with Amazon Linux 1 end-of-life, scheduled for December 31, 2023. Customers using Go with Lambda should migrate their functions to the provided.al2 runtime. Benefits include support for AWS Graviton2 processors with better price-performance, and a streamlined invoke path with faster performance.

For more serverless learning resources, visit Serverless Land.