Serverless COBOL: Rejuvenating legacy code with open source software — Part 2

The benefits of the serverless architecture are not reserved to newly written applications. Legacy code can be combined with leading-edge technologies by deploying them in a cloud platform. This will ensure reuse of the massive existing legacy assets and further extend their life, thus using them in new ways.

Serverless COBOL: Rejuvenating legacy code with open source software — Part 1 explained why and how GnuCOBOL, together with AWS Lambda, can be used to extend the life of legacy code. It also explained the benefits of open source when looking to modernize your legacy applications.

As explained in part 1 of this two-part series, GnuCOBOL is a free and open source implementation of the COBOL programming language. COBOL is a compiled English-like computer programming language designed for business use. It is an imperative, procedural and, since 2002, object-oriented language. COBOL is primarily used in business, finance, and administrative systems for companies and governments.

AWS Lambda is a serverless compute service that lets you run code without provisioning or managing servers, creating workload-aware cluster scaling logic, maintaining event integrations, or managing runtimes.

In Part 2, we will walk through a detailed technical implementation that uses an open source project. The open source project contains the COBOL code and the CI/CD scripts that are available from the GitHub repository.

CI/CD (continuous integration and continuous deployment) is a DevOps component. CI/CD allows building, testing, and deploying applications rapidly and reliably, while improving quality and reducing time to market.

GitHub is an AWS Partner Network (APN) with the AWS DevOps Competency.

Figure 1: Architecture and process flow

Figure 1 depicts the architecture of the solution, together with the process flow of the technical implementation that we are discussing in this post.

We will walk through each component of this architecture and process flow.

1. GitHub repository, GitHub secrets, GitHub Actions, GitHub workflows

GitHub repository: Our source code is ultimately always pushed from our local machine to the GitHub repository. The GitHub repository contains many features, some of which we have used in this technical implementation. We will go into detail about some of these features later on.
GitHub secrets: Secrets are encrypted environment variables that you create in an organization, repository, or repository environment. The secrets that you create are available to use in GitHub Actions workflows.
GitHub Actions: A GitHub Action is an individual unit of functionality that can be combined with other GitHub Actions to create workflows, which are triggered in response to certain GitHub events—for example, pull, push, or commit.
GitHub workflows: Workflows run inside managed environments on GitHub-hosted servers. By constructing a workflow using GitHub Actions, we can deliver CI/CD functionality for our COBOL program deployment on Lambda, directly from GitHub.

GitHub secrets

When forking the repository to run it on your own, you must create three GitHub secrets in your own repository for workflows to work properly:

${{ secrets.AWS_ACCESS_KEY_ID }}: The access key under which the workflow will run
${{ secrets.AWS_SECRET_ACCESS_KEY }}: The secret key validating the use of the preceding access key
${{ secrets.AWS_REGION }}: The Region in which you want the workflow to be deployed and run

The credentials given to the identity of the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY via AWS IAM must grant permissions to deploy and run Lambda functions and to create corresponding Amazon API Gateway definitions. Additionally, an Amazon Simple Storage Service (Amazon S3) bucket must be created to import the uploaded artifacts in AWS when they get deployed.

GitHub Actions and workflows

Implemented as a GitHub Action, the workflow—scripted in lambda-cobol.sh—includes the following:

A Docker image is constructed to install the GnuCOBOL compiler and its dependencies on top of the base Amazon Linux image. The purpose of such a container is to take advantage of the isolation provided by containers. Consequently, the build environment is fully controlled.
This COBOL builder imports the source code of hello-world.cob and compiles it to generate an x86 native binary named hello-world.

Note: The results of the various executions of the workflow in this repo can be seen in the Actions tab.

2. AWS SAM CLI for SAM build and SAM deploy

The AWS Serverless Application Model (AWS SAM) CLI provides a Lambda-like execution environment to allow locally building, testing, and debugging applications defined by SAM templates or through the AWS Cloud Development Kit (AWS CDK).

The following are also contained in the workflow as scripted in lambda-cobol.sh:

The binary is packaged, via the AWS SAM CLI, with other required runtime artifacts. The libcob library is required by GnuCOBOL. The shell script bootstrap implements the requirements of custom Lambda runtimes.
This package is deployed on the Lambda service via the AWS SAM CLI. The AWS SAM CLI is used to check proper deployment; it invokes the function synchronously.

3. CloudFormation stack

As can also be seen in the lambda-cobol.sh script, the AWS SAM description is processed by AWS Lambda and AWS CloudFormation to deploy the function and the API. The function and the API are defined as CloudFormation stacks.

The AWS SAM model for our COBOL Lambda instance is in the file lambda-cobol-sam.yaml. The file contains a single AWS::Serverless::Function with its parameters. During deployment, AWS SAM and CloudFormation expand it into six more granular resources that can be located with Resource creation Initiated as a ResourceStatusReason of the CloudFormation stack.

4. The hello-world binary and the API

As previously mentioned, the compiled COBOL source code is deployed as a binary in AWS Lambda, and published as an API on the Amazon API Gateway.

The deployed COBOL program is now accessible over HTTP via the definition of a REST service on the Amazon API Gateway.

The curl command calls the URL with the obtained DNS to validate the proper execution of the newly deployed Lambda instance. The URL for curl is built following the syntax: https://$API_ID.execute-api.$AWS_REGION.amazonaws.com/Prod/$LAMBDA_NAME. $API_ID, which is dynamic and obtained via an Amazon apigateway get-rest-apis CLI command.

If a previous deployment of the CloudFormation stack is active, it gets deleted before the AWS SAM build in order to start the entire CloudFormation deployment process all over again.

Note: The version of GnuCOBOL currently used is v2.2. A version 3.1 was published in late December 2020; however, its libcob runtime library has hardwired dependencies on recent Linux system libraries that are not yet available with proper version in Lambda runtime. We’ll bump to newest GnuCOBOL when Lambda runtime gets updated.

Lastly, here is an excerpt of the logs of the last execution related to the Lambda service operated from the AWS SAM CLI:

### execution date: Thu Jul  8 01:59:59 UTC 2021
 
### Check existing Lambdas functions...
{
    "Functions": [
        {
            "FunctionName": "Hello-world-Python",
            "FunctionArn": "arn:aws:lambda:us-east-1:514764745669:function:Hello-world-Python",
            "Runtime": "python3.8",
            "Role": "arn:aws:iam::514764745669:role/service-role/Hello-world-Python-role-lyqky200",
            "Handler": "lambda_function.lambda_handler",
            "CodeSize": 299,
            "Description": "",
            "Timeout": 3,
            "MemorySize": 128,
            "LastModified": "2021-02-06T10:48:38.267+0000",
            "CodeSha256": "fI06ZlRH/KN6Ra3twvdRllUYaxv182Tjx0qNWNlKIhI=",
            "Version": "$LATEST",
            "TracingConfig": {
                "Mode": "PassThrough"
            },
            "RevisionId": "d90d1b6d-667c-46d9-b9d5-e7fdefdfc004",
            "PackageType": "Zip"
        }
    ]
}
 
### Starting SAM build...

Build Succeeded

Built Artifacts  : build
Built Template   : build/template.yaml

Commands you can use next
=========================
[*] Invoke Function: sam local invoke -t build/template.yaml
[*] Deploy: sam deploy --guided --template-file build/template.yaml
    
 
### Starting SAM deployment...

	Deploying with following values
	===============================
	Stack name                   : lambda-cobol-stack
	Region                       : us-east-1
	Confirm changeset            : False
	Deployment s3 bucket         : net.didier-durand.lambda-code
	Capabilities                 : ["CAPABILITY_IAM"]
	Parameter overrides          : {}
	Signing Profiles             : {}

Initiating deployment
=====================

Waiting for changeset to be created..

CloudFormation stack changeset
-------------------------------------------------------------------------------------------------
Operation                LogicalResourceId        ResourceType             Replacement            
-------------------------------------------------------------------------------------------------
+ Add                    HelloWorldCobolGetReso   AWS::Lambda::Permissio   N/A                    
                         urcePermissionProd       n                                               
+ Add                    HelloWorldCobolRole      AWS::IAM::Role           N/A                    
+ Add                    HelloWorldCobol          AWS::Lambda::Function    N/A                    
+ Add                    ServerlessRestApiDeplo   AWS::ApiGateway::Deplo   N/A                    
                         ymentaf1c952223          yment                                           
+ Add                    ServerlessRestApiProdS   AWS::ApiGateway::Stage   N/A                    
                         tage                                                                     
+ Add                    ServerlessRestApi        AWS::ApiGateway::RestA   N/A                    
                                                  pi                                              
-------------------------------------------------------------------------------------------------

Changeset created successfully. arn:aws:cloudformation:us-east-1:514764745669:changeSet/samcli-deploy1625709703/799c3553-8361-4c7a-8b88-dc16def5a578


2021-07-08 02:01:54 - Waiting for stack create/update to complete

CloudFormation events from changeset
-------------------------------------------------------------------------------------------------
ResourceStatus           ResourceType             LogicalResourceId        ResourceStatusReason   
-------------------------------------------------------------------------------------------------
CREATE_IN_PROGRESS       AWS::IAM::Role           HelloWorldCobolRole      -                      
CREATE_IN_PROGRESS       AWS::IAM::Role           HelloWorldCobolRole      Resource creation      
                                                                           Initiated              
CREATE_COMPLETE          AWS::IAM::Role           HelloWorldCobolRole      -                      
CREATE_IN_PROGRESS       AWS::Lambda::Function    HelloWorldCobol          -                      
CREATE_IN_PROGRESS       AWS::Lambda::Function    HelloWorldCobol          Resource creation      
                                                                           Initiated              
CREATE_COMPLETE          AWS::Lambda::Function    HelloWorldCobol          -                      
CREATE_IN_PROGRESS       AWS::ApiGateway::RestA   ServerlessRestApi        -                      
                         pi                                                                       
CREATE_IN_PROGRESS       AWS::ApiGateway::RestA   ServerlessRestApi        Resource creation      
                         pi                                                Initiated              
CREATE_COMPLETE          AWS::ApiGateway::RestA   ServerlessRestApi        -                      
                         pi                                                                       
CREATE_IN_PROGRESS       AWS::Lambda::Permissio   HelloWorldCobolGetReso   Resource creation      
                         n                        urcePermissionProd       Initiated              
CREATE_IN_PROGRESS       AWS::ApiGateway::Deplo   ServerlessRestApiDeplo   -                      
                         yment                    ymentaf1c952223                                 
CREATE_IN_PROGRESS       AWS::Lambda::Permissio   HelloWorldCobolGetReso   -                      
                         n                        urcePermissionProd                              
CREATE_COMPLETE          AWS::ApiGateway::Deplo   ServerlessRestApiDeplo   -                      
                         yment                    ymentaf1c952223                                 
CREATE_IN_PROGRESS       AWS::ApiGateway::Deplo   ServerlessRestApiDeplo   Resource creation      
                         yment                    ymentaf1c952223          Initiated              
CREATE_IN_PROGRESS       AWS::ApiGateway::Stage   ServerlessRestApiProdS   -                      
                                                  tage                                            
CREATE_IN_PROGRESS       AWS::ApiGateway::Stage   ServerlessRestApiProdS   Resource creation      
                                                  tage                     Initiated              
CREATE_COMPLETE          AWS::ApiGateway::Stage   ServerlessRestApiProdS   -                      
                                                  tage                                            
CREATE_COMPLETE          AWS::Lambda::Permissio   HelloWorldCobolGetReso   -                      
                         n                        urcePermissionProd                              
CREATE_COMPLETE          AWS::CloudFormation::S   lambda-cobol-stack       -                      
                         tack                                                                     
-------------------------------------------------------------------------------------------------

Successfully created/updated stack - lambda-cobol-stack in us-east-1

 
 
### Inkoking deployed Lambda synchronously from CLI...
{
    "StatusCode": 200,
    "ExecutedVersion": "$LATEST"
}
invocation result:

  {
    "isBase64Encoded": false,
    "statusCode": 200, 
    "body": "Hello World from COBOL!" 
  } 
 
### Obtaining API gateway config...
{
    "items": [
        {
            "id": "azi318aybb",
            "name": "lambda-cobol-stack",
            "createdDate": "2021-07-08T02:02:21+00:00",
            "version": "1.0",
            "apiKeySource": "HEADER",
            "endpointConfiguration": {
                "types": [
                    "EDGE"
                ]
            },
            "tags": {
                "aws:cloudformation:logical-id": "ServerlessRestApi",
                "aws:cloudformation:stack-id": "arn:aws:cloudformation:us-east-1:514764745669:stack/lambda-cobol-stack/7191d740-df90-11eb-8764-0e69690dd155",
                "aws:cloudformation:stack-name": "lambda-cobol-stack"
            },
            "disableExecuteApiEndpoint": false
        }
    ]
}
api id: azi318aybb
 
### Running curl https request to https://azi318aybb.execute-api.us-east-1.amazonaws.com/Prod/lambda-cobol-hello-world ...
Hello World from COBOL!

Feel free to fork and replicate this repo in your own environment following the setup detailed in this post. We welcome all feedback and suggestions for extensions.

AWS Open Source Blog