AWS Cloud Operations Blog
Leveraging Infrastructure as Code for AWS Mainframe Modernization with AWS CloudFormation
AWS Mainframe Modernization service supports AWS CloudFormation templates to manage environments and applications, allowing customers to leverage best practices of Infrastructure-as-Code (IaC) and DevOps. This post defines the architectural improvements fostered by CloudFormation for mainframe applications modernized with AWS cloud. We’ll then cover the benefits of IaC in specific contexts for modernized mainframe applications. We’ll wrap up by highlighting some example use cases supported by the AWS Mainframe Modernization service when deploying and managing the service using CloudFormation.
New Architectural Capabilities Delivered by AWS Mainframe Modernization
AWS Mainframe Modernization service is a cloud-native platform to modernize, migrate, execute and operate mainframe applications. In addition to analysis and transformation tools, it provides a fully-managed and resilient runtime environment making it easier to run and operate modernized applications. It brings deep new architectural possibilities. Mainframe logical partitions (LPARs) usually execute large monoliths tying together numerous applications. With AWS Mainframe Modernization service, customers can replace those entangled and large LPARs with a set of smaller and independent environments and applications. These execute separately a fraction of the initial LPAR workload. Each of these pairs is managed in full independence of all others, potentially in separate AWS accounts to further increase isolation and improve security. This isolation improves the operational efficiency while offering new opportunities for more finely customized service level agreements (SLAs). CloudFormation allows for efficient management of this potentially large set of AWS resources.
Figure 1: Multi-region application management with CloudFormation StackSets
Figure 1 above represents a use case where a single application fragment of the initial monolith has been separated from the global workload. It is now managed independently from all other applications. A Codebuild pipeline produces its artifacts that are automatically distributed across all regions where the application can be hosted closer to its users. This artefact replication happens automatically via Simple Storage Service (Amazon S3) replication mechanisms throughout a set of buckets. They are configured for replication and defined in a single template instantiated as a unified StackSet. The cross-region features of the CloudFormation service transparently take care of their geographical distribution. Similarly, all pairs (application and environment) are created from a single template instantiated and geographically distributed by the CloudFormation service. Updates to running application instances are then easily coordinated to the unified CloudFormation StackSet template defining the geographically dispersed application instances.
This increased automation and system granularity improves cost efficiency of the target system. Both the cost of consumed cloud resources and the involvement of human resources are reduced:
- The pay-as-you-go model of AWS is more easily leveraged. When an environment and application pair is not needed, it can be stopped while other pairs keep running to serve their users. Different instances of transactional applications (CICS, IMS) for different geographical regions or lines of business are able to stop corresponding pairs after office hours.
- Each environment and application pair can be configured with different resources (compute instance type, Amazon Aurora cluster size, etc.) to simultaneously optimize its SLA and costs. For example, long-running batch jobs can be executed in instances with higher CPU clock frequency to reduce execution time. Such pairs can be stopped when the batch jobs do not need to run.
- Developers have increased personal productivity via independent environments and applications, avoiding frictions with other teams or colleagues. The separation improves security: no more need for shared development LPARs where confidential business data becomes more widely accessible than needed. Similarly, each member of the operations team can easily start its own environment and application. This helps validating a new system configuration on an exact replica of the production environment. Tests become more relevant and quality is improved.
Support of CloudFormation templates fosters mainframe application architecture composed of granular environments and applications. They are optimally defined to minimize costs and improve SLA, while reducing operational complexity.
Mainframe Infrastructure as Code via AWS CloudFormation
CloudFormation gives you an easy way to model a collection of related AWS and third-party resources, and provision them quickly and consistently. They are managed throughout their lifecycles, by treating them as Infrastructure-as-Code. A CloudFormation template describes your desired resources and their dependencies so you can launch and configure them together as a stack. You can manage and provision stacks across multiple AWS accounts and AWS Regions.
CloudFormation as an IaC provider, allows efficient management of the numerous components of this granular architecture while delivering auditability, repeatability and increased agility. AWS customers are using IaC for a more efficient, cost-effective, and higher quality management of large server fleets. This best practice can also be applied to mainframe applications.
Benefits
IaC can now be extensively applied to AWS Mainframe Modernization. Let’s review the benefits of using CloudFormation to manage infrastructure as code for mainframe applications.
Holistic Integration and homogeneous management of Cloud Services
Many AWS customers managing large cloud infrastructures rely extensively on IaC via CloudFormation to keep operations under control and meet their SLAs. A single CloudFormation template can manage the deployment of one or multiple resources across one or multiple AWS services. Managing multiple AWS services with one template makes it easier to integrate these AWS services together. Having most of the cloud resources definition stored and versioned in one single code repository like git, allows for better resource dependencies control and increased productivity. It is also self-documenting. Versioning allows more easily locating the last configuration changes to faster understand how they may relate to an issue being investigated. Managing migrated mainframe applications via CloudFormation removes silos for system management. It delivers a better quality of service by using the same processes across applications.
Increased Agility via Faster Deployments
Instantiating via a CloudFormation template is faster and less error-prone than defining configurations of AWS services via the AWS Management Console or even AWS Command Line Interface (AWS CLI) commands. Additional agility and productivity gains are also achieved via CloudFormation Stacksets. If the customer has an architecture where each developer has an independent environment under a single StackSet, they can all be updated once via the automated deployment of a new version of the StackSet. Similarly, when a customer has clones of the same application for different business lines or geographical branches, they can be managed as members of the same StackSet using the same unique CloudFormation template.
Reliability via immutable infrastructure
One AWS Cloud best practice to manage resources is to use immutable infrastructure. If a server needs an update or a fix, new servers are deployed instead of updating the ones in use. Instead of logging into a server and updating the software version, every change starts with a software push to the code repository. Since in-place changes are not allowed in immutable infrastructures, you can be sure about the state of the deployed system. Immutable infrastructures are inherently more consistent, reliable, and predictable, simplifying many aspects of software development and operations. It is listed as a best practice under the AWS Well Architected Reliability pillar, as well as following the DevOps best practice of managing “Cattle Not Pets”. The CloudFormation for AWS Mainframe Modernization service facilitates the use of IaC for creating and managing an immutable infrastructure. CloudFormation can also help detect configuration drift.
Improved Quality via Consistency
Improving quality requires repeatability. CloudFormation is foundational for it. Configuration definitions via the console may introduce human errors. Such manual activity should be avoided to ensure optimal quality of production environments. A customer can replicate the production configuration of an online or batch application to more efficiently diagnose an issue. For this, they would instantiate another CloudFormation stack based on the exact same template of code as the one used in production. The reproduction of the context of the issue is better ensured vs. a console-based manual rebuild. This also applies to testing new versions of the AWS Mainframe Modernization cloud-native fully-managed runtimes. The same cloning mechanism will ensure that the testing environment is identical to production. Consequently, the relevance of testing is increased and its results are of much higher accuracy. A SysOps team testing a new version of the AWS Mainframe Modernization runtime engine will better comprehend what will happen when the productive deployment is triggered.
Audit and Change Management
By enforcing CloudFormation as the main tool to manage environments and applications, configurations of the mainframe applications as a whole are visible in the templates. They can contain the configuration of the applications and their associated resources (database, Amazon Elastic File System (Amazon EFS), Amazon Simple Queue Service (Amazon SQS) queues, etc.). Definitions stored in a source code repository trace changes as subsequent versions over time. Practicing infrastructure as code means applying the same rigor of application code development to infrastructure provisioning. All configurations should be defined in a declarative way and stored in a source control system. With Infrastructure-as-Code, this code is also tested in the form of unit testing, functional testing, and integration testing. This allows better root cause analysis when new issues happen. Both the changes to the application source code and the application definitions can be reviewed altogether. They are reviewed in chronological order to better understand an issue and resolve it faster.
Compliance
From a compliance perspective, the chronological and unified journal of changes in conjunction with additional services like AWS CloudTrail is highly valuable. It allows SysOps teams to better demonstrate to auditors that systems remain compliant with stringent regulations (PCI DSS, SOC, GDPR, etc.). With AWS CloudFormation Guard, you can keep your AWS infrastructure and application resources in compliance with your company policy guidelines. It provides compliance administrators with a simple, policy-as-code language to define rules that can check for both required and prohibited resource configurations. It enables developers to validate their CloudFormation templates against those rules. With AWS CloudFormation Hooks, you can run code before creating, updating, or deleting a resource. The result of the code can be either to trigger a warning message, or prevent the deployment of a resource. Customers choose CloudFormation Hooks to provide the automatic and proactive enforcement of business requirements through a series of checks during provisioning. Resources are only provisioned if they pass these checks, effectively lowering security and compliance risks, reducing overhead of fixing compliance issues, and optimizing costs.
CloudFormation in action for AWS Mainframe Modernization
With CloudFormation templates, you can work with a broad set of AWS services, such as Amazon Simple Storage Service (Amazon S3), AWS Auto Scaling, Amazon CloudFront, Amazon DynamoDB, Amazon Elastic Compute Cloud (Amazon EC2), Amazon ElastiCache, AWS Relational Database Service (Amazon RDS), Elastic Load Balancing (ELB), AWS Identity and Access Management (IAM), and Amazon Virtual Private Cloud (Amazon VPC). Now let’s focus on some CloudFormation specifics for the AWS Mainframe Modernization service.
CloudFormation enables developers to create AWS resources in an orderly and predictable fashion. Resources are written in text files using JavaScript Object Notation (JSON) or Yet Another Markup Language (YAML) format. The templates require a specific syntax and structure that depends on the types of resources being created and managed. Resources are authored in JSON or YAML with any code editor such as AWS Cloud9. After checking code into a version control system, CloudFormation builds the specified services in a safe, repeatable manner.
A CloudFormation template is deployed into the AWS environment as a stack. Stacks are managed through the AWS Management Console, AWS CLI, or CloudFormation APIs. If you need to make changes to the running resources in a stack, update the stack via an updated template. Before making changes to resources, you can generate a change set, which is a summary of proposed changes. Change sets allow you to see how changes might impact running resources before implementing them.
Typically, the simplest definitions of AWS Mainframe Modernization service resources specify only required fields and assume system defaults for all optional resource properties.
For example, minimal definitions for an application and an environment with AWS Blu Age engine in a CloudFormation template can be:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "AWS M2 Environment and M2 Application for Blu Age engine with system defaults",
"Resources": {
"M2ApplicationBluAge": {
"Type": "AWS::M2::Application",
"Properties": {
"Name": "m2-app-bluage",
"EngineType": "bluage",
"Definition": {
"S3Location": "s3://bluage-bucket/planets/v1/application_definition_planets.json"
}
}
},
"M2EnvironmentBluAge": {
"Type": "AWS::M2::Environment",
"Properties": {
"Name": "m2-env-bluage",
"EngineType": "bluage",
"InstanceType": "M2.m5.large"
}
}
}
}
This template defines an application for the AWS Blu Age engine with its application definition found at the specified S3 location that you own. The environment has a default AWS Blu Age engine version, no storage file system, an AWS-managed encryption key, and a single host instance. It uses the default VPC, subnets, and security groups. Any applications deployed are not publicly accessible from outside the VPC.
A similar minimal resource definition can be achieved for resources with the Micro Focus engine:
{
"AWSTemplateFormatVersion": "2010-09-09T00:00:00.000Z",
"Description": "AWS M2 Environment and M2 Application for Micro Focus engine with system defaults",
"Resources": {
"M2ApplicationMicroFocus": {
"Type": "AWS::M2::Application",
"Properties": {
"Name": "m2-app-microfocus",
"EngineType": "microfocus",
"Definition": {
"S3Location": "s3://microfocus-bucket/carddemo/v1/application_definition_carddemo.json"
}
}
},
"M2EnvironmentMicroFocus": {
"Type": "AWS::M2::Environment",
"Properties": {
"Name": "m2-env-microfocus",
"EngineType": "microfocus",
"InstanceType": "M2.m5.large"
}
}
}
}
More fine-grained control over resources can be achieved by explicitly configuring various properties, especially in the case of environments. And using CloudFormation functionality like stack parameters and outputs increases the reusability of templates:
Yaml format can be used if preferred instead of Json for CloudFormation template definitions. The CloudFormation Designer is also a helpful tool. You can find supported AWS Mainframe Modernization resource types and properties for AWS CloudFormation in the user guide.
Go build CloudFormation templates
AWS Mainframe Modernization service enables granular deployments to better optimize cost, and better operate modernized and migrated mainframe applications. CloudFormation facilitates managing those systems composed of a larger number of AWS resources at scale. If you don’t use CloudFormation for AWS Mainframe Modernization service yet, you are encouraged to do so. The use cases outlined in this blog can help your organization improve its operational efficiency!
To learn more, please visit the AWS Mainframe Modernization Resources page.
About the authors: