Resizing volumes and instances using ServiceNow and AWS

The AWS Service Management Connector for ServiceNow enables ServiceNow end users to provision, manage, and operate AWS resources natively through ServiceNow. This lets our customers connect a technical operation with a business workflow, perhaps requiring approvals from management or other teams. The key in all of this is empowering and enabling end-users, thereby removing manual processes.

This post shows work we have done with our customer – Terex Corporation – to automate the common processes of resizing Amazon Elastic Compute Cloud (Amazon EC2) Instances and Amazon Elastic Block Storage (Amazon EBS) volumes using their internal IT Service Management tool. This has resulted in self-service ability for users, while meeting corporate requirements. In this post, we will demonstrate sample workflows using custom AWS Systems Manager Automation runbooks and integrating these with ServiceNow.

Terex Corporation is a diversified global manufacturer of a broad range of equipment that is focused on delivering reliable, customer-driven solutions for many applications. This includes the construction, infrastructure, quarrying, mining, transportation, refining, energy, utility, and manufacturing industries. Terex reports in two business segments: Aerial Work Platforms and Materials Processing.

Customer Challenge

At Terex, it is important that that development and production environments are correctly managed. Many of Terex’s workloads run on Amazon EC2 Windows Instances with Amazon EBS volumes. For example, Amazon EC2 instance resizing is a common reoccurring task which was previously an operational overhead and time-consuming process. This is primarily due to reliability and recoverability aspects being incorporated into Terex’s resizing operations runbooks. For example, these aspects include creating an AMI as a backup and making sure that the instance will include the latest drivers and configurations that support the most recent generation instance types as described in the AWS user guide for Windows Instances.

The two key challenges that have arisen since adopting the cloud at Terex is the ability to flex resources up and down quickly while focusing on cost initiatives. Amazon EBS volumes are charged by the amount of storage that you provision in GB per month until you release the storage. Therefore, Terex was looking for a solution that could help them provision smaller Amazon EBS volumes while letting them programmatically expand storage according to usage thus decreasing Amazon EBS volume operating costs.

Specifically, the expansion of root volumes (used by the operating systems) is required to prevent crashes and failures due to lack of space. This includes the benefit of having an automated process without requiring user intervention if this occurs outside of business hours.

Terex has seen an opportunity for improvement of existing processes through its ITSM tool. Previously, requests from end-users were sent to the Cloud Centre of Excellence (CCoE) team to change the specifications of the Amazon EC2 instances and/or Amazon EBS volumes. As these changes have cost implications, we worked with Terex to implement a process. This includes the required business approvals, along with a technical workflow to implement these changes. The previous process required end-users to open a ticket, and then have an engineer review it and manually execute the task. Varying workloads and schedules meant that these requests could take hours or even a few days. Moving to an automated process that is driven by tickets has brought this down to minutes. Furthermore, although AWS Systems Manager contains existing Automation runbooks with actions to execute these tasks, Terex was looking to adapt their aforementioned custom operational runbooks to satisfy their CCoE standards that are critical for the solution.

The solution discussed in the following sections allows end-users freedom within the definitions provided by the business. Then, this allows for flexibility along with cost-conscious considerations to be embedded within it.

Solution Overview

In the following solution, we will leverage ServiceNow as the IT Service Management workflow:

1. A set of AWS Systems Manager Automation Documents are set up to implement the tasks of resizing Amazon EBS volumes and Amazon EC2 Instances.
2. The documents are surfaced in ServiceNow using the AWS Service Management Connector.
3. Additional components used by the documents are deployed, and these include: an Amazon Simple Notification Service (Amazon SNS) Topic and required AWS Identity and Access Management (IAM) Roles.
4. We also provide an optional integration with AWS Service Catalog.

When users submit a request for an operation in ServiceNow, the following workflow is executed:

• Approvals from within the company are required
• Once approved, the relevant Systems Manager Document is triggered
  • The Systems Manager Document contains the business logic, what, and how to apply changes
• Users are notified upon completion of the request

Although the logic is quite similar, each workflow has slightly different logic applied within it:

Workflow 1 – Resizing of Amazon EC2 Instances

This automation will provide the ability to resize an Amazon EC2 Instance to a desired instance type. Before doing so, it will create snapshots of its root Amazon EBS volumes for backup purposes. Moreover, it will update the AWS NVMe, ENA, and AWS PV Drivers, which will make sure that the most recent generation instance types are supported. The workflow includes the following steps:

• Take a snapshot of the root Amazon EBS volume and verify the snapshot
• Install and upgrade AWS PV drivers.
• Install and upgrade ENA.
• Upgrade AWS NVMe drivers.
• Stop the instance.
• Update the instance type
• Start the instance
• Notify using Amazon SNS (optional)

Workflow 2 – Auto-Expansion of Windows root Volumes

This automation that provides the Instance ID will initially discover the Instance root volume, take an Amazon EBS Snapshot, modify the root Amazon EBS volume (/dev/sda1), and expand the Root Volume logical partition inside of the operating system. These are the steps in this workflow:

• Amazon EBS Snapshot of the volume is taken
• A tag is applied to the snapshot. The team will delete the tag programmatically once the instance hasn’t experienced any issues within the last 24 hours.
• The Amazon EBS volume is extended (within defined limits).
• The logical volume is extended inside of the Operating System.
• An optional Amazon SNS Notification is provided.

Prerequisites

To deploy and run through the process described in this post, you’ll need the following:

• An AWS Account with appropriate permissions to setup the connector (as follows) and the services mentioned in this exercise.
• A clean ServiceNow Instance, or a Personal Developer Instance – see ServiceNow’s documentation on how to obtain a free instance if you don’t already have one.
• An installed and configured AWS Service Management Connector for ServiceNow.
  • Make sure that you check both the AWS Service Catalog and AWS Systems Manager under the AWS service integrations.
• Familiarity with AWS Systems Manager and AWS Service Catalog.

Walkthrough

As described above, the solution leverages AWS Systems Manager Documents to perform the logic on a requested Amazon EC2 Instance or Amazon EBS volume. For your convenience, we have created an AWS CloudFormation template that will deploy those automatically into your account in a specific region.

Step 1: Launch a CloudFormation template to deploy the required AWS resources

We have built a CloudFormation template that will deploy the above components into your AWS account in an automated fashion.

• Log in to your AWS Account
• Launch Stack
• Select Next
• For parameters:
  • Optionally modify the default name for the provided Amazon SNS Topic
• Select Next, Next
• Check the box I acknowledge that AWS CloudFormation might create IAM resources, and select Create stack
• Wait until the Stack status changes to CREATE_COMPLETE

Step 2: Connect to your ServiceNow environment

The connector synchronizes data every 31 minutes, so you can either wait or manually trigger the sync job. A sync job will surface all of your AWS Systems Manager Documents in ServiceNow. Therefore, we recommend that you deactivate the documents that aren’t needed to make browsing and managing these on your ServiceNow side easier (see the related section in our documentation).

Navigate to your ServiceNow service portal, where under Categories you should see an AWS Systems Manager category.

• Select either Resize EC2 Instance or Expand EBS Root Volume
• Provide the required inputs
  • When selecting Resize EC2 Instance, type the desired instance type, i.e., t3.large, and when selecting Expand EBS Root Volume, the workflow will expand the Amazon EBS volume and Windows OS Partition by 20% programmatically.
• Select Order Now, and if prompted, Checkout
• You will be taken to the Request Summary page

Within a few minutes, you will see the following outputs:

To view more detailed information, log in to your AWS account and navigate to AWS Systems Manager and then select Run Command. If the action has completed, you will find it under the Command history tab, otherwise, it’ll appear on the default running Commands tab.

Service Actions

AWS Service Catalog provides a function called Service Actions. In turn, this lets an end-user (a “consumer” of a set of Products) trigger certain actions against something that they have provisioned. This could be a simple create/update/delete, or a greatly extended leveraging of the AWS System Manager Automation Documents.

Different types of users will interact differently regarding AWS resource consumption. Some will leverage an ITSM tool, such as ServiceNow, some will use the console, while others will consume via programmatic interfaces (Infrastructure-as-Code, AWS Command Line Interface (AWS CLI), or AWS SDK).

Service Actions let all of these types of users run those predefined actions against a provisioned product. Terex surfaces these actions in the console as well. This is so that users who provision Service Catalog Products can also interact and apply changes to them in a tested and resilient fashion.

Further enhancements

• There are many additional use cases for ITSM-driven workflows that enable end-users. These include launching Amazon WorkSpaces, Amazon SageMaker instances and more.
• Auto-expansion and increasing Amazon EBS volume space can be automated using Amazon CloudWatch Alarms thus removing the requirement for a request or approval, or possibly only as per defined thresholds.

Conclusion

In this post, we have described a real-life scenario where a CCoE has enabled end-users to customize their AWS resource usage with a self-service approach. This allows users move at their own pace while still meeting the company’s guidelines and requirements to properly manage their environment.  Go to the documentation to get started with the AWS Service Management Connector for ServiceNow.

Authors: