Provision products and raise patch change requests in AWS via ServiceNow

ServiceNow is a popular cloud-based IT Service Management (ITSM) platform. Organizations use ServiceNow to manage incidents, track scheduled and planned infrastructure changes, manage new service requests and track configuration items across IT systems. Common questions I’ve had from customers include how they can use ServiceNow to provision new instances. Or, how to use ServiceNow to implement patches for AWS resources while still using ServiceNow defined change management workflows. Fortunately, AWS provides the ‘AWS Service Management Connector (SMC) for ServiceNow’, where customers can implement these use cases natively in ServiceNow. The AWS SMC also supports use cases including incident and configuration management. You can refer to AWS Service Management Connector for ServiceNow documentation for more details.

In this blog, we walk through the steps to set up and configure a web server, then patch the instance using the pre-configured ServiceNow workflow. For this use case, we provision a new NGINX web server product, raise a change request to patch the web server instance, and view the associated AWS events.

Overview of solution

The solution is composed of the following components:

1. The ServiceNow Personal Developer Instance (PDI) provides the user interface to provision a new product and raise a change request.
2. The AWS Service Management Connector (SMC) application will be installed and configured in the ServiceNow PDI.
3. When we order a new web server product within the ServiceNow service catalog, the AWS SMC calls AWS Service Catalog to provision a new EC2 instance.
4. When we raise a change request in ServiceNow, the AWS SMC calls AWS Systems Manager Change Manager to initiate a change request and run an automation document (runbook) to patch the instance. The runbook uses AWS CloudFormation templates and AWS Lambda functions.
5. AWS Systems Manager also captures events associated with the change request in an AWS CloudTrail event store, which can be queried via SQL.

Walkthrough

The steps to deploy the solution can be summarized in the following order:

• Deploy the CloudFormation template to set up the baseline permissions and IAM users.
• Install the AWS Service Management Connector application in ServiceNow.
• Configure AWS Service Catalog integration in ServiceNow required to provision a new product.
• Configure AWS Systems Manager Automation and Systems Manager Change Manager integration required to raise a new patch management change request.
• Validate the solution by provisioning a new NGINX web server and raising a change request to patch the web server instance.
• Validate the change request details.
• Clean up resources.

Prerequisites

For this walkthrough, you should have the following prerequisites:

• An AWS account
• A ServiceNow Personal Developer Instance (PDI)

Create baseline permission and IAM users in AWS

1. To create the users and permissions, use the CloudFormation template from the AWS Service Management Connector documentation.
2. The template creates an AWS Sync User and AWS End User. Make note of the SCEndUserAccessKey, SCEndUserSecretAccessKey, SCSyncUserAccessKey, and SCSyncUserSecretAccessKey values from the stack output. These will be used when configuring the AWS Account within ServiceNow.

Configure ServiceNow components

1. The ServiceNow steps are performed using the admin user credentials created as part of obtaining a PDI process.
2. Activate the ‘User Criteria Scoped API’ plugin by navigating to the system ‘Plugins’ page. Search for ‘User Criteria’. If the plugin is already installed, select the plugin to verify it is active. Note: Clear the ServiceNow platform cache first.
3. Activate the ‘Discovery and Service Mapping Patterns’ plugin by searching for the ‘Discovery and Service Mapping Patterns’ application and installing it.
4. Activate the ‘Change Management – Change Model Foundation Data’ plugin by searching for ‘Change Management’. If the plugin is already installed, select the plugin to verify it’s active.

Install ServiceNow Connector scoped application

1. From the ServiceNow PDI, we will use an ‘Update Set’ to install the AWS Service Management Connector.
2. Download the ServiceNow Update Set. From the ‘Retrieved Update Sets’, select the ‘Import Update Set from XML’ and upload the XML file.
3. Preview and ‘Commit the Retrieved Update Set’ to install the connector.

Configure AWS Service Catalog integration in ServiceNow

1. Under ‘System Definition’ > ‘Choice Lists’, create discovery source and configure platform tables, by adding an entry into the Configuration Item (CI) table for the AWS SMC to report discovered CIs into the CMDB.
2. Set permissions on ServiceNow Platform by modifying the ‘Application Access’ for the ServiceNow Service Catalog table so that the AWS products will display under AWS portfolios as subcategories.
3. Under ‘System Definition’ > ‘Tables’, search for and select the ‘Catalog Item Category’ table. Check the boxes ‘Can create’, ‘Can update’, ‘Can delete’.

Configure ServiceNow permissions and add AWS products in ServiceNow

1. Under ‘System Security’ > ‘Users and Groups’ > ‘Users’, edit the user’s role by adding ServiceNow permissions required for managing automation and change manager templates, accessing CloudTrail events, searching AWS Service Catalog products and portfolios, and managing AWS accounts.
2. Under ‘Self-Service’ > ‘Service Catalog’, add the ‘AWS Service Catalog’ product to the ServiceNow Service Catalog.

Configure Systems Manager Change Manager Integration in ServiceNow

1. Under ‘System Definition’ > ‘Choice Lists’, add an entry into the Change Request table with a label of ‘AWS Change Request’ and value of ‘AWSChangeRequest’.
2. Enable AWS Systems Manager Change Manager integration Change models by setting the following properties under ‘sys_properties.list’:

• • ‘com.snc.change_management.change_model.type_compatibility’ = ‘true’.
  • ‘com.snc.change_management.change_model.hide’ = ‘false’.

Configure AWS Accounts in ServiceNow to synchronize data with AWS

1. Under ‘AWS Service Management Connector’ > ‘Setup’ > ‘AWS Accounts’, create a new AWS Account (for example, ‘smc-cloudops’). Use the access key and secret key for the ‘SyncUser’ and ‘EndUser’ (created in the CloudFormation stack creation step earlier). Check the boxes to integrate with AWS Service Catalog, AWS Config, AWS Systems Manager Automation, and Systems Manager Change Manager.
2. Validate the connectivity of the newly created AWS Account.

Verify the ‘SMC Example Portfolio’ in AWS

1. In the AWS Management Console, under ‘AWS Service Catalog’ > ‘Portfolios’, there will be a ‘SMC Example Portfolio’ that was created when the CloudFormation template ‘smc-iam-setup.json’ was run earlier.
2. Selecting the ‘SMC Example Portfolio’ will show two products: ‘Simple S3 Private Bucket’ and ‘Demo NGINX Webserver’.

Configure permissions for sample end user (Abel Tuter) in ServiceNow

1. In the ServiceNow PDI, under ‘Users and Groups’ > ‘Groups’, create a new group record called ‘Order_AWS_Products’ and add the sample user ‘Abel Tuter’ to the group.
2. Under ‘AWS Service Management Connector’ > ‘AWS Service Catalog’ > ‘Portfolios’, add a new group called ‘Order_AWS_Products’ to the ‘SMC Example Portfolio’.

Provision a new Web Server in AWS via the ServiceNow Service Catalog

1. We’re now ready to provision a new AWS service via ServiceNow. Log in to the ServiceNow PDI using the sample user ‘abel.tuter’. From the landing page, select ‘Service Catalog’ under ‘Self-Service’.
2. Select the ‘Demo NGINX Webserver’.

3. The NGINX web server product details page will request input parameters. These include the product name, the Virtual Private Cloud (VPC) to deploy the web server, the instance type, and any required tagging information. Click on the “Create plan” to view the resources to be provisioned.

4. The Resource Changes shows that a new security group and EC2 instance will be created as part of this product. Select ‘Order Now’, to provision the product.

5. Confirmation page showing that the request has been submitted. Select the ‘Request Number’ link, and view details of the request.

6. Requested Item page showing that it has been approved and currently in ‘Fulfilment’ stage.

7. Log in to the AWS Management Console. Under the EC2 service, confirm that a new EC2 instance running NGINX web server has been successfully created. Copy the public IPv4 DNS and open it in a new browser window.

8. The browser shows the default landing page. We have now successfully created a NGINX web server.

Configure AWS Systems Manager Automation integration in ServiceNow

1. Now that we have provisioned an Amazon EC2 instance via the ServiceNow Service Catalog, this next section will describe how to raise a change request in ServiceNow. This will help you to apply patch updates to the web server. First, we will configure ServiceNow integration with AWS Systems Manager Automation, which is required in order to run the patch update automation document.
2. In the PDI, under ‘System Properties’ > ‘AWS Systems Manager’ > ‘Systems Manager Automation’, verify that the ‘AWS Systems Manager – Execute Automation Document’ workflow exists by navigating to ‘Workflow Editor’.
3. Filter workflows by ‘AWS’ and view the ‘AWS Systems Manager – Execute Automation Document’ workflow, along with other workflows created by the Service Management Connector Application.

4. Assign additional permissions to the ‘SCEndUser’ to stop and update EC2 instances. In the AWS Management Console, under ‘IAM’ > ‘Users’, add an inline policy that contains permissions allowing the ‘SCEndUser’ to stop/start EC2 instances and retrieve AWS Systems Manager documents.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "ec2:StopInstances",
                "ec2:StartInstances",
                "ssm:GetDocument"
            ],
            "Resource": "*",
            "Effect": "Allow"
        }
    ]
}

5. Under ‘Systems Manager’ > ‘Change Manager’, create a standard change template to patch an EC2 instance using the ‘AWS-PatchInstanceWithRollback’ runbook provided by AWS. Note: If Change Manager has not been set up for the account, refer to Setting up Change Manager for details.
6. Enable auto-approval, as this is required for the connector to synchronize change templates and add an IAM user as the first level approver. Submit the change template for review.
7. The template must be in an ‘Approved’ state before it is synchronized by the connector, so navigate to the ‘blog-patch-template’ and select ‘Approve’.

Configure AWS Systems Manager Change Manager integration in ServiceNow

1. In the AWS Management Console, under ‘CloudTrail’ > ‘Lake’ > ‘Event data stores’, create event data store. Check the ‘Management events’ box.
2. In the ServiceNow PDI, under ‘System Properties’ > ‘AWS Systems Manager’ > ‘AWS Systems Manager Change Manager’, verify that the AWS Systems Manager Change Manager role matches the ‘ServiceNowChangeManager’ role that was created as part of the CloudFormation template ‘smc-iam-setup.json’ executed earlier. Update the “Event Data Store Name’ with the event data store created in the previous steps.
3. Under ‘AWS Systems Manager’ > ‘Change Templates’, verify the ‘blog-patch-template’ has been synchronized from AWS Systems Manager Change Manager.

Create a new change request from ServiceNow

1. We’re now ready to create a new change request to patch our web server via ServiceNow. Log in to the Personal Developer Instance (PDI) using the ‘admin’ user and select ‘Change’ > ‘Create New’.

2. Select the option to make changes to AWS resources using Change Manager Templates.

3. Select the ‘blog-patch-template’ as the change request template.

4. Enter the instance ID of the NGINX web server created earlier and ‘Submit’.

5. In the change request form, select an ‘Assignment group’ and then select ‘Request Approval’.

6. Approve the change request.

7. The change request progresses to the ‘Scheduled’ stage. Select ‘Implement’.

8. In the ‘Change Tasks’ tab at the bottom, the implementation task state is ‘In Progress’.

9. You can also view the workflow execution states in graphical form.

10. After the change request completes, check the ‘AWS Status’ attribute for ‘CompleteWith Success’.

11. You can also verify that the automation execution associated with the change request was successfully implemented by navigating to ‘Automation Executions’.

12. The Automation Executions show that the ‘blog-patch-template’ document ‘Completed with Success’.

View the change request details in AWS

1. We can view the change request details in AWS by querying the ‘smc-change-store-1’ event store we created earlier. In the AWS Management Console, select ‘CloudTrail’ > ‘Lake’. In the Editor, issue a query on the event store where the ‘sourceIPAddress’ equals ‘ssm.amazonaws.com’ to view the events generated by the change request.

2. Additionally, you can view the events for the ‘PatchInstanceWithRollback’ stack in CloudFormation.

Cleaning up

To avoid incurring future charges, remove the resources by deleting NGINX web server instance and disabling change requests/scheduled jobs in ServiceNow.

Conclusion

Customers using ServiceNow as their ITSM solution can leverage the AWS Service Management Connector to provision, manage, and operate AWS resources natively in ServiceNow. In this blog, we’ve shown how to implement steps to provision an AWS product, and raise a change request to patch an instance through the ServiceNow interface. In doing so, customers can accelerate migration and AWS adoption at scale through oversight and governance in their declared operational tooling and system of record.

About the Author