AWS Cloud Operations Blog

Orchestrating multi-step, custom patch processes using AWS Systems Manager Patch Manager

The ongoing management of operating system and application-level patching is critical for ensuring that your organization’s software is up to date and meets compliance policies. Patching is not always a straightforward process. You often need to orchestrate custom procedures, workflows, and scripts to ensure that applications can be safely stopped, started, and verified during the patching process.

AWS Systems Manager Patch Manager has been helping AWS customers manage and automate the process of patching their Linux and Windows managed instances. But complex, multi-step processes continue to present challenges when customers attempt to orchestrate the end-to-end patch workflow.

I am excited to highlight the release of a new Patch Manager feature, patch lifecycle hooks, that makes the orchestration of these multi-step processes simpler. Patch lifestyle hooks extend existing Patch Manager functionality to include new pre-patching and post-patching hooks that allow custom, customer-specified steps to be run at different phases of the patching workflow.

For example, consider a customer with a custom-built application that requires a procedure to start and stop the application. The customer can use patch lifecycle hooks to run a pre-patching custom script to safely shut down the application before performing the patching process. After patching is complete and the server has been rebooted, a post-patching custom script can be run to start the application and perform validation testing to ensure it is operating as expected before signaling success of the overall patching process.

Orchestrating custom patch processes with patch lifecycle hooks

With the release of patch lifecycle hooks, AWS introduces a new AWS Systems Manager document (SSM document), AWS-RunPatchBaselineWithHooks. This document is a wrapper document that uses the existing AWS-RunPatchBaseline SSM document to compose more complex patch installation scenarios, allowing custom scripts to be run as pre-installation, post-installation, and post-reboot hooks.

With these new hooks, you can specify the execution of SSM documents at pre- and post-installation stages of the patching workflow. The SSM documents you choose as the hooks can be AWS pre-defined or custom. For information about creating custom SSM documents, see creating Systems Manager documents in the AWS Systems Manager user guide.

You can review the newly released AWS-RunPatchBaselineWithHooks document in the AWS Systems Manager console. From the left navigation pane, choose Documents, and then search on the Owned by Amazon tab.

The Documents page provides four tabs: Owned by Amazon, Owned by me, Shared with me, and All documents. The Owned by Amazon tab is selected. AWS-Run-PatchBaselineWithHooks is entered in the search field.

Figure 1: Searching AWS Systems Manager console for SSM documents

To use patch lifecycle hooks, you can perform on-demand patching in the AWS Systems Manager console or you can adjust an existing Patch Manager configuration by changing the current task being run by the AWS Systems Manager maintenance window. For more information, check AWS Systems Manager Patch Manager in the AWS Systems Manager User Guide or the Patching your Windows EC2 instances using AWS Systems Manager Patch Manager blog post on the subject.

Prerequisites

Make sure that AWS Systems Manager is set up and the AWS Systems Manager agent has been installed and updated on all of the instances you want to patch. Patch lifecycle hooks are available for Systems Manager Agent (SSM Agent) version 3.0.502 and higher.

To proceed with the following walkthrough of adjusting an existing Patch Manager configuration, a previously created Patch Manager patching configuration must be available with scheduling being managed by a Systems Manager maintenance window.

Perform on-demand patching using patch lifecycle hooks

Patch lifecycle hooks can be used when performing on-demand patching using Patch Manager in the AWS Systems Manager console.

  1. Begin within the AWS Systems Manager console. From the left navigation pane, choose Patch Manager, and then choose Patch Now.

Advanced options let you configure on-instance orchestration for complex patching scenarios. The lifecycle hooks feature allows you to use additional SSM documents to prepare your instances before and after installation.

Figure 2: Patch now advanced options

  1. In Advanced Options, select Use lifecycle hooks, and then select the SSM documents that will be run at various stages throughout the patching process.

Under Lifecycle hooks selection, for Before install, PatchingDemo-StopAppGracefully is selected. For After install, PatchingDemo-UpdateApp is selected. For On Exit, PatchingDemo-StartAppandVerify is selected.

Figure 3: Selecting SSM documents to use as lifecycle hooks

  1. Follow the steps in Patching instances on demand to complete the patch now configuration.

Update an existing Patch Manager configuration to use patch lifecycle hooks

To edit an existing Patch Manager configuration to use patch lifecycle hooks, start by editing the maintenance window.

  1. In the AWS Systems Manager console, choose Maintenance Windows, and then choose the Window ID for the maintenance window you would like to update.

Under Maintenance windows, the window ID with the name BlogPostDemo and a state of Enabled is highlighted.

Figure 4: Maintenance windows console

  1. In the details page for the maintenance window, choose Tasks. Select the radio button next to the Window task ID, and then choose Edit.

The window task named PatchingTask with a priority of 1 and a type of RUN_COMMAND is selected.

Figure 5: Maintenance window task ID

  1. In the Command document section, search for the AWS-RunPatchBaselineWithHooks document and then select the radio button next to the document.

AWS-RunPatchBaselineWithHooks is selected. Its owner is Amazon. Its platform types are Windows and Linux.

Figure 6: Updating maintenance window task with new SSM document

  1. In the Parameters section, you will see the new hook parameters where you can specify the SSM documents to be run at different stages of the patching process.

Three new parameters have been added:

Pre-Install Hook Doc Name: The SSM document is run before patches are installed.

Post-Install Hook Doc Name: The SSM document is run after patching, but before reboot.

On-Exit Hook Doc Name: The SSM document is run after the reboot is complete.

The SSM documents can be owned by Amazon or your custom documents.

In Parameters, for the pre-install hook document name, PatchingDemo-StopAppGracefully is displayed. For the post-install hook document name, PatchingDemo-UpdateApp is displayed. For the on-exit hook document name, PatchingDemo-StartAppAandVerify is displayed.

Figure 7: Populating AWS-RunPatchBaselineWithHooks document parameters

  1. After you complete the Parameters section, choose Edit Run command task.

On the Tasks menu for the updated maintenance window, you will see the new AWS-RunPatchBaselineWithHooks selection under Task ARN.

Under Tasks, a windows task ID named PatchingTask with a priority of 1 and a type of RUN-COMMAND is displayed. Its task ARN is AWS-RunPatchBaselineWithHooks.

Figure 8: Maintenance window task ARN, AWS-RunPatchBaselineWithHooks

You’ve now configured the patch lifecycle hooks feature. The next run of your maintenance window will perform patching using the new AWS-RunPatchBaselineWithHooks document.

Conclusion

Patch lifecycle hooks are the recommended way to run multi-step processes directly on instances to be patched. For the orchestration of custom steps that do not require execution on the instance to be patched, such as removing an instance from an Elastic Loading Balancing Application Load Balancer or updating an Amazon DynamoDB table, the recommended solution is to continue using AWS Systems Manager Automation.

With the introduction of the new Patch Manager patch lifecycle hooks feature, you can now use AWS Systems Manager Patch Manager to automate your most complex patching workflows, reducing the manual intervention required to perform this critical task.

For more information about Patch Manager patch lifecycle hooks, see About the AWS-RunPatchBaselineWithHooks SSM document in the AWS Systems Manager user guide.

About the Author

Ryan Stebich is a Senior Solutions Architect with Amazon Web Services (AWS) based out of North Carolina. He leverages his broad range of IT experience and technical knowledge to design Cloud based solutions to help customers solve their business challenges.