Setting up Windows Server Failover Cluster shared storage on AWS Outposts rack

In this blog post, I will highlight how to use Microsoft Windows Server Storage Spaces Direct to create Clustered Shared Volumes when running Windows Server Failover Clusters on an AWS Outposts rack.

Introduction

Customers advancing in their cloud migration journey are utilizing the cloud for nearly every type of workload. However, some use cases still cannot be fully migrated to the cloud. These may include latency-sensitive business applications where the workload needs to be connected to another device or system on the local network or applications that need to process large datasets created or stored at an edge location. Examples include workloads running on factory floors for automated operations in manufacturing, real-time patient diagnosis, or medical imaging. For these use cases, constantly transferring data to and from the cloud can be too slow, too resource-intensive, or even prohibited due to regulatory constraints.

AWS Outposts racks were developed with these types of use cases in mind. Outposts rack is a fully managed service that extends the same Amazon Web Services (AWS) infrastructure, services, APIs, and tools to your on-premises environment for a consistent hybrid experience. Outposts rack allows you to migrate these applications to AWS while meeting on-premises locality requirements.

Frequently, Microsoft workloads that are migrated to AWS rely on Windows Server Failover Cluster (WSFC) technology to provide resiliency for Windows-based workloads. Many applications that run on WSFCs require storage to be accessible to all cluster nodes in a shared manner. This can include software such as Microsoft SQL Server, Windows File Servers, or other commercial software. These Cluster Shared Volumes (CSV) are required to provide continued availability of storage during cluster impacting events.

When running WSFCs in an AWS region, multiple options are available for providing shared storage to cluster nodes. Examples include Amazon EBS Multi-Attach, Amazon FSx, and partner ISV solutions. Outposts rack only supports a subset of the services that are available in AWS regions, so these options are not available. When running WSFCs on Outposts rack, a solution needs to be implemented to allow storage to be shared between the Amazon Elastic Cloud Compute (Amazon EC2) instances running as cluster nodes.

One option for addressing this challenge is to use Storage Spaces Direct, a feature of Windows Server that allows you to combine storage drives on a cluster of servers into a software-defined storage pool. This storage pool can then be used to create shared storage that can be accessed across WSFC cluster nodes.

Solution overview

Storage Spaces Direct (S2D) is included in the Datacenter version of Windows Server 2016 and later. S2D is a software-defined storage solution that works in tandem with WSFCs to create highly available and scalable storage systems using locally attached storage drives on physical or virtual servers. S2D aggregates the drives across a WSFC cluster of nodes into a single storage pool, from which virtual disks are created and presented to the cluster as cluster shared volumes. To ensure the resiliency and availability of the storage pool, S2D employs mirroring and parity by distributing data across the cluster nodes. This allows for failures at an individual server or underlying storage level to be handled without compromising the storage pool.

When running S2D on an AWS Outposts rack, you will use Amazon EC2 instances and associated Amazon Elastic Block Store (Amazon EBS) GP2 volumes. The Windows Server EC2 instances will be configured as a WSFC. Each EC2 instance will have a minimum of two EBS volumes attached, specifically dedicated for S2D usage. This setup allows S2D to create a virtual storage pool across the nodes in the cluster (Figure 1). Clustered shared volumes will be created on top of this storage pool and will be accessible to all nodes in the cluster. Applications running on the cluster can then use this shared storage as needed, ensuring the high availability of your critical workloads. When running workloads on AWS Outposts, especially in single-rack environments, it is important to consider additional factors to ensure the high availability and reliability of those workloads. For guidance on designing highly available application environments using AWS Outposts, review, AWS Outposts High Availability Design and Architecture Considerations .

Figure 1 – Solution reference architecture

Prerequisites

1. Fully configured AWS Outposts rack. Refer to the User Guide for Outpost racks.
2. Two or more Amazon EC2 instances running Windows Server 2016 or later configured into a Window Server Failover Cluster (WSFC).
3. Amazon EC2 Instances must be configured as defined in the Microsoft S2D documentation.
4. Each EC2 instance should have a minimum of two non-root EBS GP2 volumes attached. Volumes must be empty with no old partitions or other data.

Walkthrough

This walkthrough will guide you through the process of enabling Storage Space Direct and creating a CSV for your existing WSFC running on Outposts rack.

Step 1 – Verify attached EBS storage

• Each EC2 instance in your cluster will need to have a minimum of two non-root EBS GP2 volumes attached. These volumes must be raw and dedicated to S2D usage. The volumes should be a consistent size across the nodes.
• Run a Windows PowerShell command on each node to verify storage (Figure 2).
• In an elevated Administrator PowerShell prompt run the following command.
  • Get-Disk

Figure 2 – PowerShell Get-Disk output

• Volumes dedicated to S2D usage should be shown in the PowerShell output with a Partition Style of RAW.

Step 2 – Enable Storage Spaces Direct

• S2D is enabled via a PowerShell command. You will define a storage pool friendly name and disable the cache via parameters. On Outposts rack with EBS GP2 volumes, the cache feature of S2D is not supported.
• On one of your cluster nodes, in an elevated administrator PowerShell prompt run the following command. Replace BLOGDEMO with your desired storage pool name.
  • Enable-ClusterStorageSpacesDirect -PoolFriendlyName 'BLOGDEMO' -CacheState Disabled
• You will be asked to verify whether you want to perform this action. Enter Y to continue.
• You will be shown an output with your cluster node name and path to a logging report (Figure 3).

Figure 3 – Enabling S2D with PowerShell

• Once S2D has been enabled on your cluster, a WSFC storage pool will have been created. This storage pool will use the underlying EBS volumes from your EC2 instances.
• You can verify this configuration from the Microsoft Failover Cluster Manager administration tool (Figures 4, 5, and 6).

Figure 4 – S2D Storage Pool visible in Failover Cluster Manager administration tool

Figure 5 – Details of S2D Storage Pool

Figure 6 – Physical Disks supporting S2D Storage Pool

Step 3 – Create Cluster Shared Volume

• Once your cluster storage pool is available, you can create CSVs from this available storage. Volumes can be created in Failover Cluster Manager or via PowerShell.
• On one of your cluster nodes, in an elevated Administrator PowerShell prompt run the following command. Replace your individualized parameters for volume name, storage pool name, and storage volume size.
  • New-Volume -FriendlyName "BLOGDEMOVOL1" -FileSystem CSVFS_ReFS -StoragePoolFriendlyName BLOGDEMO -Size 25GB
• Upon successful execution of the command, you will see an output similar to Figure 7.

Figure 7 – New-Volume PowerShell command output

• You can verify the new CSV in Failover Cluster Manager (Figure 8).

Figure 8 – Newly created CSV

Step 4 – Test failover of CSV between cluster nodes

• After creation of the new CSV, you will want to test failover of the volume to other nodes in the cluster. This can be accomplished in Failover Cluster Manager or PowerShell.
• To perform in Failover Cluster Manager.
  • Take note of the existing owner node (Figure 9).
  • Right-click on the newly created CSV. Select Move, then Select Node…. Choose the cluster node to move the CSV to (Figure 10).
• Verify the new owner is as expected (Figure 11). If new owner is as expected and the volume status is Online then the test has been successful.

Figure 9 – Current CSV owner node

Figure 10 – Choosing destination node for moving CSV

Figure 11- New CSV owner node

Cleanup

Cleanup not required as no additional AWS resources were created during this walkthrough.

Conclusion

The solution outlined in this post demonstrates how customers can leverage the capabilities of S2D to enable highly available, software-defined storage on Outposts racks. By using S2D to create a virtual storage pool from EBS volumes, you can ensure that your critical Windows-based applications have the shared, resilient storage they require, even in an on-premises Outposts rack deployment.

Ultimately, the ability to extend the AWS platform into the on-premises data center through Outposts rack and using complementary technologies like WSFC and S2D allow for a wide range of enterprise use cases to be migrated to the cloud. This flexibility and choice will enable you to modernize your IT environments at your own pace, while still benefiting from the innovations in AWS services, tools, and capabilities.

Visit the Microsoft documentation on Storage Spaces Direct for full details on deploying and managing the feature.

AWS has significantly more services, and more features within those services, than any other cloud provider, making it faster, easier, and more cost effective to move your existing applications to the cloud and build nearly anything you can imagine. Give your Microsoft applications the infrastructure they need to drive the business outcomes you want. Visit our .NET on AWS and AWS Database blogs for additional guidance and options for your Microsoft workloads. Contact us to start your migration and modernization journey today.