AWS Storage Blog

Deploying Amazon FSx for NetApp ONTAP using AWS CloudFormation

In today’s fast-paced digital landscape, the efficient management and accessibility of data stand as pivotal challenge for businesses. For a global team to efficiently collaborate on a project, they require a shared file storage that supports multiple protocols, robust data management features, and dependable data integrity. When deploying infrastructure repeatedly or in different environments, it is important that infrastructure is consistently reproducible, reducing the risk of configuration errors.

Amazon FSx for NetApp ONTAP is ideal for those looking for a fully managed solution for multi-protocol access, as it allows you to focus on core business activities instead of managing complex infrastructure. An effective way of deploying infrastructure consistently and without errors is to use Infrastructure-as-Code (IaC) tools – and you can deploy FSx for ONTAP with several.

In this blog, we discuss deployment using AWS CloudFormation, and deploy an FSx for ONTAP file system using it. CloudFormation lets you provision related AWS and third-party resources quickly and consistently, and manage them throughout their life cycles, by treating infrastructure as code. You can incorporate the code in this blog in an existing template to create, update, and delete an entire stack as a single unit. Using Drift Detection, you can keep track of changes to resources outside CloudFormation, making sure you always have the most up-to-date picture of your infrastructure.

Infrastructure-as-Code (IaC) tools

By definition IaC is the process of managing and provisioning computer data centers through machine-readable definition files, rather than physical hardware configuration or interactive configuration tools. IaC enables you to maintain the configuration or state of the infrastructure in a known way.

It is recommended to use AWS CloudFormation when all the resources you are deploying are in AWS. If you need to orchestrate multi-cloud environments, or need to use HashiCorp Terraform for other reasons, you can refer to the blog Deploying Amazon FSx for NetApp ONTAP using HashiCorp Terraform. If you already have a file system deployed and are looking for a configuration management tool to maintain the configuration in a known state, Ansible will be able to help. You can refer to How to manage Amazon FSx for NetApp ONTAP with Ansible for a brief introduction to Ansible. If you need more flexibility and want to perform operations not supported by AWS CloudFormation, or want to developer friendly automation refer to the blog Deploying Amazon FSx for NetApp ONTAP using AWS CloudFormation.

There are wide variety of tools in the Infrastructure-as-Code (IaC), configuration management and automation market. AWS CloudFormation and HashiCorp Terraform are provisioning tools. Ansible is a configuration management tool. The tool you should use depends on the challenges you need to solve and your posture in terms of automation. For performing Day-0 activities like provisioning it is recommended to use either AWS CloudFormation or HashiCorp Terraform. If customer is totally in AWS, we recommend using AWS CloudFormation because of its integration with other AWS services. If you have a need to build and manage hybrid or multi-cloud, we recommend using HashiCorp Terraform.

For performing Day-1 activities like configuring storage, we recommend that you use Ansible. Ansible offers certified modules for managing NetApp. For complete list of NetApp modules refer the collection documentation FSx for NetApp Ontap. If you need more flexibility than what Ansible can provide, we recommend using the REST API. In some cases, you might have to use a combination of these tools. Combinations that we recommend for managing FSx for ONTAP are provisioning and configuration management (AWS CloudFormation/Terraform + Ansible) or provisioning and Orchestration (AWS CloudFormation/Terraform + REST API). Refer to the “Additional reading” section to get more insight into the tools mentioned.

Overview of components

Amazon FSx for NetApp ONTAP is a fully managed service that provides highly reliable, scalable, high-performing, and feature-rich file storage built on NetApp’s popular ONTAP file system. FSx for ONTAP provides a seamless solution for multi-protocol access, allowing the organization to focus on their core business activities instead of managing complex infrastructure. FSx for ONTAP file systems are similar to on-premises NetApp clusters. The following list identifies the main components in FSx for ONTAP:

  • File system: A file system is the primary Amazon FSx resource, analogous to an on-premises ONTAP cluster
  • Storage Virtual Machine (SVM): An SVM is an isolated file server with its own administrative credentials and endpoints for administering and accessing data
  • Volumes: ONTAP serves data to clients and hosts from logical containers called volumes

AWS CloudFormation is an AWS native IaC service that lets you model, provision, and manage AWS and third-party resources. It uses either JSON or YAML templates. CloudFormation has many advantages including; CloudFormation is a fully managed AWS service, and state management is a built in feature. It is native to AWS, so support for new services is available faster. It’s easy to use, has custom resource support, and advanced features like drift detection and rollback capabilities are available.

It is important to note that CloudFormation is proprietary to AWS. So it only supports provisioning on AWS. CloudFormation templates also has a size limit of 51 MB or 500 resources per template. This limit can be overcome with the help of nested stacks.

The following table provides an overview of the CloudFormation feature set:

Feature AWS CloudFormation
Support Native managed service
Scope AWS Services*
Type of tool Orchestration tool
State management Yes (manages state in stacks)
State file format JSON or YAML
License and support Free and covered under AWS support
Change verification Yes, using Change sets
API Model Native FSx API
Language Declarative language
Infrastructure Immutable*
Rollback Yes
External wait conditions Yes

Deploying FSx for ONTAP file system using AWS CloudFormation

AWS CloudFormation enables you to provision and manage AWS and third-party resources by treating infrastructure-as-code. It gives developers and businesses an easy way to create a collection of related resources, and provision and manage them in an orderly and predictable fashion. With a few clicks, you can use a CloudFormation template to pre-configure and deploy FSx for ONTAP resources like file systems, storage virtual machines, and volumes in a standardized and repeatable way across multiple regions and accounts. Customers benefit from this by being able to architect their deployments in a variety of ways. They have more control over provisioning and extending the architecture of the FSx for ONTAP with IaC with the same configuration in multiple environments.

Prerequisites

  • You will need an AWS account and associated credentials that allow you to create resources
  • Amazon Virtual Private Cloud (Amazon VPC) and two associated subnets we utilize for creating the file system. We are discussing Multi-AZ file system in rest of the blog which is deployed across multiple Availability Zones (AZ)

Procedure

  1. AWS CloudFormation resource type AWS::FSx::FileSystem creates a file system in FSx for ONTAP. It takes input from the user on parameters like StorageCapacity, SubnetIds, etc. The deployed file system’s configuration will depend on the parameter values provided. The listed template will create a multi-AZ file system named OntapMultiAzFileSystemWithAllConfigs.
  2. AWS CloudFormation resource type AWS::FSx::StorageVirtualMachine creates a storage virtual machine (SVM) for an Amazon FSx for ONTAP file system. The listed template creates a Storage Virtual Machine (SVM) named OntapStorageVirtualMachineWithAllConfigs. An SVM is a virtual, isolated file server with its own administrative credentials and IP address for accessing data. In this example, we are using Active Directory to manage access to the file shares. Note that Active Directory is not required, as FSx for ONTAP supports Workgroup mode as well. It takes user credentials from AWS Secrets Manager. OntapStorageVirtualMachineWithAllConfigs refers to the previously created file system OntapMultiAzFileSystemWithAllConfigs as one of the attributes.
  3. AWS::FSx::Volume creates a FSx for ONTAP storage volume. The template provided creates a volume called OntapVolumeWithAllConfigs. It refers to the previously created resource OntapStorageVirtualMachineWithAllConfigs as one of the attributes. The following parameterized template furnishes all the components needed to establish an FSx for ONTAP file system, including an SVM and a volume:
Description: Creates your Amazon FSx for NetApp ONTAP.

Metadata:
  Authors:
    Description: Amazon Web Services
  AWS::CloudFormation::Interface:
    ParameterGroups:
    - Label:
        default: Amazon FSx for NetApp ONTAP file system parameters
      Parameters:
        - FSxNName
        - StorageCapacity
        - SubnetIds
        - SecurityGroupIds
        - Iops
        - AutomaticBackupRetentionDays
        - PreferredSubnetId
        - RouteTableIds
        - EndpointIpAddressRange
        - ThroughputCapacity
        - FsxAdminPassword
        - KmsKeyId
    - Label:
        default: Storage Virtual Machine for Amazon FSx for NetApp ONTAP file system parameters
      Parameters:
        - SVMname
        - NetBiosName
        - DnsIps
        - DomainName
        - OrganizationalUnitDistinguishedName
        - ADsecretname
        - RootVolumeSecurityStyle
    ParameterLabels:
      FSxNName:
        default: Name of the Filesystem
      StorageCapacity:
        default: Storage Capacity
      SubnetIds:
        default: Subnet Ids
      SecurityGroupIds:
        default: Security group ID
      Iops:
        default: Iops
      AutomaticBackupRetentionDays:
        default: AutomaticBackupRetentionDays
      PreferredSubnetId:
        default: Preferred Subnet Id for FSxN
      RouteTableIds:
        default: Preferred route table for FSxN
      EndpointIpAddressRange:
        default: floating ip EndpointIpAddressRange
      ThroughputCapacity:
        default: Preferred ThroughputCapacity for FSxN
      FsxAdminPassword:
        default: Default password for FSxN.
      KmsKeyId:
        default: AWS KMS key ID for FSxN Encyption at Data at Rest
      SVMname:
        default: Name of the SVM for FSxN.
      NetBiosName:
        default: Preferred NetBiosName for FSxN
      DnsIps:
        default: DNS ips for the FSxN
      DomainName:
        default: Domain Name of the ActiveDirectoryConfiguration
      OrganizationalUnitDistinguishedName:
        default: Organization unit for FSxN
      ADsecretname:
        default: AWS Secrets Manager Secret Name
      RootVolumeSecurityStyle:
        default: Security Style of the Root volume.

Parameters:
  FSxNName:
    Description: Name of the Filesystem
    Type: String
  StorageCapacity:
    Description: StorageCapacity for FSxN FileSystem
    Default: 1024
    Type: Number
  SubnetIds:
    Description: Subnet ids for FSxN FileSystem in comma seperated values.
    Type: CommaDelimitedList
  SecurityGroupIds:
    Description: Security group ID for the FSxN FileSystem.
    Type: CommaDelimitedList
  Iops:
    Description: Choose Iops of your Amazon FSx for NetApp ONTAP file system.
    Default: 3072
    Type: Number
  AutomaticBackupRetentionDays:
    Type: Number
    Description: Backup of your Amazon FSx for NetApp ONTAP file system's Volumes.
  PreferredSubnetId:
    Description: Provide a subnet that will be preferred node of Amazon FSx for NetApp ONTAP file system's Volumes.
    Type: String
  RouteTableIds:
    Description: Route table for the FSxN
    Type: CommaDelimitedList
  EndpointIpAddressRange:
    Description: Private floating endpoint EndpointIpAddressRange for FSxN.
    Type: String
  ThroughputCapacity:
    Description: Throughput for FSxN
    AllowedValues: [128, 256, 512, 1024, 2048]
    Type: Number
  FsxAdminPassword:
    Description: AWS Secrets Manager Secret Name containing Default password to login to FSxN using fsxadmin username.
    Type: String
  KmsKeyId:
    Description: AWS KMS key ID for FSxN Encyption at Data at Rest, if you dont have Customer Managed Key Leave it black. As AWS Fsx Service create a Fsx Key by default for Encyption at rest.
    Type: String
    Default: ""
  SVMname:
    Description: Name of the SVM for FSxN
    Type: String
  NetBiosName:
    Description: Preferred NetBiosName for FSxN,
    Type: String
  DnsIps:
    Description: DNS ips for the FSxN. If Multiple provide comma seperated.
    Type: CommaDelimitedList
  DomainName:
    Description: Domain Name of the ActiveDirectoryConfiguration
    Type: String
  OrganizationalUnitDistinguishedName:
    Description: Organization unit for FSxN.
    Type: String
  ADsecretname:
    Description: 'AWS Secrets Manager Secret Name containing password for the AD user". Validate secret is stored in format {"username" : "<your username>", "password" : "<your password>"}'
    Type: String
  RootVolumeSecurityStyle:
    Description: Security Style of the Root volume. Either in [MIXED, UNIX, NTFS].
    Type: String
    AllowedValues: [MIXED, UNIX, NTFS]

Conditions:
  IsKMSKeyprovided: !Not [!Equals [ !Ref KmsKeyId, "" ]]


Resources:
  OntapMultiAzFileSystemWithAllConfigs:
    Type: "AWS::FSx::FileSystem"
    Properties:
      FileSystemType: "ONTAP"
      KmsKeyId:  !If [ IsKMSKeyprovided, !Ref KmsKeyId, !Ref "AWS::NoValue"]
      StorageCapacity: !Ref StorageCapacity
      StorageType: SSD
      SubnetIds: !Ref SubnetIds
      SecurityGroupIds: !Ref SecurityGroupIds
      OntapConfiguration:
        AutomaticBackupRetentionDays: !Ref AutomaticBackupRetentionDays
        DailyAutomaticBackupStartTime: "07:00"
        DeploymentType: "MULTI_AZ_1"
        DiskIopsConfiguration:
          Iops: !Ref Iops
          Mode: "USER_PROVISIONED"
        PreferredSubnetId: !Ref PreferredSubnetId
        RouteTableIds: !Ref RouteTableIds
        EndpointIpAddressRange: !Ref EndpointIpAddressRange
        ThroughputCapacity: !Ref ThroughputCapacity
        WeeklyMaintenanceStartTime: "4:16:30"
        FsxAdminPassword: !Join
              - ':'
              - - '{{resolve:secretsmanager'
                - !Ref FsxAdminPassword
                - 'SecretString:password}}'
      Tags:
        - Key: "Name"
          Value: !Ref FSxNName
  OntapStorageVirtualMachineWithAllConfigs:
    Type: "AWS::FSx::StorageVirtualMachine"
    Properties:
        ActiveDirectoryConfiguration:
          NetBiosName: !Ref NetBiosName
          SelfManagedActiveDirectoryConfiguration:
            DnsIps: !Ref DnsIps
            DomainName: !Ref DomainName
            OrganizationalUnitDistinguishedName: !Ref OrganizationalUnitDistinguishedName
            Password: !Join
              - ':'
              - - '{{resolve:secretsmanager'
                - !Ref ADsecretname
                - 'SecretString:password}}'
            UserName: !Join
              - ':'
              - - '{{resolve:secretsmanager'
                - !Ref ADsecretname
                - 'SecretString:username}}'
        FileSystemId: !Ref OntapMultiAzFileSystemWithAllConfigs
        Name: !Ref SVMname
        RootVolumeSecurityStyle: !Ref RootVolumeSecurityStyle
        Tags:
          - Key: "Name"
            Value: "OntapSvm"
  OntapVolumeWithAllConfigs:
    Type: "AWS::FSx::Volume"
    Properties:
      Name: "volume1"
      OntapConfiguration:
        JunctionPath: "/volume1" 
        SecurityStyle: "NTFS"
        SizeInMegabytes: 10240
        StorageEfficiencyEnabled: True
        StorageVirtualMachineId: !Ref OntapStorageVirtualMachineWithAllConfigs
        TieringPolicy:
          CoolingPeriod: 31
          Name: "AUTO"
      Tags:
        - Key: "Name"
          Value: "OntapVolume"
      VolumeType: "ONTAP"

You can extend the functionality of the template beyond what is natively supported by AWS CloudFormation using custom resource and resource types. For example, if you want to create a CIFS share, which is not supported natively by AWS CloudFormation, you can do that using custom resources. For more information on it you can refer to the documentation on managing resources using AWS CloudFormation resource types.

As new features and services are released rapidly, sometimes they are released without CloudFormation support. Refer to the CloudFormation Coverage Roadmap which is used as a primary source of input to our development process.

As mentioned, the CloudFormation template provided in this post creates an FSx for ONTAP file system, an SVM, and a volume. You can mount it on any machine using NFS protocol. To make it accessible over SMB/CIFS, a CIFS share needs to be created.

Cleaning up

To delete the resources created in this post, go to the AWS CloudFormation console in the management account. Choose the stack you created, and then choose Delete. For more information, see the documentation on deleting a stack on the AWS CloudFormation console.

Conclusion

In this blog, we provided an overview of AWS CloudFormation and how to create a FSx for ONTAP File system, SVM, and a volume using it. Automation not only reduces the potential for manual errors, but also saves valuable time for IT teams, enabling them to focus on more strategic tasks than managing infrastructure. In exceptional scenarios where AWS CloudFormation cannot be used, FSx for ONTAP supports other tools. More information is available in the following “additional reading.”

If you have any comments or questions, don’t hesitate to leave them in the comments section.

TAGS: , ,
Madhu Vinod Diwakar

Madhu Vinod Diwakar

Madhu is a Cloud Infrastructure Architect at Amazon Web Services (AWS), focusing on storage migration, performance, and optimization for customer workloads. Outside of work, Madhu likes to spend time playing racket sports like table tennis or badminton.

Sandeep Vadapalli

Sandeep Vadapalli

Sandeep Vadapalli is a Cloud Infrastructure Architect at Amazon Web Services Professional Services. In this role, Sandeep collaborates directly with clients to facilitate and expedite their migration to cloud-based solutions through the processes of building, designing, and architecting cloud-based solutions. Outside of work, Sandeep finds enjoyment in hiking and spending time in nature.