Introducing a New Blog Series, Service Spotlight for Financial Services: Featuring Amazon SageMaker Notebook Instances

As we wrap up the 3-week long re:Invent virtual conference, we are pleased to launch the first blog post of the Service Spotlight for Financial Services. This series aims to provide financial services customers a deep dive into the five key considerations of a particular service to help streamline service approval based on our experience with global financial institutions, including:

1. Achieving compliance
2. Data protection
3. Isolation of compute environments
4. Automating audits with APIs
5. Operational access and security

Each of these five areas will include specific guidance that can help you streamline service approval for the particular service, which may need to be adapted to your specific use case and environment.

We are starting the series with Amazon SageMaker Notebook Instances because of the large growth of SageMaker usage among FSI customers. Financial institutions are increasingly adopting Amazon Machine Learning services to easily build and train machine learning models to effectively detect online payment fraud, such as Fraud.net or build a recommendations platform that provides more value to customers and enables data scientists to move projects from design to production quickly, which is what NerdWallet is accomplishing. The organization uses Amazon Machine Learning to provide more than 20 machine learning models and relies on Amazon DynamoDB and AWS Lambda to run code without provisioning or managing servers.

An Amazon SageMaker notebook instance is a machine learning (ML) EC2 instance running the Jupyter Notebook App. SageMaker manages creating the instance and related resources for you. You use Jupyter notebooks in your notebook instance to prepare and process data, write code to train models, deploy models to SageMaker hosting, and test or validate your models. The Amazon SageMaker notebook instance removes the undifferentiated heavy lifting of providing a best-in-class Jupyter notebook experience to your developers and data scientists, furthermore these notebooks instances benefit from deep integration with S3 storage, Amazon CloudWatch Logs to detect anomalous behaviors, set alarms, and discover insights to keep the SageMaker notebooks running more smoothly, and a choice of instance types with various CPU/GPU choices.

We will now dive deep into the five key considerations outlined previously.

Security and Compliance with SageMaker Notebook Instances

Security is a shared responsibility between AWS and you. AWS is responsible for protecting the infrastructure that runs AWS services in the AWS Cloud and also provides you with services that you can use securely. Your responsibility is determined by the AWS service that you use. On the customer’s side of the shared responsibility model, customers should first determine their requirements for network connectivity, encryption and access to other AWS resources. We will dive deeper into those topics in the upcoming sections.

Amazon SageMaker is included in the scope of the following compliance programs with regards to AWS’s side of the shared responsibility model. In following sections, we will cover topics on the customer side of the shared responsibility model.

• SOC 1,2,3
• PCI
• IRAP Protected
• ISO/IEC 27001:2013, 27017:2015, 27018:2019, and ISO/IEC 9001:2015
• OSPAR
• MTCS (Regions: US-East, US-West, Singapore, Seoul)
• C5

Data Protection with SageMaker Notebook Instances

Encryption, both in transit and at rest, is a best practice to ensure the confidentiality and integrity of the data being processed by SageMaker notebook instances. FSI customers typically encrypt their data on AWS, to protect sensitive data from unauthorized access, meet compliance/regulatory requirements, and align with cloud security best practices.

AWS by default encrypts the presigned URL using HTTPS for accessing SageMaker notebook instances over the network.

Customers can configure encryption to protect the SageMaker notebook instance data in:

1. Notebook instance storage volume data at rest (If you don’t specify an AWS Key Management Service (AWS KMS) key, SageMaker encrypts storage volumes with a transient key and discards it immediately after encrypting the storage EBS volume. For notebook instances, if you don’t specify a KMS key, SageMaker encrypts both OS volumes and ML data volumes with a system-managed KMS key)
2. At rest in Amazon Simple Storage Service (Amazon S3) buckets
3. In transit for Amazon S3 by using the aws:SecureTransport condition on Amazon S3 bucket policies. We recommend implementing a detective control using the s3-bucket-ssl-requests-only managed AWS Config rule to ensure that all objects in the specified S3 bucket, are accessed with secure transport.
4. In transit by using Inter-container traffic encryption (data transmitted between instances, when performing distributed training)

In addition to the enterprise-wide Amazon S3 security best practices, customers can choose to further protect their SageMaker notebook instances-related S3 buckets by:

1. Creating bucket and IAM policies consistent with the principle of least privilege and public access block enabled
2. Using VPC endpoints for access, ensure SageMaker can only access permissible S3 buckets
3. Enable cross-Region replication if you have regulatory requirements that data be replicated at distances larger than multiple geographically diverse Availability Zones
4. Using AWS services like Macie to discover and protect your sensitive data, and GuardDuty to monitor object-level API operations and identify potential security risks

The following IAM conditions can be used to enforce data protection when working with SageMaker notebook instances:

Isolation of Compute Environments with SageMaker Notebook Instances

Amazon SageMaker provides managed EC2-based services such as Jupyter Notebooks. SageMaker runs the infrastructure for these components using EC2 resources dedicated to you. These EC2 resources can have a network interface in your VPC environment allowing you to apply your network level controls, such as security groups, to the notebooks, training jobs, and hosted ML models.

Amazon SageMaker does this by creating an elastic network interface (ENI) in your specified VPC and attaching it to the SageMaker managed instance. Using this pattern, the service gives you control over the network-level access of the services you run on Amazon SageMaker.

To disable direct internet access, you can specify a VPC for your notebook instance that does not have access to the internet in your AWS account, you can limit the internet access in the service account by setting the sagemaker:DirectInternetAccess IAM condition key or by setting the Direct Internet Access flag in the console. By doing so, you prevent SageMaker from providing internet access to your notebook instance. As a result, the notebook instance won’t be able to train or host models unless your VPC has an interface endpoint (PrivateLink) or a NAT gateway, and your security groups allow outbound connections.

If you enable network isolation, the containers can’t make calls to external networks, even to other AWS services such as Amazon S3.

You can connect to your notebook instance from your VPC through an interface endpoint in your Virtual Private Cloud (VPC) instead of connecting over the internet.

SageMaker notebook instances support Amazon Virtual Private Cloud (Amazon VPC) interface endpoints that are powered by AWS PrivateLink. Each VPC endpoint is represented by one or more Elastic Network Interfaces with private IP addresses in your VPC subnets.

Automating Audits with APIs with SageMaker Notebook Instances

There are several AWS Config rules that can be implemented to ensure compliance with specific configurations. AWS Config monitors the configuration of resources and provides some out of the box rules to alert when resources fall into a non-compliant state. AWS Config has three SageMaker specific managed config rules out of the box. The first is sagemaker-notebook-no-direct-internet-access that checks whether direct internet access is disabled for an Amazon SageMaker notebook instance. The second is sagemaker-notebook-instance-kms-key-configured that checks whether an AWS KMS key is configured for the Amazon SageMaker Notebook Instance. The last managed config rule is sagemaker-endpoint-configuration-kms-key-configured that checks whether an AWS KMS key is configured for an Amazon SageMaker endpoint configuration. You can view more details on these managed config rules here.

API calls made to the SageMaker service are recorded in AWS CloudTrail. CloudTrail provides an aggregated repository of AWS API calls and changes to resources for over 160 AWS services. When it comes to SageMaker notebook instances there are a few key APIs that should be monitored to ensure only approved notebooks have been created. For SageMaker notebook instances:

• The CreateNotebookInstance API can be used to create notebooks.
• The DeleteNotebookInstance API can be used to delete existing notebooks.
• The CreatePresignedNotebookInstanceUrl API can be used to generate a URL that you can use to connect to the Jupyter server from the notebook instance, once a notebook is created. (You can restrict access to this API and to the URL that it returns to a list of IP addresses that you specify.)

Once the notebooks have been created, there are a few key APIs to ensure that they don’t deviate from defined standards.

• The StartNotebookInstance and StopNotebookInstance API calls can be used to start and stop existing notebooks and should be monitored to ensure the crucial work of developers and data scientists isn’t abruptly interrupted.
• The CreateNotebookInstanceLifecycleConfig API can be used to configure a collection of shell scripts that run when you create or start a notebook. These scripts can be used to integrate with private package repos or pre-install custom packages, connect to a proxy server, get enhanced logging and monitoring on notebooks etc. This ensures only approved lifecycle configs are created and run should be monitored (DeleteNotebookInstanceLifecycleConfig API performs the opposite operation).
• Monitoring the UpdateNotebookInstanceLifecycleConfig API to validate that no unintended modifications to an existing lifecycle config is key to ensuring compliance to set standards.
• The UpdateNotebookInstanceAPI can be used to make modifications to an existing notebook such as changing the instance type or associating/disassociating a lifecycle config with a notebook.

These APIs should all be monitored to ensure that only appropriate actions are being made against your SageMaker notebook instances and leveraging CloudTrail can help achieve this goal.

For a complete list of SageMaker APIs, not only related to SageMaker notebook instances, review the Amazon SageMaker API Reference.

Here is an example of what a CloudTrail log looks like for the CreateNotebookInstance API:

{
    "eventVersion": "1.05",
    "userIdentity": {
        "type": "AssumedRole",
        "principalId": "XXXXXXXXXXXXXXXXXXXXXX:NAME",
        "arn": "arn:aws:sts::xxx:assumed-role/admin/NAME",
        "accountId": "XXX",
        "accessKeyId": "XXX",
        "sessionContext": {
            "sessionIssuer": {
                "type": "Role",
                "principalId": " XXXXXXXXXXXXXXXXXXXXXX ",
                "arn": "arn:aws:iam::XXX:role/NAME",
                "accountId": "XXX",
                "userName": "NAME"
            },
            "webIdFederationData": {},
            "attributes": {
                "mfaAuthenticated": "false",
                "creationDate": "2020-10-29T12:40:04Z"
            }
        }
    },
    "eventTime": "2020-10-29T12:41:32Z",
    "eventSource": "sagemaker.amazonaws.com",
    "eventName": "CreateNotebookInstance",
    "awsRegion": "us-east-1",
    "sourceIPAddress": "72.21.198.65",
    "userAgent": "aws-internal/3 aws-sdk-java/1.11.864 Linux/4.9.217-0.3.ac.206.84.332.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.262-b10 java/1.8.0_262 vendor/Oracle_Corporation",
    "requestParameters": {
        "notebookInstanceName": "servicespotlight",
        "instanceType": "ml.t2.medium",
        "subnetId": "subnet-efd911ce",
        "securityGroupIds": [
            "sg-4XXXXX68"
        ],
        "roleArn": "arn:aws:iam::XXX:role/service-role/AmazonSageMaker-ExecutionRole-20201029T084187",
        "directInternetAccess": "Enabled",
        "volumeSizeInGB": 5,
        "rootAccess": "Enabled"
    },
    "responseElements": {
        "notebookInstanceArn": "arn:aws:sagemaker:us-east-1:XXX:notebook-instance/servicespotlight"
    },
    "requestID": "8063007f-fc31-4d1b-ae28-53c8b144c85e",
    "eventID": "1c677e93-7742-48fd-a34c-4ce100aabf66",
    "eventType": "AwsApiCall",
    "recipientAccountId": "XXX"
}

Operational Access and Security with SageMaker Notebook Instances

We know that customers care deeply about privacy and data security. That’s why AWS gives you ownership and control over your content through simple, powerful tools that allow you to determine where your content will be stored, secure your content in transit and at rest, and manage your access to AWS services and resources for your users. We also implement responsible and sophisticated technical and physical controls that are designed to prevent unauthorized access to or disclosure of your content.

AWS has developed a security assurance program that uses best practices for global privacy and data protection to help you operate securely within AWS, and to make the best use of our security control environment. These security protections and control processes are independently validated by multiple third-party independent assessments.

Customers can review third-party auditor reports in the AWS SOC 2 Type II report in AWS Artifact.

Access to a SageMaker Jupyter notebook instance is governed by AWS IAM. In order to open a Jupyter notebook instance, users will need access to the CreatePresignedNotebookInstanceUrl API call. This API call creates a presigned URL which can be followed to obtain Web UI access to the Jupyter notebook server. To secure this interface, you use IAM policy statements to wrap conditions around calling the API, for example who can invoke the API and from what IP address.

{
   "Id":"notebook-example-1",
   "Version":"2012-10-17",
   "Statement":[
      {
         "Sid":"Enable Notebook Access",
         "Effect":"Allow",
         "Action":[
            "sagemaker:CreatePresignedNotebookInstanceUrl",
            "sagemaker:DescribeNotebookInstance"
         ],
         "Resource":"*",
         "Condition":{
            "ForAnyValue:StringEquals":{
               "aws:sourceVpce":[
                  "vpce-111bbccc",
                  "vpce-111bbddd"
               ]
            }
         }
      }
   ]
}

The preceding IAM policy states that someone can only communicate with a notebook if they do so from within a VPC and through specific VPC endpoints. Using mechanisms like the preceding example, you can explicitly control who can interact with a notebook server.

Customers can also restrict the SageMaker notebook instance types to the instance types that are built on The AWS Nitro System. Nitro System is a collection of AWS-built hardware and software components that enable high performance, high availability, and high security. The Nitro System’s security model is locked down and prohibits interactive access, reducing the possibility of human error and tampering. The list of EC2 instances that are built on the Nitro System can be found here.

Service control policies (SCPs) are a type of organization policy that you can use to manage permissions in your organization. SCPs offer central control over the maximum available permissions for all accounts in your organization. SCPs help you to ensure your accounts stay within your organization’s access control guidelines.

A sample SCP is shown in the following example that will prevent creation of notebook instances unless they built on the Nitro System. This SCP can be applied to the desired Organizational Unit(s) or Accounts where SageMaker notebook instances will be created.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "RestrictSagemakerInstanceTypestoNitro",
            "Effect": "Deny",
            "Action": "sagemaker:CreateNotebookInstance",
            "Resource": [
                "arn:aws:sagemaker:*:*:notebook-instance/*"
            ],
            "Condition": {
                "ForAnyValue:StringNotEquals": {
                    "sagemaker:InstanceTypes": [
                        "ml.c5.xlarge",
                        "ml.c5.2xlarge",
                        "ml.c5.4xlarge"
                        "ml.c5.xlarge",
                        "ml.c5.2xlarge",
                        "ml.c5.4xlarge",
                        "ml.c5.9xlarge",
                        "ml.c5.18xlarge",
                        "ml.c5d.xlarge",
                        "ml.c5d.2xlarge",
                        "ml.c5d.4xlarge",
                        "ml.c5d.9xlarge",
                        "ml.c5d.18xlarge"
                    ]
                }
            }
        }
    ]
}

Conclusion

In this post, we reviewed SageMaker notebook instances and highlighted key information that can help FSI customers accelerate the approval of the service within these five categories: achieving compliance, data protection, isolation of compute environments, automating audits with APIs, and operational access and security. While not a one-size-fits-all approach, the guidance can be adapted to meet your organization’s security and compliance requirements and provide a consolidated list of key areas for SageMaker notebook instances.

In the meantime, be sure to visit our AWS Industries blog channel and stay tuned for more financial services news and best practices.