Integrating Research and Engineering Studio in Trusted Research Environments built on AWS

This post was contributed by Balaji Raman, Amit Samal, Steven Johnston and Viji Ravichandran

Enterprise and academic research organizations are moving their compute workloads to the cloud for their research needs. Security and compliance are key considerations for customers dealing with highly sensitive data in healthcare, financial, and education industry segments, among others. The demand for a standard computing environment for research with built-in security controls, user access management, information governance, and data management are increasing globally.

Organizations call these environments Trusted Research Environments (TREs). Specifications like SATRE in the UK and FISMA in the US provide minimum requirements for information systems handling sensitive data in their compute environments. These specs typically describe processes and controls on handling of data, network, compute isolation, access controls and other aspects which apply to these controlled environments. They drive consistency across organizations making it easier for users, operators, and developers to work with sensitive data.

Anyone building bespoke technical controls for compute workloads within TREs find maintenance, ops, and compliance burdensome. Research and Engineering Studio on AWS (RES), lets you configure and customize various controls to remove the need for a bespoke implementation.

In this blog post, we’ll describe how to integrate RES, an AWS-supported, open-source product, to meet the needs for compute workloads in a TRE. RES enables admins to provide a portal for scientists and engineers to run technical computing workloads quickly on AWS. We’ll show you how to use the controls available in RES to build and integrate it into a TRE using an example use case.

Virtual research environment within a trusted research environment

TREs are designed to provide a secure and controlled space for authorized users to access, store, analyze, and share sensitive data, analysis, models, and other artifacts while maintaining data privacy, security, and compliance with governance protocols to enable advanced research and development. The virtual research environment (VRE) within a TRE facilitates access to compute resources and provides the technical controls needed to align with TRE design objectives.

Figure 1 illustrates the functional blocks of a typical TRE and highlights the virtual research environment where you can use RES.

Figure 1- Shows the various functional blocks in a typical TRE. RES integrates into the Virtual Research Environment functional block and interacts with other functional blocks for data handling, including ingress and egress from the compute environments.

The VRE enables researchers to work with data assigned to their project, using compute environments they provision to conduct their studies and will be maintained by a TRE Operator – a persona described in SATRE as an infrastructure management role.

A high-level data flow for ingress and egress out of VRE is as follows:

1. Data from data sources is moved into a Data Staging environment by researchers or an organization.
2. Information Governance manager reviews and approves data in the Airlock component and moves it to a secure Data Environment.
3. Data manager controls linking approved datasets requested by researchers to their respective projects in VRE.
4. Researchers are allocated logically isolated projects in VRE with access to their datasets for conducting research. Researchers request review from Information Governance Manager and potentially other reviewers to egress their analysis out of TRE. One and/or both approve data egress in the Airlock component similar to Ingress.

For more in-depth guidance on best-practices and standardized architectures, refer to SATRE, which provides a comprehensive framework for TRE operators and developers. In this post, we’ll focus our attention on the VRE functional block and how RES fits into this function and integrates with TRE.

Integrating RES in TRE

RES has a concept of projects, which act as virtual collaboration spaces for authorized users. We will walk through deployment considerations and a use case where the administrator creates two projects (Project A and Project B) in RES, with logical separation for users, data, network, and compute.

For securing data egress, we will demonstrate how global permissions in RES restrict data movement from VDIs and set up Amazon Virtual Private Cloud (VPC) end points to prevent data exfiltration from VDIs. For data isolation, Project A and Project B will have access to separate research datasets from an Amazon Simple Storage Service (Amazon S3) bucket and separate Amazon Elastic File System (EFS) file system as home directories for data isolation by project for the VDI.

For compute isolation, Project A and Project B will use customized VDI with Red Hat Linux 9 (RHEL9) and Amazon Linux 2 (AL2) respectively. Further, we will demonstrate several project level configurations including isolation of users with roles, network isolation of project resources through security groups, scripts to customize VDI, and access to other AWS services from the VDI through Identity and Access Management (IAM) policies.

Step 1. Deployment consideration for RES

• Review the deployment plan and create any external resources required before deploying RES.
• Optionally, setup custom permission boundaries to restrict access to AWS resources from the RES environment.
• For deployment in private VPC and keep the traffic private, you need to prepare the Amazon Machine Image (AMI) for infrastructure hosts to work in isolated subnets and setup VPC end points.To deploy RES in a private VPC, follow the pre-requisites here. The instructions will help to set up the VPC endpoints for keeping the traffic private and prepare the Amazon Machine Image (AMI) for infrastructure hosts to work in isolated subnets without internet traffic.
• Follow the documentation to deploy RES. For any additional deployment controls for highly regulated workloads, please refer to Landing Zone Accelerator on AWS. As part of deployment configuration, you can restrict access at the network level to the web portal through the CIDR block and prefix list.

Step 2. Identity setup

RES uses any SAML 2.0/OIDC compliant identity provider to authenticate user access to the RES Portal. Another one of our blog posts can help you understand the different identity setup RES provides that best fit your deployment. RES allows you to manage users and groups from your Identity Provider.

Step 3. Global permissions to restrict data egress from VDI

The following are the different configurations administrators set to restrict data egress from VDI at the global level. Permissions set at the global level will take effect for every project in RES.

a. Desktop Sharing Profile

While organizations encourage collaboration between researchers, they still have obligations to meet compliance policies to restrict certain functionality from the VDI, such as clipboard copy/paste or upload/download of files from/to their VDI. RES allows administrators to restrict these permissions centrally through the use of Desktop Sharing Profile.

b. File Browser

RES allows users to download data from the File Browser available in the web portal. If you want to restrict this functionality, you achieve this by disabling File Browser access to users.

c. SSH

If, to meet organizational compliance, you don’t want anyone to access your VDI through SSH you can achieve this by configuring SSH access from RES to block SSH access to RES environment.

Step 4. Permission Profiles for Project level resources

RES comes with default two roles: Project Member and Project Owner. If you further want to customize the permissions and restrict certain access to resources at the project level, such as creation of sessions, you achieve this through the use of custom permission profiles.

Step 5. Onboard S3 buckets and Filesystems

• Onboard the Amazon S3 buckets, namely Bucket A and Bucket B, for the two projects. To prevent users from exporting data from S3 buckets in TRE into their own AWS account containing S3 buckets outside TRE, attach a VPC endpoint with IAM policies to your private VPC. For more instructions on setup, see here.
• Onboard the Amazon EFS filesystems, EFS1 and EFS2, for the two projects. This keeps the user data segregated on separate filesystems for both projects.

We will configure in Step 7 on how to restrict access for Bucket A/EFS1 only to Project A and Bucket B/EFS2 to Project B. Buckets and filesystem associated with a project will mount on the VDI that belongs to the project during the start of session, ensuring data isolation across projects and users.

Step 6. Compute customization for project and deployment in isolated subnets

Researchers have diverse needs for tools and applications to analyze datasets and vary by project. With RES-ready Amazon Machine Images (AMIs), administrators pre-install RES dependencies for virtual desktop instances (VDIs) on custom AMIs. RES-ready AMI allows you to deploy in isolated subnets with no traffic to public internet.

RES manages AMI as software stacks. Customize the two different AMIs, one with RHEL9, and another with AL2, and register them as Software Stack 1 and Software Stack 2 in RES. We will demonstrate in step 7 on how to associate the software stacks with projects.

Step 7. Project-level isolation of resources and users

Projects form a boundary for virtual desktops, teams, and budgets. In this example, we will create two projects, Project A and Project B. We will isolate access at the project level for data in Amazon S3 buckets, separate filesystems as home directory, user access, network isolation through security groups and software stacks for VDIs.

During the project setup, administrators can perform the following configurations.

a. User isolation

Assign users and/or groups the role (Project Member or Project Owner). See Default permissions profiles for the actions each role can take. RES also allows you to create a custom role and assign to users or group.

b. Network isolation

Add a security group to control the egress and ingress of network traffic for VDI instances under your project.

c. Dataset isolation

Select Bucket 1 for Project A and Bucket 2 for Project B to restrict access to buckets by projects. Buckets will mount only to those VDIs associated with a project.

d. Per-project home directory for data isolation

For project level isolation, select EFS1 filesystem for Project A and EFS2 filesystem for Project B to mount as home directory.

e. Optional Configuration for Projects

If your project needs to provide access to other AWS Services, you do so by adding an IAM policy to control VDI access to other AWS resources deployed under your project. For e.g. access to services such as Amazon RedShift for data. Further, you can add launch scripts during the session start for VDI within your project to further customize your VDI environment. RES allows script initiation for Linux and Microsoft Windows. For example, launch scripts can acquire a license for a tool running in your VDI.

Step 8. Associate Software Stack to Project and launch VDI

a. Register Software Stack to Projects

Associate Software Stack 1 to Project A and Software Stack 2 to Project B in order to launch the VDI by following the instructions here.

Step 9. Launch VDI in Projects

Users belonging to Project A can now launch RHEL9 based VDI and similarly users belonging to Project B can launch AL2 based VDI. VDI belonging to Project A will mount Bucket 1 and EFS1 as home filesystem. VDI belonging to Project B will mount Bucket 2 and EFS2 as home filesystem.

Conclusion

In this blog post, we showed how to set up a RES environment with two projects that have data, network and compute level isolation and access control to resources.

RES provides flexibility to use architecture that supports customization for your specific TRE functional requirements and allows integration with the rest of your TRE environment including mechanisms for data ingress and egress out of your VRE.

Customers remain responsible for validating the technical controls provided by RES against their organizational compliance standards and accreditation requirements. The guidance provided in this post will help you integrate RES in TRE environments, but if you want additional assistance, your AWS account team can provide advice and help from our Solutions Architects who have experience designing TREs using RES.

AWS Professional Services also has experience deploying TREs using RES in production, and can help assess your TRE requirements, provide architectural guidance and implementation services.

Finally, we can refer you to our amazing APN partners who can help with integration support, periodic maintenance, and upgrades of your TRE environments.

If you have questions or feedback for us, please reach out to ask-hpc@amazon.com.