AWS Partner Network (APN) Blog
Building Customer 360 Experiences Through Secure, Zero-Copy Data Collaboration Using AWS Clean Rooms
By Kasi Muthu, Sr. Partner Solutions Architect, Data and AI – AWS
By Rajib Deb, Specialist Leader, AI and Data Engineering – Deloitte
By Ashok Thambi, Sr. Manager, AI and Data Engineering – Deloitte
 |
| Deloitte |
 |
Technology advances have enabled organizations to harvest massive amounts of data, and rich insights from this data enables hyper-personalization through which companies can offer relevant products and services to their customers. To fully engage with customers and produce meaningful experiences, creating a 360-degree view of them is essential.
The information needed to build this view requires internal and external sources of data. Therefore, building a 360-degree view involves collecting and aggregating data from disparate applications, data sources, and marketing channels into a centralized location. It also involves complex data movement pipelines.
In this post, we will introduce the concept of data clean rooms and showcase how AWS Clean Rooms helps customers and their partners more easily and securely collaborate and analyze their collective datasets—without sharing or copying one another’s underlying data. We’ll also discuss how AWS Clean Rooms makes building a 360-degree experience secure and seamless by combining various data sources without having to move any data around.
Deloitte is an AWS Premier Tier Services Partner and Managed Service Provider (MSP) that’s recognized as an AWS Clean Rooms Partner. Deloitte’s end-to-end capabilities and understanding of your business and industry help amplify the transformative value of cloud.
The Concept of Clean Rooms
A data clean room, in simple terms, is a secure, private, and governed data exchange where multiple parties share and collaborate with their data while maintaining privacy and data integrity. With a clean room, enterprises combine and aggregate data without revealing individual customer data. In this setup, each provider has authoritative controls over how much data they want to share.
Clean rooms were initially adopted by the pharmaceutical industry to allow companies and researchers to access aggregate medical data while helping to ensure patient privacy. This is now widely adopted across industry sectors like advertising, financial, retail, and manufacturing. Data clean rooms enable the capability to democratize collaboration while maintaining quality, trust, security, and auditability of the data.
Reference Architecture
A typical data clean room architecture will consist of the following components.
Figure 1 – Data clean room logical architecture.
Organizations that want to take part in the collaboration form the vertices here. Let’s look at the other components:
- Data share: The data clean room solution needs to have an easy abstraction to be able to share any type of data assets, such as tables, views, or stored procedures. This component allows addition or removal of data assets dynamically and as required.
- Data access policies: This component allows companies to add access constraints on the data as well as on the data operations. It’s not sufficient to only restrict which data elements will be visible to the consumer; you also need to be able to add constraints on what type of operations the consumer can do on the data.
- Logging and usage metrics: This component acts as the observatory layer which enables logging and auditing the data usage. It also allows organizations to measure the usage metrics per consumer so it may be monetized based on the amount of usage.
AWS Clean Rooms
AWS customers often want to share and collaborate on data with another enterprise or within a siloed business unit in the same enterprise. To make this happen, however, there are challenges in terms of data protection imperatives and data governance standards. There’s also the need to build and manage a data movement ecosystem.
AWS Clean Rooms provides a secure environment for data sharing and ensures data is kept private and confidential throughout the collaboration process. It helps enterprises perform privacy-centric analysis on decentralized datasets. It also helps keeping the underlying data and schema of the data entirely private.
AWs Clean Rooms enables customers to collaborate with any other enterprise on the AWS cloud in minutes. This allows for use cases such as unique insights, advertising campaigns, investment decisions, and data enrichment. Each of the collaborators can restrict the type of analysis allowed on their data, thereby providing them full control of what can be shared. There’s also an option to pre-encrypt the data so even when an analysis is being run, it’s run on encrypted data.
The following diagram shows a typical reference architecture of a data clean room in the AWS ecosystem.
Figure 2 – AWS Clean Rooms reference architecture.
Customer 360 Use Case
To demonstrate the power of AWS Clean Rooms, let’s take an example of a telco enterprise that wants to build a 360-degree view of its customers. The enterprise wants to dive deeper into the customer experience by analyzing their interactions with support and also know more about their preferences.
For the sake of simplicity, let’s consider the following data across three platforms:
- First-party (1P): Customer data collected and stored directly by the telco enterprise.
- Second-party (2P): Data collected, stored, and operated by a partner of the telco that handles support tickets.
- Third-party (3P): A commercial data agency that provides supplemental information about the enterprise’s customers.
The following diagram shows the data elements that make up this sample dataset we’ll use for demonstration purposes.
Figure 3 – Sample data elements for a telco enterprise.
Solution Architecture
The next diagram shows the architecture of a data clean room that can be deployed for a customer 360 solution.
Figure 4 – Solution architecture for customer 360 solution for a telco enterprise.
On top is the first-party AWS account which is owned by the telco enterprise. It accumulates data needed for the customer 360 use cases and lands them in Amazon Simple Storage Services (Amazon S3). Once the data in S3 is cataloged in an AWS Glue table, the telco creates a collaboration in AWS Clean Rooms. Once this is done, the organization invites the second-party and third-party AWS accounts as collaborators.
Similarly, the second and third-party accounts land and catalog the data they are willing to share on a collaboration to S3. Once the first-party AWS account sends the invite for collaboration, it shows up on the AWS Clean Rooms console for second- and third-party accounts, as shown in the following screenshot. They can choose to accept it.
Figure 5 – Clean rooms collaboration invite.
With AWS Clean Rooms, only one of the collaborators are able to query and view the results. The first-party account, an account owned and operated by the telco enterprise in this example, keeps that ability. The following screenshot shows how this is set up on the first-party account.
Figure 6 – Clean rooms setup.
After this is done, the second- and third-party accounts receive an invite for collaboration from the first-party account on their Clean Rooms console. They can choose to accept the collaboration.
Configuring Tables
The next step is to configure tables for collaboration. Here is where all parties can decide how much or how little they want to share in a collaboration. You can decide to choose either all columns in AWS Glue Catalog table or a custom list of limited columns.
In this customer 360 use case, all three parties choose to share the columns that are relevant for the use case.
Configuring Analysis Rules
Once a table is configured, you must configure analysis rules so it can be used in queries. There are two kinds of analysis rules that AWS Clean Rooms supports today:
- Aggregation: This allows queries that aggregate statistics using SUM, AVG, and COUNT along optional dimensions. This supports use cases such as segment analysis, measurement, and attribution.
- List: This allows queries that list the overlap between this table and the table of the member who can query. It supports use cases such as enrichment and segment building.
Since we are interested in enrichment with the customer 360 use case, we pick the List option.
The next step is to specify the join column, a column that overlaps with the member who wants to query. You can also specify the columns that can be used in the SELECT and WHERE clauses of the query. The following screenshot shows how it’s set up for the first-party account, the member with query privileges.
Figure 7 – Join controls for analysis.
A similar approach can be followed in configuring and setting up analysis rules in the second- and third-party accounts.
Querying
Once all of the required tables across all accounts are configured, the account with the query permissions can simply start executing queries. There’s no need to build extract, transform, load (ETL) pipelines, move data around, or share schemas.
The following screenshot shows how the query window looks like. It lists all of the members in the collaborations, the associated tables, and even the columns and controls that are in place for querying.
Figure 8 – Querying data across all parties.
The output to this query that combines customer data from three parties land in Amazon S3. As shown in the solution architecture diagram in Figure 4 above, it can then be loaded into Amazon Redshift for reporting, or fed into Amazon SageMaker for data science use cases such as predicting customer churn.
Cleanup
To clean up the resources you created, go to the collaborations page and delete the collaboration on all three accounts.
Conclusion
An ecosystem is stronger than each participant within the ecosystem, and the same is true for data collaboration. Each participant within a data collaboration ecosystem benefits by sharing data insights from each other.
AWS Clean Rooms enables that collaboration in a secure and governed way. Implementing a data clean room helps organizations amplify their business growth by creating a data exchange with a group of data providers and enabling them to exchange data insights. This is done while protecting sensitive data and ensuring compliance with data protection guidelines
AWS and Deloitte empower customers to transform their business, innovate faster, and grow ahead of the curve. The combination of AWS’s industry-leading cloud technologies with Deloitte’s deep industry experience and established customer relationships makes this possible.
Deloitte is a strategic global systems integrator with thousands of certified AWS practitioners across the globe. It continues to raise the bar through participation in the AWS Competency Program with 21 expert specializations. Learn more by visiting the AWS and Deloitte page.
.
.
Deloitte – AWS Partner Spotlight
Deloitte is an AWS Premier Consulting Partner and MSP. Through a network of professionals, industry specialists, and an ecosystem of alliances, they assist clients in turning complex business issues into opportunities for growth, helping organizations transform in the digital era.