Replicate SAP to AWS in Real-Time with Business Logic Intact Using BryteFlow

By Nimish Jacob, Solutions Architect – Bryte Systems
By Faraz Masood, Sr. Solutions Architect – AWS

Bryte Systems

Organizations are no longer constrained by storage capacities and no longer need to prioritize data streams and sources.

Migrating to the cloud helps accelerate data-driven initiatives for modern organizations, and provides a unified view of data and an economical way to store unstructured, semi-structured, and structured data in one centralized location.

Getting SAP data into Amazon Web Services (AWS) in real-time enables insights for better business decisions, realizes competitive advantages, enhances sharing and collaboration, and improves operational performance. It also provides the opportunity to integrate data from SAP and non-SAP sources.

This post examines how to extract and integrate SAP data on AWS for use cases like analytics, reporting, artificial intelligence (AI), machine learning (ML), and Internet of Things (IoT) in real-time, using the BryteFlow SAP Data Lake Builder on AWS.

Bryte Systems is an AWS Partner and AWS Marketplace Seller with Competencies in Migration as well as Data and Analytics Migration. BryteFlow provides real-time, no-code replication to cloud and on-premises platforms.

Let’s look at how the BryteFlow SAP Data Lake Builder enables you to integrate SAP data on AWS and realize more value from SAP data.

Looking Beyond SAP BW

SAP BW, or the SAP Business Warehouse, is a reliable, scalable, and secure platform that provides tools and functionalities to use and interpret business data for reporting and analytics. It’s a software system that enables companies to collect, integrate, transform, and analyze data from SAP applications and external data sources in a single, centralized data repository.

Data in SAP BW can be modeled for complex querying, analytics, and reporting for taking informed business decisions. If your implementation loads data from SAP systems to SAP BW, you may have realized it involves multiple steps and configurations as part of a complex process.

Operational reporting capabilities in SAP make use of embedded analytics within enterprise resource planning (ERP) applications. However, there’s an increasing demand from users for data integration between SAP and non-SAP data, data from social media, IoT, and software-as-a-service (SaaS) applications. These users need to create modern business models with machine learning and increase operational efficiencies.

Why Take SAP Data to AWS?

In addition to leveraging SAP BW, it’s recommended to get your SAP data to AWS, as users can build complete enterprise analytics workflows using data lakes and analytics tools on AWS.

The AWS ecosystem has a host of applications that can help you realize more value from your SAP data. AWS offers multiple ways to put your data to work and not just for business intelligence (BI), reporting, and analytics, but also for AI/ML, IoT, and other use cases.

Integrating SAP Data on AWS for Multiple Use Cases

AWS provides over 200 integrated applications for infrastructure technologies like compute, storage, and databases, and emerging technologies like AI/ML, data lakes, analytics, and IoT. You can easily integrate your SAP ERP data and data from other sources to drive these initiatives, in addition to real-time reporting and analytics.

Some AWS services and their use cases include:

Cost-Effective Storage for All Your Data

Large volumes of SAP data and other data can be stored on Amazon Simple Storage Service (Amazon S3) at affordable rates. Your data resides in S3 and you can take across to the data warehouse only what you actually need for the compute.

Scalability to Process Large Volumes of Data

The AWS platform is highly scalable and enables scaling up and down of compute resources as per requirements. There are no infrastructure limitations.

Flexibility to Select the Services You Need

You have the freedom to select the services you need for your SAP data management and analytics and pay only for what is consumed. This provides the flexibility to experiment with different tools and services and adapt to changing business requirements.

High Availability

SAP data is mission-critical, and AWS ensures data is always accessible and secure by providing multiple Availability zones and regions. Automated backups, disaster recovery, and data replication features ensure business continuity.

Data Remains Highly Secure

AWS provides multiple layers of security including network security, identity and access management, encryption, and compliance certifications.

BryteFlow SAP Data Lake Builder on AWS

SAP data, by its nature, is inherently complex. SAP’s proprietary ABAP language, logic, and processes can be challenging to integrate with data from other sources. ABAP replication is batch only, so the issue of replication in real-time also needs to be worked around.

SAP tables could number in thousands, with complex relationships, proprietary formats, and customized modules, which could prove a hurdle in data extraction and integration. This issue can be addressed with the inclusion of the BryteFlow SAP Data Lake Builder in your SAP integration stack.

The BryteFlow SAP Data Lake Builder is a third-party SAP extract, transform, load (ETL) tool that integrates seamlessly with the AWS ecosystem. It extracts SAP ERP data at the application level with business logic intact from applications like SAP ECC, S/4HANA, SAP BW, SAP HANA using the Operational Data Provisioning (ODP) framework, and OData services. It provides real-time data on destinations like Amazon S3 and Amazon Redshift, syncing data with source using change data capture (CDC).

The BryteFlow SAP Data Lake Builder is a no-code tool and provides a completely automated setup of data extraction and automated analysis of the SAP source application. It delivers data that’s ready-to-use on the target for various uses cases including analytics and machine learning.

Figure 1 – Replicating SAP data to AWS in real-time with BryteFlow.

Key Features

• Connects to SAP at the application level. The tool can connect to ODP extractors, SAP SLT (if licensed), or core data service (CDS) views to extract data and build the data lake.
• Provides flexible connections for SAP including database logs, ECC, HANA, S/4HANA, and SAP data services.
• Automates data extraction, using OData services to extract the data—both initial, incremental, or deltas—and syncs data in real-time using CDC.
• Captures the inserts, updates, and deletes as they happen and merges them with data in real-time.
• Routinely replicates over a million rows of data in 30 seconds.
• No coding is required for any process including data extraction, masking, SCD Type2, or data merging.
• Upsert on Amazon S3 is handled without any coding with stateless EMR clusters and full support for configurable partitioning, compression and file types with no coding, and integration into the AWS big data ecosystem.
• Automates integration with Amazon Athena and AWS Glue Data Catalog (at API level).
• Provide data transformation on S3, Redshift, Amazon Aurora, and Snowflake in collaboration with BryteFlow Blend.
• Automates integration with AWS services as required. This includes S3, EMR clusters, Amazon Athena, AWS Glue Data Catalog, and Redshift. It also uses AWS Identity and Access Management (IAM), AWS Key Management Service (AWS KMS), Amazon CloudWatch Logs, and Amazon Simple Notification Service (SNS) for monitoring and security.
• Encrypts the data at rest and in transit. It uses SSL to connect to data warehouses and databases, and can mask sensitive data and integrates with AWS KMS to encrypt data for high-level security.
• Highly secure and installed in the customer’s virtual private cloud (VPC) with no outbound access. All security and data access is controlled by the customer.
• BryteFlow software can be configured for IAM, AWS KMS, and other AWS security best practices, as per the customer’s requirements.
• Provides automated configuration for smart partitioning, file types, and compression while ingesting data.
• Enables querying on the data lake itself with Amazon Athena or Amazon Redshift Spectrum.
• Provides an enterprise-grade data preparation workbench that separates raw data and enriched or curated data into multiple S3 folders while managing jobs and dependencies. It categorizes data into different levels of security classifications and maturity–from raw data to highly-curated data marts.
• All data definition languages (DDLs), data mapping, and data conversion are automatically handled.
• Provides automated SCD Type 2 history of the data.
• End-to-end monitoring of workflows is provided with BryteFlow ControlRoom.
• Automatic network catch-up mode resumes where it left off when conditions normalize.

Finally, the BryteFlow SAP Data Lake Builder delivers data to AWS with the following methods:

• To Amazon S3 directly using EMR
• To S3 then to Redshift using EMR
• To Redshift directly
• To AWS S3 and Athena

Figure 2 – Select how you want to connect to SAP from the drop-down.

BryteFlow SAP Data Lake Builder offers various methods to connect to SAP. Users can select the tables needed for replication (with SCD history if required).

Figure 3 – Select checkbox to define table for replication and mask sensitive data, if required.

BryteFlow Blend for Data Transformation on AWS

In case you have a requirement to join, model, or transform SAP data with non-SAP data, BryteFlow Blend provides a convenient way to do this. It has a drag-and-drop interface for data enrichment and joining of SAP and non-SAP sources on Snowflake, Amazon S3, Amazon Redshift, and PostgreSQL.

BryteFlow Blend integrates with the BryteFlow SAP Data Lake Builder seamlessly for transformation jobs. It joins and merges data from legacy databases, device and sensor data, and application data and transforms it using simple SQL.

BryteFlow Blend provides smart partitioning and compression of data and enables users to use BI tools to consume the data. Data transformation automatically maintains full metadata and data lineage, and the transformation software creates a data-as-a-service environment where business users can self-serve.

ALDO Group Unlocks SAP Data to AWS

ALDO is a prominent retailer that needed to unlock data in SAP BW and take it to AWS for new insights and use cases. Replicating the data in SAP BW to AWS would liberate the data and enable it to be used by multiple tools for several use cases.

The BryteFlow SAP Data Lake Builder was used as a no-code solution to extract the ALDO Group’s data from the SAP HANA application layer to the AWS data lake using CDS Views and standard extractors.

ALDO Group can now access data from SAP in an elastic, scalable data lake on AWS for analytics and AI/ML use cases. No data engineer was required during the implementation and the solution was deployed at least 90% faster than comparable solutions. To learn more, read BryteFlow’s ALDO customer success story.

“We used the BryteFlow software to replicate our data from our SAP HANA application layer using CDS views and standard extractors to the AWS data lake,” said Tony Laroche, Business Intelligence Architect at ALDO Group. “The tool has simplified data engineering and is simple to use. It delivers our data to an AWS data lake with data organized and ready to use.”

Summary

This post explains how the BryteFlow SAP Data Lake Builder can replicate data from SAP applications to AWS, keeping business logic intact, and saving both effort and time.

The no-code tool can be used to integrate SAP data on the AWS data lake in real-time to modernize data and enable use cases like analytics, reporting, AI/ML, and IoT. BryteFlow SAP Data Lake Builder integrates completely with the AWS ecosystem and additional AWS services can be recruited with just a few clicks.

For more information on how the BryteFlow SAP Data Lake Builder can support real-time SAP data integration from multiple sources, get a free proof of concept (POC) with screen sharing and AWS best practice recommendations.

You can also try out BryteFlow SAP Builder in AWS Marketplace or contact Bryte Systems for a demo.

.

.

Bryte Systems – AWS Partner Spotlight

Bryte Systems is an AWS Partner with Competencies in Migration as well as Data and Analytics Migration. BryteFlow provides real-time, no-code replication to cloud and on-premises platforms.

Contact Bryte Systems | Partner Overview | AWS Marketplace