Magellan Rx Reduces ETL Time by Over 70% Using Amazon Redshift

Magellan Rx Management (Magellan Rx), a division of Magellan Health Inc., is a next-generation pharmacy organization that delivers meaningful solutions to the people it serves. Magellan Rx’s customers include health plans, employers, third-party administrators, union groups, and government agencies. To support its growing data needs and meet its service-level agreements, the organization needed to migrate from its on-premises data warehouse to a high-performing data hosting solution. 

In 2020, Magellan Rx migrated to Amazon Web Services (AWS). The organization now hosts its data using Amazon Redshift, a fully managed petabyte-scale data warehouse service in the cloud that enables users to query and combine exabytes of structured and semistructured data across data warehouses, operational databases, and data lakes using standard structured query language (SQL). Using AWS services, Magellan Rx reduced operational costs, shortened extract, transform, and load (ETL) times, and scaled operations.

Magellan Rx chose to migrate to AWS based on the seamless compatibility of AWS services and other cloud-based tools. Cloud-based architecture would also provide Magellan Rx with significant savings in licensing and support fees, help the organization scale to continue to meet and exceed demand, and create a more resilient data environment. Plus, Magellan Rx was an early adopter of the cloud, so its team of data engineers did not require extensive training to execute the migration. Magellan Rx began its migration to AWS in January 2020 and went live in December. 

On AWS, Magellan Rx can uphold the level of service its customers expect. The solution uses multitenant architecture and is completely configurable, enabling Magellan Rx to onboard new customer data in less than 1 hour. Amazon Simple Storage Service (Amazon S3)—an object storage service that offers industry-leading scalability, data availability, security, and performance—houses the data placed in its Amazon Redshift warehouse. Magellan Rx can preserve, retrieve, and restore every version of every object stored in its Amazon S3 buckets, fulfilling its disaster recovery requirements. “We have built a bedrock of modern computing,” says Vinesh Kolpe, Vice President of Information Technology at Magellan Rx. “We are not only exceeding our service-level agreements but also scaling without compromising them.”

Using AWS services, Magellan Rx created a centralized data solution and decreased the total cost of ownership of its database environments by 30 percent. Access to near-real-time data from transactional systems on the cloud created the foundation that the organization needed for quicker data discovery and advanced analytics. Magellan Rx can store its data in one location on Amazon Redshift, enabling sound data hygiene practices across the organization. As a result, its stakeholders can access relevant information with ease and are able to execute data-driven decisions to improve processes. “On the new architecture, we have greater visibility of raw and curated data,” says Srini Ponnada, Director of Information Technology at Magellan Rx. “With this system, we are able to cut down the load on our online transaction processing systems and in turn provide timely information to our users.” 

Because Amazon Redshift operates using standard SQL, Magellan Rx staff members do not need to learn new skills to query data. Amazon Route 53, a highly available and scalable cloud Domain Name System web service, helps route customers into the system for queries running ETL jobs. Using this service, Magellan Rx reduced its ETL time from 11 hours to 3 hours, or by over 70 percent. 

In addition to reducing on-premises licensing and support fees, Magellan Rx has seen cost savings in other areas of its business. “On Amazon Redshift, we reduced our operational costs by 20 percent,” says Kolpe. “That is a significant win from our previous stack.”

By migrating to AWS, Magellan Rx reduced its operational costs, shortened ETL and analytics query times, and gained the ability to onboard new customer data faster. Magellan Rx can now take advantage of AWS tools to implement machine learning and artificial intelligence into its pipelines.

In the future, Magellan Rx plans to continue building solutions using AWS tools and services, such as Amazon SageMaker, a service that developers can use to build, train, and deploy machine learning models for virtually any use case, and Amazon Redshift ML, which makes it simple for data analysts and database developers to create, train, and apply machine learning models using familiar SQL commands in Amazon Redshift data warehouses. Using these solutions will help Magellan Rx support its customers and expand its business intelligence capabilities. 

“This project laid the foundation for data strategy—a road map to the future—for Magellan Rx,” says Deepti Bhanti, Senior Director of Information Technology at Magellan Rx. “Accessing raw and near-real-time data will unlock the power of analytics and enable us to perform machine learning at scale.”