Amazon MSK enhances cross-cluster replication with support for identical topic names

Posted on: Sep 10, 2024

Amazon MSK Replicator now supports a new configuration that enables you to preserve original Kafka topic names while replicating streaming data across Amazon Managed Streaming for Apache Kafka (Amazon MSK) clusters. Amazon MSK Replicator is a feature of Amazon MSK that lets you reliably replicate data across MSK clusters in the same or different AWS region(s) with just a few clicks. The new configuration reduces the need for you to reconfigure client applications during setup and makes it even more simple to operate multi-cluster streaming architectures, while continuing to benefit from MSK Replicator’s reliability.

With Amazon MSK Replicator, you can easily build regionally resilient streaming applications for business continuity, share data with partners, aggregate data from multiple clusters for analytics, and serve clients globally with lower latency. With the new configuration, you can retain topic names during replication while automatically avoiding the risk of infinite replication loops that comes with using third-party or open-source tools for replication. If you setup active-passive cluster architecture to build regionally resilient streaming applications, where one cluster handles live traffic while another acts as a standby, the new configuration also streamlines the failover process. Applications can seamlessly failover to the standby cluster without requiring reconfiguration, as topic names remain intact.

Support for the new configuration is available in all regions where Amazon MSK Replicator is available. To see all the regions where Amazon MSK Replicator is available, see the AWS Region table. To learn more, visit our developer guide or product page.