AWS Big Data Blog
Category: Advanced (300)
Querying raw log data using SQL and PPL with the optimized engine in Amazon OpenSearch Service
Learn how to run fast analytical queries directly against raw log and trace data in Amazon OpenSearch Service using PPL and SQL. Follow a single incident investigation, one query at a time, and see how the new optimized engine answers each question directly from raw spans.
How Zepto powers sub-second search using OpenSearch Service OR2 instances
Learn how Zepto, India’s fast-growing quick-commerce platform, migrated Amazon OpenSearch Service to OpenSearch Optimized (OR2) instances to scale sub-second product search across hundreds of delivery hubs, achieving over 100% higher indexing throughput and 30% cost savings while serving the same workload on two-thirds the data nodes.
IAM authentication with OAuth 2.0 for Amazon MQ for RabbitMQ
IAM authentication with OAuth 2.0 lets clients connect to Amazon MQ for RabbitMQ using their existing IAM identity instead of static broker-local credentials. This post covers the key rabbitmq.conf configuration for using AWS IAM as an OAuth 2.0 provider and shows a multi-tenant example with vhost isolation enforced by IAM roles and broker scope aliases.
OAuth 2.0, LDAP, and HTTP auth for Amazon MQ for RabbitMQ
Amazon MQ for RabbitMQ supports OAuth 2.0, LDAP, and HTTP-based authentication backends so you can connect your broker to the identity infrastructure you already use. This post explains how each approach works and helps you decide which one fits your use case.
Mutual TLS and SSL certificate authentication for Amazon MQ for RabbitMQ
Learn how to add certificate-based identity verification to Amazon MQ for RabbitMQ. This post explains SSL certificate authentication for passwordless login through the EXTERNAL SASL mechanism and mutual TLS (mTLS) for two-way certificate verification, highlights the key rabbitmq.conf settings, and helps you decide which approach fits your compliance requirements.
Authentication and authorization options for Amazon MQ for RabbitMQ
Amazon MQ for RabbitMQ supports multiple authentication and authorization methods, so you can connect your broker to the identity infrastructure you already use. This post introduces the available options and helps you choose the right one for your use case.
Trace cascading decision failures with a blame graph on Amazon OpenSearch Service
When multiple AI agents collaborate on a decision and get it wrong, standard logs can’t tell you which agent caused it. This post shows how to build a blame graph on Amazon OpenSearch Service and Amazon Bedrock that measures influence between agents and walks backward from a failed decision to find the root cause.
How AppFolio transformed its data streaming architecture with Amazon MSK Express brokers
Learn how AppFolio transformed its data streaming architecture by adopting Amazon MSK Express brokers, replacing hours-long rebalances and manual storage planning with a platform that scales automatically across workload-isolated clusters.
Centralized CloudTrail monitoring across 100+ AWS accounts
Learn how to build a centralized AWS CloudTrail monitoring solution on Amazon OpenSearch Service, with Terraform managing the full stack. It handles 200 GB/day of logs across 100+ accounts, provides automated threat detection, delivers on-demand SOC 2, PCI DSS, and HIPAA compliance reporting, and gives four teams isolated access.
Scaling fine-grained access control for enterprise lakehouse using SageMaker Unified Studio and AWS Lake Formation
As enterprise lakehouses grow to thousands of tables across business domains and regions, fine-grained access control becomes a governance bottleneck. This post shows how to combine AWS IAM Identity Center, AWS Lake Formation tag-based access control, and trusted identity propagation in Amazon SageMaker Unified Studio for automated, auditable, least-privilege access.









