AWS Big Data Blog

Category: Advanced (300)

Querying raw log data with SQL and PPL with the optimized engine in Amazon OpenSearch Service

Querying raw log data using SQL and PPL with the optimized engine in Amazon OpenSearch Service

Learn how to run fast analytical queries directly against raw log and trace data in Amazon OpenSearch Service using PPL and SQL. Follow a single incident investigation, one query at a time, and see how the new optimized engine answers each question directly from raw spans.

How Zepto powers sub-second search using OpenSearch Service OR2 instances

How Zepto powers sub-second search using OpenSearch Service OR2 instances

Learn how Zepto, India’s fast-growing quick-commerce platform, migrated Amazon OpenSearch Service to OpenSearch Optimized (OR2) instances to scale sub-second product search across hundreds of delivery hubs, achieving over 100% higher indexing throughput and 30% cost savings while serving the same workload on two-thirds the data nodes.

IAM authentication with OAuth 2.0 for Amazon MQ for RabbitMQ

IAM authentication with OAuth 2.0 lets clients connect to Amazon MQ for RabbitMQ using their existing IAM identity instead of static broker-local credentials. This post covers the key rabbitmq.conf configuration for using AWS IAM as an OAuth 2.0 provider and shows a multi-tenant example with vhost isolation enforced by IAM roles and broker scope aliases.

Mutual TLS and SSL certificate authentication for Amazon MQ for RabbitMQ

Mutual TLS and SSL certificate authentication for Amazon MQ for RabbitMQ

Learn how to add certificate-based identity verification to Amazon MQ for RabbitMQ. This post explains SSL certificate authentication for passwordless login through the EXTERNAL SASL mechanism and mutual TLS (mTLS) for two-way certificate verification, highlights the key rabbitmq.conf settings, and helps you decide which approach fits your compliance requirements.

Trace cascading decision failures with a blame graph on Amazon OpenSearch

Trace cascading decision failures with a blame graph on Amazon OpenSearch Service

When multiple AI agents collaborate on a decision and get it wrong, standard logs can’t tell you which agent caused it. This post shows how to build a blame graph on Amazon OpenSearch Service and Amazon Bedrock that measures influence between agents and walks backward from a failed decision to find the root cause.

How AppFolio transformed its data streaming architecture with Amazon MSK Express brokers

How AppFolio transformed its data streaming architecture with Amazon MSK Express brokers

Learn how AppFolio transformed its data streaming architecture by adopting Amazon MSK Express brokers, replacing hours-long rebalances and manual storage planning with a platform that scales automatically across workload-isolated clusters.

Centralized CloudTrail monitoring across 100+ AWS accounts

Centralized CloudTrail monitoring across 100+ AWS accounts

Learn how to build a centralized AWS CloudTrail monitoring solution on Amazon OpenSearch Service, with Terraform managing the full stack. It handles 200 GB/day of logs across 100+ accounts, provides automated threat detection, delivers on-demand SOC 2, PCI DSS, and HIPAA compliance reporting, and gives four teams isolated access.

Scaling fine-grained access control for enterprise lakehouse using SageMaker Unified Studio and AWS Lake Formation

Scaling fine-grained access control for enterprise lakehouse using SageMaker Unified Studio and AWS Lake Formation

As enterprise lakehouses grow to thousands of tables across business domains and regions, fine-grained access control becomes a governance bottleneck. This post shows how to combine AWS IAM Identity Center, AWS Lake Formation tag-based access control, and trusted identity propagation in Amazon SageMaker Unified Studio for automated, auditable, least-privilege access.