AWS Big Data Blog

Category: Announcements

Mutual TLS and SSL certificate authentication for Amazon MQ for RabbitMQ

Mutual TLS and SSL certificate authentication for Amazon MQ for RabbitMQ

Learn how to add certificate-based identity verification to Amazon MQ for RabbitMQ. This post explains SSL certificate authentication for passwordless login through the EXTERNAL SASL mechanism and mutual TLS (mTLS) for two-way certificate verification, highlights the key rabbitmq.conf settings, and helps you decide which approach fits your compliance requirements.

Amazon OpenSearch Service extends version lifecycle support timelines

Amazon OpenSearch Service extends version lifecycle support timelines

In November 2024, we announced Standard and Extended Support dates for legacy Elasticsearch and OpenSearch versions on Amazon OpenSearch Service. We are extending security and operating system patch coverage for these versions by 12 months, through November 7, 2027, and announcing support dates for additional Elasticsearch and OpenSearch versions.

Introducing Apache Spark troubleshooting agent for Amazon EMR on EKS

In this post, we show you how to set up the agent for Amazon EMR on EKS and walk through troubleshooting a failed job run. We demonstrate the workflow from both the Amazon EMR console and an AI assistant that supports the Model Context Protocol (MCP), an open standard for connecting AI assistants to external tools and data.

Deliver Apache Kafka data to streaming tables for Apache Iceberg with Amazon MSK Express brokers

Deliver Apache Kafka data to streaming tables for Apache Iceberg with Amazon MSK Express brokers

Announcing delivery to streaming tables on Apache Iceberg for Amazon MSK Express brokers, a fully managed capability that continuously materializes your Kafka streaming data as queryable Iceberg tables on Amazon S3 Tables. No connectors, Flink jobs, or custom consumers to manage, and no code to write.

Amazon EMR Serverless now supports 32 vCPU workers for the most demanding Spark jobs

Accelerate Spark on EMR Serverless with larger workers and shuffle-optimized disks

Amazon EMR Serverless now supports a 32 vCPU / 244 GB worker configuration for the most demanding Spark jobs. Across 126 TPC-DS and TPC-H queries, larger workers delivered an average 29% faster query execution and 29% lower cost, with the biggest gains on shuffle-heavy, multi-table join queries.

Zero Copy access to Apache Iceberg tables in Amazon S3 from Salesforce Data 360 using the Iceberg REST endpoint from AWS Glue Data Catalog

In this post, we demonstrate how AWS and Salesforce customers can access their enterprise data lakes on AWS from Salesforce Data 360 using zero-copy file federation.

Introducing Apache Spark Connect support in AWS Glue interactive sessions

Apache Spark Connect bridges the gap between these two worlds: you develop in local Python, but execute on AWS Glue against actual data. Today, AWS Glue interactive sessions support Spark Connect natively. You can connect from any environment that supports the PySpark remote() API, including VS Code, PyCharm, Amazon SageMaker Unified Studio notebooks, and standalone Python applications. You don’t need to install specialized kernels or manage cluster infrastructure.