AWS Compute Blog

Category: Analytics

Architecture overview

Deploying an EMR cluster on AWS Outposts to process data from an on-premises database

This post is written by Eder de Mattos, Sr. Cloud Security Consultant, AWS and Fernando Galves, Outpost Solutions Architect, AWS. In this post, you will learn how to deploy an Amazon EMR cluster on AWS Outposts and use it to process data from an on-premises database. Many organizations have regulatory, contractual, or corporate policy requirements […]

Architecture Diagram

Triggering AWS Lambda function from a cross-account Amazon Managed Streaming for Apache Kafka

This post is written by Subham Rakshit, Senior Specialist Solutions Architect, and Ismail Makhlouf, Senior Specialist Solutions Architect. Many organizations use a multi-account strategy for stream processing applications. This involves decomposing the overall architecture into a single producer account and many consumer accounts. Within AWS, in the producer account, you can use Amazon Managed Streaming for […]

Lambda runtimes

Managing AWS Lambda runtime upgrades

This post is written by Julian Wood, Principal Developer Advocate, and Dan Fox, Principal Specialist Serverless Solutions Architect. AWS Lambda supports multiple programming languages through the use of runtimes. A Lambda runtime provides a language-specific execution environment, which provides the OS, language support, and additional settings, such as environment variables and certificates that you can access from […]

Converting Apache Kafka events from Avro to JSON using EventBridge Pipes

This post is written by Pascal Vogel, Solutions Architect, and Philipp Klose, Global Solutions Architect. Event streaming with Apache Kafka has become an important element of modern data-oriented and event-driven architectures (EDAs), unlocking use cases such as real-time analytics of user behavior, anomaly and fraud detection, and Internet of Things event processing. Stream producers and consumers […]

Lambda updated initial scaling

Scaling improvements when processing Apache Kafka with AWS Lambda

AWS Lambda is improving the automatic scaling behavior when processing data from Apache Kafka event-sources. Lambda is increasing the default number of initial consumers, improving how quickly consumers scale up, and helping to ensure that consumers don’t scale down too quickly. There is no additional action that you must take, and there is no additional […]

A/B Configuration: Mix CPU architecture with ALB ingress connected to two separate CPU services

Mixing AWS Graviton with x86 CPUs to optimize cost and resiliency using Amazon EKS

This post is written by Yahav Biran, Principal SA, and Yuval Dovrat, Israel Head Compute SA. This post shows you how to integrate AWS Graviton-based Amazon EC2 instances into an existing Amazon Elastic Kubernetes Service (Amazon EKS) environment running on x86-based Amazon EC2 instances. Customers use mixed-CPU architectures to enable their application to utilize a wide selection […]

ICYMI2023Q1

Serverless ICYMI Q1 2023

February 12, 2024: Amazon Kinesis Data Firehose has been renamed to Amazon Data Firehose. Read the AWS What’s New post to learn more. Welcome to the 21st edition of the AWS Serverless ICYMI (in case you missed it) quarterly recap. Every quarter, we share all the most recent product launches, feature enhancements, blog posts, webinars, […]

Implementing architectural patterns with Amazon EventBridge Pipes

This post is written by Dominik Richter (Solutions Architect) Architectural patterns help you solve recurring challenges in software design. They are blueprints that have been used and tested many times. When you design distributed applications, enterprise integration patterns (EIP) help you integrate distributed components. For example, they describe how to integrate third-party services into your […]

Deploying Local Gateway Ingress Routing on AWS Outposts

This post is written by Leonardo Solano, Senior Hybrid Cloud Solution Architect and Chris Lunsford, Senior Specialist Solutions Architect, AWS Outposts. AWS Outposts lets customers use the same Amazon Virtual Private Cloud (VPC) security mechanisms, such as security groups and network access control lists, to control traffic flows for on-premises applications running on Outposts. Some […]

Reference architecture

Using custom consumer group ID support for AWS Lambda event sources for MSK and self-managed Kafka

This post shows how to use the new custom consumer group ID feature of the Lambda event source mapping for Amazon MSK and self-managed Kafka. This feature can be used to consume messages with Lambda starting at a specific timestamp or offset within a Kafka topic. It can also be used to consume messages from a consumer group that is replicated from another Kafka cluster using MirrorMaker v2.