Getting started with Amazon Managed Service for Apache Flink

Why Managed Service for Apache Flink?

With Amazon Managed Service for Apache Flink, you can transform and analyze streaming data in real time using Apache Flink. Amazon Managed Service for Apache Flink simplifies building and managing Apache Flink workloads and allows you to integrate applications with other AWS services.

Build streaming applications with Apache Flink and Apache Beam

1

Step 1: Download the open source libraries into your favorite IDE

You can start by downloading the open source libraries including the AWS SDK, Apache Flink, and connectors for AWS services.

2

Step 2: Build a sample application using Apache Flink or Apache Beam

You write your Apache Flink application code using data streams and stream operators. Application data streams are the data structure you perform processing against using your application code. Data continuously flows from the sources into application data streams. One or more stream operators are used to define your processing on the application data streams.

3

Step 3: Upload your code to Amazon Managed Service for Apache Flink

Once built, upload your code to Amazon Managed Service for Apache Flink. The service takes care of everything required to continuously run your real-time applications, including automatically scaling to match the volume and throughput of your incoming data. 

Intuitively use Amazon Managed Service for Apache Flink Studio

1

Step 1: Create an Amazon Managed Service for Apache Flink Studio application

Start from the Amazon Managed Service for Apache Flink, Amazon MSK, or Amazon Kinesis Data Streams console. Use custom connectors to connect to any other data source.

2

Step 2: Write code in the serverless notebook in SQL, Python, and Scala and develop Apache Flink applications

Run individual paragraphs in the notebook, view results in context, and use Apache Zeppelin’s built-in visualization to accelerate development. You can also use user-defined functions in your code. 

3

Step 3: Build and deploy as an Amazon Managed Service for Apache Flink streaming application

Deploy your code as a continuously running stream processing application in just few steps. Your deployed application will be an Amazon Managed Service for Apache Flink application with durable state and auto scaling. You will also have the opportunity to change sources, destinations, logging, and monitoring levels before you productionize your code. 

E-Learning

In this workshop, you will build an end-to-end streaming architecture to ingest, analyze, and visualize streaming data in near real time. You’ll set out to optimize fleet operations of a taxi company in New York City by analyzing the telemetry data of the taxi fleet in near real time.

Learn more

Use a prebuilt solution to quickly get started. With AWS Streaming Data Solution for Amazon Managed Service for Apache Flink, you can solve for real-time streaming use cases such as capturing high-volume application logs, analyzing clickstream data, continuously delivering to a data lake, and more. 

Explore

In this workshop, you will create a stream processing application to identify customers entering your casino who are betting big. The application will send you an email when big spenders sit down at a gambling table. You will also identify tables that need a refill on chips, and identify potential cheaters in your casino through their betting and winning patterns. 

Learn more

In this five part learning series you will build a modern data streaming architecture for a real-time gaming leaderboard using Amazon Managed Service for Apache Flink Studio. This architecture includes data ingestion, real-time enrichment with database change data capture (CDC), data processing, as well as computing, storing and visualizing the results.

Watch series playlist