We use essential cookies and similar tools that are necessary to provide our site and services. We use performance cookies to collect anonymous statistics, so we can understand how customers use our site and make improvements. Essential cookies cannot be deactivated, but you can choose “Customize” or “Decline” to decline performance cookies.
If you agree, AWS and approved third parties will also use cookies to provide useful site features, remember your preferences, and display relevant content, including relevant advertising. To accept or decline all non-essential cookies, choose “Accept” or “Decline.” To make more detailed choices, choose “Customize.”
Customize cookie preferences
We use cookies and similar tools (collectively, "cookies") for the following purposes.
Essential
Essential cookies are necessary to provide our site and services and cannot be deactivated. They are usually set in response to your actions on the site, such as setting your privacy preferences, signing in, or filling in forms.
Performance
Performance cookies provide anonymous statistics about how customers navigate our site so we can improve site experience and performance. Approved third parties may perform analytics on our behalf, but they cannot use the data for their own purposes.
Allowed
Functional
Functional cookies help us provide useful site features, remember your preferences, and display relevant content. Approved third parties may set these cookies to provide certain site features. If you do not allow these cookies, then some or all of these services may not function properly.
Allowed
Advertising
Advertising cookies may be set through our site by us or our advertising partners and help us deliver relevant marketing content. If you do not allow these cookies, you will experience less relevant advertising.
Allowed
Blocking some types of cookies may impact your experience of our sites. You may review and change your choices at any time by selecting Cookie preferences in the footer of this site. We and selected third-parties use cookies or similar technologies as specified in the AWS Cookie Notice.
Your privacy choices
We display ads relevant to your interests on AWS sites and on other properties, including cross-context behavioral advertising. Cross-context behavioral advertising uses data from one site or app to advertise to you on a different company’s site or app.
To not allow AWS cross-context behavioral advertising based on cookies or similar technologies, select “Don't allow” and “Save privacy choices” below, or visit an AWS site with a legally-recognized decline signal enabled, such as the Global Privacy Control. If you delete your cookies or visit this site from a different browser or device, you will need to make your selection again. For more information about cookies and how we use them, please read our AWS Cookie Notice.
Amazon Data Firehose is the easiest way to load streaming data into data stores and analytics tools. Data Firehose is a fully managed service that makes it easy to capture, transform, and load massive volumes of streaming data from hundreds of thousands of sources into Amazon S3, Amazon Redshift, Amazon OpenSearch Service, Snowflake, Apache Iceberg tables, Amazon S3 Tables (preview), generic HTTP endpoints, and service providers like Datadog, New Relic, MongoDB, and Splunk, enabling real-time analytics and insights.
You can launch Amazon Data Firehose and create a delivery stream to load data into Amazon S3, Amazon Redshift, Amazon OpenSearch Service, Snowflake, Apache Iceberg tables, Amazon S3 Tables (preview), HTTP endpoints, Datadog, New Relic, MongoDB, or Splunk with just a few clicks in the AWS Management Console. You can send data to the delivery stream by calling the Firehose API, or running the Linux agent we provide on the data source. Data Firehose then continuously loads the data into the specified destinations.
Elastic scaling to handle varying data throughput
Once launched, your Firehose streams automatically scale up to handle gigabytes per second or more of input data rate, and maintain data latency at levels you specify for the stream, within the limits. No intervention or maintenance is needed.
Load new data in seconds
You can specify a batch size or batch interval to control how quickly data is uploaded to destinations. For example, you can set the batch interval anywhere from zero seconds to 15 minutes. Additionally, you can specify whether data should be compressed or not. The service supports common compression algorithms including GZip, Hadoop-Compatible Snappy, Zip, and Snappy. Batching and compressing data before uploading enables you to control how quickly you receive new data at the destinations.
Support for multiple data sources
Firehose reads data easily from 20+ data sources, including Amazon MSK and MSK Serverless clusters, Amazon Kinesis Data Streams, Databases (preview), Amazon CloudWatch Logs, Amazon SNS, AWS IoT Core, and more.
Apache Parquet or ORC format conversion
Firehose supports columnar data formats such as Apache Parquet and Apache ORC are optimized for cost-effective storage and analytics using services such as Amazon Athena, Amazon Redshift Spectrum, Amazon EMR, and other Hadoop based tools. Firehose can convert the format of incoming data from JSON to Parquet or ORC formats before storing the data in Amazon S3, so you can save storage and analytics costs.
Deliver partitioned data to S3
Dynamically partition your streaming data before delivery to S3 using static or dynamically defined keys like “customer_id” or “transaction_id”. Firehose groups data by these keys and delivers into key-unique S3 prefixes, making it easier for you to perform high performance, cost efficient analytics in S3 using Athena, EMR, and Redshift Spectrum.Learn more
Integrated data transformations
You can configure Amazon Data Firehose to prepare your streaming data before it is loaded to data stores. Simply select an AWS Lambda function from the Amazon Data Firehose stream configuration tab in the AWS Management console. Amazon Data Firehose will automatically apply that function to every input data record and load the transformed data to destinations. Amazon Data Firehose provides pre-built Lambda blueprints for converting common data sources such as Apache logs and system logs to JSON and CSV formats. You can use these pre-built blueprints without any change, or customize them further, or write your own custom functions. You can also configure Amazon Data Firehose to automatically retry failed jobs and back up the raw streaming data. Learn more
Support for multiple data destinations
Firehose reads data easily from 20+ data sources, including Amazon MSK and MSK Serverless clusters, Amazon Kinesis Data Streams, Amazon CloudWatch Logs, Amazon SNS, AWS IoT Core, and more.Amazon Data Firehose currently supports Amazon S3, Amazon Redshift, Amazon OpenSearch Service, Snowflake, Apache Iceberg tables, Amazon S3 Tables (preview), HTTP endpoints, Datadog, New Relic, MongoDB, and Splunk as destinations. You can specify the destination Amazon S3 bucket, the Amazon Redshift table, the Amazon OpenSearch Service domain, generic HTTP endpoints, or a service provider where the data should be loaded.
Optional automatic encryption
Amazon Data Firehose provides you the option to have your data automatically encrypted after it is uploaded to the destination. As part of the Firehose stream configuration, you can specify an AWS Key Management System (KMS) encryption key.
Metrics for monitoring performance
Amazon Data Firehose exposes several metrics through the console, as well as Amazon CloudWatch, including volume of data submitted, volume of data uploaded to destination, time from source to destination, the Firehose stream limits, throttled records number and upload success rate.
Pay-as-you-go pricing
With Amazon Data Firehose, you pay only for the volume of data you transmit through the service, and if applicable, for data format conversion. You also pay for Amazon VPC delivery and data transfer when applicable. There are no minimum fees or upfront commitments. You don’t need staff to operate, scale, and maintain infrastructure or custom applications to capture and load streaming data.