Generate your cost optimization reports with Data Exports for Cost Optimization Hub

We’re excited to announce that starting today AWS Billing and Cost Management provides Data Exports for Cost Optimization Hub. With Data Exports for Cost Optimization Hub, you can deliver consolidated cost optimization recommendations in CSV or parquet format to Amazon S3 on a recurring basis. You can define what data you want to include in the export using basic SQL query or column/row filters in the console. You can share exports with stakeholders, load them into your BI dashboards, or add to your Cloud Financial Management data lake.

Double the power with Cost Optimization Hub and Data Exports

Since we launched Cost Optimization Hub at 2023 re:Invent, many of you have started using this free feature to review cost saving opportunities for your organization, such as Amazon EC2 instance rightsizing, AWS Graviton migration, and spend and usage commitments (e.g. Reserved Instances, Savings Plans). The dashboard experience gives you a quick overview of all the recommendations with estimated savings without duplication. You can zoom into any opportunities by filtering and grouping with fields such as account and region, and prioritize them by savings and implementation effort.

Around the same time, we made Data Exports available for you in AWS Billing and Cost Management. Data Exports gives you the ability to customize your export of billing and cost management data by selecting only the data you want to process, removing any sensitive cost information, and controlling the output schema of your exports.

Now, Data Exports for Cost Optimization Hub enables you to configure exports of your cost optimization recommendations to Amazon S3 with the data filters and savings deduplication preference applied. You can always get the most up-to-date cost saving recommendations on a daily basis. With Data Exports for Cost Optimization Hub, you’ll receive your recommendations in easy-to-ingest data files, instead of by paginated APIs, which simplifies creating reports or dashboards.

“Cost optimization is a strategic priority for our customers and the Cost Optimization Hub has made it easier for customers to access all optimization recommendations that AWS provides. Data Exports for Cost Optimization Hub makes it easier for customers to systematically receive and integrate optimization datasets into your data pipelines and tooling. This way, you can act upon these recommendations and maximize the amount of value they receive from our all of optimization offerings.”

John Phillips, General Manager of Cloud Financial Management Engineering and Product Management, AWS

How can you access Data Exports for Cost Optimization Hub?

If you haven’t used Cost Optimization Hub in the past, you need to first opt-in via the console or API. You will also need to enable AWS Compute Optimizer to receive rightsizing and idle recommendations. You can read detailed instructions in the user guide “Getting-started with Cost Optimization Hub“.

You can create exports for Cost Optimization Hub by going to the Billing and Cost Management Console and clicking on “Data Exports” or “Cost Optimization Hub” on the left-side navigation bar. You can set up the export by clicking on “Create export” from either of the console pages and using the export creation workflow (see instructions in this user guide “API reference_CreateExport”).

Option 1: To create an export via the Data Exports console, click the “Create” button on the “Data Exports” page and select the “Cost optimization recommendations” table. You will need to specify the export name and S3 delivery location. You can optionally select columns and add row filters for the content you want to include in the export. If you want to deduplicate cost savings and only export recommendations that lead to the highest savings per resource, you can leave “include all recommendations” unchecked.

Figure 1. Sample screenshot of creating an export for Cost Optimization Hub in the Data Exports console

Option 2: To create an export via the Cost Optimization Hub console, click on the “Create recurring export to Amazon S3” button at the top right of the page. You will be brought to the create export page of the Data Exports console to complete the export. The data table content settings and filters preference you’ve selected in Data Exports will be carried over to the export creation page.

Figure 2. Sample screenshot of creating an export in Cost Optimization Hub console

Once the export is created, Data Exports starts to deliver data according to the specified frequency. If this is your first time creating a data export, you must create a service-linked role to give Data Exports access to the Cost Optimization Hub data (see more details in this user guide “Creating data exports”).

Anything you should be aware of?

Cost Optimization Hub consolidates recommendations from multiple sources and carries over the preferences you set in these sources. For example, any preferences you set on recommendations in Compute Optimizer will be reflected in the recommendation results and savings amount in the Cost Optimization Hub.

To avoid double counting the potential savings for your cost savings recommendations, you can leave the “Include all recommendations” box unchecked in the Data Exports console. You can also do this by specifying “FALSE” for the “INCLUDE_ALL_RECOMMENDATIONS” table preference when using the CLI/SDK. You should still be aware that if you take a recommendation provided by Cost Optimization Hub, you may affect the potential savings of related recommendations. For example, if you take a recommendation to rightsize an EC2 instance, the potential savings for a Savings Plan covering that instance would decrease.

You can also manage your discount visibility by using Cost Optimization Hub preferences: member-account-discount-visibility, and savings-estimation-mode. Cost saving estimates will be populated according to your preference in the “*_after_discount” and “*_before_discount” columns.

You can create up to 2 exports for Cost Optimization Hub. Your first export will take up to 24 hours to be delivered. Subsequent export refreshes will occur once a day. After you change a preference in Cost Optimization Hub, the new preference will take effect in your next export as long as the preference was changed more than 10 minutes before the next export starts.

Conclusion

Data Exports for Cost Optimization Hub gives you the ability to create exports of your cost saving recommendations on the filtered views that you create in the Cost Optimization Hub or Data Exports. Get your most updated recommendations today and share it with your team to act on these opportunities sooner.