Announcing General Availability of AWS Cost Anomaly Detection

We are excited to announce that AWS Cost Anomaly Detection is now generally available. This service joined many other AWS services that take care of the complex monitoring and reporting tasks, so you can focus on important business targets. As Werner Vogels, CTO of AWS, emphasized in his re:Invent keynote yesterday, there are three main pillars of monitoring: 1) metrics, 2) logging, and 3) tracking. AWS Cost Anomaly Detection uses a multi-layered machine learning model that learns your unique, historic spend patterns to detect one-time cost spike and/or continuous cost increases, without you having to define your thresholds. Every anomaly detected will be available in the detection history tab. We send you the anomaly detection report with root-cause analysis. And the service also comes at no cost to our customers.

Since its preview (read this blog post for guides and tips) back in September, we’ve worked with hundreds of customers to collect user feedback. Every one of you can help train our machine learning model by submitting assessments, as you evaluate the anomalies detected for your usage. In this blog post, Lutong Yang, our Sr. Product Manager, will provide the service overview and introduce new features available.

It is easy to set up the AWS Cost Anomaly Detection. The first step is to create a cost monitor. A cost monitor allows you to create spend segments.  According to your defined segments, AWS will evaluate spend anomalies in a preferred granular level. For example, if you want to evaluate your production accounts separately from your development accounts, you would choose a specific linked/member account as your cost monitor. There are four types of cost monitors you can choose from: 1) AWS services, 2) linked/member accounts, 3) cost allocation tags, and 4) Cost Categories. After selecting your cost monitors, the second step is to set alert subscriptions. You can customize the alert threshold, the alert frequency, and whom to notify. The threshold you set is to define your alerting preferences only, our machine learning models will automatically determine your anomaly threshold. To be notified, you have the option to choose email notification or SNS topic. After cost monitors and alert subscriptions are created, Cost Anomaly Detection will begin to work within 24 hours and you will be notified based on your alert subscription preference. You can see a list of every anomaly detected through the console or API. As you evaluate the anomalies detected, you can also submit an assessment. As mentioned, submitting assessments will train the machine learning models to tailor to your specific spend patterns.

“At Atlassian, our billing data is millions of data points across many AWS accounts and services, and cloud spend anomalies can often go undetected. AWS Cost Anomaly Detection has enabled the Atlassian Cloud FinOps team to be aware of the changes in costs which need our attention. By reducing the time between an unexpected change in cloud costs and the resolution we are able to keep our cloud spend on track.” Mike Fuller, Principal Engineer & Head of Cloud FinOps, Atlassian

We understand that each customer’s organization structure and business needs are different. Starting today, customers have the ability to create multiple alert subscriptions for the same cost monitor and/or attach multiple cost monitors to one alert subscription. To explain how to leverage this new feature, we will walk through two common scenarios below.

Scenario 1: One cost monitor with several alert subscriptions

If your organization has different departments and each department is interested in a different alert threshold and frequency, then you can create multiple alert subscriptions that listen to anomalies detected by the same monitor. For example: you may create a single service monitor to monitor all AWS Services. As anomalies are detected, you may want to create an ‘Immediate Alert’ subscription, so that your Engineering team is immediately notified of all anomalies. At the same time, you can create a separate subscription for your DevOps team to receive a ‘Daily Summary’ report each day for any anomaly that exceeds $100 in impact. Similarly, while listening to the same Cost Monitor, you can create a ‘Weekly Summary’ subscription for your Finance stakeholders so they are aware of any anomaly that exceeds a $1,000 impact. With this new feature, several of your stakeholders can now subscribe to the same cost monitor with their own alert preferences.

Scenario 2: Multiple cost monitors and one alert subscription

Imagine your organization has several business units and each business unit has their own cost monitors and alert subscriptions. If leadership is interested in a summary report across all business units and want to be alerted when the cost reaches a specific threshold, you can create a separate alert subscription for your leadership without needing to add them individually to each monitor. To achieve this, go to the Alert subscriptions tab and attach cost monitors to your new subscription.

Alternatively, you can attach multiple cost monitors to a single alert subscription during the ‘create cost monitor’ workflow. Clicking on ‘Choose an existing subscription’ will allow you to select a list of already created alert subscriptions. This means that if multiple monitors triggered anomalies in a week, and your subscription was for a ‘Weekly Summary’ frequency, then you will receive a single email with all anomalies across all monitors.

After you have created cost monitors and alert subscriptions, the next steps remain the same as before. Cost Anomaly Detection will start to notify you of anomalies based on the threshold and frequency that you have set up.  You can find every anomaly detected in the detection history tab. To help you pin-point potential cost drivers, Cost Anomaly Detection provides root causes analysis per detection. The root cause analysis includes account ID, the service that is responsible for the anomaly, severity, duration, etc. (see detailed definitions from this user guide).

Anomalies can be planned or unplanned, to improve upon future anomaly detection, you have the option to submit assessments per detection. The submission will further improve the machine learning models to customize to your business needs.

Conclusion

AWS Cost Anomaly Detection allows you to easily set up cost monitors and alert subscriptions to detect unusual spend across your organization. We will notify you when anomalies are detected and also provide insights into potential root causes. AWS Cost Anomaly Detection has made it easy to customize your cost monitors and alert subscriptions to meet the needs of your organization. Now with its general availability, we look forward to receiving your feedback and assessment, and improving our model to tailor to your unique needs.