AWS Cloud Operations Blog

KPIs – Enterprise Journey from Technology to Business

As discussed in this blog post, AWS sees organizations with well-defined, tracked and aligned business key performance indicators (KPIs) thrive in their cloud transformation journey. However, it is a challenge to define and track these KPIs. Even when organizations align to track outcomes and there is value in doing so, some encounter difficulties focusing on those KPIs from the start.

In this blog post, we will walk through a fictional company’s (AnyCompany) KPI adoption journey, based on collective real-world experience working with multiple customers. AnyCompany, a large enterprise organization, aimed to boost operational profits and revenue. They started by eliminating on-premises data centers and moving their applications to the cloud to achieve cost optimization, reduce total cost of ownership, retire technical debt and build a foundation for future capabilities. The future state they adopted drove their business’s growth and transformation.

Tracking the effectiveness of cloud adoption using business KPIs

Understanding AnyCompany’s business goal and their current culture, processes, and governance, we recommended a crawl, walk and run approach to drive the adoption of business KPIs. It starts with technology KPIs in the first phase (crawl), introducing business KPIs in the next phase (walk) to overcome initial friction and gradually shifting to all business KPIs in the last phase (run). The focus in the crawl phase is to mitigate technical debt by moving workloads to the cloud. Tactical KPIs ensure effective tracking and progress reporting of the re-platform and re-architecture of workloads. The walk phase ties cloud adoption technology and cost savings outcomes to the business outcomes of the business units. In the run phase, after an organization has achieved success through internal marketing and positive feedback from internal business application development teams, external client facing teams get motivated with the prospect of achieving similar outcomes for their clients. This creates a positive flywheel effect that powers the overall transformation of the organization from a people, process, technology, and business perspective.

Crawl phase – Track and measure technical KPIs

In this phase, the primary focus is on technology – reduce technical debt and increase cloud adoption. KPIs in this phase focus on measurement of benefits such as cost reduction, number of trained staff, number of applications migrated or operational excellence, as shown in the following table (figure 1). Organizations focus on internal KPIs in this stage and readily communicate progress to internal users outside of IT. This creates the muscle memory of measuring, tracking, and sharing KPIs. KPIs are effective when the time frame to achieve completion spans multiple quarters or years. While AnyCompany required this step because of their siloed structure, other organizations may move quickly to the next stage and don’t require the crawl stage to build momentum.

Benefits Objectives Measurments Impacts
Cost Reduction Eliminate presence in hosted data center Servers decommissioned from Managed Service Provider (MSP) datacenter

Detailed view of IT spend, allows targeted optimization efforts

More accurate forecasting for infrastructure spend

Cost savings on-premises vs. cloud (business case)

Servers on-premises vs. cloud
Cost utilization on-premises vs. cloud
YoY cost comparison
Reduce Downtime Simplify application footprint by retiring applications not in use % of applications retired

Proactive resolution before technical issues impact customer

No performance degradation during peak application use (autoscaling)

Decreased application outages

Eliminate technical debt % of applications on supported OS
% of applications on supported DB
Improve operational metrics Baseline MTTD on-premises vs. cloud
Improve performance for platform and rehost Baseline application performance at peak load on-premises vs. cloud
Decrease number of unplanned outages Baseline number of outages on-premises vs. cloud

AnyCompany retired approximately 25% of their on-premises applications and 50% of costly legacy databases and servers by using the KPIs in the preceding table (figure 1) during the crawl phase. In addition, they closed multiple data centers, improved security and observability, and created a modern application development platform.

Walk phase – Gradually introduce business KPIs

In this stage, an organization links business outcomes to technology drivers. It is a gradual shift as you expand on the KPIs established in the crawl phase. Note that the goal is not to complete all KPIs during the crawl phase, but to focus on the teams’ utilization of metrics and their ability to transition to a state of continual improvement and transparency with stakeholders. As shown in the following table (figure 2), you are taking the initial set of KPIs and expanding on them, such as adding automated environment provisioning. This frees developers from managing infrastructure and reduces failed deployments, both of which allow them to focus on feature enhancements and more valuable tasks. Here you want to start to measure business impact as a secondary goal. Look for opportunities to introduce these KPIs to teams as the initial ones have a clear path to completion and have teams use these new KPIs in their decision-making process.

AnyCompany was now well into their cloud migration and looking beyond the migration as part of this phase. They added business outcomes to the technology KPIs that were already being measured as part of their cloud migration journey. By continuing communication with the entire company, corporate stakeholders have made a connection of how their business is being positively affected by the migration. Figure 2 highlights the business impact of the technology KPIs being tracked.

Benefits Objectives Measurments Impacts
Reduce downtime Increase architecture delivered as code Automated environment provisioning

More developer time spent working on feature enhancements

Fewer failed deployments – Reduction in repeating work

Faster response time to reported issues

Deployment Efficiency Automated application deployment
Deployment success rate
Streamline deployment governance Deployment governance process evaluation
% of test automation
Deployment time on-premises vs. cloud vs. cloud native platform
Cost Reduction Modernize applications ready to replatform or refactor by onboarding to cloud native platform % applications on cloud native platform

Increased response to reported defects in production

Decreased number of defects in production

Increased delivery of product enhancements to customers

Eliminate downtime during deployments

Improved application security (automated security scans)

% AWS managed databases
Feature cycle time (how long does it take to push a feature request to production)
# deployments YoY
# defects reported in production

Run phase – Track, measure, evolve business KPIs over time

In this stage, the long-term business benefits are being realized and the first part of the migration is coming to a close. The organization is mature in the cloud at this point. Early tactical measures are being replaced with KPIs that have a direct impact on the organization’s business, as shown in the following table (figure 3). This includes top line growth, operational excellence and efficiency, increased operating profit, increased market share and new revenue segments. The organization understands the connection between outputs, such as feature releases and outcomes that drive the business. It also measures outcomes while looking at what capabilities need to be added to deliver those.

Benefits Objectives Measurments Impacts
Efficiency Developer non-billable hours reduced Demonstrations and proof of concepts are now done with reduced effort, tracking pre-sales hours Employee productivity efficiency improved
Reusable assets, such as AWS CloudFormation templates, that will get leveraged across teams for common workload stacks, having teams focus on value-added vs undifferentiated work Assets tagged and access tracked
Agility Ability to blend junior and senior level engineers on projects, providing stronger guardrails and paved paths. They can now ramp less experienced staff more quickly Introduce new tools such as Amazon Code Whisperer and measure output Faster onboarding process for new employees
Innovation Able to add new capabilities to their client offerings based on cloud native solutions (globalization, resiliency, and sustainability) Feedback scores of new features by customers New revenue streams

At this stage, AnyCompany achieved a level of maturity with Cloud transformation from a technology perspective and is now focused on linking and measuring business KPIs that are connected to the work they are completing. Mechanisms and governance are in place to ensure that technical debt does not go unchecked while they focus on functionality and features that allow the business to transform. This phase is an ongoing journey and the connection between business and IT strategy will help AnyCompany position itself to be a market leader and improve collaboration.

Conclusion

This blog post has shown the approach taken by AnyCompany from the start of their transformation journey. Regardless of your current situation, you can follow these steps to maintain or regain momentum and achieve consistent progress. We discussed a phased approach to build mechanisms to measure and share technical KPIs that can be gradually replaced with business KPIs with the goal of enhancing KPIs continuously.

About the authors:

Lijan Kuniyil

Lijan Kuniyil is a Senior Technical Account Manager at AWS. Lijan enjoys helping AWS enterprise customers build highly reliable and cost-effective systems with operational excellence. Lijan has more than 25 years of experience in developing solutions for financial and consulting companies.

Bryan Long

Bryan Long is part of the Enterprise Technologist team composed of former leaders and customers who have hands-on experience leading complex business transformations utilizing AWS. He works with key accounts to help customers with their journey by leaning in and understanding the strategic goals and current challenges as well as potential road blocks that might occur and how to mitigate them.

Eric Macrina

Eric Macrina is a Principal Sales Representative at AWS based out of Boston, MA. He currently works with Enterprise Northeast for the Business Services Advisory Consulting team. He’s responsible for building strategic relationships and generate business value within the accounts and implementing a broad strategy for earning customer acceptance and service implementation.

Prathima Prasad

Prathima Prasad is a Sr TAM at AWS working with Cross Industry Enterprise and Energy customers, she is passionate about helping customers in their well architected journey focusing on operational excellence, cost and sustainability.

Ramesh Dwarakanath

Ramesh Dwarakanath is a Principal Solutions Architect at AWS based out of Boston, MA. He works with Enterprises in the Northeast area on their Cloud journey. His areas of interest are Containers and DevOps. In his spare time, Ramesh enjoys tennis, racquetball.