AWS Cloud Operations Blog
KPIs – Enterprise Journey from Technology to Business
As discussed in this blog post, AWS sees organizations with well-defined, tracked and aligned business key performance indicators (KPIs) thrive in their cloud transformation journey. However, it is a challenge to define and track these KPIs. Even when organizations align to track outcomes and there is value in doing so, some encounter difficulties focusing on those KPIs from the start.
In this blog post, we will walk through a fictional company’s (AnyCompany) KPI adoption journey, based on collective real-world experience working with multiple customers. AnyCompany, a large enterprise organization, aimed to boost operational profits and revenue. They started by eliminating on-premises data centers and moving their applications to the cloud to achieve cost optimization, reduce total cost of ownership, retire technical debt and build a foundation for future capabilities. The future state they adopted drove their business’s growth and transformation.
Tracking the effectiveness of cloud adoption using business KPIs
Understanding AnyCompany’s business goal and their current culture, processes, and governance, we recommended a crawl, walk and run approach to drive the adoption of business KPIs. It starts with technology KPIs in the first phase (crawl), introducing business KPIs in the next phase (walk) to overcome initial friction and gradually shifting to all business KPIs in the last phase (run). The focus in the crawl phase is to mitigate technical debt by moving workloads to the cloud. Tactical KPIs ensure effective tracking and progress reporting of the re-platform and re-architecture of workloads. The walk phase ties cloud adoption technology and cost savings outcomes to the business outcomes of the business units. In the run phase, after an organization has achieved success through internal marketing and positive feedback from internal business application development teams, external client facing teams get motivated with the prospect of achieving similar outcomes for their clients. This creates a positive flywheel effect that powers the overall transformation of the organization from a people, process, technology, and business perspective.
Crawl phase – Track and measure technical KPIs
In this phase, the primary focus is on technology – reduce technical debt and increase cloud adoption. KPIs in this phase focus on measurement of benefits such as cost reduction, number of trained staff, number of applications migrated or operational excellence, as shown in the following table (figure 1). Organizations focus on internal KPIs in this stage and readily communicate progress to internal users outside of IT. This creates the muscle memory of measuring, tracking, and sharing KPIs. KPIs are effective when the time frame to achieve completion spans multiple quarters or years. While AnyCompany required this step because of their siloed structure, other organizations may move quickly to the next stage and don’t require the crawl stage to build momentum.
| Benefits | Objectives | Measurments | Impacts |
|---|---|---|---|
| Cost Reduction | Eliminate presence in hosted data center | Servers decommissioned from Managed Service Provider (MSP) datacenter | Detailed view of IT spend, allows targeted optimization efforts More accurate forecasting for infrastructure spend Cost savings on-premises vs. cloud (business case) |
| Servers on-premises vs. cloud | |||
| Cost utilization on-premises vs. cloud | |||
| YoY cost comparison | |||
| Reduce Downtime | Simplify application footprint by retiring applications not in use | % of applications retired | Proactive resolution before technical issues impact customer No performance degradation during peak application use (autoscaling) Decreased application outages |
| Eliminate technical debt | % of applications on supported OS | ||
| % of applications on supported DB | |||
| Improve operational metrics | Baseline MTTD on-premises vs. cloud | ||
| Improve performance for platform and rehost | Baseline application performance at peak load on-premises vs. cloud | ||
| Decrease number of unplanned outages | Baseline number of outages on-premises vs. cloud |
AnyCompany retired approximately 25% of their on-premises applications and 50% of costly legacy databases and servers by using the KPIs in the preceding table (figure 1) during the crawl phase. In addition, they closed multiple data centers, improved security and observability, and created a modern application development platform.
Walk phase – Gradually introduce business KPIs
In this stage, an organization links business outcomes to technology drivers. It is a gradual shift as you expand on the KPIs established in the crawl phase. Note that the goal is not to complete all KPIs during the crawl phase, but to focus on the teams’ utilization of metrics and their ability to transition to a state of continual improvement and transparency with stakeholders. As shown in the following table (figure 2), you are taking the initial set of KPIs and expanding on them, such as adding automated environment provisioning. This frees developers from managing infrastructure and reduces failed deployments, both of which allow them to focus on feature enhancements and more valuable tasks. Here you want to start to measure business impact as a secondary goal. Look for opportunities to introduce these KPIs to teams as the initial ones have a clear path to completion and have teams use these new KPIs in their decision-making process.
AnyCompany was now well into their cloud migration and looking beyond the migration as part of this phase. They added business outcomes to the technology KPIs that were already being measured as part of their cloud migration journey. By continuing communication with the entire company, corporate stakeholders have made a connection of how their business is being positively affected by the migration. Figure 2 highlights the business impact of the technology KPIs being tracked.
| Benefits | Objectives | Measurments | Impacts |
|---|---|---|---|
| Reduce downtime | Increase architecture delivered as code | Automated environment provisioning | More developer time spent working on feature enhancements Fewer failed deployments – Reduction in repeating work Faster response time to reported issues |
| Deployment Efficiency | Automated application deployment | ||
| Deployment success rate | |||
| Streamline deployment governance | Deployment governance process evaluation | ||
| % of test automation | |||
| Deployment time on-premises vs. cloud vs. cloud native platform | |||
| Cost Reduction | Modernize applications ready to replatform or refactor by onboarding to cloud native platform | % applications on cloud native platform | Increased response to reported defects in production Decreased number of defects in production Increased delivery of product enhancements to customers Eliminate downtime during deployments Improved application security (automated security scans) |
| % AWS managed databases | |||
| Feature cycle time (how long does it take to push a feature request to production) | |||
| # deployments YoY | |||
| # defects reported in production |
Run phase – Track, measure, evolve business KPIs over time
In this stage, the long-term business benefits are being realized and the first part of the migration is coming to a close. The organization is mature in the cloud at this point. Early tactical measures are being replaced with KPIs that have a direct impact on the organization’s business, as shown in the following table (figure 3). This includes top line growth, operational excellence and efficiency, increased operating profit, increased market share and new revenue segments. The organization understands the connection between outputs, such as feature releases and outcomes that drive the business. It also measures outcomes while looking at what capabilities need to be added to deliver those.
| Benefits | Objectives | Measurments | Impacts |
|---|---|---|---|
| Efficiency | Developer non-billable hours reduced | Demonstrations and proof of concepts are now done with reduced effort, tracking pre-sales hours | Employee productivity efficiency improved |
| Reusable assets, such as AWS CloudFormation templates, that will get leveraged across teams for common workload stacks, having teams focus on value-added vs undifferentiated work | Assets tagged and access tracked | ||
| Agility | Ability to blend junior and senior level engineers on projects, providing stronger guardrails and paved paths. They can now ramp less experienced staff more quickly | Introduce new tools such as Amazon Code Whisperer and measure output | Faster onboarding process for new employees |
| Innovation | Able to add new capabilities to their client offerings based on cloud native solutions (globalization, resiliency, and sustainability) | Feedback scores of new features by customers | New revenue streams |
At this stage, AnyCompany achieved a level of maturity with Cloud transformation from a technology perspective and is now focused on linking and measuring business KPIs that are connected to the work they are completing. Mechanisms and governance are in place to ensure that technical debt does not go unchecked while they focus on functionality and features that allow the business to transform. This phase is an ongoing journey and the connection between business and IT strategy will help AnyCompany position itself to be a market leader and improve collaboration.
Conclusion
This blog post has shown the approach taken by AnyCompany from the start of their transformation journey. Regardless of your current situation, you can follow these steps to maintain or regain momentum and achieve consistent progress. We discussed a phased approach to build mechanisms to measure and share technical KPIs that can be gradually replaced with business KPIs with the goal of enhancing KPIs continuously.
About the authors: