Business Value is IT’s Primary Measure of Progress

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
—”The 12 Principles Behind the Agile Manifesto”

In the early days of my career, the success of my IT projects was measured by delivering on time, within budget, and with sufficiently good quality. “Sufficient” often meant that hundreds of errors, classified according to various criteria, were still open. I don’t even want to think about load and performance tests. Working software was the primary measure of progress. And that was difficult enough!

At that time, the basic assumption was that specifications would meet customer needs and create added value for the company. No one really asked about the actual outcomes or benefits for the company and its customers; if someone had asked me, I wouldn’t have been able to answer. We didn’t measure those values—we weren’t even capable of doing so.

That has changed. As an industry, IT solved the issue of working software at least a decade ago. And in many cases, companies across all industries— especially digital native businesses—are capable of measuring the outcomes and business values of their IT initiatives. They use this knowledge to strategically and tactically design their product roadmaps and deliver accordingly.

Measuring what value, if any, my organization’s releases created made a big difference for me. This ability helped us to specify requirements differently. In fact, we were no longer specifying but experimenting in a fast build-measure-learn cycle. This rapid learning led us to better satisfy customer needs. We were able to demonstrate that fundamental business metrics changed as a result of our activities.

Once we were demonstrably contributing to the company’s success, my organization was suddenly perceived as a profit center rather than a cost center—a profit center that delivered not just features but actual business value. So investments were made in my organization, and talent availability became my biggest problem. I had to continue optimizing costs, especially in operations, but funding a new product development team was no longer a big issue.

I had earned my seat at the table and was no longer perceived and treated like a (mediocre) supplier but as an important driver of business innovation.

The primary measure of progress for an IT initiative and organization is the business value it delivers.

How do you measure business value? With business key performance indicators (KPIs) supported by customer anecdotes—not with the number of features you release. But for many organizations, determining the value of their IT initiatives (or any initiative) is not easy.

How do you do it?

1. Join Forces between Business and IT

Many organizations perceive and treat IT as an external service provider, keeping it distant from business and customers. Sometimes IT organizations reinforce this perception by behaving like external service providers. This is often the case when an IT organization outsources a large portion of its applications and acts as an IT purchasing and coordination department.

If you want to change this misperception, introduce cross-functional teams and departments that contain all the roles needed to satisfy customer needs. Give these teams shared values, principles, and objectives. Finally, reconsider your outsourcing strategy and insource business-relevant applications determined by business value and frequency of change.

2. Start with the KPI in Mind

KPIs make customer behavior and feedback tangible. Whenever you want to build something for your customers, ask yourself:

• What does good look like?
• How do we know that it is really good?
• Which KPI needs to change in the right direction?

Define these before you even start thinking about a solution. Such KPIs could be the strategic north stars for all your organization’s efforts.

3. Democratize Access to Analytics

The backbone of this approach is making analytics capabilities available to everyone in your combined IT / business product development organization. Try to generate the right questions and answers from your current analytics infrastructure. Then use this experience to adapt the infrastructure in an evolutionary way. Only start a project that builds a new revolutionary infrastructure if you have fundamental structural problems.

Invest in your people too. Your team should have basic statistical knowledge that exceeds averages and sums, including percentiles, distribution, cohorts, and medians. The ability to read, analyze, and visualize data correctly is important, as is advanced knowledge of spreadsheets and available business intelligence tools. Organizations often purchase great tools but use just a fraction of their functions.

4. Make A/B Testing a Standard Procedure

It is really, really hard to know if a change, feature, or service makes a difference to your customers. Many things influence your KPIs: weather, season, assortment, etc. One of the best and most feasible approaches I have discovered in my career is A/B testing in all variants.

A/B testing is a methodology in which you present different versions of a piece of software to your users at the same time. Each user sees only one version of the feature during the experiment. This creates two groups you can compare well regarding the relevant key figures. For example, does a change in product search on an e-commerce site lead to more or fewer sales? A/B testing allows direct comparison between features. Invest in your A/B testing capabilities from technical, cultural, and process perspectives.

If you can measure the success of your IT projects and releases, the next step is to use small, isolated, and measurable experiments and releases to drive your roadmap prioritization. – Matthias