Author Note: In my previous post, I described the role of KPIs in business strategy. In this post, I’ll explore KPIs more deeply.
Looking closer, are there any specific KPIs that organizations should track when data, analytics, and AI are used to deliver upon the strategic plan? Perhaps there are.
The analyst firm Gartner recently published a research note titled “5 Data and Analytics KPIs Every Executive Should Track,” They include:
- The business impact of data, analytics, and AI
- Time from data creation to insight, and ultimately action
- Data quality
- Data literacy levels
- Data, analytics, and AI risks
The indicators will help you get more out of your AI programs and avoid wasting your data as you try to use it.
1. Track the business impact of data and analytics
This is the great, defining indicator because it answers the question, “How much has the needle moved because of our efforts in data and analytics?” Almost everybody in the organization wants to know whether the investment in data, analytics, and AI is paying off, and this is the shorthand answer.
But projects can make an impact without making the cash register ring right away. Think about a few different shades of business impact:
Indirect — You can’t directly link all of the decisions you’re basing on data analytics to bottom-line results, but you may still feel you’re moving in the right direction. Or maybe you haven’t yet discovered the right metrics to link your data-driven decisions to business outcomes; when you discover those metrics, the business impact won’t seem indirect anymore.
Leading — Some outcomes predict other outcomes. Suppose a low net promoter score among your employees two years ago convinced you to add headcount in Human Resources last year, and productivity is rising this year. A concrete outcome leading to another concrete outcome is a tangible business impact.
Anecdotal — You can’t be certain that working from home is the reason your employees are more productive, but many of them are telling you that it is the reason. It’s easy to measure time spent in the office before and during the pandemic. But with few recognized ways of measuring all the variables, you must rely on the anecdotal evidence that working from home is linked to the productivity boost.
Direct — How close to the scientific method — hypothesize, measure, change variables, measure, conclude — can you get? Online advertising, for example, has vast potential for A/B testing and refining with thousands or millions of data points.
Touched — When multiple teams, factors, and AI efforts are at work, it becomes difficult to attribute a given outcome to any single element. I like to think about this: How do I measure the value of electricity in my house? Without electricity, I may not see a wall in the dark, not be able to charge my devices, and not be able to binge my favorite shows. But quantifying the ROI of that electricity is quite difficult. Still, both the efforts and the impact are measurable, so, like electricity, you can say confidently that data, analytics, and AI have touched certain outcomes.
Given those different gradations, how do you report on the business impact of data, analytics, and AI? Consider this structure as an example:
Business impact
- Target total for this fiscal year: $25M vs. $2M actualized YTD
Progress
- Use case 1: 80% underway, on track
- Use case 2: 20% underway, 3 months’ delay
- Use case 3: 100% complete
2. Monitor time from insight to action
Over time, data can go stale for several reasons: competitors get the drop on you, customer tastes turn on a dime, and uncontrollable economic factors come into play.
That’s why the second indicator looks at how quickly your organization can derive insights from data and turn them into action. That agility — where preparation meets opportunity — is your competitive differentiator, whether your point of departure is a clear business question or an unguided hunt for insights hidden in your data.
Here’s guidance for tracking and reporting on the time that elapses between insight and action in your organization:
Time from insight to action
- Currently: 45 days
- Target for end of year: 7 days
Process improvement
- Speeding up approval process: 80% complete
- Improving data literacy: 55% staff trained
- Accelerating IT involvement: 10% complete
3. Measure data quality
“How do we know the data’s any good?”
That’s the logical question to ask anytime you arrive at a new insight through data, analytics, and AI, especially if the insight is unexpected. It can be only as valid as the data on which it is based.
The key to reporting this indicator is not to simply name the data source but to quantify the current trustworthiness of the data and to show the path to improvement:
Data quality
- Currently: 70% of sales forecast based on unverified data
- Target for EOY: 10%
Process improvement
- Remove manual reconciliation process (prospect data): 100% complete
- Train data entry associates (customer data): 25% complete
4. Track data literacy levels
When most companies hear “data-driven,” they usually think “technology.” But success in data, analytics, and AI depends more on skills and people than on technology.
That’s why it’s important to track and report on your company’s progress in data literacy across all functions of the enterprise. It’s no secret that professionally trained data scientists are in short supply, so smart companies instead upskill their current workforce to make better-informed decisions more quickly. Training in data literacy shows people how to retrieve, analyze, and derive insights from data, then communicate those insights and convert them to action. Data literacy extends from advanced data scientists to citizen data scientists, and it’s at the heart of digital transformation.
Consider this structure for your reporting of this indicator:
Data literacy rate
- Currently: 35% of staff have completed the program
- Target for EOY: 100%
Progress
- Uptake in self-service reporting and dashboards
- Staff survey results on the usefulness of their new skills
5. Quantify data, analytics, and AI risk
What’s the chance that one or more of your data, analytics, and AI projects will be misused? What’s your company’s exposure if your work becomes a mess, goes negative in value, and turns into waste?
Sensitive data usually rubs up against regulations and industry standards. If you’re maintaining a warehouse of customer data and somebody forgets to enable encryption, how much time, money, and effort could it cost you in remedies and reputation?
Here’s one perspective on quantifying your risk of exposure:
Data, analytics, and AI risk
- Currently: $17,000,000 data risk
- Target for EOY: $5,000,000
Progress
- Projects undergoing ethics review: 30%
- Projects undergoing privacy impact assessment: 80%
- Projects undergoing data protection impact assessment: 55%
Conclusion
Data, analytics, and AI are part of your competitive advantage, part of the promise you make to your customers and part of your perceived value to investors and prospects.
Smart leaders focus on managing their company’s data as a strategic resource. The indicators outlined above are designed to keep leaders’ fingers on the pulse of their data and keep it from ever getting messy.
You can use this self-assessment to evaluate your organization’s analytic maturity, receive a personalized report, and get use cases to help you start planning KPIs for your next project.