Are your KPIs and OKRs good, poor or ugly?
How would you know? What criteria would you use to make your decision? If you answered, “they would be SMART”, that is just not enough. There is no guarantee that just because it follows a simple framework, it is truly a good performance measure (aka KPIs, OKRs, indicators, metrics).
Let’s test your measurement chops with a Quick Quiz!
Here’s how it works:
- We provide five categories (one per quiz)
- Each category has 2-3 potential measurement examples. Once you have looked at them, decide if you think that the grouping of measures represent:
- A Good measure, as it gives us evidence on how we are improving.
- A Poor measure, but with a modification it could give us evidence.
- An Ugly measure, asit will never be able to give us the evidence we need.
Question #1: Are these potential measures good, poor, or ugly?
- Develop partnerships for the delivery of training with major universities.
- Implementation of a measurement system to improve fruit quality.
Answer: Ugly
Why? Because performance measures are not actions or initiatives! Any organization can get stuff done, but high performing organizations want to know if those actions had any impact on achieving their strategic priorities.
Question #2: Are these potential measures good, poor, or ugly?
- Win Banksia Award for environmental sustainability.
- Leverage the Women in Business Certification.
Answer: Ugly
Why? Awards, certifications, and events are not performance measures! Sure, enter competitions to boost workplace morale and achieve certifications for credibility, but awards and certifications are not real performance measures.
Question #3: Are these potential measures good, poor, or ugly?
- 85% of staff have been briefed on the marketing plan by the end of 2023.
- Delivery of all website upgrades completed each quarter.
- Implement Customer Relationship Management system (CRM) by June 22.
Answer: Ugly
Why? Milestones help you gauge progress of a project keeping to a schedule, but they don’t tell you anything about the performance or impact of that project. Milestones and activity completion are not performance measures and are more likely to be project management milestones.
*Note: You could make it into a better performance measure if, for example, an organization wants to measure how well they do project management overall. They could do this by looking at a collection of projects over time to see how well, on average, the organization is achieving the result of keeping projects on time and on budget.
Question #4: Are these potential measures good, poor, or ugly?
- Customer survey
- Occupational Health and Safety incident reports
Answer: Poor
Why? Sources of data and data collection tools are not performance measures.
However, if you became a little clearer on what question you were hoping to answer on your customer survey, it could get better.
Question #5: Are these potential measures good, poor, or ugly?
- Employee Engagement
- Environmental Footprint
- Website Visitation
Answer: Ugly
Why? Vague measure names are not performance measures. Watch out for measures that come to mind first and the group thinks are easy to get data for. There is a high likelihood that they have very low relevance to your original strategic goal. Language is critical so be careful of words that can mean different things to different people. If you asked your peers, “How many ways are there to measure how engaged an employee is?” you likely will hear a whole variety of ideas.
How do we know that these examples we just reviewed are not performance measures?
Because here is the definition of what a performance measure really is, courtesy of performance measure specialist Stacey Barr.
Use these criteria to explore if your performance measures are good, poor, or ugly!
What does it mean for a performance measure to be Quantified?
Quantification means your performance measure must be a statistic of some kind (a count, average, percentage, ratio, sum, etc.) and being quantitative means, you can compare your current performance against past performance and against desired future performance (also called targets).
What does it mean for a performance measure to be Objective?
Objective means your performance measure allows everyone to draw the same conclusion when they use it. When it is objective, the measure has a consistent calculation (rather than a bunch of different versions) using the same data, and that data is representative of the goal or result you are measuring. Objective also means that is it not influenced by personal feelings or opinions when representing the facts.
What does it mean for a performance measure to be Evidence?
Evidence means your performance measure is based on directly observable facts about the performance result you are trying to achieve (not hearsay!). It cannot be based on data that is for a different result. For example, Net Profit is not evidence of Customer Satisfaction, even though they’re related. Evidence is directly observable in real life, with our physical senses (meaning we could see, hear, or touch it in real life).
What does it mean for a performance measure to have Degree?
Degree means your performance measure is not simply counting your Yes or No responses. Rather, it reveals how much something is changing on a continuous scale, not just whether it happened or not. It must be sensitive enough to pick up smaller changes that can provide the feedback you need as you execute actions over time. For example, if you measure average stakeholder satisfaction on a scale of 1-10, you can detect smaller changes than if you measure with a percentage. It lets us know HOW satisfied the stakeholder is, not just IF they were satisfied.
What is a Performance Result?
A Performance Result describes what your team wants to achieve; the impact you want from your actions, not the action itself. A result is “end-in-mind” thinking. Your team won’t know what activities to adjust or change if you only measure completion and milestones. Measuring the result (even if your team can only influence it and not control it 100%), is still more valuable than just counting stuff you do.
What does it mean for a performance measure to be Over Time?
Over Time means that everything in our organizations happen through time: our business planning, target setting, decision-making and action implementation. Unless we measure PROGRESS toward our desired result, through time, we don’t know whether we are heading in the right direction until it is too late. Great measures give us warning signals. You don’t want to have to wait until the end of the year to find out if your change strategies have the desired impact. We want to be able to test if our projects are working or not.
Question #6: Are these potential measures Good, Poor, or Ugly?
- Overall Customer Satisfaction = a quarterly average satisfaction rating provided by active customers about their overall experience with us.
- Net Promoter Score = the percentage of Promoters (who rate 9-10) minus the percentage of Detractors (who rate 1-6) for how likely they are to recommend us to a friend or colleague, calculated monthly from active customers.
Answer: Good
Why? Because they both match the criteria of what a performance measure really is.
So why are so many of our performance measures more often Poor and Ugly than they are Good?
- Could it be that to develop performance measures that satisfy this definition, we need to be rigorous and deliberate, and we just don’t know how?
- Could it be that we want short cuts and silver bullets like buying software, searching up measures on the internet, or hiring consultants?
- Could it be that we are stuck in old ways that are just not serving our organizations, but are fearful of change?
Or maybe, you feel super excited to change how and what you measure!
Click here to download the Whitepaper Measure what Matters and discover more about the 8-step performance measure process called The PuMP® Blueprint.
The PuMP® Performance Measure Blueprint was created by Australia’s performance measure specialist Stacey Barr. Louise Watson of Adura Strategy is North American Official Partner and Licensed PuMP® Blueprint consultant.
