Donald Wheeler, author, statistician, and quality control expert (and the inspiration and source of today’s article), tells a story of a president of a shoe company who had a chart on display, similar to the one below. One day, one of his managers asked why he had this chart displayed so prominently. The president replied, “I want to see how the plant is doing”. The manager then asked “Well, tell me, how we are doing?”. The president paused, as if no one had asked him before, and then said, “Well, some days are better than others”.
What’s revealed here is that, even though the president had data in a suitable format, he had no way to analyze the values and interpret them for decision-making about how to improve performance. It is incredibly common that many management reports are full of limited comparisons, using only two points of data for their analysis; but this analysis approach is full of dangerous distortions with potentially serious consequences.
Dr. Walter Shewhart, a pioneering physicist, engineer and statistician, called the father of quality control in his day, established rules to help leaders more effectively understand and communicate data. These rules can be paraphrased into two over-riding truths.
Truth #1 for understanding and communicating data
Data doesn’t have meaning without context
The context of the performance measures’ data should be completely and fully described, including details on who collected the data, how the data was collected, what the values represent, how the values are computed, and whether the formula ever changed over time?
If leaders want to be able to use their data to make decisions, heed this advice from Dr. Wheeler:
- Trust no one who cannot, or will not, provide the context of their figures.
- Stop reporting comparisons between pairs of values (two points of data, such as this month compared to same time last month) except when you can also show it as part of a broader comparison of data points.
- Start using graphs which allow you to present current values in context. Read more about this kind of chart.
Truth #2 for understanding and communicating data
While every data set contains noise, some data sets may contain signals of change
Before we can use data from performance measures to justify any action during strategy execution, we must be able to detect a signal of change within our data. Otherwise, we are likely to make mistakes within our analysis.
This distinction between signals (patterns within our data that reveal a change in performance has occurred) and noise (the naturally occurring variation within performance) is the foundation for every meaningful analysis of data and defines two mistakes which can be made when we analyze data.
- Mistake 1: interpreting noise in our data (naturally occurring or routine variation) as a meaningful departure from past performance. We want to avoid this mistake as it could lead us to take actions which are, at best, inappropriate, or, at worst, completely contrary to the proper course of action. This mistake leads to waste and loss in organizations every day.
- Mistake 2: failing to recognize when a change has occurred in our performance and failing to celebrate success and achievement within our organizations. Successfully being able to detect a signal of change in your data is critical to driving effective strategy execution across departments and teams.
Dr. Wheeler’s key reminders when we want more value from our data:
- The purpose of analysis is insight
- The best analysis is the simplest and most accurate analysis which gives the needed insight
- For more meaningful analysis, we have to be aware of the best tools (such as those provided within the PuMP Performance Measurement process (link) and have the right data to use within the right charts in order to help us differentiate the signals from the noise.
Ultimately, we need more Statistical Thinking in our organizations, not just more data!
Special thanks to Dr. Donald J. Wheeler. Much of the content in this article originally appeared originally in his book Understanding Variation, Second Edition.
The PuMP® Performance Measure Blueprint was created by Australia’s performance measure specialist Stacey Barr to help overcome measurement challenges like the ones identified in this article. Louise Watson of Adura Strategy is Canada’s Official Partner and Licensed PuMP® Blueprint consultant and trainer.
