“If you lack self-awareness, you can’t change. Why should you? As far you’re concerned you’re doing everything right”
Jim Whitt, author and founder, Purpose Unlimited.
In today’s article, I share an excerpt from Steven G. Rogelberg’s book The Surprising Science of Meetings, Oxford University Press 2018, with the intent to help leaders and their teams better understand how unknowingly over-inflating their self-assessment of the quality of their organization’s performance measures may be putting their decision-making at risk, with potentially dire consequences.
Excerpt: “Prairie Home Companion” was a live radio program introduced in 1974 on Minnesota Public Radio. The show was set in Lake Wobegon, a fictional town. The town was described as a place where “all women are strong, all the men are good looking, and all the children are above average.” From this, psychology professor David Myers coined the term the “Lake Wobegon Effect”, which refers to the robust human tendency to overestimate – relative to others – one’s own knowledge, skills, abilities and personality traits. In other words, most people believe they are well above average, which clearly is a statistical impossibility.
One of the first studies on this effect was conducted by researchers at the College Board, the publisher of the SAT exam. They attached a research survey to the SAT exam asking students to provide self-ratings on a variety of personal characteristics, such as leadership and the ability to get along well with others. Seventy percent of the students rated themselves as being above or median on leadership. Regarding the ability to get along with others, 85% indicated they were above median, and a whopping 25% indicated they were in the top 1%.
One reaction to these data is to, perhaps, assume that this is just another example of the narcissistic teenage mindset and that these results would not extend to more grounded adults. Well, not so fast. Faculty at the University of Nebraska were surveyed about their teaching ability. Over 90% rated themselves as above average and 68% indicated they were in the top 25%. Another research project looked at self-perceptions of driving skills relative to others. When asked about driving safety, 88% of a US sample indicated they were in the top 50% of drivers.
Most recently, a unique study was conducted with prisoners in England, the majority of whom had been convicted of either violence against people or robbery. The participants were asked to compare their standing, relative to other prisoners and non-prisoners, on nine traits, such as kindness, generosity, self-control, morality, and law-abidingness.
In light of the other results presented here, it should come as no surprise that the vast majority of participants rated themselves as “better than the average inmate” on all traits. What is most interesting is that when the inmates compared themselves to non-prisoners, they still rated themselves as BETTER on all dimensions, with one exception: they rated themselves as equal to non-prisoners on law-abidingness.
For our non-US readers who may think it does not apply in their country, research has found this “better-than-average bias” in diverse sets of samples from Germany, Israel, Japan and Australia.”
Want to read the whole book? Here is the Link to Amazon.ca
What does this have to do with Performance Measurement?
Although direct tests of the self-inflation bias with regard to organizational performance measurement skills and abilities do not yet exist (to the best of my knowledge anyway), 13 years in the performance measurement field inspired me to want to share Mr. Rogelberg’s research to increase awareness of the risks that go with it when it comes to measuring performance (also called metrics, indicators etc.).
As we know, the purpose of measurement is to produce statistically sound insights that help leaders make decisions about how to improve the performance of their organization’s strategic priorities over time. However, if they are unknowingly making these decisions using poorly designed measures with unreliable data producing weak insights, the risks are high for making the wrong decisions. These wrong decisions can have considerable consequences to their organization’s strategic execution including wasting considerable time and money, not to mention the potential loss of reputation and trust.
If we apply Rogelberg’s research to our own organization’s leadership, do you think the same over-inflation of how leaders assess the quality of their performance measures would occur as it does in Rogelberg’s examples? Based on his research, the likelihood is very high, and the risks of poor decisions based on them is even higher.
Ready to Improve how Teams and Leaders Self-assess the Quality of their Measures?
Apply the following 5 criteria, to one measure at a time, and experience how they drive more meaningful assessments of your measures.
- Criteria 1: Is it a strong indicator? Does the measure provide direct evidence of the result occurring and tell the story of change and improvement over time?
- Criteria 2: Is the indicator adequately quantified?Do the words describing the indicator allude to the mathematical formula needed to bring the indicator to life (ie. does it produce consistent data points)?
- Criteria 3: Is the indicator using the appropriate statistic? (count vs average, for example)
- Criteria 4: Is the indicator feasible? How easy or hard is it to get the data, and can we bring the data in frequently enough to show change?
- Criteria 5: Could this measure drive any unintended consequences?
The path to more accurate self-awareness (leadership as well as system-wide awareness) of what it really takes to make evidence-based decisions, is multi-pronged. First and foremost, leaders need more robust measurement skills.
Interested in exploring leadership training? Check-out our Evidence-Based Leadership program.
Louise Watson is the PuMP Partner for Canada, and these criteria are inspired by Stacey Barr’s PuMP Performance Measurement Process. The full definition of what makes a real performance measure can be found here.
Special acknowledgement to Steven G. Rogelberg’s book The Surprising Science of Meetings, Oxford University Press 2018 and the excerpt incorporated into this blog.
