I had a fantastic day Sunday last; good lunch, stimulating diverse conversation, rugby in Wales, and all in the company of Dave Snowden.
The conversation covered philosophy, literature, the strength and frailties of human nature, the needs of (all of us in) management to have a degree of certainty in decision outcomes.
Now, in certain contexts, exclusively quantitative measurements are powerful decision aids, but we fall into trouble when we extend the context inappropriately, yet still maintain our absolute belief in the power of quantitative measurement.
I was fortunate to play school-boy rugby at a reasonable level, aided in no small part by a maths teacher coach who brought us training by “deploying the science of goals and measurement”, an extremely radical approach some 40 years ago. While most teams were told to simply “go out and play” or “run harder,” we had our tackle count, sprint times, times to ‘break-down’ logged, with appropriate improvement goals provided. This caused some of the less motivated and, shall I say, less mobile players to self de-select, and the overall impact was to produce a more motivated, mobile, speedier rugby team. Naturally, the team did well by local standards, and the ‘scientific method’ was emulated by a number of the local authorities.
Fast forward to a tour of the Antipodes commencing with an opening game in Tonga. The 80 minute game was terminated after half an hour as we ran out of substitute players. We were just as fast, just as motivated as they were but we were also lighter by an average of 15 kg per player. The picture at the top says it all.
We did not adjust. We had statistics on our side; we just had to try harder—and ultimately we failed. We had stuck to our plan because we were confident that measurements were tied to the cause and effect of success, and we refused to play what was in front us.
I have witnessed a similar mindset in clients who love the statistical out of the Net Promoter Score, and pay scant regard to the supporting narrative research. They get a warm feeling from the quantified record of historical performance, yet ignore the rich data that should give them powerful insight into the challenges and opportunities ahead.
Now, I often no longer provide the intellectual argument. Intellectual stimulation provokes thought, emotional stimulation provokes action.
Sometimes you need to let the management team play the Tongans.
For companies that have NPS (or pilot NPS) data, this is how I play ‘Management v Tonga’.
1. Get the leadership team to list out the 3-5 major challenges facing the business over the next 12-18 months. You can place these on the Cynefin framework if you want some extra spice.
2. Take a sample set of about 20 responses and lay out the raw scores on stickies on one side of the meeting room wall, and stickies with the supporting narrative on the other.
3. Ask the group to examine both sets of data, and get them to generate and ‘dots and patterns’ they see emerging.
4. Ask the group which of the two sets of data would be the more useful in helping to resolve the output from step 1 in a manner that is practicable, actionable, and with a degree of confidence.
5. Tell the group that the latest HBR edition has pointed out an error in Fred Reichold’s calculations, and that the statistical link between NPS and profit is spurious. Would the group still find value in the narrative?
In my experience running this exercise, it reveals to senior management the real power of narrative research, and then prompts further discussion on the potential danger of making a measure a goal, and how measurement and flexibility should always be in harness. You have to play what’s in front of you.