The Two White Lies in Every Analytics Story

Every analytics story contains two little white lies. If you can see them, they do no harm; but the unwary will be led astray.

The First Little Lie: “It was amazing”

If you are giving a presentation in front of hundreds of people you want your analytics project to sound a little bit amazing. You are more inclined to say you used “predictive analytics” to forecast costs, rather than say that you made an estimate based on past trends.

Scott Adams captured this dynamic in a comic where Dilbert is trying to impress his mother. Dilbert says, “Today I debugged a TCP/IP driver for an application that runs over ISDN with bonding.” She replies, “You mean all you did was slap a BRI analyzer on a circuit and look for bad packets?” Busted.

The signal that you are facing the “it was amazing” white lie, is that you (unlike Dilbert’s mom) can’t quite tell what the person did. In other words, if you were asked to do it yourself, you wouldn’t know what to do.

When his mom dismisses Dilbert’s work, he responds “Yeah, but it’s really hard”. I have a good deal of sympathy for that. It may be overblown to say “We used predictive analytics”, but it may have been really hard to make a credible estimate. Furthermore, the point isn’t how amazing the analysis was, the point is that it added value.

In the end, your customer shouldn’t really care how you did your analytics, just if you helped them solve a problem.

The Second Little Lie: “We followed a linear path to the answer”

Here’s how a typical analytics story sounds: “We identified issues. Ranked them. Gathered the relevant data. Used regression to find key drivers. Addressed the biggest drivers.”

Here’s what really happened: “We start working on retention and half-way through someone said there were more pressing issues. We got a list of issues, but then the CFO said the strategy had changed again. We went back to the drawing board and found new priorities. We mucked around with different kinds of analysis before we realized that the only thing we needed to do was a simple regression.”

The white lie of making the story much more linear than it was makes some sense. You’ve got limited time, so you strip out the “irrelevancies” and tell a straightforward story that is easy to follow.

The trouble with stripping out the irrelevances is that working through the missteps is actually the hard part. If analytics was a linear process we’d all be doing it without breaking into a sweat. You constantly hear people saying that the most important thing is asking the right question. Well okay, I buy that, but how do you find the right question? You find it by going on this meandering cycle of work and conversations until you get enough clarity to know what question matters most.

What you should do

It’s entirely fair to push back on the first little white lie and get the speaker to tell you, step by step, what they actually did. The point is not to spoil their moment, it’s to learn enough that you can do the same thing in your own organization.

As to the second white lie when someone is sharing their story you probably don’t want to hear the long winding tale. However, as you embark on your own analytics project, recognize that all the missteps are a part of the process, maybe in fact “they are the process”. Remember that the hard part is the messy exploration needed to find the right question.