The truth about metrics

The truth about metrics

Sometimes your mind is one big dichotomy, and right now I’m battling with with the different variables that affect metrics.

I’m a numbers guy.

I like using financial spreadsheets to play out numerical scenarios. I don’t often need to refer to what I create, but the process of thinking through “what-ifs” is invaluable in the long run. When you sit down and think through things carefully in advance you are primed to respond faster and more adaptively when the time comes. People love to talk about their ability to be dynamic and adjust, but it’s often #fakenews. Most actually do very little to plan, predict and advantageously position themselves for future events. It’s easy to “go with the flow” if you never actually do anything different.

As a leader, having metrics is all the rage. It’s in every leadership book out there. I have often used the phrase “if you can’t measure it, you can’t fix it”. I believe that to be true across most business processes and I have helped companies profit in the millions by doing that very thing with technology.

However, when it comes to managing other leaders, one observation over the years has held true, over and over again. One that I couldn’t understand. I have actually changed my behavior because of this observation but had no words to describe it until now:

Goodhart’s law and the law of observation.

Goodhart’s law, in simple terms, says once a measurement becomes a target, it fails to have value as a measurement.

The observation law states that once people are being observed in relation to a goal, the outcome is that you start to get the behaviors that you want, not the actual results that you want.

As a leader of leaders, I can say that this is the case more often than people want to admit. For example, if you create a sales goal or operations goal, in daily use, what usually happens is that the goal itself becomes meaningless instantly. In reality once people know they are being measured, they will either almost always hit this goal or almost always miss it, and rarely do they actually make a significant change to hit the desired results. They either change their behavior to “hit their number at all costs” or they attribute it to a bad metric. Either way, you do not get the results you need.

The goal of business leaders is results-oriented so when you introduce a single tangible metric you run the risk of mistaking behaviors for results. People are very quick to adjust their behavior in the pursuit of the goal, despite their new behavior having little to no impact on the desired results. Ask any product manager what happens when they put a goal of working X hours on a product. Usually the hours are hit, but the results (or lack of) never change. It’s because people are focused on the metric and not the goal.

Let’s take an example of billable hours, a common metric used to measure service-based businesses. Let’s say you and your leadership team decide to bill 100 hours/week.

Chances are your team will hit this number, but so what? Could you have done 150 instead? I doubt the goal is to hit 100 and stop. What about billing 80 hours but putting significant effort towards billing 150 next year? Wouldn’t that be a better result for the business? What if you could have targeted 90 and taken those extra 10 towards making the next 12 months easier for you and your customer? What if, in order to hit your “target”, you did things you shouldn’t have, creating bad debt (either relationship debt or technical debt) that you will have to pay for later ten-fold?

So what do you do?

Personally I have found it most effective to use a hybrid approach. I still utilize external metrics quite a bit, but only if they will create meaningful action by whoever I am measuring. If I’m the only one that is truly taking results-based action using these numbers, I find it more effective to discretely measure results, not being as transparent as I would like. I may say “I am measuring billable hours”, but I’ll refrain from putting a specific number knowing what the end result will be. I have learned to be careful in treating everything like a financial transaction.

But again, this is such a difficult thing for me because I really do believe both in metrics and transparency. I want to give my leadership team targets and I almost always find being transparent to be a better approach. However I have simply found that oftentimes doing so creates the opposite effect than the one that I was trying to achieve in the first place.

One of my favorite songs probably says it best.

And all the real estate in my mind is for sale
It’s all been subdivided
Divided into reasons why
My two opposing thoughts at once are fine

– Jack Johnson, My Mind Is For Sale

Related Posts