Press "Enter" to skip to content

Where do you want to go? Measure your way is half the work done.

Where do you want to go? Measure your way is half the work done.

You’ll get what you measure!

The tweet that led to this article.

Indicators are great. Because when you measure something, it will auto-magically improve.

Yes, that’s true! That’s because we are not invisible observers: the very fact to observe the system, influences the system.

Let’s say that at some point in time you rule that knowing the amount of code covered by unit tests is important.

So you start measuring the code coverage by the unit tests. Even better, you display it openly in the open space and during big events with the stakeholders.

At that point, you’re not yet seeking for improvement. You just want people to know and care.

And it works! They now care about the code coverage by the unit tests. Those who were not familiar with the concept now are. The team members start checking code coverage with every new line of code they add.

And guess what happens? Code coverage by the unit tests improves. Yet you did not ask that from anyone! You only measured it, you did not request it to improve.

Yet it improves. By taking the time to measure it, you implied that it was important.

The situation is even more tricky than implying it: our minds want to improve metrics, whatever they are.

This is actually at the core of gamification: having a score, we instinctively want to improve this score. Even if there is no prize. Even if there is no competition with other people, only competing against yourself. Earning abstract points if enough to motivate our minds!!!

Beware of what you ask for; your wishes might be fulfilled.

Be careful of vanity metrics: what you measure will increase, but that might not be a guarantee of getting what you actually want.

Back to the previous example: code coverage by unit tests is now improving, and that’s great.

But… Is that really a good thing? Or is it a vanity metric?

It is indeed totally possible to write dubious unit tests that improve code coverage… While not really guaranteeing anything about the quality and reliability of the code itself.

Hey!!! That’s not what you wanted!

Yet it’s a completely possible outcome of the indicator.

And this is no fantasy: every developer crossed the path of some assert(true) joke that just makes sure that coverage measure is good while not testing anything. Yes, this is happening everyday in the real life.

It can always get worse…

Sometimes, some metrics are doing more harm than simply measuring the bad thing: they have a negative or destructive effect on the system.

Need an example? Let’s say that you start paying bonuses for each bug fixed. That should motivate those developers to fix all bugs, right? Well, it might also convince them to create more bugs…

Most vanity metrics can have a negative effect: just because they make you believe that everything is doing fine, you end up caught off-guard. Back to the example of the fake code coverage given earlier, you may build a false sense of safety which would lead to over-ambitious decisions… Which will inevitably backfire given the real state of affairs.

So choose widely your indicators

One might argue that relying on only one indicator is stupid.

Well, maybe only one indicator is too few… But don’t follow too many metrics either. Like everything else, focus is key. You should be able to prioritize.

What works well is focusing on the high-level, business indicators. These are the hardest to game, they hardly become vanity metrics.

Start from these. Then create other, finer-grained indicators for your team but never lose sight from the global business indicators.

Laisser un commentaire

Votre adresse e-mail ne sera pas publiée. Les champs obligatoires sont indiqués avec *