You are still reading after such a title? you are brave. If I tell you that I will talk about bees, Kant and zombies, do you still stay here? good, let’s begin.
First of all, like a good philosopher, we have to define concepts. What is a bug? We will choose the largest possible definition. It can be expressed in two ways:
It is everything that annoys me in my experience using a software.
It is everything that threatens the value of my product.
As you may have noticed, they are derived from the famous quote from Jerry Weinberg: “Quality is value to someone”. What do you notice in such a definition? Bugs are a matter of perception, they are in the eye of the user or the stakeholder. They are also a judgement like in deciding if two parallel lines can cross each other (is it true or false) or like in deciding if the Picasso’s bulls are beautiful (or not).
Colors don’t exist
Let’s make an analogy. If you say at a family dinner that colors don’t exist, it is more likely that you will receive only silence and the conversation will soon change to another subject. It is something too weird to imagine that colors don’t really exist while I see them all the time. In reality, external objects just send us luminous rays and it is our brain, with the cones inside our eyes that find meaning in those rays. Think about those people who cannot see some colors like most of us. On the other side, dogs for example don’t see colors, they only see shades of grey. And the evolution has created so many and beautiful varieties of flowers because bees see colors, just like us.
It’s the same with bugs. They are not a thing that is inside the product. They are a judgement made by the user or the stakeholder. Nothing else. If you haven’t understood that point you cannot deal with bugs economy and set up a pragmatic strategy about quality.
If bugs are only an evaluation or a categorization (true/false, annoying/negligible, weird or not, ugly or not…) then you have to deal with some important consequences.
You cannot count things that don’t exist. Or trying to count them is as useful as counting colors. It may show you a tendency: we test more, then we find more bugs = good; we test more, then we find less bugs because they are handled before the functional review = good too. But in the end, having 5, 50 or 500 bugs in your backlog does not mean anything by itself. It is relative to your context, your legacy, your functional scope, your audience.
Bugs nature changes with time. Sometimes, you decide that some user flow is optimised, but when you look at users searching for that link or button, you understand you were wrong. There are also zombie bugs: they were already in the code, inactive, and they wake up with some new configuration. Same with performance issues: as your website is more and more used, when do you decide that it is slow enough to treat it as a bug? (Here the frog metaphor).
Root cause analysis is useless. Or better said guilty cause analysis is useless. Is it the stakeholder that miss-specified her need? Is it the developer who is too lazy to handle all cases? Is it the tester guilty, because she didn’t test enough? Is it the user who is too foolish to understand our disruptive UX? Well, I don’t mind. When something annoys the user or the stakeholder, we fix it. Right now. And of course we write the checks that will most likely catch the bug if it tries to come back.
You cannot please everyone. Colors are a matter of taste you know. Same that Picasso. I don’t like some behaviours in some apps or websites, other just don’t care. We have to make choices and assume them. If we decide that something is a bug because it threatens the value of my product, then we fix it. On the other case, we don’t fix it at all.
A product without bugs is a non sense. Once again, bugs are in your head, not on the screen. You can do user research, you can do exploratory testing, you can automate all the things, you can be data driven, but you cannot be sure how the end users will use your product or even if they want to use it — and even less what will be buggy for them.
You cannot build your strategy based on bugs hunting. It’s like fighting against zombies, ghosts and shadows. Because those things don’t really exist, you remember? But you can — and you should — run your strategy based on risk analysis (what threatens my product the most?) and on resilience. Examples of things to do to React to issues:
- you detect unexpected behaviours with automation and real time monitoring,
- with blue/green deployment,
- with feature flags,
- reverting to previous version or fixing if it is easy and with poor impact.
Same with performance: the system should handle double or triple load by design.
Here was some consequences of dealing with bugs as judgements, relative to each one of us but with real impact on your business. There are probably more in your context. And yes, I didn’t mentioned Kant after the introduction, but you probably already noted that all my demonstration is based on his Criticism.
Did you learn something? Did you find this article interesting? Share it 🙂