Press "Enter" to skip to content

Let’s dive into this risk mitigation strategy

This risk mitigation strategy is made of several parts :

  1. The changelog
  2. Risk analysis
  3. (Internal to the team) Regression testing plan
  4. (External to the team) Validation by partners
  5. Push to production strategy

Changelog

What have we changed?

The changelog is the exhaustive set of changes made by this release.

It was at first written by hand by the developers but later they just made sure they write proper GIT commit messages and now this changelog is simply a dump of the set of commit messages.

Risk analysis

What can be possibly broken or impacted by this set of known changes?

Performing a risk analysis is a very special train of thought, one that is typically alien to developers.

If you ask this question to developers, they will immediately think of how to validate the changes which is completely missing the point: these new changes are probably the most thoroughly tested parts of the software right now as everybody tested them over and over before getting to the point where the changes will be released.

Instead you want to check everything that is around the change but is not directly part of the change itself. To manage impacts.

This is one of the most valuable skills of the testers: performing risk analysis is at the core of the tester’s skill set. Developers can learn this skill too (which will make them better developers) but having a tester around to teach the whole team is a big plus. Pairing is your best friend.

Regression testing plan

What tests will we perform to mitigate the risks that we identified?

Again, the goal is not to validate the changes, but to make sure that the changes have not broken anything.

The regression testing plan is devised from the risk analysis:

  • The set of test scenarios that must be tested to cover the area at risk
  • The set of devices/OS/browsers that must be tested, from a device compatibility point of view

This could also include exploratory testing charters.

Validation by partners

What checks do we recommend partners to do?

We are not working alone. We have several kinds of partners:

  • We are producing a library that will be used by internal customers
  • Some features require a formal validation by some third-party company to certify that our product follow the market specifications
  • Some features are critical to the business path of partners, so they want to make sure we don’t break anything on the critical business features

How do we handle these partners? How do we involve them just enough? While we can’t bypass them, we don’t want to involve them too much either as it would otherwise impede our speed. Indeed, validation by partners is usually a lengthy, blocking process.

In this section we propose a compromise between full involvement and total avoidance of the partners. In any case, this is only a recommendation: it is up to the partners to decide if they want to follow it or not.

It is essential to provide the partners with the full risk mitigation strategy. Otherwise they won’t be able to choose between following and not following our recommendation. It is only with all the information in hand that they are able to compromise and not to go the full regression testing route.

Push to production strategy

Will we push to production incrementally or all-at-once? If we go the incremental way, what are the steps?

Finally, we propose a push to production strategy.

At the moment our tooling is not the most appropriate for real incremental deployment, so we might want to avoid the hassle for trivial changes and speed-up the release pace instead.

There are also cases where incremental push to production does not really help detecting problems early.

It also happened that our partners asked explicitly for an incremental push to production instead of running any regression testing on their side.

Good thing our partners have complete faith in our own tests! But then they ask for an incremental push to production instead 😋

Communicating with the ops team

This is actually an extract from a JIRA ticket.

Hey! This is a JIRA ticket.

It is indeed. This is because the current process of the ops team is based on JIRA. Also it just made sense to share risk mitigation strategies with the ops team and since we were looking for a place to write and store them, then this mandatory JIRA ticket was the logical place.

As a side-effect, tying the risk-analysis with creating the ops team JIRA ticket was a way to ensure the JIRA ticket was not created at the last possible moment. Win-win situation!

Communicating with our partners

Since the validation of our partners can block our process, it is very important to put the appropriate effort in managing these dependencies.

To reduce the risk of blocking, we thus suggest them the validation they should do. We follow the same pattern of risk-based testing. But even more important, we share all the elements that lead us to this suggestion.

If we don’t give them this information (changelog + risk-analysis + what tests will be performed by us ahead of them) the only option they’ll have will be performing a full regression testing session. Which is again what we want to avoid at all cost to keep moving fast.

If we want to avoid black-box testing, we have to open the box.

Communicating with the whole company

Finally, we keep the whole company up-to-date with the last changes in the product.

The raw release pace makes it infeasible to send an e-mail at each release. Instead the team created a specific channel in the company’s Slack and announces there the releases. As usual, the full risk mitigation strategy is provided.

In conclusion…

Make a true risk analysis but, even more important, share it with the world!

You simply cannot ask other teams how they should test your product if they don’t know what you’ve done to the product.

Communication is key. Share the information you have.

Laisser un commentaire

Votre adresse e-mail ne sera pas publiée. Les champs obligatoires sont indiqués avec *

Top