Just digged up this old LinkedIn group discussion and I felt that I should rewrite my answer on my blog:
What is the best way to move a business over from Manual to Automated Testing?
From my experience, you must include the developpers into the automated tests. One way I like to do it, is to start easy with acceptance tests on new developments, only to cover regression test cases later. Why this approach?
Automating tests is easy if you wrote the code to be testable
One fundamental characteristic of automated tests is that the underlying code has to be written in a way that make it (easily) testable. If that’s not the case, the test automation attempt will fail, either at short term or at longer term.
- Failure at short term: tests are too costly to write and run
- Failure at longer term: tests are too hard to maintain, and takes too much time to run
This is usually obvious to developers who practice TDD (Test-Driven Development).
By writing unit tests at the same time than writing production code, the production code gets naturally written in a way that can be easily covered by unit tests.
On the other hand, if you go with writing unit tests after the production code is written, you’ll face many issues you need to fix before you get working tests. (mainly, breaking dependencies in the code so that you can run the pieces individually)
The same applies to higher-level tests; if you write code with automated tests in mind, you’ll write a different code that will be easily testable and whom tests will be maintanable. Otherwise you’ll either have to refactor the code until you can test properly (costly) or you’ll have to write bad tests that cover existing code as-is (costly + hard to maintain).
Acceptance testing as an entry door: start with getting the methodology right
That’s why I’d rather introduce automated testing as acceptance tests on new developments, to promote writing code in a way that enables good tests:
- easy to write,
- easy to maintain, and
- run quickly.
Let’s not forget that this will be a new way of doing for the team, and maybe new tools will be added to the mix. That’s already enough trouble for the team at the moment.
In addition, going into acceptance tests helps jelling the whole team together as it creates an environment where developers, testers and business analysts (PO) work all together hand in hand. This is invaluable by itself.
As the team matures, start covering legacy code
Let’s fast forward later in the project. The developers are writing their unit tests by themselves, and all new developments have acceptance tests written with the help of testers and PO. But you still want to cover all the code that was written before that. We could call it legacy code. That’s probably these test cases that you wanted to automate in the first place, actually. Well, now’s the time.
Now that the team is accustomed to this new way of doing, now that they master the new tools and processes, they can go ahead with regression testing on legacy code.
This time the hardest part is not learning new tools: it’s getting the old code in a shape that gets it testable. Yep, refactoring.
At that point, maybe this is just code that nobody wants to change in fear of breaking it. That’s OK, really; do not change anything before you write any characterization tests! What you’re doing is not called refactoring unless you are sure you don’t break anything: you need tests beforehand.
Write bad tests to enable refactoring
OK so you must refactor to enable testing, but you need tests before you refactor.
How to break the loop? One way of doing would be to write big, ugly tests that test the code as-is.
I call them bad tests, because they will be costly to write, run and maintain as their scope will be much bigger than what is actually needed. Still, these tests come handy to make sure you do not break anything when performing your first refactoring.
Don’t keep those bad tests
But then, do not forget that these are bad tests! As soon as you’ve done some refactoring, then write the proper tests, just like you would with new code.
Once the code is refactored and properly covered by tests, consider again the big tests you wrote earlier. Your goal is to get rid of them.
They sure were very useful to get out of the bad situation, but to be sustainable automated tests must run quickly and be easy to maintain. If they do not fit with this definition, either rewrite them or simply get rid of them. If the feature is covered by the new tests, that’s OK!
I have since written another article entirely dedicated to test automation pitfalls.