A response to "Failures in Isolation"

Following Corey Haines on twitter is such a good idea. Today he dropped this gem:

Please read, I'm not going to paraphrase it. Nevermind, I will. But it will be really short. Poorly designed code that is tested with a high level of isolation causes a problem. You can't refactor it with confidence because the tests, or the mocks, are blocking your progress. So let's talk about how they impede refactoring first.

Tests prevent behavior from flowing 'up'. For instance, if you mistakenly coupled a model class to a view tier class you could not 'refactor' this problem without breaking the tests for your model. Most likely, the test will fail because the API of the class has changed, you no longer pass the request.

For Mocks it is the opposite. You cannot move behavior 'down'. Let's say I have leaked some model behavior into my controller and I wish to move it to the appropriate model. If my test uses the real model, not a mock, this is not a problem. I can move the behavior and my controller test will still pass. Now I can safely write model tests for the new behavior and delete the controller test that now duplicates it.

If I have mocked the model, however, my controller test will fail when I attempt to move the behavior 'down' into the model. One offense is not an issue. I was involved in one system where an entire test suite was thrown away because of improperly used mocks. Death by a thousand paper cuts.

I have no universal solution for this problem - as it is a problem of my skill as a designer and the skill of my peers. However, I have come up with two helpful guidelines.

Only directly test code you like

I dislike code when I'm unsure of its design. I also dislike code that appears volatile. As my confidence has grown I find myself more comfortable with my early design decisions and more willing to glorify my creations with their own test sooner. When I am not confident I keep my tests at a high enough level where I can refactor freely. As my confidence grows in a system I move towards higher levels of isolation.

Only mock code you like

For all the same reasons as above. Mocks crystalize an API for a class and can cause a lot of problems if the class is used often in a system and the class it is poorly designed. As the ugliness and volatility of code increase, mock it less.

In Conclusion

As with everything, the value of isolation in testing is not an absolute. For younger developers I suggest beginning with a state-based testing approach over using mocks pervasively. A state-based style will afford you maximum flexibility to refactor your design. As your experience grows begin to use mocks to solve some of the problems that state-based testing will invariably bring about.

That is the path I travelled and it worked out alright for me.