Holy Over Mocking Batman: A Natural Progression
Posted by coreycoogan on January 28, 2010
Uncle Bob has been on an interesting kick lately, writing thought provoking posts with some controversial undertones. A few days ago, he posted “Mocking Mocking and Testing Outcomes“, where he mocked the [over?]use of mocking frameworks. It’s an interesting article and worth a read.
I can certainly see where he is coming from with this post. The part that really grabbed me was the last paragraph:
“Also, why do you need to verify that “verify(manager).createCredential” was called? If you really get the credentials you’re expecting from the Authenticator, why would you need to check where they come from? Isn’t it one more way to couple your test to the implementation?”
I think this is the single most common pitfall of anyone who has been doing TDD with mocking for a short period of time. The initial tendency is to test each and every interaction between the objects. It usually starts out with large tests that make too many assertions. It’s not uncommon in these tests to see the developer assert that:
- The application service layer gets called
- A transaction is started
- The appropriate repository method[s] are called, which can be multiple repositories
- Some business logic is executed
- The right value is returned
Such a test is going to be very brittle and break upon any serious refactoring. As Uncle Bob points out, do we really care about each of these interactions? Sure, in some cases it may be important to make sure a transaction is started, but in most cases all you care about is that the right value is returned. This is the natural progression of implementing TDD, especially when mocking gets introduced. I think it takes the pains of these brittle tests to discover that asserting all those interactions is really not necessary.
The next progression starts is when the developer begins to understand what one assertion, or logical assertion group, per test really means. This phase is a little less painful because at least now there is a separate test for each interaction, which is still typically unnecessary. Now that the tests are broken up into logical units, the tests are easier to fix when refactoring breaks the interactions.
Now the final progression – the pains of these fragile tests lead to the notion that all these tests really don’t matter. They really don’t prove anything about the quality of the software. The tests match the implementation and the system depends a set of very brittle tests that could make refactoring too painful. That’s when the tests get simpler. That’s when the realization that a mock isn’t needed for everything and ever interaction doesn’t need to be tested.
If you are reading this and it sounds like you are in phase 1 or 2, I highly recommend you read the post “Three simple Rhino Mocks rules” by Jimmy Bogard. It really influenced the way I write tests.
James Thigpen said
So I’m at the realization now that interaction based tests with too many mocks yields fragile tests, but in the blog you don’t really describe what that next step is, just that “tests get simpler”. How do they get simpler? What do they look like?
coreycoogan said
Thanks for the comment James. Your tests should test the core logic that you are getting at. For example, a typical layered architecture might include some domain services that interact with entities. Don’t bother testing how the service method gets the entity, just test the logic and state changed by the service method. You may then have Application Services that are responsible for getting entities from repository and feeding them into the domain service. From that standpoint, an interaction test to ensure that the app service is opening a transaction or calling the proper domain service may be appropriate, but you wouldn’t need to assert any calls on your Repository and could Mock your domain service. Does this make sense? Let me know if you’d like to see some code and I will write a short post to include some.