Is there an isolation framework (Moq, RhinoMock, etc.) to check for exceeding?
In Osherove's great book, The Art of Unit Testing, one of the test anti-patterns is the over-specification, which is basically the same as checking the internal state of an object rather than the expected result. In my experience, using isolation frameworks can cause the same unwanted side effects as testing internal behavior, because each only strives to implement the behavior necessary for your stub to interact with the object under test. Now, if your implementation changes later (but the contract remains the same), your test will suddenly break because you are expecting some data from a stub that was not implemented.
So what do you think is the best approach to dealing with this?
1) Fully implement your stubs / mockups, this has the negative side effect of potentially making your test less readable and also specifying more than is necessary to pass the test.
2) Thanks for handcrafted, fully realized fakes.
3) Implement your stubs / fakes so they skip your test, then handle the fragility this might introduce.
a source to share
I don't think you should approve manual testing - unless you want to test instead of code.
Instead, you have another option - if you are testing functionality rather than implementation, try to avoid testing private methods (which can be refactored) and write less fragile tests altogether, you will see that using a fake / isolating framework does not require you to refine the system and does not cause your tests to become more fragile.
In short - writing brittle tests can be done with or without fakes / mockups and vice versa.
a source to share
I usually use mocks instead of cropped / fake objects. I find them a lot less of a problem and they have better control over the control of the test code because it's not cluttered with all kinds of semi-arid implementations. They also help clarify what is being tested.
Another advantage is that I only need to indicate where the class under test needs something specific from the layout. So I don't need to point out where it doesn't matter. As far as validation goes, again I only need very much calls from the class under test to the mock that I care about and consider the important aspects of the test.
a source to share
I think the problem is always the same, although it comes in different flavors: if you have tests that somehow cover the internals of a class, then you will break tests that cover that inner code.
IMHO there are two ways to handle this:
- Your tests only cover the public contract of the class - a testing strategy that's widely adopted for this exact reason: you don't need to change your tests as long as the public contract remains constant. Unfortunately, this is not the case, which you will do in test-driven development.
- If your tests come from the TDD process, they will regularly cover non-public code. This means they will break if you change the code. The only way to keep in sync here is to "fix" the tests along with the code. This means more maintenance is required during the development process. There's no recipe to deal with this easily (other than throwing out the dough, of course ...).
My personal "way out" is the concept of "code elements", not just code. A code element consists of three parts: Documentation, Test, Code. So if you change one part of an element, you also need to adjust the other two - otherwise, you will leave the broken code element behind.
a source to share