Some time ago, I've read a
great post by Amir Kolsky and Scott Bain about why Test First technique is so important in TDD. I'd like to use it to elaborate on the few reasons I consider Test First to be an essential practice:
1. You don't know whether the test can ever fail.
This is one of the main points by Amir and Scott. When your write your test after the fact, you don't even know whether the test can ever fail even when the behavior described by the test is broken/changed.
The first time I encountered this argument (and it was long before the Amir and Scott's post), it quickly raised my self-defense mechanism: "But how can it be? I'm a wise person, I know what code I'm writing. If I make my unit tests small enough, it's self-evident that I'm describing the correct behavior. This is paranoid". However, this turned out not to be true. Let me describe, from my experience, just three ways (I know there are more, I just forgot the rest :-D) one can really put in a unit test that never fails:
a) Accidentially skipping addition to test suite.
However funny this may sound, it happens. The example I'm going to give is from C#, but almost every unit testing framework in almost every language has some kind of machanism of marking methods as "tests", whether by attributes (C#) or annotations (Java) or with macros (C and C++) or by inheriting from common class, or just a naming convention.
So, in my example, I'm using nUnit. In nUnit, to make a method "a test", you mark its class with [TestFixture] attribute, and every test must be marked with [Test] attribute in the following way:
[TestFixture]
public class CalculatorSpecification
{
[Test]
public void ShouldDisplayAdditionResultAsSumOfArguments()
{
//...
}
}
Now, imagine that you're doing post-factum unit testing in an environment that has, let's say, more than thirty unit tests - you've written the code, now you're just making test after test "to ensure" (as you see, this is not my favourite reason for writing unit tests) the code works.Test - pass, test - pass, test-pass. You almost always launch the whole suite, since it's usually painful to point out each time, which test you wanna run, plus, you don't want to introduce a regression. So, this is really: Test - all pass, test - all pass, test - all pass... Hopefully, you use some kind of snippets mechanism for creating new unit tests, but if not (and many don't actually do this, myself included :-( ), once in a while, you do something like this:
[TestFixture]
public class CalculatorSpecification
{
//... some unit tests here
public void ShouldDisplayZeroWhenResetIsPerformed()
{
//...
}
}
And you don't even notice that this is not added to your test suite, because there are so many unit tests already that it's almost irrational to search for your new test in the test list. Also, this fact that you have omitted the addition, does not disturb your work flow: test - all pass, test all pass, test - all pass... So, what you end up is a test that not only will never fail - it will never be executed.
So, why does Test First help here? Because, in Test First, a test passing right away is what DOES disturb your work flow. In TDD, the work flow is: test - fail - pass (ok, and refactor, but for the sake of THIS discussion, it doesn't matter so much), test - fail - pass, test - fail - pass...
Once in a while, I stumble upon a situation where Test First saves me from this trap.
b) Misplacing mock setup
Ok, this may sound even funnier (well, honestly, every mistake sounds funny), but it happened to me a couple of times, so it's beneficial to mention this. The example I'm going to show uses manual mocks, but, just not to jump into "dynamic vs handmade mocks" discussion here, this can happen with dynamic mocks as well, especially if you're in a hurry.
So, without further ado, the code (yes, this is a stupid example I made up in two minutes, so don't criticize the design here, please ;-):
[Test]
public void ShouldRecognizeTimeSlotAboveMaximumAllowedAsInvalid()
{
var frame = new FrameMock();
var validation = new Validation();
var timeSlotAboveMaximumAllowed = TimeSlot.MaxAllowed + 1;
var result = validation.PerformForTimeSlotIn(frame);
frame.GetTimeSlot_Returns = timeSlotAboveMaximumAllowed;
Assert.IsFalse(result.Passed);
Assert.AreEqual(
ValidationFailureReasons.AboveAcceptableLimit,
result.Reason);
}
Note how the tested method (which is PerformForTimeSlotIn()) is called BEFORE the mock is actually set up and the set up return value is never taken into account. So, how did it happen that, despite this fact, the call yielded correct result? This sometimes happens, and it happens most often in case of various boundary values (nulls etc.).
c) Using static data inside production code.
Once in a while, you have to jump in and add some tests and logic to code that's written by someone else. Imagine this code is a wrapper around your product XML configuration file. You decide to write your unit tests after applying the changes ("well", you can say, "I'm all protected by the suite that's already in place, so I can make my change without risking regression, then just test my changes and it's all good...").
So, you start writing a test. The test suite class contains a member like this:
XmlConfiguration config = new XmlConfiguration(xmlFixtureString);
What it does is to set up a global object used by all the tests. BUT, what you can also see, it uses the same fixture object every time. What you need to write tests for is a little corner case that does not need all this crap that already got into the global fixture. So, you decide to start fresh and write your own fixture. Your test begins like this:
string customFixture = CreateMyOwnFixtureForThisTestOnly();
var configuration = new XmlConfiguration(customFixture);
...
And it passes. Ok, what's wrong with this? Nothing big, unless you read the source code of XmlConfiguration class carefully. Inside, you can see, where the xml string is stored:
private static string xmlText; //note the static keyword!
What the...? Well, well, here's what happened: the author of this class coded in a small little optimization. He thought: "The configuration is only changed by the field group and to do it, they have to shut down the system, so, there is no need to read the XML file every time an XmlConfiguration object is created. I can save some cycles and I/O operations by reading it only once when the first object is created. Another created object will just use the same XML!". Good for him, not so good for you. Why? Because (unless your test runs first), your custom fixture will never be used!
This was the last way I wanted to mention, now to the second point.
2) "Test After" ends up as "Test Never"
Once in a while, I come into argument with a coworker of mine. What those guys usually say is: "Ok, I know that unit testing is good, I just don't buy the Test First part". When I raise the argument mentioned in point 1, they say: "I can make it without Test First - after I write each unit test, I modify production code on purpose to make sure this test fails - then I get the same value as you do, without the Test First absurd". What I usually do then is to take them to my desk and show them a code coverage report for a single file. All of the methods are covered in green as covered, and one single method is in red as not covered. Then I say: "Guess which parts of this code was written Test First and which is written Test After". I like it when they discover (or I tell them) that this uncovered code is Test After that ended up as Test Never.
Let's be honest - we're all in a hurry, we're all under pressure and when this pressure is too high, it triggers heroic behaviors in us, especially when there's a risk of not making it with the iteration commitment. Such heroic behavior usually goes by the following rules: drop all the "baggage", stop learning and experimenting, revert to all of the old "safe" behaviors and "save what we can!". If tests are written last, they're considered "baggage", since the code is already written, "and it will be tested anyway" by real tests (box testing, smoke testing, sanity testing etc. comes into play). It is quite the contrary when using Test First, where failing test is a reason to write any code. To write the code, you need the reason and thus, unit tests become irremovable part of your development. By the way, I bet in big corporations no one sane ever thinks they can abandon checking in the code to source control, at the same time treating unit tests as "an optional addition", but that's a topic for another post.
3) Not doing Test First is a waste of time.
The first time I thought about it, I thought it was only about missing all the benefits that Test First brings you: the ability for a test to fail, ability to do a clean-sheet analysis, ability to do Need Driven Design etc., however, now I think there is more to it. If you're reading the
Sustainable Test Driven Development blog regularly, you know that Amir and Scott value
testability as design quality, along with cohesion, encapsulation and others. Also, they state that in order to make TDD (and even plain unit testing for that matter) sustainable, the code must have this testability quality on a very high level. How can we use this valuable insight to identify the waste? Let's see how testability looks like in Test First workflow (let's assume that we're creating new code, not adding stuff to dirty, ugly legacy code):
- Write unit test that fails (The code has high testability)
- Write code that satisfies the test
Now, how does it usually look like in Test After approach (from what I saw in various situations):
- Write some code (probably spans few classes until we're satisfied).
- Start writing unit tests
- Notice that unit testing the whole set of classes is cumbersome and unsustainable and contains high redundancy.
- Refactor the code to be able to isolate objects and inject some mocks (The code has high testability)
- Write proper unit tests.
As you may have noticed, I emphasized few steps that are additional in Test After approach. What's their equivalent in Test First? Nothing! Doing these things is a waste of time! And, this is a waste of time I'm seeing done over and over again!
Anyway, because of this, let me say it clearly: if you've got a guy in your team that's doing TDD and he's the only one, and you're just "coding and shipping" carelessly even without unit tests, be informed, that this guy will ALWAYS have to corect your code to make it testable and, for that matter, better designed. So, if you really wanna stay on the dark side and not do what's right and ethical (which is to do TDD), at least buy him a soft drink from time to time.
Ok, that's it. Enjoy your weekend!
