Friday 27 July 2012

If you treat your unit tests like garbage, they'll eventually become just that. And they will smell bad...

Some time ago, I was TDDing a piece of code. My coworker glanced from the rear and, seeing a name of the spec I was writing, which was something like:

ShouldReportAllErrorsSortedAlphabeticallyWhenItEncountersErrorsDuringSearch()
he said:

  • - What a crappy names you've got for your unit tests! They're so unreadable!
  • - Why do you think they're crappy?
  • - These are too long! I can't read this!

I then suggested that we looked at the names of the unit tests he and his team were writing. I stopped at a unit test named

TrySendingHttpRequest()
and the following dialog occured:

  • - See this one?
  • - Yep.
  • - When will it pass or fail? And what are you trying to achieve in this test by sending the HTTP request?
  • - I don't know. I don't care. If it fails, I'll have to fix it, if not, I can forget it. That's all there is to it.
  • - Well then, if my unit test names are unreadable, yours are useless.

Then I said: "let's now look at the output from CruiseControl". I navigated to the cruise control unit test output web page of the project I was in. The unit test name I was writing was there, but written in a more human-readable form: "Should report all errors sorted alphabetically when it encounters errors during search". "See this?" - I said - this is our living documentation. He discarded this argument as "bringing unnecessary details" into the unit test suite.

The next day, I sent him an excerpt from Growing Object Oriented Software Guided By Tests book, which recommends to give unit tests names such as

notifiesListenersThatServerIsUnavailableWhenCannotConnectToItsMonitoringPort()
It did not convince him, however. The answer I received was: "We do it the way we do because it's comfortable for us. Just because we do something different than you doesn't mean we're doing it wrong. And whatever evidence you have that suggests long unit test names make sense, I will just keep doing what I'm doing right now. Period."

The lesson

In case you didn't notice - this was not only a debate about long and short names of unit test names, this was a debate about what's the place of unit tests in team and project's ecosystem.

While I personally value unit tests as a design tool, a feedback tool and a living documentation on micro level, there are people who just don't care - for them, unit tests are some crappy code that passes through the production code to make sure nothing breaks. Those guys are not aware of the value unit tests AKA unit-level specification can bring to the team and the project.

There are, however, downsides of this. If you treat your unit tests like garbage, they'll eventually become just this. E.g. for anyone that does not care about how unit tests are named (I assume that this someone would not even dare to use TDD - it's even more difficult than thinking of a descriptive name), here are a few things to consider:

  1. how do you know what situation does the test describe?
  2. how do you know whether the test describes a single situation, or few of them at the same time?
  3. how do you know what is the right assertion/set of assertions for the test?
  4. how do you know whether the test should stay or be removed when you modify the functionality covered by the test?
  5. how do you know whether the test or production code is at fault when you see unit test failure?
  6. how do you know that you will not introduce a duplicate test for a behavior that's already covered by another test when adding to test suite originally created by another team member?
  7. by looking at the unit testing tool output, how do you know whether the fix will be easy or not?
  8. how do you plan to maintain the test suite in the long term?
  9. how would you answer a new developer in your team who asks you " what is this test for"
  10. what's your plan on documenting the code you're writing?
  11. before writing the test, do you know what will you be testing? If so, then how do you know it?
  12. when writing a new suite of unit tests, how dou you keep track of behaviors/scenarios already covered and ones that still need to be covered?

I could keep adding to the list, but let's stay with this for now. Feel free to comment on the questions above. If there's any demand, I'll post my answers from the perspective of a person doing TDD. What needs to be noted is that a person treating unit tests like garbage will probably have an answer to all of the above questions. The funny part is what these answers usually are.

Apart from naming, there are other aspects of unit test that make it stink. That, however, I will leave for another time. See ya!

No comments: