Test Driven Development - objections, part 2

Welcome to part 2 of my post about TDD objections.

By the way, I found an interesting analysis on slideshare on the same topic. It's an interesting read, so be sure to take a look.

Where did we leave the last time?

Ok, let's take on the next set of objections from the list:

1. Not enough time
2. TDD will slow us down, because we'll have to create a lot of additional code
3. TDD will slow us down, because we'll have to maintain a lot of additional code
4. We don't have the necessary skills or experience
5. We don't have the necessary tools
6. (This one is unit testing specific) There are already other kinds of tests, we don't need unit tests?
7. It's very hard to perform with legacy code
8. I already know how to write a testable code, TDD will not really improve my design
9. We've got enough quality and we're doing fine without TDD
10. My manager won't let me do TDD
11. There's no "scientific proof" and enough research on whether TDD really provides a return of investment

The ones I'm gonna talk about today are the one marked with strong text. Let's go then!

4. We don't have the necessary skills or experience

This is a valid impediment. However, its a common one in any skill. You see, it's an extremely rare situation that someone is proficient using a technique for the first time (think of the times when you learned how to use a keyboard and a mouse). Most techniques require some time and knowledge to master - the same is with TDD. Let's say that TDD is like a flower, where the core is the "Red-Green-Refactor" cycle and there are a lot of petals - good practices and heuristics. These include: need-driven design, triangulation, mocking, listening to tests etc.

Thankfully, lack of skills can be dealt with by providing training, mentoring, books, doing katas, randoris and by the teams drawing conclusions out of their own experience. In other words, this is a temporary obstacle. Of course, together with staff rotation in your team comes the need to renew part of the investment in skills, knowledge and experience.

5. We don't have the necessary tools

This too is a valid impediment. TDD is a kind of process that can be made a lot easier with tools. There are many areas where tools can improve the overall performance with TDD. These include:

1. Code navigation
2. Running unit tests
3. Automated refactoring
4. Code Analysis
5. Quick fixes (like "generate method signature from its use)
6. Continuous Testing
7. Code generation

...etc.

If the issue is "money related" (e.g. there are tools on the market, but hey have to be bought), then the management should consider buying the best tools that are on the market and are available within the company budget. Thus, it's always good to have management buy-in for TDD. Thankfully, most of these tools (excluding the ones related to continuous testing) give a performance boost regardless of whether someone is using TDD or not, so many teams have such tools anyway.

If the issue is "technology related" (there are no tools on the market that work well with the technology your team is using), the good news is that TDD is usable even without these tools and still provides a lot of its benefits, since it is about analysis, design and specification. The tools only (well, "only" :-)) help in keeping focus and go faster through the whole cycle.

6. (This one is unit testing specific) There are already other kinds of tests, we don't need unit tests?

Although TDD is not solely limited to unit-level specs/tests, they form a crucial part of it. While some people believe that we can skip higher level specs in TDD, there's no one I know who says you can skip unit level.

Anyway, let's get to the point. this objection is based on a premise that TDD is about testing and "unit tests" produced with it are merely another kinds of tests. When I hear people raising this argument, they talk a lot about coverage, defect discovery, strengthening the testing net etc. Such arguments, for the most part, miss the point, since they ignore the biggest benefits of TDD which do not lie in its testing aspect (which I already discussed).

Dealing with such objections is really hard, because it requires others to accept a different point of view than the one they kept believing in so far. Sure, the books and authorities are on your side, but hey, the guys that read books and listen to authorities don't usually need convincing! So what to do in such case?

Remember, this argument is usually raised in teams that don't even do unit testing, let alone TDD. The following options are available:

1. Point the opponents to the blog posts and literature - be careful with this. If you do it wrong, the other side may take it as questioning their professionalism. Also, they may just question the authorities you believe in - this is rather easy in software engineering world - they can just say "that doesn't convince me at all" and the game is over. You have to somehow show that you're passionate about TDD and point at such sources as to something that made you passionate. In other words, if you want to lead someone out of their biases, tell them that you too were led out before them and tell them how. This leaves them in a position of "look, there's a way to do things better" instead of "you're all idiots and only I know the truth". I'm hardly an expert on this matter, however, there are some "change patterns" that are worth checking out.
2. Talk about your experiences from your previous projects if you have any or bring someone else if you don't. There are not many arguments as convincing as "I have experienced it myself", especially to the guys that don't have any experience in a certain field and are in a process of evaluating whether it makes sense to enter this field. Also, some people are more convinced by "soft" arguments ("we did it like this before and everybody in the team said that they feel in improvement in their coding style as never before"), while others are better convinced by "hard" arguments ("we did it like this before and we were able to get the best results of code complexity analysis in the whole project plus we were able to demonstrate a print out of documentation of 40 pages that just came out for free as we did TDD."). Also, better than words are living proofs ("you can just ask those guys I worked with and they show you what our living documentation looks like" or "Just ask the guys that were there with me on how they feel about it").
3. Create an internal training. This has one big advantage and two small disadvantages. The advantage is that you have a lot of time (at least longer than on regular meetings) to lead people by hand through your argumentation, reasons, examples and so on. In other words, you're given a chance to give more full explanation of the idea. The first disadvantage is that usually you get to prepare for such training in your free time (since management usually approves only spending time on giving the training, not on preparing it). The second disadvantage is that the people that you want to convince can simply ignore the invitation to the training and not attend at all or (if they're forced to attend) they can spend the whole training doing their stuff on their laptops.

7. It's very hard to perform with legacy code

The version of this objection that I find valid is: "It's harder to perform with legacy code". Why? Because TDD requires the code to have a quality called "testability". Legacy code doesn't usually have it. Thus, the code has to be refactored, risking breaking something that already works (if you know how to do this, then the risk is significantly lower, but there's still some risk). On the other hand, there is this temptation to just "insert a little if clause right in the middle of this mess and be done with".

Anyway, the strategy to deal with this objection is to somehow make is clear that it's either "we don't have the necessary skills" or "we don't have the necessary tools" objection, just dressed differently. In the first case, take a look at the following books:

1. Refactoring by Martin Fowler
2. Dealing Effectively With Legacy Code by Michael Feathers
3. Behead Your Legacy Beast. Refactor and Restructure Relentlessly With The Mikado Method (free e-book) by Daniel Brolund and Ola Ellnestam

and in the second case, talk to your boss and make sure that your team has the tools it needs.

Part 3 is already published! Go ahead and read it!

In the meantime, I'll be happy to hear your comments. I'm not an oracle and would gladly learn more about what objections do people receive and how they dispel them.

Test Driven Development - objections, part 1

Since you're reading this blog, you probably have your own, more or less informed, view on TDD. Maybe you already read a book or two, maybe you've got two (or twelve) years of TDD practice on your back, but maybe you've heard about TDD and about it being "cool" only recently and are merely striving to learn more? If that's the case you've probably got a lot of doubts on whether TDD is really beneficial or whether it will prove beneficial in the specific environment you happen to work in. You're eager to get started, but you wonder whether the time and effort spent on learning TDD will prove itself to be well spent.

During my adventure with TDD, I encountered many objections, either from myself (and they were dispelled by others) or from other engineers (these I tried to dispel myself). Today, I'd like to comment on some of those objections I met with when talking about TDD. In particular:

1. Not enough time
2. TDD will slow us down, because we'll have to create a lot of additional code
3. TDD will slow us down, because we'll have to maintain a lot of additional code
4. We don't have the necessary skills or experience
5. We don't have the necessary tools
6. (This one is unit testing specific) There are already other kinds of tests, we don't need unit tests?
7. It's very hard to perform with legacy code
8. I already know how to write a testable code, TDD will not really improve my design
9. We've got enough quality and we're doing fine without TDD
10. My manager won't let me do TDD
11. There's no "scientific proof" and enough research on whether TDD really provides a return of investment

This post is around the first three points, marked with strong text. The later points will be discussed in part 2 of this post. Ok, let's go!

1. Not enough time

This is my favorite objection, just because dispelling it is so fun and easy. I usually approach this kind of objections with Gojko Adzic's argument on how to solve Not Enough Time. You see, "not enough time" is always a facade reason - it is there only to hide the true one. The only direct solution to "not enough time" is to "make more time", which, by the way, is impossible. Luckily, we can restate it into something solvable like "something prevents me from allocating time for TDD". The further actions depend on what is this "something". Maybe it's a boss that will punish their employees for not fulfilling the short term goals? Maybe it's lack of appropriate knowledge or training? Anyway, these issues can be dealt with. "Not enough time" can't.

2. TDD will slow us down, because we'll have to create a lot of additional code

This is actually a (partially) valid argument. TDD really does lead to creating more code, which costs time. However, this does not necessarily mean that the overall development process takes more time than not writing this additional code. This is because this "test code" is not being written just for the sake of "being written" :-). The act of writing and the code that is created as a result of this writing provide a great value to a single developer and an entire team.

TDD aids developers in the process of analysis. What would otherwise be a mental effort to make everything up inside your head, turns into concrete failing or impossible-to-write code. This generates questions that are usually better asked sooner than later. Thanks to this, there's no escape from facing uncertainty.

The case is different (at least for me) when developing without TDD. In this approach, I discovered that I tend to write what I know first, leaving what I don't know for later (in hope that maybe I'll learn something new along the way that will answer my questions and lower uncertainty). While the process of learning is surely valuable, the uncertainty must be embraced instead of being avoided. Getting away from answering some key questions and leaving them for later generates a lot of rework, usually at the end of iteration, where it's very dangerous.

When I do TDD and I encounter a behavior that I don't know how to specify, I go talk with stakeholders ("Hey, what should happen when someone creates a subscription for magazines already issued?"). Also, If someone asks me to clarify my question, I can show them the spec/test I'm trying to write for the behavior and say "look, this does not work" or "so the output should be what?". This way, I can get many issues if not solved, then at least on the table much sooner than in case of non-TDD coding. It helps me eliminate rework, save some time and make the whole process more predictable.

TDD also provides a big help when designing. It lets you do your design outside-in, beginning with the domain and its work-flows, not the reusable, versatile, framework-like, one-size-fits-all utility objects that end up having only 30% of its logic ever used. This way, TDD lets you avoid over-design (by the way, code reuse is a good thing, it's just that preliminary generalization is as dangerous as preliminary optimization).

On the other hand, by promoting a quality called "testability", it promotes loose coupling, high cohesion and small, focused, well encapsulated objects. I already did some posts on that, so I'm not gonna delve more into this topic here. Anyway, striving for high testability helps avoid under-design.

Another way to think about the process of doing TDD is that you're actually documenting your design, its assumptions, legal and illegal behaviors. Others will be able to read it and learn from it when they face the task of using your abstractions in a new context.

3. TDD will slow us down, because we'll have to maintain a lot of additional code

It's true that, when done wrong, TDD produces a mass of code that is hard to maintain. Refactoring becomes a pain, each added functionality breaks dozens of existing specs/tests and the teams seriously consider abandoning the practice.

This happens more often when the teams do not adopt TDD, but rather stick with unit tests and do it only for the sake of testing.

The truth is that TDD, when done right, helps avoid such situations. Also, this help is actually one of its goals! To achieve this, however, you need two things.

The first one is knowing TDD good practices that help you write only the specs/tests that you need, focus on behavior and discover interactions between different objects, limiting an impact of a change to a small number of specs/tests. I actually didn't write about it yet on my blog, but there are some other sources of information. Anyway, this issue is easily solvable by a combination of training, mentoring, books and experience.

the second thing is "listening to the tests", covered both by Steve Freeman & Nat Pryce (they call it Synaesthesia) and Amir Kolsky & Scott Bain (they call it Test Reflexology). The big idea is that difficulties in writing and maintaining specs/tests are a very much desired feedback on quality of your design (also make sure to look at James Grenning's post on TDD as a design rot radar).

In other words, as long as the design is good and you know how to write tests/specs, this whole objection is not a problem. Of course, there is still a code to maintain, but I found it to be an easy task.

Another thing to keep in mind is that by maintaining specs/tests, you're actually maintaining a living documentation on several levels (because TDD is not solely limited to unit level). Just think how much effort it takes to keep an NDoc/JDoc/Doxygen documentation up to date - and you never actually know whether such documentation speaks the truth after a year of maintenance. Things get better with tests/specs, which can be compared with the code just by running them, so the maintenance is easier.

Part 2 is already written! Read it!

Also, feel free to leave your comment. How do you deal with these objections when you encounter them? Do you have any patterns you can share?

Reusing code in Test Data Builders in C#

This is a C# version of a post I did a while ago. Hope it's of any use.

Test Data Builders

Sometimes, when writing unit or acceptance tests, it's a good idea to use Test Data Builder. For example, let's take a network frame that has two fields - one for source, one for destination. A builder for such frame could look like this:

public class FrameBuilder
{
  private string _source;
  private string _destination;

  public FrameBuilder Source(string newSource)
  {
    _source = newSource;
    return this;
  }

  public FrameBuilder Destination(string newDestination)
  {
    _destination = newDestination;
    return this;
  }

  public Frame Build()
  {
    var frame = new Frame()
    {
      Source = _source,
      Destination = _destination,
    };
    return frame;
  }
}

and it can be used like this:

var frame = new FrameBuilder().Source("A").Destination("B").Build();

The issue with Test Data Builder method reuse

The pattern is fairly easy, but things get complicated when we have a whole family of frames, each sharing the same set of fields. If we wanted to write a separate builder for each frame, we'd end up duplicating a lot of code. So another idea is inheritance. However, taking the naive approach gets us into some trouble. Let's see it in action:

public abstract class FrameBuilder
{
  protected string _source;
  protected string _destination;
  
  public FrameBuilder Source(string newSource)
  {
    _source = newSource;
    return this;
  }
  
  public FrameBuilder Destination(string newDestination)
  {
    _destination = newDestination;
    return this;
  }
  
  public abstract Frame Build();
};

public class AuthorizationFrameBuilder : FrameBuilder
{
  private string _password;
  public AuthorizationFrameBuilder Password(string newPassword)
  {
    _password = newPassword;
    return this;
  }
  
  public override Frame Build()
  {
    var authorizationFrame = new AuthorizationFrame()
    {
      Source = _source,
      Destination = _destination,
      Password = _password,
    };
    return authorizationFrame;
  }
}

The difficulty with this approach is that all calls to FrameBuilder methods return a reference to FrameBuilder, not an AuthorizationFrameBuilder, so we cannot use calls from the latter after calls from the first. E.g. we cannot make a chain like this:

new AuthorizationFrameBuilder().Source("b").Password("a").Build();
  

This is because Source() method returns FrameBuilder, which doesn't include a method called Password() at all. Such chains cause compile errors.

Generics to the rescue!

Fortuntely, there's a solution for this. Generics! Yes, they can help us here, but in order to do this, we have to use a trick that in C++ world is called "Curiously Recurring Template Pattern" (don't know if it has any name in the C# world). By using the trick, we'll force the FrameBuilder (superclass) methods to return reference to its subclass (AuthorizationFrameBuilder) - this will allow us to mix methods from FrameBuilder and AuthorizationFrameBuilder in any order in a chain, because each method, not matter which of the classes it is defined in, returns a reference to a subclass.

Let's see how this works out in the following example:

//thankfully, the following is legal :-)
public class FrameBuilder<T> where T : FrameBuilder<T>
{
  protected string _source;
  protected string _destination;

  public T Source(string newSource)
  {
    _source = newSource;
    return this as T;
  }
  
  public T Destination(string newDestination)
  {
    _destination = newDestination;
    return this as T;
  }
}

public class AuthorizationFrameBuilder 
  : FrameBuilder<AuthorizationFrameBuilder>
{
  string _password;

  public AuthorizationFrameBuilder Password(string password)
  {
    _password = password;
    return this;
  }
  
  public AuthorizationFrame Build()
  {
    var frame = new AuthorizationFrame()
    {
      Source = _source,
      Destination = _destination,
      Password = _password,
    };
    return frame;
  }
}

Note that in FrameBuilder, the this pointer is cast to its generic type, which happens to be the sublass on which the methods are actually called. this cast is identical in every method of FrameBuilder, so it can be captured inside a separate method or property like this:

  T This 
  {
    get { return this as T; }
  }
    
  public T Source(string newSource)
  {
    _source = newSource;
    return This;
  }

Summary

This solution makes it easy to reuse any number of methods in any number of different builders, so it's a real treasure when we've got many data structures that happen to share some common fields.

That's all for today - Hope you enjoy the C#-specific version of the solution. By the way, in contrast to C++, C# allows one or two solutions other than chained methods to achieve the same data-building result. Can you name them?

Explaining TDD to high school students

Some time ago, I took on a challenge to try and prepare a short introduction to TDD that would make a good presentation for high school students (let's say 45 minutes). As TDD is hard to grasp and get used to even by experienced programmers, I thought that this should be really basic, just to put the point across.

I've pretty much prepared the flow of the presentation. Today I'd like to publish a draft of the presentation in hope to get some constructive feedback and make it even better.

Oh, by the way, I put content associated with each slide below it. Have fun!

Here we go!

Let's start with something obvious. In every activity, it's good to have a goal. Whether you're a race car driver, or a painter or a golf player, or even a tourist laying on the beach to get some sun and rest, you tend to have a goal. Without a goal, it's hard to determine things such as: when am I finished? How do I know that I failed? What can cause me to break the activity? How often do I need to repeat the activity? There are some other questions that can be derived from "having a goal".

We, software engineers, also need to have goals when writing software.

Let's take an exemplary student that is thinking about writing an application "from scratch". His goal may be to create an application that looks and works like Thunderbird e-mail client.

And so, he sets on a journey to make his dream come true. However, the goal is a distant one and so reaching it may be very difficult. This is because there's nothing along the road to tell him whether he's actually closer to or further away from the goal. It's almost like embarking on a journey with eyes closed. In software engineering world, we talk about a project having a high risk associated to it.

However, there's an easy way out of this inconvenience.

If we can split the path into several shorter ones, by choosing a subset of functionality to deliver first, we can arrive at the goal faster and with lower risk. It's like stating "ok, first I need to get to the Statue Of Liberty, and from there...".

Using this approach, we can slowly, but reliably arrive...

...at the full goal. This ability is, of course influenced by another factor - whether or not we have the ability not to mistakenly go back to the point of start. In other words, if our journey is composed of three points: A, B and C, we want to be sure whether from B, we're going to C, not back to A. In order to do this, we need some kind of "landmarks". In software engineering terms, we talk about "a way to confirm existing functionality and avoid introducing regression".

Thus, it makes sense to draw two conclusions.

The first one is that wherever we go and whatever software we write, we cannot take it "in one bite" - we have to split the overall goals into shorter goals to get feedback earlier.

Also, we want to be sure whether we really reached the goal or is it just our imagination. We'd like to be be able to tell whether all the goals are achieved, and if not, we'd like to know what's the next goal we must fulfill in order to get the work done.

Also, we don't want to come back and re-add the functionality we already put in. When we reach one goal, we want to get right to the next one and so on until the application is finished. If we can't completely prevent breaking what already works when adding new functionality, we want to at least know instantly when it happens to address it right away when it's a little trouble to do so.

Thus, we need the goals to be easily and reliably verifiable, so that the first time we arrive at the goal, we want to be sure that we really did. Later, we'd also want to re-verify these goals easily to make sure we didn't lose them accidentally while trying to achieve further goals.

These were the conclusions upon which TDD is built. Now let's see how it looks like in practice, taking on a naive example.

Let's imagine that we're trying to write a function that will raise a number to a given power. Also, let's keep the problem at the primary school level, because that's approximately where I left my math skills :-).

Anyway, we know how the signature would look like - we take one number, and another one and raise the first to the power of another. Now we need to state some goals that will help us arrive at the target.

We can come up with few examples of how properly implemented power function should behave. Each of these examples describes a more general rule. Such examples are called "Key Examples". The first one tells us what about a special case when we take anything and raise it to the power of 0. The second one describes a special case when we take anything and raise it to the power of 1. The third one illustrates the general rule that when we raise something to the power of N, we multiply it N times.

A set of key examples together with more general description forms a specification. Long gone are the times when the best way to write specification was to develop an abstract model of it. Nowadays, we want each rule implemented by the system to be illustrated by a concrete example. That's because...

...it's very easy to translate the speification made up this way into code, making it executable. This way, we're achieving the so-desired verifiability - if the specification is written as code that actually invokes the developed logic, we can reliably verify whether we really achieved the specified goals. Also, we can re-verify it later in no time, since code executes blazingly fast compared to a human that would need to read the specification and compare it with the actual code.

Ok, so let's take on the first statement of this specification and try to put it in the context.

In order to write and run specifications, we use special tools that provide some infrastructure upon which we can build. Such tools will automate many tasks for us, so we just need to write what we want and the tool will take care of gathering the specifications, executing them and reporting the result. For this example, we're gonna use a framework that's called XUnit.Net.

XUnit.Net allows us to state "facts" about the developed systems, by creating methods and marking them with [Fact] attribute and stating what is the expected behavior of the developed system. If the result is in sync with what we expect, the tool will mark such example as "passed" (usually using green color). If the result is not in sync, the tool will make the example as "failed" (usually using red color). Also, the specification needs a name. We try to name it after a description of the general rule illustrated by the example, so that we can easily come back to it later and read it as a "living documentation".

Now that we have prepared the necessary infrastructure, let's try adding some content to the example.

First, we state our assumptions. We often think about examples in executable specifications in terms of three sections: "Given" (assumptions), "When" (action), "Then" (desired result). We treat it as a general template for a behavior description. In this case, our assumption is that we have any number which happens to be 3 (but the name points that it can be any other number as well). We understand the code we just wrote as "given any number".

Now for the action, or "When". In our case, the action that we want to describe is taking something to the power of 0. Note that we use 0 explicitly and not give it any name. This is to stress that the described behavior takes place only when we use 0 here. This part should be understood as "When we raise it to the power of 0".

And now, the desired result, or "Then". Here, we state what should happen when the behavior is in place - in other words, what will make the example "passed" (green). In our case, we say "Then the result should be equal to 1". If this is not true, the example will be marked as "failed" (red).

Ok, let's quickly recap by trying to read the whole example. It reads like this:

Given any number
When I raise it to the power of 0
Then the result should be equal to 1

Our goal is to make this example "pass" (green) - when we it happens, the tool will display a message like the one on the slide. Note that the goal fulfills the criteria that we defined earlier. It is:

• short
• incremental - covers a well defined part of the functionality
• verifiable - we can compile it, run it and in a second, we'll get a response on whether this goal is achieved or not.

By the way, this goal is so far unfulfilled. We don't have any code to even get past the compilation stage...

So let's add some. Note that we're deliberately putting in an implementation that will make this example "failed" (displayed in red). This is to make sure that the goal is well-stated. One can, by mistake, make an example that will always "pass", and we want ourselves protected from this kind of errors. Thus, we make the first implementation as naive as possible just to compile it and watch it not fulfilling our current specification.

The example we have created seems state the goal well. As the system does not work the way this example describes, it shows "failure" (in red), meaning that the goal is not achieved yet. Our task is to achieve it.

Thankfully, this simple goal could be achieved by changing one character in the original implementation. This is just enough implementation to put the desired behavior in place. Done.

"Wait a minute" - you may say - "this isn't a proper implementation of power algorithm! It's cheating!". And you may give some examples where the current implementation won't work...

...like 2 to the power of 2. If you really said this, all of this would actually be correct, except for the "cheating" part :-).

That's because TDD process consists of small cycles, where we do provide the simplest implementation possible and expand it when we expand the specification with new examples.

This process is usually called "Red - Green - Refactor" and consists of three stages:

1. Red - named after the color that the tool for running executable specification shows you when the goal stated with an example is not achieved. We saw this when we made out Pow() method return 0 instead of expected 1.
2. Green - named after the color that the tool shows you when the goal stated with example is achieved by the current implementation. We saw this when we put in the correct implementation for "anything to the power of 0" scenario.
3. Refactor - After achieving the goal and making sure no previous goals were lost, it's a good moment to take a step back and look at the current design. Maybe the behaviors added on "one by one" basis can be generalized? Maybe something other can be cleaned up? In our case, no refactoring was needed, since there was hardly any design, however, in real-life scenarios, this is a crucial step.

When we finish the whole cycle, we take on another goal and do over until we run out of goals. That's the core of the TDD process.

Now's the time for a confession - you thought this presentation was about Test-Driven Development, but until now, I didn't even mention the word "test" - I was only talking about goals, specifications and examples. So where are the tests?

Ok, here's the thing: we use the examples to state our goals and verify their achievement up to the moment when the logic is in place. After this happens, we don't throw out these examples - they take on the role of micro-level tests that ensure all behaviors persist when we add more logic. These "tests" are a by-product of the TDD process.

The example we went through just now was a simplified case of a single function. As you surely know, the real-world projects, especially those object oriented ones, are not like this. They consist of a web of objects, collaborating together to achieve a task.

Most of these objects know about other objects and use them. In other words, object depend on other objects. Some say that object orientation is all about managing dependencies. How does TDD fit here?

Using examples, we can specify how an object should interact with its collaborators, even those that do not exist yet. "What?" - you may ask - "how am I supposed to create an example of an object that uses something which does not exist?". True, it's impossible per se, but there are ways to deal with that.

The trick is to develop fake objects that look like the real ones on the surface, but underneath, they allow us to set up their behavior, so that the specified object "thinks" different things happen in the system. It's like taking over all media in real life and broadcasting fake auditions about earthquake in New York - people in other countries, who know about current situation in New York from media only will believe the lies and act like they were real. Here, we wanna do the same thing - cheat an object about what happens in its surrounding to write examples on how it should behave.

In order to do it, we can use polymorphism. Let's take a look at two examples of such fake objects.

Sometimes, we want to state that the specified object should communicate something to its collaborators. Let's say we have a drive full of music and we want to show an example where our object makes an attempt to play the music. If we used the real drive, the music playing or not could result from many different conditions (like file privileges, format, whether there are actually any files on the drive etc.) which are out of scope of this example. To make things easier, we use a fake object, cheating our specified one into thinking that it's dealing with a real drive.

This is the second type, that allows us to set up a value returned by a method. Using this object, we can cheat the users of the drive into thinking that it's read only. If we used a real drive, we would probably have to invoke some complex logic to set it in this state. With a fake, we can just pre-program an answer that will be issued when the question gets asked.

The End

That's all I have for now, I'd be grateful for any feedback on this draft. Bye!

Reusing code in Test Data Builders in C++

Test Data Builders

Sometimes, when writing unit or acceptance tests, it's a good idea to use Test Data Builder. For example, let's take a network frame that has two fields - one for source, one for destination. A builder for such frame could look like this:

class FrameBuilder
{
protected:
  std::string _source;
  std::string _destination;
public:
  FrameBuilder& source(const std::string& newSource)
  {
    _source = newSource;
    return *this;
  }

  FrameBuilder& destination(const std::string& newDestination)
  {
    _destination = newDestination;
    return *this;
  }

  Frame build()
  {
    Frame frame;
    frame.source = _source;
    frame.destination = _destination;
    return frame;
  }
};

and it can be used like this:

auto frame = FrameBuilder().source("A").destination("B").build();

The issue with Test Data Builder method reuse

The pattern is fairly easy, but things get complicated when we have a whole family of frames, each sharing the same set of fields. If we wanted to write a separate builder for each frame, we'd end up duplicating a lot of code. So another idea is inheritance. However, taking the naive approach gets us into some trouble. Let's see it in action:

class FrameBuilder
{
protected:
  std::string _source;
  std::string _destination;
public:
  FrameBuilder& source(const std::string& newSource)
  {
    _source = newSource;
    return *this;
  }

  FrameBuilder& destination(const std::string& newDestination)
  {
    _destination = newDestination;
    return *this;
  }

  virtual Frame* build() = 0;
};

class AuthorizationFrameBuilder : public FrameBuilder
{
private:
  std::string _password;
public:
  AuthorizationFrameBuilder& password(const std::string& newPassword)
  {
    _password = newPassword;
    return *this;
  }

  Frame* build()
  {
    auto authorizationFrame = new AuthorizationFrame();
    authorizationFrame->source = _source;
    authorizationFrame->destination = _destination;
    authorizationFrame->password = _password;
    return authorizationFrame;
  }
}

Note that there are two difficulties with this approach:

1. We need the build() method to return a pointer, or we'll never be able to use methods from FrameBuilder in the chain (because each of the methods from FrameBuilder returns a reference to FrameBuilder, which only knows how to create frames, not how to create authorization frames). So, we'll need the polymorphism to be able to perform chains like:

   AuthorizationFrameBuilder().password("a").source("b").build()

2. Because FrameBuilder calls return a reference to FrameBuilder, not an AuthorizationFrameBuilder, we cannot use calls from the latter after calls from the first. E.g. we cannot make a chain like this:

   AuthorizationFrameBuilder().source("b").password("a").build()

   This is because source() method returns FrameBuilder, that doesn't include a method called password() at all. Such chains end up in compile errors.

Templates to the rescue!

Fortuntely, there's a solution for this. Templates! Yes, they can help us here, but in order to do this, we have to use the Curiously Recurring Template Pattern. This way we'll force the FrameBuilder methods to return reference to its subclass - this will allow us to mix methods from FrameBuilder and AuthorizationFrameBuilder in any order in a chain.

Here's an example code for the solution:

template<typename T> class FrameBuilder
{
protected:
  std::string _source;
  std::string _destination;
public:
  T& source(const std::string& newSource)
  {
    _source = newSource;
    return *(reinterpret_cast<T*>(this));
  }

  T& destination(const std::string& newDestination)
  {
    _destination = newDestination;
    return *(reinterpret_cast<T*>(this));
  }
};

class AuthorizationFrameBuilder 
: public FrameBuilder<AuthorizationFrameBuilder>
{
private:
  std::string _password;
public:
  AuthorizationFrameBuilder& password(const std::string& password)
  {
    _password = password;
    return *this;
  }

  AuthorizationFrame build()
  {
    AuthorizationFrame frame;
    frame.source = _source;
    frame.destination = _destination;
    frame.password = _password;
    return frame;
  }
};

Note that in FrameBuilder, the this pointer is cast to its template type, which happens to be the sublass on which the methods are actually called. this cast is identical in every method of FrameBuilder, so it can be turned into a separate method like this:

  T& thisInstance()
  {
    return *(reinterpret_cast<T*>(this));
  }

  T& source(const std::string& newSource)
  {
    _source = newSource;
    return thisInstance();
  }

Summary

This solution makes it easy to reuse any number of methods in any number of different builders, so it's a real treasure when we've got many data structures that happen to share some common fields.

That's all for today - if you'd like to, please use the comments section to share your solution to this problem for other programming languages.

Bye!

Don't use setup and teardown, or I will...

...write a blog post.

There - I did it. I told you I would!

This time, I'm going to share some of my reasons why I tend not to use Setup and Teardown mechanism at all.

First, a disclaimer - my point here is NOT that Setup and Teardown lead to inevitable catastrophe. My point here is that Setup and Teardown are so misunderstood and so easy to abuse, that I'd rather not use them at all. There are some other reasons why I actually prefer not having Setups and Teardowns even when they are used properly, but I'll save this for another post. This time, I'd like to focus only on the ways this mechanism is most often abused.

What's Setup and Teardown?

As everyone knows, a Setup method is a special kind of method that is executed by unit testing tools before each unit test in the suite. Such methods are commonly used to set the stage before a unit test begins. Analogously, "Teardown" method is a method that is always run after unit test execution and is usually used to perform cleanup after the test finishes. So, given this code (example uses NUnit):

[Setup]
public void Setup()
{
  Console.WriteLine("SETUP");
}

[Teardown]
public void Setup()
{
  Console.WriteLine("TEARDOWN");
}

[Test]
public void Test2()
{
  Console.WriteLine("TEST");
}

[Test]
public void Test1()
{
  Console.WriteLine("TEST");
}

... when it is run by a unit testing tool, it produces the following output:

SETUP
TEST
TEARDOWN
SETUP
TEST
TEARDOWN

While having the common logic for "setting the stage" and "cleaning up" in test suite looks like adhering to DRY, avoiding duplication etc., there are, however, certain dangers when using this kind of mechanism, some of which I'd like to list below.

Issues with Setup and Teardown

Because we're talking mostly about C#, we're mainly going to examine Setup, because Teardown is seldom used for unit tests in such languages. By the way, the examples provided are to explain what kind of abuse I have in mind. I tried to keep them simple - this way they're more understandable, but do not show how bad can it get with real code and real project - you'll have to believe :-). Let's go!

Joint Fixtures

Imagine that someone has a set of unit tests evolving around the same setup (let's call it "A"):

[Setup]
public void SetUp()
{
  emptyList = new List<int>(); //A
}

[Test] //A
public void ShouldHaveElementCountOf0AfterBeingCreated() 
{
 Assert.AreEqual(0, emptyList.Count());
}

[Test] //A
public void ShouldBeEmptyAfterBeingCreated()
{
  Assert.True(emptyList.IsEmpty());
}

One day, another set of unit tests must be added for the same class that requires the setup to be handled a little bit differently (let's call it "B"). What this person might do is to just add the necessary setup besides the first one.

[Setup]
public void SetUp()
{
  emptyList = new List<int>(); //A
  listWithElement = new List<int>() { anyNumber }; //B
}

[Test] //A
public void ShouldHaveElementCountOf0AfterBeingCreated() 
{
 Assert.AreEqual(0, emptyList.Count());
}

[Test] //A
public void ShouldBeEmptyAfterBeingCreated()
{
  Assert.True(emptyList.IsEmpty());
}

[Test] //B
public void ShouldNotContainElementRemovedFromIt()
{
  listWithElement.Remove(anyNumber);
  Assert.False(listWithElement.Contains(anyNumber));
}

[Test] //B
public void ShouldIncrementElementCountEachTimeTheSameElementIsAdded()
{
  var previousCount = listWithElement.Count();
  listWithElement.Add(anyNumber);
  Assert.AreEqual(previousCount + 1, listWithElement.Count());
}

The downside is that another person striving to understand one unit test will have to read both through the related setup and the unrelated one. And in real life scenarios such orthogonal setups tend to be longer that in this toy example.

Of course, this may be fixed by separating this class into two - each having its own setup. There are, however, two issues with this: one is that this is almost never done by novices, and the second is that it usually complicates navigation through the unit tests.

Locally Corrected Fixtures

Another temptation a novice may face is when one day they need an object that's slightly different than the one already set up in the Setup. The temptation is to, instead of create new object configured separately, undo some of the changes made by the Setup just for this single test. Let's see a simplified example of a situation where one needs a radio set up slightly differently each time:

[Setup]
public void Setup()
{
  radio = new HandRadio();
  radio.SetFrequency(100);
  radio.TurnOn();
  radio.SetToSecureModeTo(true);
}

[Test]
public void ShouldDecryptReceivedContentWhenInSecureMode()
{
  var decryptedContent = radio.Receive(encryptedContent);
  Assert.AreEqual("I love my wife", decryptedContent);
}

[Test]
public void ShouldThrowExceptionWhenTryingToSendWhileItIsNotTurnedOn()
{
  radio.TurnOff(); //undo turning on!!

  Assert.Throws<Exception>(() =>
    radio.Send(Any.String())
  );
}

[Test]
public void ShouldTransferDataLiterallyWhenReceivingInNonSecureMode()
{
  radio.SetSecureModeTo(false); //undo secure mode setting!!

  var inputSignal = Any.String();
  var receivedSignal = radio.Receive(inputSignal);

  Assert.AreEqual(inputSignal, receivedSignal);
}

Overloaded fixtures

Let's stick with the hand radio example from previous section. Consider the following unit test:

[Test]
public void ShouldAllowGettingFrequencyThatWasSet()
{
  radio.SetFrequency(220);
  Assert.AreEqual(220, radio.GetFrequency());
}

Ok, looks good - we specify that we should be able to read the frequency that we set on the radio, but... note that the radio is turned on in the Setup, so this is actually getting a frequency from a radio that's turned on. What about if it's turned off? Does it work the same? In the real world, some radios are analog and they allow manual regulation of frequency with a knob. On the other hand, some radios are digital - you can set their parameters only after you turn them on and they become active. Which case is it this time? We don't know.

Ok, I actually lied a little. In fact, if someone was doing TDD, we can assume that if the scenario was different when a radio is off, another unit test would exist to document that. But in order to determine that, we have to: 1) look through all the unit tests written for this class, and 2) really, really believe that the author(s) developed all the features test-first. An alternative is to look at code coverage or at the code itself. However, all these ways of dealing with the issue require some examination that we have to perform. In such case, the Setup gets in the way of understanding a minimum activities required for a functionality to work as described.

Scope hiding

It's quite common to see unit tests grow too big in scope as the design gets worse and worse and it's harder and harder to separate different bits and pieces for unit testing. Many people think that Setup and Teardown are THE feature to deal with this. I'm sorry, they are not. If you have issues with your design, the right way to go is to SOLVE them, not to HIDE them. Unfortunately, this kind of Setup abuse tends to grow into multi-level inherited Setups (do class names like "TestBase" ring a bell?), where no one really knows anymore what's going on.

Evading already used values

Consider the following example of a user registration process that may take place in various situations (e.g. creating sessions, presidential election etc.):

[Setup]
public void Setup()
{
  registration = new UserRegistration();
  registration.PerformFor("user1");
  registration.PerformFor("user2");
}

[Test]
public void ShouldBeActiveForRegisteredUsers()
{
  Assert.True(registration.IsActiveFor("user1"));
}

[Test]
public void ShouldBeInactiveForUnregisteredUsers()
{
  //cannot be "user1" or "user2" or it fails!
  Assert.False(registration.IsActiveFor("unregisteredUser"));
}

Why "unregisteredUser" was chosen as a value for unregistered user? That's because someone writing a unit test wanted to evade the values used in Setup. While that's not as bad when reading those unit tests, it's a pain when writing new ones - in order to avoid conflict, you have to always look back at what users are already registered and either use the same values or deliberately pick different ones. Thus, writing every new test begins with trying to understand what the Setup already does and trying to get around that. What's worse, when putting another value in the Setup, we have to go through all existing unit tests to make sure that we pick a value different that already used in any of them. This hurts maintainability.

Why do I tell teams to avoid Setup and Teardown

As Roy Osherove once told me:

I don't use my frameworks to teach me design.

and while it was in a different context, I can paraphrase it into saying that avoiding using a feature instead of learning how not to misuse it is not something that will teach you good design and TDD.

Why then do I insist that people new to unit testing hold back from using features such as Setup and Teardown? How is it helpful?

In my opinion, holding back from using Setup and Teardown exposes us to a diagnostic pain - when we feel the pain and we cannot work around it, we have to learn to avoid it. It's never sufficient to say "don't use Setup and Teardown" or put in any other rule (e.g. "no getters") without following up on experience from applying this rule. When the team gets into difficulties, there has to be someone to explain what's the root cause. Only then do such rules make sense. If not for this, the team would just find another workaround, e.g. with helper methods (which are better than Setup and Teardown in that you always get to choose every time which ones to use and which not, but can also be abused). Thus, when I say "no Setup/Teardown" to a team, I always wait for a reaction like "you're nuts!" - through such experiences, I'm preparing them to acknowledge that TDD is something different that they initially thought (which is usually something like "execute a lot of production code" and "get a high coverage").

How about experienced developers? Do they need to adhere to the same rule? Well, let me share my experience. I told myself once - if I ever run into a situation where it makes perfect sense to use Setup and Teardown, I will not hold myself back. Guess what - since then, I used it only once (because one technique of dependency injection required it) and I was regretting it later. Other than this, I never really felt the need to use this feature, since I was doing even better without it. There are some other advantages (like keeping each unit test a "one part story") that make me never want to go back. Where I work, we've got over a thousand of unit tests and no Setup and Teardown at all.

There are actually more smells and difficulties associated with Setup and Teardown (like impeding certain refactorings) and I could also write something more about Setup and Teardown vs helper methods, but everything has to have an end, doesn't it? Till the next time!

Starting off with unit testing and TDD? Here's my advice...

Thanks to Mark Mishaev for inspiration.

Sometimes, I hear that someone is trying to start off with unit testing or TDD and they ask me for some good advice on where to start - some general guidelines that would help them along the way. I thought I'd summarize my usual answer here for everyone to benefit from.

So, if you're starting off with unit testing or TDD, here's my advice:

1. Read at least these blogs to grasp some good practices:
   • Sustainable Test Driven Development by Amir Kolsky and Scott Bain
   • My own blog
   • There are some other good ones, just look around, I recommend blogs by Steeve Freeman, Roy Osherove, Dan North, Liz Keogh, Robert C. Martin, James W. Grenning, Kent Beck, Mark Seemann, James Newkirk...
2. Pick up some books on the subject. The topic of unit testing and TDD might seem straightforward (there are a lot of simplifications around, such as "unit test is just a small code that creates a class, invokes a method and performs assertions on its output" and "TDD means you write your test first, then write the code"), but it actually requires quite a bit of knowledge to get things right. Also, don't limit yourself to a single book - read at least three or four - many TDD gurus have more or less different styles of doing TDD and writing unit tests and your own technique will get more and more flexible as you manage to understand more of them.
3. Try writing your tests first. For a list of benefits, take a look at:
   1. The importance of test failure
   2. Test First - why is it so important?
4. For new code, try to follow Need Driven Development
5. To drive your code and design, try exposing yourself to “diagnostic pain”. This means putting some limitations onto yourself and when you’re tempted to break them, it means that the code or design is at fault and you should refactor. To give you a quick example, I do not use Setup and TearDown kinds of methods in unit tests (not mentioning Action Attributes from NUnit), which means all the objects used in the unit test are created in the body of the test method itself. I sometimes use helper methods when they improve readability and understandability of the test (but not to hide redundancy - that's the catch). At first it might look awkward, but I have created a lot of unit tests without Setup and Teardown and I'm very happy with the result. So it’s not like I don’t know how to use a unit testing framework - I deliberately hold back from using many of its features
6. Ideally, a maintainable unit test has to fulfill three conditions, as pointed by Scott Bain:
   1. It fails reliably when behavior specified by it is broken
   2. It fails only when behavior specified by it is broken (so breaking another behavior should not break this test)
   3. No other unit tests will fail when behavior described by this test is broken
   This is the ideal - try to keep as close to it as possible.
7. Try using a technique called Constrained Non-Determinism. You can either use tools that help with this (like AutoFixture for C#) directly, or you can wrap it in your own class (if you're reading my blog, you probably know that I like to wrap creating anonymous values in a class called Any). Alternatively, just roll out your own mini library or a set of functions.
8. Use continuous testing tools, (examples for .NET include Mighty Moose (free) or NCrunch (paid)) for running your unit tests. If no such solution exists for your programming language of choice, you can always write a script that performs continuous compilation and running unit tests and lets you know of the results. There are some tools like Watchr that can make writing such scripts easier (especially when integrated with your operating system's default user notification mechanism).
9. Always specify class behaviors (not methods or classes) with unit tests – to help you with this, try using the following convention of naming your tests: ShouldXYZ() where XYZ is the description of the behavior the class should supply. The convention is taken from Dan North’s post and helps to avoid:
   1. Check-it-all unit tests – where one does a single setup and verifies multiple behaviors.
   2. Names that do not make sense. I saw one a unit test named: SendFrameMethodSendsFrame() which tells almost nothing about the behavior. The “Should” naming forces us to rethink the name and come up with something like: ShouldPassReceivedValidFrameThroughTheMessageQueue() which works better.
   3. Also, pick names that read well when converted from "ThisNamingConvention" (or "This_naming_convention" - whatever you choose to apply to your unit test methods) to "this naming convention" – it lets you use the unit test results report in Cruise Control or Jenkins as a living documentation of your code after you apply a little post-procesing to it.
   4. You can look at my blog post that discusses the repercussions of bad naming.
10. When choosing a mocking framework for your project, put your money on ease of use and readability. For example, many mocking frameworks for C# use lambda expressions everywhere, but many programmers are still not used to thinking in terms of expressions and are a bit confused. When developers find using such framework awkward, they're more likely to reject the idea of unit testing or TDD. Thus, as a main mocking framework, I like to choose NSubstitute.
11. Always keep handy a "fallback mock framework" - there are some features that your main mocking framework of choice does not support or that are difficult to achieve using this framework. For example, NSubstitute does not yet support partial mocks. That's why it's good to have a secondary mock framework of choice that can make up for the main one's weaknesses in some rare cases. For such a fallback mock framework, I like to use Moq, but that may as well be FakeItEasy or Rhino Mocks or something else - depending on what you need.
12. Try to stay as close as possible to SOLID principles and adhere to the Law Of Demeter (which is a conclusion from those principles, specifically - from Open Closed principle) – all of this lets you write better, more focused and more maintainable unit tests. For a list of smells that show the problem is in the design rather than in tests, see:
    1. Test Reflexology - part 1
    2. Test Reflexology part 2
    3. TDD and Single Responsibility Principle
    4. Mocking method chains, part 1: when not to
    5. A kata challenge to try your TDD skills on - failure and what we can learn from it
13. Watch out for unit tests execution times – unit tests that run for more than 2-3 seconds (to tell you the truth, many should even run in less than one second), almost always have a smell attached to them – either they touch external resources or are not unit tests. Staying away from external resources may be tricky. For one such example where this actually does get tricky and one may be tempted to break the isolation, see my example related to timers
14. Avoid the "The code is so straightforward that there’s no need to test it" thinking. When the code is complex, the solution is refactoring, not unit-testing (of course, tests can be written, but they have to be used as coverings). Unit tests are about specification, not coverage, and short specifications are good.
15. Perform a kata or two on your own. One such kata and its exemplary solution are posted on this blog
16. There may be a temptation to write more coarse-grained tests/specifications that are tied to the code (they’re sometimes called white-box tests). This is possible, however, I recommend not to do it if it’s not very well thought. I’ve been in projects where coarse-grained, so called “white-box tests” led to situations where correcting and debugging the tests took twice as long as changing the code. Again, it’s possible to write good coarse-grained tests, it just that it’s hard – it requires reading some books and doing few dry runs to grasp many best practices that are different than in case of unit tests. On short discussion of how things may go wrong, be sure to look at my post: Perfect from the start.
17. Also, there may be a temptation to "test private methods". This is a sign that things are going a little bit offroad.
18. Attaining X% code coverage should not be a goal. High coverage is good when it’s a side effect.

I could go on writing, but I think these are the most important, non-obvious things I learned throughout my adventure with TDD. As always, feel free to add your advice in the comments.

See ya!