Moving to NUnit from MsTest - experience report

Recently, I was a part of migration of some tests from MsTest to NUnit. I was in a situation where some of the tests were written in NUnit and some (legacy ones) in MsTest and it was too costly to maintain both tools, so the money were put on NUnit. The goal was to convert existing tests as fast as possible, leaving refactoring and cleanup of test and production code for later. I didn't know much about MsTest before, and here's what I learned about it and the transition:

Assertions

Much to my astonishment, the number of assertions MsTest supports is rather small, compared to NUnit. You cannot assert something is greater than something else, cannot assert a value is in a certain range etc. This way, a lot of assertions looked like this (remember this was ugly legacy test code :-)):

Assert.True(numberOfMessages <= 1000 
  && numberOfMessages >= 950, "numberOfMessages")

This, at failure, gives you the a message like: "Expected true, got false". Pretty helpful, isn't it (sarcasm)? Sure, you can make the message better, but you have to work hard on it yourself. The NUnit's (better) equivalent is:

Assert.That(numberOfMessages, Is.InRange(950, 1000));

Usually, when writing good unit tests, I don't really need this kind of assertions, but this was legacy code and it tested many kinds of weird situations over a large set of objects put together.

On the other hand, MsTest has a strongly typed assertions in form of Assert.AreEqual<T>() which are missing from NUnit. This, however, is not a big deal, because creating a custom wrapping assertions which will give you this benefit in NUnit is trivial. For example:

public class XAssert
{
  public static void AreEqual<T>(T expected, T actual)
  {
    NUnit.Framework.Assert.AreEqual(expected, actual);
  }
}

Lessons learned:

1. If you ever use MsTest, do yourself a favor and pick an external assertion library.
2. If you use NUnit, writing strongly typed wrappers over common assertions may be a good idea.

Deployment Items in .testsettings file

MsTest has this little feature of deployment items that can be configured in solution-wide .testsettings file to be copied to the output directory before a test is run (usually these are some extra files etc.).

But wait! Why on earth would someone be willing to use that kind of feature in unit tests at all?

Unfortunately, this was my case - the deployment items were used to load an XML configuration :-(. The problem with Deployment Items is that some 3rd party runners support it, and some don't (namely: NCrunch and Gallio, which makes running tools like Sonar over such tests pretty difficult). Also, some 3rd party unit testing tools do not run tests from the build output directory, but create their owne custom directories somewhere on the hard drive (e.g. NCrunch does), but the paths coded in Deployment Items are not adjusted and get gopied to the same directories as always, leading to failed runs.

Lessons learned:

1. If you're using Deployment Items in MsTest for unit tests, you don't even give me a chance to think you're serious.
2. When Deployment Items are used to copy some additional files to the build output, this mechanism can be replaced with including these config files in test project as links and setting their "Copy Local" property to copy them to the build output directory.
3. Do not write or load files in unit tests!!!

PrivateType and PrivateObject

I was shocked when I discovered that MsTest includes helpers for accessing private instance/static methods: PrivateType and PrivateObject. Even for legacy code, there are better strategies to apply. Anyway, what shocks me more is that anyone can even think that using these may be a good idea (it is not). Thus, when converting to NUnit, I found such constructs in the code. They lead to brittle tests that are accessing members using strings and reflection. It seems like MsTest allows generating accessors that are strongly typed and use strings and reflection under the hood, but this is still coupling to implementation details, plus I saw PrivateType and PrivateObject being used directly many times.

Lessons Learned:

1. When writing new tests, forget that PrivateType and PrivateObject exist. Look for the real problem.
2. PrivateType and PrivateObject are just helpers over a reflection API, so they don't require being executed under MsTest to work properly. During transition to NUnit, you can leave references to MsTest assembly and change all code references to include namespace e.g. from PrivateObject to Microsoft.VisualStudio.TestTools.UnitTesting.PrivateObject. The code will still work.

Cost of Transition

In general, it's a lot simpler to migrate from MsTest to NUnit than the other way round. This is because NUnit requires only that an assembly references nunit.framework.dll and marks certain methods with [Test] attribute. On the other hand, MsTest requires a special type of project (a Test Project) to hold the tests. Also, as I said, MsTest's assertions are for the most part a subset of NUnit's. Anyway, to convert a project from MsTest to NUnit, the following steps are usually sufficient:

1. Reference nunit.framework.dll from the test project.
2. Convert Deployment Items into links in project copied to the build output folder.
3. Search-Replace the following:
   1. [TestClass] -> [TestFixture]
   2. [TestMethod] -> [Test]
   3. [TestInitialize] -> [SetUp]
   4. [TestCleanup] -> [TearDown]
   5. [TestClassInitialize] -> [TestFixtureSetUp]
   6. [TestClassCleanup] -> [TestFixtureTearDown]
   7. using Microsoft.VisualStudio.TestTools.UnitTesting; -> using NUnit.Framework;
4. Change Assert.IsInstanceOfType(x, typeof(X)) to Assert.IsInstanceOf<X>(x);
5. Change Assert.AreEqual<X>(x, y) to Assert.AreEqual(x, y)
6. Fix references to some MsTest-specific classes (like PrivateObject)

Summary

And that's it. These were the lessons learned and tips for dealing with the transition as fast as possible. I advise to do a real cleanup of the test and production code sooner than later, but the tips I outlined here let you defer this cost.

Developing TDD style by example

(this post is adapted from my work-in-progress open source TDD tutorial)

(Note that I use a term "Statement" instead of "test" and "Specification" instead of "Test Suite" in this post)

Throughout this blog, you have seen me using quite extensively a utility class called Any. The time has come to explain a little bit more carefully what principles lie under this technique and tool. I'll also use this technique as a case study to show you how one develops a style of Test-Driven Development.

A style?

Yep. Why am I wasting your time writing about style instead of giving you the hardcore technical details? The answer is simple. Before I started writing this tutorial, I read four or five books solely on TDD and maybe two others that contain chapters on TDD. All of this sums up to about two or three thousands of paper pages, plus numerous posts on many blogs. And you know what I noticed? No two authors use exactly the same sets of techniques for test-driving their code! I mean, sometimes, when you look techniques they're suggesting, the suggestions from two authorities contradict each other. As each authority has their followers, it's not uncommon to observe and take part in discussions about whether this or that technique is better than a competing one or which one leads to trouble in the long run.

I did this, too. I also tried to understand how come people praise techniques I KNEW were wrong and led to disaster. Then, Finally, I got it. I understood that it's not a "technique A vs. technique B" debate. There are certain sets of techniques that work together and choosing one technique leaves us with issues we have to resolve by adopting other techniques. This is how a style is created.

Developing a style starts with underlying set of principles. These principles lead us to adopt our first technique, which makes us adopt another one and, ultimately, a coherent style emerges. Using Constrained Non-Determinism as an example, I'll try to show you how part of a style gets derived from a technique that is derived from a principle.

Principle: Tests As Specification

As I already stressed, I strongly believe that unit tests constitute an executable specification. Thus, they should not only pass input values to an object and assert on the output, they should also convey to their reader the rules according to which objects and functions work. The following oversimplified example shows a Statement where it's not explicitly stated what's the relationship between input and output:

[Fact] public void 
ShouldCreateBackupFileNameContainingPassedHostName()
{
  //GIVEN
  var fileNamePattern = new BackupFileNamePattern();
  
  //WHEN
  var name = fileNamePattern.ApplyTo("MY_HOST_NAME");
  
  //THEN
  Assert.Equal("backup_MY_HOST_NAME.zip");
}

Although the relationship can be guessed quite easily, remember it's just an example. Also, seeing code like that makes me ask questions like: is the "backup_" prefix always applied? What if I actually pass "backup_" instead of "MY_HOST_NAME"? Will the name be "backup_backup_.zip", or "backup_.zip"? Also, is this object responsible for any validation of passed string?

This makes me invent a first technique to provide my Statements with better support for the principle I believe in.

First technique: Anonymous Input

I can wrap the actual value "MY_HOST_NAME" with a method and give it a name that better documents the constraints imposed on it by the specified functionality. In our case, we can pass whatever string we want (the object is not responsible for input validation), so we'll name our method AnyString():

[Fact] public void 
ShouldCreateBackupFileNameContainingPassedHostName()
{
  //GIVEN
  var hostName = AnyString();
  var fileNamePattern = new BackupFileNamePattern();
  
  //WHEN
  var name = fileNamePattern.ApplyTo(hostName);
  
  //THEN
  Assert.Equal("backup_MY_HOST_NAME.zip");
}

public string AnyString()
{
  return "MY_HOST_NAME";
}

By using anonymous input, we provide a better documentation of the input value. Here, I wrote AnyString(), but of course, there can be a situation where I use more constrained data, e.g. AnyAlphaNumericString() when I need a string that does not contain any characters other than letters and digits. Note that this technique is applicable only when the particular value of the variable is not important, but rather its "trait". Taking authorization as an example, when a certain behavior occurs only when the input value is Users.Admin, there's no sense making it anonymous. On the other hand, for a behavior that occurs for all values other than Users.Admin, it makes sense to use a method like AnyUserOtherThan(Users.Admin) or even AnyNonAdminUser().

Now that the Statement itself is freed from the knowledge of the concrete value of hostName variable, the concrete value of "backup_MY_HOST_NAME.zip" looks kinda weird. There's no clear indication of the kind of relationship between input and output and whether there is any at all (one may reason whether the output is always the same string or maybe it depends on the string length). It is unclear which part is added by the production code and which part depends on the input we pass to the method. This leads us to another technique.

Second Technique: Derived Values

To better document the relationship between input and output, we have to simply derive the expected value we assert on from the input value. Here's the same Statement with the assertion changed:

[Fact] public void 
ShouldCreateBackupFileNameContainingPassedHostName()
{
  //GIVEN
  var hostName = AnyString();
  var fileNamePattern = new BackupFileNamePattern();
  
  //WHEN
  var name = fileNamePattern.ApplyTo(hostName);
  
  //THEN
  Assert.Equal(string.Format("backup_{0}.zip", hostName);
}
public string AnyString()
{
  return "MY_HOST_NAME";
}

This looks more like a part of specification, because we're documenting the format of the backup file name and show which part of the format is variable and which part is fixed. This is something you'd probably find documented in a paper specification for the application you're writing - it would probably contain a sentence saying: "The format of a backup file should be backup_H.zip, where H is the current local host name".

Derived values are about defining expected output in terms of the input that was passed to provide a clear indication on what is the "transformation" of the input required of the specified production code.

Third technique: Distinct Generated Values

Let's assume that some time after our initial version is shipped, we're asked to make the backup feature applied locally per user only for this user's data. As the customer doesn't want to confuse files from different users, we're asked to add name of the user doing backup to the backup file name. Thus, the new format is "backup_H_U.zip", where H is still the host name and U is the user name. Our Statement for the pattern must change as well to include this information. Of course, we're trying to use the anonymous input again as a proven technique and we end up with:

[Fact] public void 
ShouldCreateBackupFileNameContainingPassedHostNameAndUserName()
{
  //GIVEN
  var hostName = AnyString();
  var userName = AnyString();
  var fileNamePattern = new BackupFileNamePattern();
  
  //WHEN
  var name = fileNamePattern.ApplyTo(hostName, userName);
  
  //THEN
  Assert.Equal(string.Format(
    "backup_{0}_{1}.zip", hostName, userName);
}

public string AnyString()
{
  return "MY_HOST_NAME";
}

Now, we can clearly see that there is something wrong with this Statement. AnyString() is used twice and each time it returns the same value, which means that evaluating the Statement does not give us any guarantee, that both values are applied and that they're applied in the correct places. For example, the Statement will be evaluated to true when user name is used instead of host name in specified production code. This means that if we still want to use the anonymous input effectively, we have to make the two values distinct, e.g. like this:

[Fact] public void 
ShouldCreateBackupFileNameContainingPassedHostNameAndUserName()
{
  //GIVEN
  var hostName = AnyString();
  var userName = AnyString2();
  var fileNamePattern = new BackupFileNamePattern();
  
  //WHEN
  var name = fileNamePattern.ApplyTo(hostName, userName);
  
  //THEN
  Assert.Equal(string.Format(
    "backup_{0}_{1}.zip", hostName, userName);
}

public string AnyString()
{
  return "MY_HOST_NAME";
}

public string AnyString2()
{
  return "MY_USER_NAME";
}

We solved the problem (for now) by introducing another helper method. However, this, as you can see, isn't a very scalable solution. Thus, let's try to reduce the amount of helper methods for string generation to one and make it return a different value each time:

[Fact] public void 
ShouldCreateBackupFileNameContainingPassedHostNameAndUserName()
{
  //GIVEN
  var hostName = AnyString();
  var userName = AnyString();
  var fileNamePattern = new BackupFileNamePattern();
  
  //WHEN
  var name = fileNamePattern.ApplyTo(hostName, userName);
  
  //THEN
  Assert.Equal(string.Format(
    "backup_{0}_{1}.zip", hostName, userName);
}

public string AnyString()
{
  return Guid.NewGuid.ToString();
}

This time, we're not returning an understandable string, but rather a guid, which gives us the fairly strong guarantee of generating distinct value each time. The string not being understandable (contrary to something like "MY_HOST_NAME") may leave you worried that maybe we're losing something, but hey, didn't we say AnyString()?

Distinct generated values means that each time we need a value of a particular type, we get something different (if possible) than the last time and each value is generated automatically using some kind of heuristics.

Fourth technique: Constant Specification

Let us consider another modification that we're requested to make - this time, the backup file name needs to contain version number of our application as well. Remembering that we want to use Derived Values, we won't hardcode the version number into our Statement. Instead, we're going to use a constant that's already defined somewhere else in the application (this way we also avoid duplication of this version number across the application):

[Fact] public void 
ShouldCreateBackupFileNameContainingPassedHostNameAndUserNameAndVersion()
{
  //GIVEN
  var hostName = AnyString();
  var userName = AnyString();
  var fileNamePattern = new BackupFileNamePattern();
  
  //WHEN
  var name = fileNamePattern.ApplyTo(hostName, userName);
  
  //THEN
  Assert.Equal(string.Format(
    "backup_{0}_{1}_{2}.zip", 
    hostName, userName, Version.Number);
}

public string AnyString()
{
  return Guid.NewGuid.ToString();
}

Note that I didn't use the literal constant value, but rather, the value inside the Version.Number constant. This allows us to use derived value, but leaves us a little worried about whether the value of the constant is correct - after all, we're using it for creation of our expected value, but it's a part of production code - i.e. is something that should be specified itself!

To keep everyone happy, we write a single Statement just for the constant to specify what the value should be:

[Fact] public void 
ShouldContainNumberEqualTo1_0()
{
  Assert.Equal("1.0", Version.Number);
}

By doing so, we make the value in the production code just echo what's in our executable Specification, which we can fully trust.

Summary of the example

In this example, I tried to show you how a style can evolve from the principles you value when doing TDD. I did so for two reasons:

1. To introduce to you a set of techniques I personally use and recommend and to do it in a fluent and logical way.
2. To help you better communicate with people that are using different styles. Instead of just throwing "you're doing it wrong" at them, try to understand their principles and how their techniques of choice support those principles.

Now, let's take a quick summary of all the techniques introduced in example:

Anonymous Input

moving the output out of the Statement code and hide it behind a method that to emphasize the constrain on the data used rather than what's its value

Derived Values

defining expected output in terms of the input in order to document the relationship between input and output

Distinct Generated Values

When using Anonymous Input, generate a distinct value each time (in case of types that have very few values, like boolean, try at least not to generate the same value twice in a row) in order to make the Statement more reliable.

Constant Specification

Write a separate Statement for a constant and use the constant instead of its literal value in all other Statements to create a Derived Value.

Constrained non-determinism

When we combine anonymous input together with distinct generated values, we get something that's called Constrained Non-Determinism. This is a term coined by Mark Seemann and basically means three things:

1. Values are anonymous i.e. we don't know the actual value we're using
2. The values are generated in as distinct as possible sequence (which means that, whenever possible, no two values generated one after another hold the same value)
3. The non-determinism in generation of the values is constrained, which means that the algorithms for generating values are carefully picked in order to provide values that are not special in any way (e.g. when generating integers, we don't allow generating '0' as it's usually a special-case-value)) and that are not "evil" (e.g. for integers, we generate small positive values first and go with bigger numbers only when we run out of those small ones).

There are multiple ways to implement constrained non-determinism. Mark Seemann himself invented the AutoFixture library for C# that's freely available to download by anyone. Here's a shortest possible snippet to generate an anonymous integer using AutoFixture:

Fixture fixture = new Fixture();
var anonymousInteger = fixture.Create<int>();

I, after Amir Kolsky and Scott Bain, like to use Any class. Any takes a slightly different approach than AutoFixture (although it shamelessly uses AutoFixture internally and actually obscures its impressive abilities). My implementation of Any class is available to download as well.

Summary

That was a long ride, wasn't it? I hope that this post gave you some understanding of how different TDD styles came into existence and why I use some of the techniques I do (and how these techniques are not just a series of random choices). In the next posts, I'll try to introduce some more techniques to help you grow a bag of neat tricks - a coherent style.

How is TDD about analysis and what's with the GIVEN-WHEN-THEN structure?

(this post is adapted from my work-in-progress open source TDD tutorial)

(Note that I use a term "Statement" instead of "test" and "Specification" instead of "Test Suite" in this post)

Is there really a commonality between analysis and TDD?

From Wikipedia:

Analysis is the process of breaking a complex topic or substance into smaller parts to gain a better understanding of it.

Thus, in order for TDD to be about analysis, it has to fulfill two conditions:

1. Be a process of breaking a complex topic into smaller parts
2. Allow gaining a better understanding of such broken topics

In the story about Johnny, Benjamin and Jane, I included a part where they analyze requirements using concrete examples. Johnny explained that this is a part of a technique called Acceptance Test-Driven Development. The process followed by the three characters fulfilled both mentioned conditions for a process to be analytical. But what about TDD itself?

Actually, I used parts of ATDD process in the story to make the analysis part more obvious, but similar things happen at pure technical levels. For example, when starting development with a failing application-wide Statement (we'll talk about levels of granularity of Statements some time later. For now the only thing you need to know is that the so called "unit tests level" is not the only level of granularity we write Statements on), we may encounter a situation where we need to call a web method and assert its result. This makes us think: what should this method be called? What are the scenarios supported? What do I need to get out of it? How should its user be notified about errors? Many times, this leads us to either a conversation (if there's another stakeholder that needs to be involved in the decision) or rethinking our assumptions. This is how we gain a better understanding of the topic we're analyzing, which makes TDD fulfill the first of the two requirements for it to be an analysis method.

But what about the first requirement? What about breaking a complex logic into smaller parts?

If you go back to our example, you'll note that both when talking to a customer and when writing code, Johnny and Benjamin used a TODO list. This list was first filled with whatever scenarios they came up with, but later, they would add a smaller unit of work. This is one way complex topics are decomposed into smaller items that all land on the TODO list (the other way is mocking, but we won't get into that yet). Thanks to this, we can focus on one thing at a time, crossing off item after item from the list after it's done. If we learn something new or encounter new issue that needs our attention, we can add it to the TODO list and get back to it later, for now continuing our work on the current point of focus.

An example TODO list from the middle of implementation may look like this (don't read trough it, I put it here only to give you a glimpse):

1. Create entry point to the module (top-level abstraction)
2. Implement main workflow of the module
3. Implement Message interface
4. Implement MessageFactory interface
5. Implement ValidationRules interface
6. Implement behavior required from Wrap method in LocationMessageFactory class
7. Implement behavior required from ValidateWith method in LocationMessage class for Speed field
8. Implement behavior required from ValidateWith method in LocationMessage class for Age field
9. Implement behavior required from ValidateWith method in LocationMessage class for Sender field

Note that some items are already crossed out as done, while other remain undone and waiting to be addressed.

Ok, That's it for the discussion. Now that we're sure that TDD is about analysis, let's focus on the tools can use to aid and inform it.

Gherkin

Hungry? Too bad, because the Gkerkin I'm gonna talk about is not edible. It's a notation and a way of thinking about behaviors of the specified piece of code. It can be applied on different levels of granularity - any behavior, whether of a whole system or a single class, may be described using it.

I actually talked about Gherkin before on this blog, just did not name it. It's the GIVEN-WHEN-THEN structure that you can see everywhere, even in code samples as comments. This time, we're stamping a name on it and analyzing it further.

In Gherkin, a behavior description consists of three parts:

1. Given - a context
2. When - a cause
3. Then - a effect

In other words, the emphasis is on causality in a given context.

As I said, there are different levels you can apply this. Here's an example for such a behavior description from the perspective of its end user (this is called acceptance-level Statement):

Given a bag of tea costs $20
When I buy two of them
Then I pay 30$ due to promotion

And here's one for unit-level (note the line starting with "And" that adds to the context):

Given a list with 2 items
When I add another item
And check count of items
Then the count should be 3

While on acceptance level we put such behavior descriptions together with code as the same artifact (If this doesn't ring a bell, look at tools like SpecFlow or Cucumber or FIT to get some examples), on the unit level the description is usually not written down literally, but in form of code only. Still, this structure is useful when thinking about behaviors required from an object or objects, as we saw when we talked about starting from Statement rather than code. I like to put the structure explicitly in my Statements - I find that it makes them more readable. So most of my unit-level Statements follow this template:

[Fact]
public void Should__BEHAVIOR__()
{
  //GIVEN
  ...context...

  //WHEN
  ...trigger...

  //THEN
  ...assertions etc....
}

Sometimes the WHEN and THEN sections are not so easily separable - then I join them, like in case of the following Statement specifying that an object throws an exception when asked to store null:

[Fact]
public void ShouldThrowExceptionWhenAskedToStoreNull()
{
  //GIVEN
  var safeList = new SafeList();

  //WHEN - THEN
  Assert.Throws<Exception>(
    () => safeList.Store(null)
  );
}

By thinking in terms of these three parts of behavior, we may arrive at different circumstances (GIVEN) at which the behavior takes place, or additional ones that are needed. The same goes for triggers (WHEN) and effects (THEN). If anything like this comes to our mind, we add it to the TODO list.

TODO list... again!

As we said previously, a TODO list is a repository for anything that comes to our mind when writing or thinking about a Statement, but is not a part o the current Statement we're writing. We don't want to forget it, neither do we want it to haunt us and distract us from our current task, so we write it down as soon as possible and continue with our current task.

Suppose you're writing a piece of small logic that allows user access when he's an employee of a zoo, but denies access if he's a guest of the zoo. Then, after starting writing a Statement it gets to you that actually any employee can be a guest as well - for example, he might choose to visit the zoo with his family during his vacation. Still, the two previous rules hold, so not to allow this case to distract you, you quickly add an item to the TODO list (like "TODO: what if someone is an employee, but comes to the zoo as a guest?") and finish the current Statement. When you're finished, you can always come back to the list and pick item to do next.

There are two important questions related to TODO lists: "what exactly should we add as a TODO list item?" and "How to efficiently manage the TODO list?". We'll take care of these two questions now.

What to put on the TODO list?

Everything that you need addressed but is not part of the current Statement. Those items may be related to implementing unimplemented methods, to add whole functionalities (such items are usually followed my more fine-grained sub tasks as soon as you start implementing the item), there might be reminders to take a better look at something (e.g. "investigate what is this component's policy for logging errors") or questions about the domain that need to get answered. If you get carried away too much in coding that you forget to eat, you can even add a reminder ("TODO: get something to eat!"). The list is yours!

How to pick items from TODO list?

Which item to choose from the TODO list when you have several? I have no clear rule, although I tend to take into account the following factors:

1. Risk - if what I learn by implementing or discussing a particular item from the list can have big impact on design or behavior of the system, I tend to pick such items first. An example of such item is that you start implementing validation of a request that arrives to your application and want to return different error depending on which part of the request is wrong. Then, during the development, you discover that more than one part of the request can be wrong at a time and you have to answer yourself a question: which error code should be returned in such case? Or maybe the return codes should be accumulated for all validations and then returned as a list?
2. Difficulty - depending on my mental condition (how tired I am, how much noise is currently around my desk etc.), I tend to pick items with difficulty that best matches this condition. For example, after finishing an item that requires a lot of thinking and figuring out things, I tend to take on some small and easy items to feel wind blowing in my sails and to rest a little bit.
3. Completeness - in simplest words, when I finish test-driving an "if" case, I usually pick up the "else" next. For example, after I finish implementing a Statement saying that something should return true for values less than 50, then the next item to pick up is the "greater or equal to 50" case. Usually, when I start test-driving a class, I take items related to this class until I run out of them, then go on to another one.

Where to put a TODO list?

There are two ways of maintaining a TODO list. The first one is on a sheet of paper, which is nice, but requires you to take your hands off the keyboard, grab a pen or pencil and then get back to coding every time you think of something. Also, the only way a TODO item written on a sheet of paper can tell you which place in code it is related to, is (obviously) by its text.

Another alternative is to use a TODO list functionality built-in into an IDE. Most IDEs, such as Visual Studio (and Resharper plugin has its own enhanced version), Xamarin Studio or eclipse-based IDEs have such functionality. The rules are simple - you put special comments in the code and a special view in your IDE aggregates them for you, allowing you to navigate to each. Such lists are great because:

1. They don't force you to take your hands off keyboard to add an item to the list.
2. You can put such item in a certain place in the code where is makes sense and then navigate back to it with a click of a mouse. This, apart from other advantages, lets you write shorter notes than if you had to do it on paper. For example, a TODO item saying "TODO: what if it throws an exception?" looks out of place on the paper, but when added as a comment to your code in the right place, it's mighty sufficient.
3. Many TODO lists automatically add items for certain things. E.g. in C#, when you are yet to implement a method that was automatically generated by a tool, its body usually consists of a line that throws a NotImplementedException exception. Guess what - NotImplementedException occurences are added to the TODO list automatically, so don't have to manually add notes to implement the methods where they occur.

There are also some downsides. The biggest is that people often add TODO items for other means than to support TDD and they never go back to such items. Some people joke that a TODO left in the code means "Once, I wanted to...". Anyway, such items may pollute your TDD-related TODO list with so much cruft that your own items are barely findable. To work around it, I tend to use a different tag than TODO (many IDEs let you define your own tags, or support multiple tag types out of the box. E.g. with Resharper, I like to use "bug" tag, because this is something no one would leave in the code) and filter by it. Another options is, of course, getting rid of the leftover TODO items - if no one addressed it for a year, then probably no one ever will.

Another downside is that when you work with multiple workspaces/solutions, your IDE will gather TODO items only from current solution/workspace, so you'll have few TODO lists - one per workspace or solution. Fortunately, this isn't usually a big deal.

How to start with a test, not implementation - part 4

(this post is adapted from my work-in-progress open source TDD tutorial)

(Note that I use "Statement" instead of "test" and "Specification" instead of "Test Suite" in this post)

This is the last installment of "how to start with a test" series. This time, we're gonna take on the last of the typical techniques, useful when there's already some existing code to fit in.

Start by invoking a method when you have one

Sometimes, we have to add a new class that implements an existing interface required by another class. The fact of implementing an interface imposes what methods should the new class support. If this point is already decided, we can start our Statement by first calling the method and then discovering what we need to supply.

A simple example

Suppose we have an application holding a lot of data that, among other things, handles importing am existing database from another instance of the application. As importing a database can be a lengthy process, a message box is displayed each time when user chooses to perform the import and this message box displays the following message: "Johnny, please sit down and enjoy your coffee for a few minutes as we take time to import your database" (given user name is Johnny). The class that implements it looks like this:

public class FriendlyMessages
{
 public string 
 HoldOnASecondWhileWeImportYourDatabase(string userName)
 {
   return string.Format("{0}, "
     + "please sit down and enjoy your coffee "
     + "for a few minutes as we take time "
     + "to import your database",
     userName);
 }
}

Now, imagine that our management told us that they need to ship a trial version with some features disabled, including importing an existing database. Among all the things we need to do to make it happen, we also need to display a different string with message saying that this is a trial version and the feature is locked. We can do it by extracting an interface from the FriendlyMessages class and using it to put in an instance of another class implementing this interface when the application discovers that it is being run as a trial version. The extracted interface looks like this:

public interface Messages
{
  string HoldOnASecondWhileWeImportYourDatabase(string userName);
}

So our new implementation is forced to support the HoldOnASecondWhileWeImportYourDatabase method. Thus, we when implementing the class, we start with the following:

public class TrialVersionMessages : Messages
{
 public string HoldOnASecondWhileWeImportYourDatabase(string userName)
 {
   throw new NotImplementedException();
 }
}

Now, we're ready to start writing a Statement. Assuming we don't know where to start, we just start with creating an object and invoking the method that needs to be implemented:

[Fact] 
public void TODO()
{
 //GIVEN
 var trialMessages = new TrialVersionMessages();
 
 //WHEN
 trialMessages.HoldOnASecondWhileWeImportYourDatabase();

 //THEN
 Assert.True(false); //to remember about it
}

As you can see, we added an assertion that always fails at the end because we don't have any assertions yet and the Statement would otherwise be already evaluated as true and we'd rather have it remind ourselves that it is not finished. Other than this, the Statement does not compile anyway, because the method HoldOnASecondWhileWeImportYourDatabase takes a string argument and we passed none. This makes us ask the question what is this argument and what's its role in the behavior triggered by the HoldOnASecondWhileWeImportYourDatabase method. Seems like it is a user name and we want it to be somewhere in the result of the method. Thus, we can add it to the Statement like this:

[Fact] 
public void TODO()
{
 //GIVEN
 var trialMessages = new TrialVersionMessages();
 var userName = Any.String();
 
 //WHEN
 trialMessages.
  HoldOnASecondWhileWeImportYourDatabase(userName);

 //THEN
 Assert.True(false); //to remember about it
}

Now, this compiles but is evaluated as false because of the guard assertion that we put at the end. Our goal is to substitute it with a real assertion for a real expected result. The return value of the HoldOnASecondWhileWeImportYourDatabase is a string message, so all we need to do is to come up with the message that we expect in case of trial version:

[Fact] 
public void TODO()
{
 //GIVEN
 var trialMessages = new TrialVersionMessages();
 var userName = Any.String();
 
 //WHEN
 var message = trialMessages.
  HoldOnASecondWhileWeImportYourDatabase(userName);

 //THEN
 var expectedMessage = 
  string.Format("{0}, better get some pocket money!", userName);

 Assert.Equal(expectedMessage, message);
}

and all that is left is to find a good name for the Statement. This isn't an issue since we already specified the desired behavior in the code, so we can just summarize it as something like ShouldYieldAMessageSayingThatFeatureIsLockedWhenAskedForImportDatabaseMessage.

How to start with a test, not implementation - part 3

(this post is adapted from my work-in-progress open source TDD tutorial)

(Note that I use "Statement" instead of "test" and "Specification" instead of "Test Suite" in this post)

In the first part, I discussed how a good name is a great start for writing a Statement when there's no production code to invoke. In the second part, I elaborated on usefulness of thinking in terms of GIVEN-WHEN-THEN structure and translating it almost literally to code. Today, I'd like to introduce to you another technique - one that may appear awkward, but is actually very useful.

Start from the end

This is a technique that I suggest to people that seem to have absolutely no idea how to start. I got it from Kent Beck's book Test Driven Development by Example. It seems funny at start, but is quite powerful. The trick is to write the Statement 'backwards', i.e. starting with what the Statement asserts to be true (in terms of the GIVEN-WHEN-THEN structure, we'd say that we start with our THEN).

This works because, while we're many times quite sure of our goal (i.e. what the outcome of the behavior should be), but are unsure of how to get there.

A simple example

Imagine we're writing a class for granting access to a reporting functionality based on roles. We don't have any idea what the API should look like and how to write our Statement, but we know one thing: in our domain the access can be either granted or denied. Let's take the successful case (just because it's the first one we can think of) and, starting backwards, start with the following assertion:

//THEN
Assert.True(accessGranted);

Ok, that part was easy, but did we make any progress with that? Of course we did - we now have a non-compiling code and the compilation error is because of the accessGranted variable. Now, in contrast to the previous approach (with translating our GIVEN-WHEN-THEN structure into a Statement), our goal is not to make this compile as soon as possible. The goal is to answer ourselves a question: how do I know whether the access is granted or not? The answer: it is the result of authorization of the allowed role. Ok, so let's just write it down, ignoring everything that stands in our way (I know that most of us have a habit to add a class or a variable as soon as we find out that we need it. If you're like that, then please turn off this habit while writing Statements - it will only throw you off the track and steal your focus from what's important. The key to doing TDD successfully is to learn to use something that does not exist yet like it existed):

//WHEN
var accessGranted 
 = authorization.IsAccessToReportingGrantedTo(
  roleAllowedToUseReporting);

Note that we do not know what roleAllowedToUseReporting is, neither do we know what's authorization, but that didn't stop us from writing this line. Also, the IsAccessToReportingGrantedTo() method is just taken from the top of our head. It's not defined anywhere, it just made sense to write it like this, because it's the most direct translation of what we had in mind.

Anyway, this new line answers the question on where do we take the accessGranted from, but makes us ask further questions:

1. Where does the authorization variable come from?
2. Where does the roleAllowedToUseReporting variable come from?

As for authorization, we don't have anything specific to say about it other than that it is an object of a class that we don't have yet. In order to proceed, let's pretend that we have such a class. How do we call it? The instance name is authorization, so it's quite straightforward to name the class Authorization and instantiate it in the simplest way we can think of:

//GIVEN
var authorization = new Authorization();

Now for the roleAllowedToUseReporting. The first question that comes to mind when looking at this is: which roles are allowed to use reporting? Let's assume that in our domain, this is either an Administrator or an Auditor. Thus, we know what's going to be the value of this variable. As for the type, there are various ways we can model a role, but the most obvious one for a type that has few possible values is an enum. So:

//GIVEN
var roleAllowedToUseReporting = Any.Of(Roles.Admin, Roles.Auditor);

And so, working our way backwards, we have arrived at the final solution (we still need to give this Statement a name, but, provided what we already know, this is an easy task):

[Fact] public void
ShouldAllowAccessToReportingWhenAskedForEitherAdministratorOrAuditor()
{
 //GIVEN
 var roleAllowedToUseReporting = Any.Of(Roles.Admin, Roles.Auditor);
 var authorization = new Authorization();

 //WHEN
 var accessGranted = authorization
  .IsAccessToReportingGrantedTo(roleAllowedToUseReporting);

 //THEN
 Assert.True(accessGranted);
}

How to start with a test, not implementation - part 2

(this post is adapted from my work-in-progress open source TDD tutorial)

(Note that I use "Statement" instead of "test" and "Specification" instead of "Test Suite" in this post)

In the previous part, I discussed how a good name is a great start for writing a Statement when there's no production code to invoke. Today, I'll introduce to you another technique.

Start by filling the GIVEN-WHEN-THEN structure with the obvious

This is applicable when you come up with a GIVEN-WHEN-THEN structure for the Statement or a good name for it (GIVEN-WHEN-THEN structure can be easily derived from a good name and vice versa). Anyway, this method is about taking the GIVEN-WHEN-THEN parts and translating them almost literally into code, then add all the missing pieces that are required for the code to compile and run.

An example speaks a thousand words

Let's take a simple example of comparing two users. We assume that a user should be equal to another when it has the same name as the other one:

GIVEN a user with any name
WHEN I compare it to another user with the same name
THEN it should appear equal to this other user

Let's start with the translation

The first line:

GIVEN a user with any name

can be translated literally to code like this:

var user = new User(anyName);

Then the second line:

WHEN I compare it to another user with the same name

can be written as:

user.Equals(anotherUserWithTheSameName);

Great! Now the last line:

THEN it should appear equal to this other user

and its translation into the code:

Assert.True(areUsersEqual);

Ok, so we've made the translation, now let's summarize this and see what's missing to make this code compile:

[Fact] public void 
ShouldAppearEqualToAnotherUserWithTheSameName()
{
  //GIVEN
  var user = new User(anyName);

  //WHEN
  user.Equals(anotherUserWithTheSameName);

  //THEN
  Assert.True(areUsersEqual);
}

As we expected, this will not compile. Notably, our compiler might point us towards the following gaps:

1. A declaration of anyName variable.
2. A declaration of anotherUserWithTheSameName object.
3. The variable areUsersEqual is both not declared and it does not hold the comparison result.
4. If this is our first Statement, we might not even have the User class defined at all.

The compiler created a kind of a small TODO list for us, which is nice. Note that while we don't have full compiling code, filling the gaps boils down to making a few trivial declarations and assignments:

1. anyName can be declared as var anyName = Any.String();
2. anotherUserWithTheSameName can be declared as var anotherUserWithTheSameName = new User(anyName);
3. areUsersEqual can be declared and assigned this way: var areUsersEqual = user.Equals(anotherUserWithTheSameName);
4. If User class does not exist, we can add it by simply stating: public class User {}

Putting it all together:

[Fact] public void 
ShouldAppearEqualToAnotherUserWithTheSameName()
{
  //GIVEN
  var anyName = Any.String();
  var user = new User(anyName);
  var anotherUserWithTheSameName = new User(anyName);

  //WHEN
  var areUsersEqual = user.Equals(anotherUserWithTheSameName);

  //THEN
  Assert.True(areUsersEqual);
}

And that's it - the Statement is complete!

How to start with a test, not implementation - part 1

(this post is adapted from my work-in-progress open source TDD tutorial)

Note that I use "Statement" instead of "test" and "Specification" instead of "Test Suite" in this post

Part 2 of this series is ready!

Whenever I sat down with a person that was about to write their first code in a Statement-first manner, the person would first stare at the screen, then at me, then would say: "what now?". It's easy to say: "You know how to write code, you know how to write a unit test for it, just this time start with the latter rather than the first", but for many people, this is something that blocks them completely. If you're one of them, don't worry - you're not alone. I decided to dedicate this series of posts solely to techniques for starting to write a Statement when there's no code. I do not use mocks in this series on purpose, to keep it simple. However, all of these techniques are proven to work when using mocks.

Start with a good name

It may sound obvious, but really some people are having trouble describing the behavior they expect from their code. If you can name such behavior, it's a great starting point.

I know some people don't pay attention to naming their Statements, mainly because they're considered as tests and second-level citizens - as long as they run and "prove the code does not contain defects", they're considered sufficient. We'll take a look at some examples of bad names and then I'd like to introduce to you some rules of good naming.

Consequences of bad naming

As I said, many people don't really care how their Statements are named. This is a symptom of treating the Specification as garbage or leftovers - something that just "runs through your code". Such situation is dangerous, because as soon as this kind of thinking is established, it leads to bad, unmaintainable Specification that looks more like lumps of accidental code put together in a haste than a living documentation. Imagine that your Specification consists of names like this:

• TrySendPacket()
• TrySendPacket2()
• testSendingManyPackets()
• testWrongPacketOrder1()
• testWrongPacketOrder2()

and try for yourself how difficult it is to answer the following questions:

1. How do you know what situation each Statement describes?
2. How do you know whether the Statement describes a single situation, or few of them at the same time?
3. How do you know whether the assertions inside those Statements are really the right ones assuming each Statement was written by someone else or was written a long time ago?
4. How do you know whether the Statement should stay or be removed when you modify the functionality it specifies?
5. If your changes in production code make a Statement evaluate to false, how do you know whether the Statement is no longer correct or the production code is wrong?
6. How do you know whether you will not introduce a duplicate Statement for a behavior that's already specified by another Statement when adding to Specification originally created by another team member?
7. How do you estimate, by looking at the runner tool report, whether the fix for failing Statement will be easy or not?
8. How do you answer a new developer in your team when they ask you "what is this Statement for?"
9. How can you keep track of the Statements already made about the specified class vs those that still need to be made?

What's in a good name?

For the name of the Statement to be of any use, it has to describe the expected behavior. At minimum, it should describe what happens at what circumstances. Let's take a look at one of the names Steve freeman and Nat Pryce came up in their great book Growing Object Oriented Software Guided By Tests:

notifiesListenersThatServerIsUnavailableWhenCannotConnectToItsMonitoringPort()

Note few things about the name of the Statement:

1. It describes a behavior of an instance of a specified class. Note that it does not contain method name, because what is specified is not a method, but a behavior that has its entry point and expected result. The name simply tells that what an instance does (notifies listeners that server is unavailable) under certain circumstances (when cannot connect to its monitoring port). It's important because such description is what you can derive from thinking about responsibilities of a class, so you don't need to know any of its methods signatures or the code that's inside of the class. Hence, this is something you can come up without before implementing - you just need to know why you created this class and feed on it.

2. The name is long. Really, really, really don't worry about it. As long as you're describing a single behavior, it's alright. I know usually people are hesitant to give long names to the Statements, because they try to apply the same rules to those names as method names in production code. Let me make it clear - these two cases are different. In case of Statements, they're not invoked by anyone besides the automatic runner applications, so they won't obfuscate any code that would need to call them. Sure, we could put all the information in the comment instead of Statement name and leave the name short, like this:

   [Fact]
   //Notifies listeners that server 
   //is unavailable when cannot connect
   //to its monitoring port
   public void Statement_002()
   {
    //...
   }
   

   There are two downsides of this solution: one is that we now have to put extra information (Statement_002) which is required only by compiler, because every method needs to have a name - there's usually no value a human could derive from such a name. The second downside is that when the Statement is evaluated to false, the automated runner shows you the following line: Statement_002: FAILED - note that all the information included in the comment isn't present in the failure report. It's really better to receive a report such as: notifiesListenersThatServerIsUnavailableWhenCannotConnectToItsMonitoringPort: FAILED, because all the information about the Statement that fails is present in the runner window.

3. Using a name that describes a (single) behavior allows you to track quickly why the Statement is false when it is. Suppose a Statement is true when you start refactoring, but in the meantime it turns out to be false and the report in the runner looks like this: TrySendingHttpRequest: FAILED - it doesn't really tell you anything more than an attempt is made to send a HTTP request, but, for instance, does not tell you whether your specified object is the sender (that should try to send this request under some circumstances) or the receiver (that should handle such request properly). To actually know what went wrong, you have to go to the code to scan its source code. Now compare it to the following name: ShouldRespondWithAnAckWheneverItReceivedAHttpRequest. Now when it evaluates to false, you can tell that what is broken is that the object no longer responds with an ACK to HTTP request. Sometimes this is enough to deduct which part of the code is in fault of this evaluation failure.

My favourite convention

There are many conventions for naming Statements appropriately. My favorite is the one developed by Dan North, which makes each Statement name begin with the word Should. So for example, I'd name a Statement: ShouldReportAllErrorsSortedAlphabeticallyWhenItEncountersErrorsDuringSearch(). The name of the Specification (i.e. class name) answers the question "who should do it?", i.e. when I have a class named SortingOperation and want to say that it "should sort all items in ascending order when performed", I say it like this:

public class SortingOperationSpecification
{
 [Fact] public void
 ShouldSortAllItemsInAscendingOrderWhenPerformed()
 {
 }
}

It's important to focus on what result is expected from an object when writing names along this convention. If you don't, you quickly find it troublesome. As an example, one of my colleague was specifying a class UserId and wrote the following name for the Statement about comparison of two identifiers: EqualOperationShouldPassForTwoInstancesWithTheSameUserName(). Note that this is not from the perspective of a single object, but rather from the perspective of an operation that's executed on it, which means that we stopped thinking in terms of object responsibilities and started thinking in terms of operation correctness, which is farther away from our assumption that we're writing a Specification consisting o Statements. This name should be changed to: ShouldReportThatItIsEqualToAnotherObjectWhenItHasTheSameUserName().

And this is the end of part 1 of the series. In the next installment, we'll take a look at another technique - brute force translation of prose description written with GIVEN-WHEN-THEN structure to code.

An expert and a novice practicing TDD - story

(this post is adapted from my work-in-progress open source TDD tutorial)

And now, a taste of things to come!

- Shang Tsung, Mortal Kombat The Movie

The above quote took place just before a fighting scene in which a nameless warrior jumped at Sub-Zero only to be frozen and broken into multiple pieces upon hitting the wall. The scene was not spectacular in terms of fighting technique or length. Also, the nameless guy didn't even try hard - the only thing he did was to jump only to be hit by a freezing ball, which, by the way, he actually saw coming. It looked a lot like the fight was set up only to showcase Sub-Zero's freezing ability. Guess what? In this post, we're gonna do roughly the same thing - set up a fake, easy scenario just to showcase some of the basic TDD elements!

As we go through the example, you might wonder how on earth could you possibly write real applications the way we'll write our simple program. Don't worry, I'm just not showing you all the tricks to spare you confusion and deliver something digestive instead of bloat of information that could kill you. In other words, the example will be as close to real world problems as the fight between Sub-Zero and nameless ninja was to real martial arts fight, but will show you some of the elements of TDD process.

Let me tell you a story

Meet Johnny and Benjamin, two developers from Buthig Company. Johnny is quite fluent in programming and Test-Driven Development, while Benjamin is an intern under Johnny's mentorship and is eager to learn TDD. They're on their way to their customer, Jane, who requested their presence as she wants them to do write a small program for her. Together with them, we'll see how they interact with the customer and how Benjamin tries to understand the basics of TDD. Just as you, Benjamin is a novice so his questions may reflect yours.

Act 1: The Car

Johnny: How do you feel on your first assignment?

Benjamin: I'm pretty excited! I hope I can learn some of the TDD stuff you promised to teach me.

Johnny: Not only TDD, but we're also gonna use some of the practices associated with a process called Acceptance Test Driven Development, albeit in a simplified form.

Benjamin: Acceptance Test-Driven Development? What is that?

Johnny: While TDD is usually referred to as a development technique, ATDD is something more of a collaboration method. Both ATDD and TDD have a bit of analysis in them and work very well together as both use the same underlying principles, just on different levels. We'll need only a small subset of what ATDD has to offer, so don't get over-excited.

Benjamin: Sure.

Act 2: The Customer

Johnny: Hi, Jane, how are you?

Jane: Thanks, I'm fine, how about you?

Johnny: Same here, thanks. So, can you tell us a bit about the software you need us to write?

Jane: Sure. Recently, I bought a new smartphone as a replacement for my old one. The thing is, I'm really used to the calculator application that was running on the previous phone and I can't find it for my current device.

Benjamin: Can't you just use another calculator app? There are plenty of them available to download from the web.

Jane: That's right. I checked them all and none has exactly the same behavior as the one I was using for my tax calculations. You know, the program was like right hand to me and it had some really nice shortcuts that made my life easier.

Johnny: So you want us to reproduce the application to run on your new device?

Jane: Yes.

Johnny: Are you aware that apart from the fancy features that you were using we'll have to allocate some effort to implement the basics that all the calculators have?

Jane: Sure, I'm OK with that. I'm so used to my calculator application so much that if I use something else for more than few months, I'll have to pay psychotherapist instead of you guys. Apart from that, writing a calculator app seems like a simple task in my mind, so the cost isn't going to be overwhelming, right?

Johnny: I think I get it. Let's get it going then. We'll be implementing the functionality incrementally, starting with the most essential ones. Which feature of the calculator would you consider the most essential?

Jane: That would be addition of numbers, I guess.

Johnny: Ok, that will be our target for the first iteration. After the iteration, we'll deliver this part of functionality for you to try out and give us some feedback. However, before we can even implement addition, we'll have to implement displaying digits on the screen as you enter them. Is that correct?

Jane: yes, I want the display stuff to work as well - it's the most basic feature, so...

Johnny: Ok then, this is a simple functionality, so let me suggest some user stories as I understand what you already said and you'll correct me where I'm wrong. Here we go:

1. In order to know that the calculator is turned on, As a tax payer I want to see "0" on the screen as soon as I turn it on.
2. In order to see what numbers I'm currently operating on, As a tax payer, I want the calculator to display the values I enter
3. In order to calculate sum of my different incomes, As a tax payer I want the calculator to enable addition of multiple numbers

What do you think?

Jane: The stories pretty much reflect what I want for the first iteration. I don't think I have any corrections to make.

Johnny: Now we'll take each story and collect some examples of how it should work.

Benjamin: Johnny, don't you think it's obvious enough to proceed with implementation straight away?

Johnny: Trust me, Benjamin, if there's one word I fear most in communication, it's "obvious". Miscommunication happens most often around things that people consider obvious, simply because other people don't.

Jane: Ok, I'm in. What do I do?

Johnny: Let's go through stories one by one and see if we can find some key examples of how the features work. The first story is...

In order to know that the calculator is turned on, As a tax payer I want to see "0" on the screen as soon as I turn it on.

Jane: I don't think there's much to talk about. If you display "0", I'll be happy. That's all.

Johnny: Let's write down this example:

key sequence	Displayed output	Notes
N/A	0	Initial displayed value

Benjamin: That makes me wonder... what should happen when I press "0" again at this stage?

Johnny: Good catch, that's what these examples are for - they make our thinking concrete. As Ken Pugh says: "Often the complete understanding of a concept does not occur until someone tries to use the concept". We'd normally put it on a TODO list, because it's part of a different story, but we're actually done with this one, so let's move straight to the story about displaying entered digits. How about it, Jane?

Jane: Agree.

In order to see what numbers I'm currently operating on, As a tax payer, I want the calculator to display the values I enter

Johnny: Let's begin with the case raised by Benjamin. What should happen when I input "0" multiple times after I have only "0" on the display?

Jane: Just one "0" should be displayed

Johnny: Do you mean this?

key sequence	Displayed output	Notes
0,0,0	0	Zero is a special case – it is displayed only once

Jane: That's right. Other than this, the digits should just show on the screen, like this:

key sequence	Displayed output	Notes
1,2,3	123	Entered digits are displayed

Benjamin: How about this:

key sequence	Displayed output	Notes
1,2,3,4,5,6,7,1,2,3,4,5,6	1234567123456?	Entered digits are displayed?

Jane: Actually, no. My old calculator app has a limit of six digits that I can enter, so it should be:

key sequence	Displayed output	Notes
1,2,3,4,5,6,7,1,2,3,4,5,6	123456	Display limited to six digits

Johnny: another good catch, Benjamin!

Benjamin: I think I'm beginning to understand why you like working with examples!

Johnny: Good. Is there anything else, Jane?

Jane: No, that's pretty much it. Let's take another story.

In order to calculate sum of my different incomes, As a tax payer I want the calculator to enable addition of multiple numbers

Johnny: Is the following all we have to support?

key sequence	Displayed output	Notes
2,+,3,+,4,=	9	Simple addition of numbers

Jane: This scenario is correct, however, there's also a case when I start with "+" without inputting any number before. This should be treated as adding to zero:

key sequence	Displayed output	Notes
+,1,=	1	Addition shortcut – treated as 0+1

Benjamin: How about when the output is a number longer than six digits limit? Is it OK that we truncate it like this?

key sequence	Displayed output	Notes
9,9,9,9,9,9,+,9,9,9,9,9,9,=	199999	Our display is limited to six digits only

Jane: Sure, I don't mind. I don't add such big numbers anyway.

Johnny: There's still one question we missed. Let's say that I input a number, then press "+" and then another number without asking for result with "=". What should I see?

Jane: Every time you press "+", the calculator should consider entering current number finished and overwrite it as soon as you press any other digit:

key sequence	Displayed output	Notes
2,+,3	3	Digits entered after + operator are treated as digits of a new number, the previous one is stored

Jane: Oh, and just asking for result after the calculator is turned on should result in "0".

key sequence	Displayed output	Notes
=	0	Result key in itself does nothing

Johnny: Let's sum up our discoveries:

key sequence	Displayed output	Notes
N/A	0	Initial displayed value
1,2,3	123	Entered digits are displayed
0,0,0	0	Zero is a special case – it is displayed only once
1,2,3,4,5,6,7	123456	Our display is limited to six digits only
2,+,3	3	Digits entered after + operator are treated as digits of a new number, the previous one is stored
=	0	Result key in itself does nothing
+,1,=	1	Addition shortcut – treated as 0+1
2,+,3,+,4,=	9	Simple addition of numbers
9,9,9,9,9,9,+,9,9,9,9,9,9,=	199999	Our display is limited to six digits only

Johnny: The limiting of digits displayed looks like a whole new feature, so I suggest we add it to the backlog and do it in another sprint. In this sprint, we won't handle such situation at all. How about that, Jane?

Jane: Fine with me. Looks like a lot of work. Nice that we discovered it up-front. For me, the limiting capability seemed so obvious that I didn't even think it would be worth mentioning.

Johnny: See? That's why I don't like the word "obvious". Jane, we'll get back to you if any more questions arise. For now, II think we know enough to implement these three stories for you.

Jane: good luck!

Act 3: Test-Driven Development

Benjamin: Wow, that was cool. Was that Acceptance Test-Driven Development?

Johnny: In a greatly simplified version, yes. The reason I took you with me was to show you the similarities between working with customer the way we did and working with the code using TDD process. They're both applying the same set of principles, just on different levels.

Benjamin: I'm dying to see it with my own eyes. Shall we start?

Johnny: Sure. If we followed the ATDD process, we'd start writing what we call acceptance-level specification. In our case, however, a unit-level specification will be enough. Let's take the first example:

Statement 1: Calculator should display 0 on creation

key sequence	Displayed output	Notes
N/A	0	Initial displayed value

Johnny: Benjamin, try to write the first Statement.

Benjamin: Boy, I don't know how to start.

Johnny: start by writing the statement in a plain English. What should the calculator do?

Benjamin: It should display "0" when I turn the application on.

Johnny: In our case, "turning on" is creating a calculator. Let's write it down as a method name:

public class CalculatorSpecification
{

[Fact] public void
ShouldDisplay0WhenCreated()
{

}

}

Benjamin: Why is the name of the class CalculatorSpecification and the name of the method ShouldDisplay0WhenCreated?

Johnny: It's a naming convention. There are many others, but this is the one that I like. The rule is that when you take the name of the class without the Specification part followed by the name of the method, it should form a legit sentence. For instance, if I apply it to what we wrote, it would make a sentence: "Calculator should display 0 when created".

Benjamin: Ah, I see now. So it's a statement of behavior, isn't it?

Johnny: That's right. Now, the second trick I can sell to you is that if you don't know what code to start with, start with the expected result. In our case, we're expecting that the behavior will end up as displaying "0", right? So let's just write it in form of an assertion.

Benjamin: You mean something like this?

public class CalculatorSpecification
{

[Fact] public void
ShouldDisplay0WhenCreated()
{
 Assert.Equal("0", displayedResult);
}

}

Johnny: Precisely.

Benjamin: But that doesn't even compile. What use is it?

Johnny: the code not compiling is the feedback that you needed to proceed. While previously you didn't know where to start, now you have a clear goal - make this code compile. Firstly, whre do you get the displayed value?

Benjamin: From the calculator display, of course!

Johnny: Then write it down how you get the value form the display.

Benjamin: like how?

Johnny: like this:

public class CalculatorSpecification
{

[Fact] public void
ShouldDisplay0WhenCreated()
{
 var displayedResult = calculator.Display();

 Assert.Equal("0", displayedResult);
}

}

Benjamin: I see. Now the calculator is not created anywhere. I need to create it somewhere now or it won't compile and this is my next step. Is this how it works?

Johnny: Yes, you're catching on quickly.

Benjamin: Ok then, here goes:

public class CalculatorSpecification
{

[Fact] public void
ShouldDisplay0WhenCreated()
{
 var calculator = new Calculator();

 var displayedResult = calculator.Display();

 Assert.Equal("0", displayedResult);
}

}

Johnny: Bravo!

Benjamin: Now the code still does not compile, because I don't have the Calculator class at all...

Johnny: sounds like a good reason to create it.

Benjamin: OK.

public class Calculator
{
}

Benjamin: Looks like the Display() method is missing too. I'll add it.

public class Calculator
{
  public string Display()
  {
    return "0";
  } 
}

Johnny: Hey hey, not so fast!

Benjamin: What?

Johnny: You already provided an implementation that will make our current Statement true. Remember its name? ShouldDisplay0WhenCreated - and that's exactly what the code you wrote does. Before we arrive at this point, let's make sure this Statement can ever be evaluates as false. So for now, let's change it to this:

public class Calculator
{
  public string Display()
  {
    return "Once upon a time in Africa...";
  } 
}

Johnny: Look, now we can run the Specification and watch that Statement evaluate to false, because it expects "0", but gets "Once upon a time in Africa...".

Benjamin: Running... Ok, it's false. By the way, do you always use such silly values to make Statements false?

Johnny: Hahaha, no, I just did it to emphasize the point. Normally, I'd write return ""; or something similarly simple. Now we can evaluate the Statement and see it evaluate to false. Now we are sure that we have not yet implemented what is required for the Statement to be true.

Benjamin: I think I get it. For now, the Statement shows that we don't have something we need and gives us a reason to add this "thing". When we do so, this Statement will show that we do have what we need. So what do we do now?

Johnny: Write the simplest thing that makes this Statement true.

Benjamin: like this?

public class Calculator
{
  public string Display()
  {
    return "0";
  } 
}

Johnny: Yes.

Benjamin: But that's not a real implementation. What's the value behind putting in a hardcoded string? The final implementation is not going to be like this for sure!

Johnny: You're right. The final implementation is most probably going to be different. What we did, however, is still valuable because:

1. You're one step closer to implementing the final solution
2. This feeling that this is not the final implementation points you towards writing more Statements. When there is enough Statements to make your implementation complete, it usually means that you have a complete Specification of class behaviors as well.
3. If you treat making every Statement true as an achievement, this practice allows you to evolve your code without losing what you already achieved. If by accident you break any of the behaviors you've already implemented, the Specification is going to tell you because one of the existing Statements that were previously true will then evaluate to false. You can then either fix it or undo your changes using version control and start over from the point where all existing Statements were true.

Benjamin: looks like quite a few benefits. Still, I'll have to get used to this kind of working.

Johnny: don't worry, it is central to TDD, so you'll grasp it in no time. Now, before we proceed to the next Statement, let's look at what we already achieved. First, we wrote a Statement that turned out false. Then, we wrote just enough code to make the Statement true. Time for a step called Refactoring. In this step, we'll take a look at the Statement and the code and remove duplication. Can you see what's duplicated between the Statement and the code?

Benjamin: both of them contain the literal "0". The Statement has it here:

Assert.Equal("0", displayedResult);

and the implementation here:

return "0";

Johnny: Good, let's eliminate this duplication by introducing a constant called InitialValue. The Statement will now look like this:

[Fact] public void
ShouldDisplayInitialValueWhenCreated()
{
 var calculator = new Calculator();

 var displayedResult = calculator.Display();

 Assert.Equal(Calculator.InitialValue, displayedResult);
}

and the implementation:

public class Calculator
{
  public const string InitialValue = "0";
  public string Display()
  {
    return InitialValue;
  } 
}

Benjamin: The code looks better and having the constant in one place will make it more maintainable, but I think the Statement in its current form is weaker than before. We could change the InitialValue to anything and the Statement would still be true, since it does not force this constant to be "0".

That's right. We need to add it to our TODO list to handle this case. Can you write it down?

Benjamin: Sure. I'll write it as "TODO: 0 should be used as an initial value."

Johnny: Ok. We should handle it now, especially that it's part of the story we're currently implementing, but I'll leave it for later just to show you the power of TODO list in TDD - whatever is on the list, we can forget and get back to when we have nothing better to do. Our next item from the list is this:

Statement 2: Calculator should display entered digits

key sequence	Displayed output	Notes
1,2,3	123	Entered digits are displayed

Johnny: Benjamin, can you come up with a Statement for this behavior?

Benjamin: I'll try. Here goes:

public void [Fact]
ShouldDisplayEnteredDigits()
{
  var calculator = new Calculator();
  
  calculator.Enter(1);
  calculator.Enter(2);
  calculator.Enter(3);
  var displayedValue = calculator.Display();

  Assert.Equal("123", displayedValue);
}

Johnny: I'm glad that you got the part about naming and writing a Statement. One thing we'll have to work on here though.

Benjamin: what is it?

Johnny: When we talked to Jane, we used examples with real values. These real values were extremely helpful in pinning down the corner cases and uncovering missing scenarios. They were easier to imagine as well, so they were a perfect suit for conversation. If we were automating these examples on acceptance level, we'd use those real values as well. When we write unit-level Statements, however, we use a different technique to get this kind of specification more abstract. First of all, let me enumerate the weaknesses of the approach you just used:

1. Making a method Enter() accept an integer value suggests that one can enter more than one digit at once, e.g. calculator.Enter(123), which is not what we want. We could detect such cases and throw exceptions if the value is outside the 0-9 range, but there are better ways when we know we'll only be supporting ten digits (0,1,2,3,4,5,6,7,8,9).
2. The Statement does not clearly show the relationship between input and output. Of course, in this simple case it's pretty self-evident that the sum is a concatenation of entered digits, we don't want anyone who will be reading this Specification in the future to have to guess.
3. The Statement suggests that what you wrote is sufficient for any value, which isn't true, since the behavior for "0" is different (no matter how many times we enter "0", the result is just "0")

Hence, I propose the following:

public void [Fact]
ShouldDisplayAllEnteredDigitsThatAreNotLeadingZeroes()
{
 //GIVEN
 var calculator = new Calculator();
 var nonZeroDigit = Any.Besides(DigitKeys.Zero);
 var anyDigit1 = Any.Of<DigitKeys>();
 var anyDigit2 = Any.Of<DigitKeys>();
          
 //WHEN
 calculator.Enter(nonZeroDigit);
 calculator.Enter(anyDigit1);
 calculator.Enter(anyDigit2);
          
 //THEN
 Assert.Equal(
  string.Format("{0}{1}{2}", 
   (int)nonZeroDigit, 
   (int)anyDigit1, 
   (int)anyDigit2
  ),
  calculator.Display()
 );
}

Benjamin: Johnny, I'm lost! Can you explain what's going on here?

Johnny: Sure, what do you want to know?

Benjamin: For instance, what is this DigitKeys type doing here?

Johnny: It's supposed to be an enumeration (note that it does not exist yet, we just assume that we have it) to hold all the possible digits user can enter, which are 0-9. This is to ensure that user will not write calculator.Enter(123). Instead of allowing to enter any number and then detecting errors, we're giving our users a choice from among only the valid values.

Benjamin: Now I get it. So how about the Any.Besides() and Any.Of()? What do they do?

Johnny: They're methods from a small utility library I'm using when writing unit-level Specifications. Any.Besides() returns any value from enumeration besides the one supplied as an argument. Hence, the call Any.Besides(DigitKeys.Zero) means "any of the values contained in DigitKeys enumeration, but not DigitKeys.Zero".

The Any.Of() is simpler - it just returns any value in an enumeration. Note that when I create the values this way, I state explicitly that this behavior occurs when first digit is non-zero. This technique of using generated values instead of literals has its own principles, but let's leave it for later. I promise to give you a detailed lecture on it. Agree?

Benjamin: You better do, because for now, I feel a bit uneasy with generating the values - it seems like the Statement we're writing is getting less deterministic this way. The last question - what about those weird comments you put in the code? Given? When? Then?

Johnny: Yes, this is a convention that I use, not only in writing, but in thinking as well. I like to think about every behavior in terms of three elements: assumptions (given), trigger (when) and expected result (then). Using the words, we can summarize the Statement we're writing in the following way: "Given a calculator, when I enter some digits first one being other than zero, then they should all be displayed in order they were entered". This is also something that I'll tell you more about later.

Benjamin: Sure, for now I need just enough detail to understand what's going on - we can talk about the principles, pros and cons later. By the way, the following sequence of casts looks a little bit ugly:

  string.Format("{0}{1}{2}", 
   (int)nonZeroDigit, 
   (int)anyDigit1, 
   (int)anyDigit2
  )

Johnny: We'll get back to it and make it more "smarter" in a second after we make this statement true. For now, we need something obvious. Something we know works. Let's evaluate this Statement. What's the result?

Benjamin: Failed, expected "331", but was "0".

Johnny: Good, now let's write some code to make this Statement true. First, let's introduce an enumeration of digits:

public enum DigitKeys
{
 Zero = 0,
 TODO1, //TODO
 TODO2, //TODO
 TODO3, //TODO
 TODO4, //TODO
}

Benjamin: What's with all those bogus values? Shouldn't we just enter all the digits we support?

Johnny: Nope, not yet. We still don't have a Statement which would say what digits are supported and thus make us add them, right?.

Benjamin: You say you need a Statement for an element to be in an enum??

Johnny: This is a specification we're writing, remember? It should say somewhere which digits we support, shouldn't it?

Benjamin: It's difficult to agree with, especially that I'm used to write unit tests, not Statements and in unit tests, I wanted to verify what I was unsure of.

Johnny: I'll try to give you more arguments later. For now, just bear with me and note that adding such Statement will be almost effortless.

Benjamin: OK.

Johnny: Now for the implementation. Just to remind you - up to now, it looked like this:

public class Calculator
{
  public const string InitialValue = "0";
  public string Display()
  {
    return InitialValue;
  } 
}

This is not enough to support displaying multiple digits (as we saw, because the Statement saying they should be supported was evaluated to false). So let's evolve the code to handle this case:

public class Calculator
{
 private int _result = 0;
      
 public void Enter(DigitKeys digit)
 {
  _result *= 10;
  _result += (int)digit; 
 }

 public string Display()
 {
  return _result.ToString();
 }
}

Johnny: Now the Statement is true so we can go back to it and make it a little bit prettier. Let's take a second look at it:

public void [Fact]
ShouldDisplayAllEnteredDigitsThatAreNotLeadingZeroes()
{
 //GIVEN
 var calculator = new Calculator();
 var nonZeroDigit = Any.Besides(DigitKeys.Zero);
 var anyDigit1 = Any.Of<DigitKeys>();
 var anyDigit2 = Any.Of<DigitKeys>();
          
 //WHEN
 calculator.Enter(nonZeroDigit);
 calculator.Enter(anyDigit1);
 calculator.Enter(anyDigit2);
          
 //THEN
 Assert.Equal(
  string.Format("{0}{1}{2}", 
   (int)nonZeroDigit, 
   (int)anyDigit1, 
   (int)anyDigit2
  ),
  calculator.Display()
 );
}

Johnny: Remember you said that you don't like the part where string.Format() is used?

Benjamin: Yeah, it seems a bit unreadable.

Johnny: Let's extract this part into a utility method and make it more general - we'll need a way of constructing expected displayed output in many of our future Statements. Here's my go at this helper method:

string StringConsistingOf(params DigitKeys[] digits)
{
 var result = string.Empty;
      
 foreach(var digit in digits) 
 {
  result += (int)digit;
 }
 return result;
}

Note that this is more general as it supports any number of parameters. And the Statement after this extraction looks like this:

public void [Fact]
ShouldDisplayAllEnteredDigitsThatAreNotLeadingZeroes()
{
 //GIVEN
 var calculator = new Calculator();
 var nonZeroDigit = Any.Besides(DigitKeys.Zero);
 var anyDigit1 = Any.Of<DigitKeys>();
 var anyDigit2 = Any.Of<DigitKeys>();
          
 //WHEN
 calculator.Enter(nonZeroDigit);
 calculator.Enter(anyDigit1);
 calculator.Enter(anyDigit2);
          
 //THEN
 Assert.AreEqual(
  StringConsistingOf(nonZeroDigit, anyDigit1, anyDigit2),
  calculator.Display()
 );
}

Benjamin: Looks better to me. The Statement is still evaluated as true, which means we got it right, didn't we?

Johnny: Not exactly. With such moves such as this one, I like to be extra careful. Let's comment out the body of the Enter() method and see if this Statement can still be made false by the implementation:

public void Enter(DigitKeys digit)
 {
  //_result *= 10;
  //_result += (int)digit; 
 }

Benjamin: Running... Ok, it is false now. Expected "243", got "0".

Johnny: good, now we're pretty sure that it works OK. Let's uncomment the lines we just commented out and move forward.

Statement 3: Calculator should display only one zero digit if it's the only entered digit even if it is entered multiple times

Johnny: Benjamin, this should be easy for you, so go ahead and try it. It's really a variation of previous Statement.

Benjamin: Let me try... ok, here it is:

[Fact] public void 
ShouldDisplayOnlyOneZeroDigitWhenItIsTheOnlyEnteredDigitEvenIfItIsEnteredMultipleTimes()
{
 var zero = DigitKeys.Zero;
 var calculator = new Calculator();
      
 calculator.Enter(zero);
 calculator.Enter(zero);      
 calculator.Enter(zero);
      
 Assert.Equal(
  StringConsistingOf(DigitKeys.Zero), 
  calculator.Display()
 );
}

Johnny: Good, you're learning fast! Let's evaluate this Statement.

Benjamin: It seems that our current code already fulfills the Statement. Should I try to comment some code to make sure this Statement can fail just like you did in the previous Statement?

Johnny: That would be wise thing to do. When a Statement is true without requiring you to change any production code, it's always suspicious. Just like you said, we have to modify production code for a second to force this Statement to be false, then undo this modification to make it true again. This isn't as obvious as previously, so let me do it. I'll mark all the added lines with //+ comment so that you can see them easily:

public class Calculator
{
 int _result = 0;
 string fakeResult = "0"; //+
      
 public void Enter(DigitKeys digit)
 {
  _result *= 10;
  _result += (int)digit; 
  if(digit == DigitKeys.Zero) //+
  {  //+
    fakeResult += "0";  //+
  } //+
 }

 public string Display()
 {
  if(_result == 0)  //+
  {  //+
   return fakeResult;  //+
  }  //+
  return _result.ToString();
 }
}

Benjamin: Wow, looks like a lot of code just to make the Statement false! Is it worth the hassle? We'll undo this whole modification in a second anyway

Johnny: Depends on how confident you want to feel. I'd say that it's usually worth it - at least you know that you got everything right. It might seem like a lot of work, but it actually took me about a minute to add this code and imagine you got it wrong and had to debug it on a production environment. _That_ would be a waste of time.

Benjamin: Ok, I think I get it. Since we saw this Statement turn false, I'll undo this modification to make it true again.

Johnny: Sure.

Epilogue

Time to leave Johnny and Benjamin, at least for now. I actually planned to make this post longer, and cover all the other operations, but I fear I'd make this too long and get you bored. You should have a feel of how the TDD cycle looks like, especially that Johnny and Benjamin had a lot of conversations on many other topics in the meantime. some of the topics are already expanded on this blog, some others I'll be touching sometime soon.

Till next time!