Sandro Mancuso & Uncle Bob comparative design episode 1 notes

Update: Sandro commented on my notes in a twitter thread.

Notes from episode 2
Notes from episode 3/4/5
Notes from episode 6

I just finished watching episode 1 of Uncle Bob and Sandro Mancuso's London vs Chicago Comparative Design Case Study and enjoyed it. I really recommend this case study to anyone who wants to learn more about TDD.

This episode 1 was about Sandro beginning writing a twitter-like application and the first tests. Sandro claims to use the London Style TDD school, which is heavily based on mocking and outside-in approach. The school originated (AFAIK) from the Growing Object Oriented Software Guided By Tests book. Because of that and because I prefer this approach to design as well, I thought it could be interesting to write down, even if only for the "future me", the notes on the differences I think I can see between what Sandro did and what GOOS would do and what I would do. Of course, this is only based on episode 1, so some things may change during the next episodes.

Disclaimer

1. This is a set of notes, not a full blown article, so its structure might be a bit too dense and unsolicited. I welcome discussion and requests to clarify stuff. If you didn't see the episode, there is a high chance some of what I write will not make sense to you.
2. This post is not a critique of Sandro's style. I am not claiming that I am better in TDD than Sandro or better in this particular style of TDD. While I may sometimes sound as if I find some ideas silly, it's just for showing this stuff from the perspective of my biases and default choices (this is why sometimes I will describe my feelings, not only rational judgement). I am by no means suggesting that my choices are/would be better. I am sharing this because I think that sharing thoughts (and feelings) leads to interesting discussions. If you find me being disrespectful, please point it out so that I can correct it.
3. While I point some of the differences I think I see between GOOS and Sandro's TDD, it's not that I do EVERYthing like in GOOS. I am not including the differences between my TDD and GOOS TDD because I figured nobody would be interested in that :-).
4. I am not an authority on what is and is not compliant with GOOS style. I just write my subjective opinion based on my observations that may be wrong. If you see me missing the point, please point it out so that I can correct it and learn from the exercise.

Differences between Sandro's TDD and GOOS:

1. GOOS worked example started with a single acceptance test, then some code was written (without unit tests) just to make this acceptance test pass, resulting with a walking skeleton. The next step would be to refactor the existing code based only on the acceptance test and then introduce some first abstractions and place to insert mocks for future behaviors. Sandro does two things differently: 
1. He has all the acceptance tests written up-front (Update: this was explained by Sandro as being an input condition for the whole exercise)
2. He drives even the very first feature with unit tests already.
2. GOOS advises writing the acceptance tests in a domain language. For that, it recommends building a testing DSL that isolate a test from the implementation details like delivery mechanism. Sandro uses plain RestAssured library calls without (almost) any abstractions on top of it. This may be because Sandro's tests are not really end-to-end, but they are tests for HTTP contracts between the front-end and the back-end (Update: this was explained by Sandro as being an input condition for the whole exercise). Also, Sandro calls his tests "integration tests", which is a term that is used for a different kind of tests in GOOS, but that might not be an issue since, as I believe, the "integration" is more about the goal of a test, not about a level or kind.
3. GOOS advises usage of strict mocking framework (JMock), while Sandro uses a more-loose-by-default mocking framework (Mockito). Usage of strict mocking framework by GOOS was not merely a choice dictated by the availability of tools at the time the book was written. On the GOOS mailing list, there is a comment by Steve Freeman that he considers strictness-by-default part of his style (cannot find it now though).
4. GOOS advises mocking interfaces, while Sandro up to now has only mocked classes. As above, the choice to introduce and mock interfaces in GOOS was not dictated by the limitation of tools at that time. It was a deliberate choice to think of interfaces as roles that objects play and GOOS authors claim they are aggressive in refactoring interfaces to make them smaller and more abstract.

Now time for the differences between Sandro's TDD and my TDD:

1. Sandro starts all his tests from a name and then an assertion. As this is one of my ways to start a failing tests, it's not the only one that I use. In my book, I describe 4 different approaches to starting with a failing test, starting from assertion being one of them.
2. Sandro uses underscores in test names and lower camel case in production code method names. It's just a matter of preference, but I find that confusing - I find it better to stick to a single convention, which has the added value of simplifying the configuration of static analysis tools that enforce naming convention and takes one dilemma away. Still, this is not a big deal, just a matter of style.
3. Sandro tends to prefer realistic test data even where this is not needed or required. In the first test, where he passes data for user registration, he chooses a realistic name, (semi)realistic password and the ID is generated as UUID and converted to string. This is not needed at all, because the object under test does not handle any validation and, as we find out later, the only class that knows that the user ID is actually a UUID is the IdGenerator - in the rest of the cases, any string would do. Personally I use dummy data wherever validating this data is not the responsibility of the object under test because I find that by doing it, I describe the contract of my class with its collaborators more precisely. Whenever I see a specific/realistic data, I interpret it as if it was required by the tested object. Also, I am confused about one more thing - if I use another ID generator or change the implementation of existing one, should I change the "realistic data" in the tests where it doesn't matter?
4. Also, I wouldn't establish relationship between test data fragments where it is clearly not required. For example, Sandro creates a registration object with "name", "password" and "about" fields and sets up a mock of a service object to return a User object with the same three values. Personally, I don't see why. The responsibility for establishing relationship between input and output values is clearly pushed to another class that is mocked (a service class), so in my mind, this is leaking responsibilities of another class to tests of class that clearly does not care. And as I understand GOOS compositional approach to design and its principle of context independence, a class and its tests should not assume in what context the class is used.
5. Sandro moves parts of the code to the setup method and parts of the data to fields of class with tests, arguing that this is cleaner. I wrote a blog post long ago explaining why I don't like this idea and why I find such tests harder to read and maintain with the "Testcase class per class" organization approach.. Looking at the video, the setup method is called "initialise", which I find a meaningless name and usually read it as "bag for everything I don't want to look at". Thus I consider it a smell and I also feel it takes away a bit of the pressure from tests to improve the design. When I see some logic that I strongly feel could be moved to a setup method and fields, this leads me to searching for a way to improve the design. I find setup methods hiding these issues for me. I understand that most probably, Sandro has a way of compensating for all the issues I described and maybe even making them irrelevant.
6. The method of UserAPI class that Sandro test-drives clearly violates the principle of command-query separation (it is a command that returns a value as well). This is often what frameworks require us to do, unfortunately. Sandro chooses to push this violation further into the application/domain layer (specifically - the UserService class). Personally, I tend to cut away from this kind of violation on controller level using a Collecting Parameter pattern and then I only deal with commands, which I find makes the rest of my tests easier.
7. In the controller test, Sandro includes parsing in the scope. I would probably push the parsing responsibility to a collaborator. I can see that Sandro can succeed with his approach with a simple, non-nested JSON with three fields, but I wonder if his approach would be the same if it was a bigger structure with several nesting levels, optional sections etc.

What's happening with my book and future plans

Hi, I haven't posted in a while, so a quick update:

• I am currently focusing on my book, that's why the updates on this blog are extremely rare.
• I have some nice topics to write blog posts about, but I'm putting them on hold for a bit. The steady increase pace of my book's content (it's already over 250 pages A4!) should make it up for you.
• Speaking about book's content, I recently finished an overhaul of part 1, which took me a long time and lots of effort.
• I'm currently fixing some bugs submitted on the book's github page for some portions of part 2
• When I finish this, I plan on continuing part 2, review the value objects chapters and probably move TDD with mocks to part 3.
• After that, I want to write something about hexagonal architecture (which will probably be part 4) and TDDing on a level of a whole component, using mentioned hexagonal approach (maybe part 5?).

There is still a lot of work ahead of me, so thank you all for your support and I hope you like the new book content. I not, well, you can still contact me with your thoughts and opinions!

New release of my book

Release notes are on the books' github page. Enjoy!

Is test isolation always a good thing?

TDD is not only about unit tests. Steve Freeman and Nat Pryce argue in their book that a TDD cycle is kickstarted using an end-to-end automated test. These tests exercise a system as a black box. This often implies difficulties and issues that well written unit tests don’t have. On of such fields of difficulty is test fixture management, especially fixture teardown.

Gerard Meszaros in his book titled XUnit test patterns provides an extensive discussion of different fixture management strategies and their implications. Using Meszaros’ terminology, what I have seen many end-to-end tests using is a combination of:

• Persistent shared immutable fixture - this is the data that’s shared by many tests - usually some initial environment setup and plumbing, uploading licenses and so on. The “immutable” part of the fixture name means that no test is allowed to modify any data that’s part of this fixture. The reason is that each test relies on this data being intact. If any test modifies the fixture, then other tests’ assumptions are broken and we cannot expect them to work reliably.
• Persistent fresh fixture - this is the data that’s needed for a single test. This may be, for example, adding a specific employee required by the test scenario to a database. As database is a persistent storage, this data must be cleaned up either at the end of the same test that created it or at the beginning of any new test (there are some ways to achieve that safely, but that’s a topic for another post). The cleanup is needed to avoid test interactions - many tests may add an employee with the same ID number, but with other data being different. Thus we need to prepare a clean bed for each test so that leftover data from previous tests doesn’t get in the way (by the way, there are some other ways of dealing with test interactions than cleaning up - although they’re quite difficult to apply).
• Transient fresh fixture - this part of data is created inside a test and removed without our intervention when the test finishes running, usually by garbage collector or other memory management mechanism. The simplest example of such fixture would be an integer variable defined in a test method - we don’t have to worry about it impacting the other tests, because its lifetime is automatically associated with the lifetime of a test.

After reading XUnit test patterns, my conclusion was obvious - life would be beautiful if we could only use transient fresh fixture. When writing unit tests for classes or small clusters of classes, this kind of fixture is the only one we ever need to use. Things change, on the other hand, when e.g. our tests require installation of the application and a database - in such situation, using only transient fixture would probably mean that each test should perform a fresh installation of an operating system. This would take ages, so we use the other fixture types as a compromise. We accept the problems they bring and accept the fact that they can introduce interactions between tests (and thus, instability) and trade these things for shorter test execution time.

I met with an opinion that this is not actually bad. What’s more, the opinion states, this can be seen as an advantage, because when a single instance of an application is exercised by many different tests, it resembles closer the way the user exercises the system when it’s deployed and, by working longer with an existing instance of a system, it may uncover some defects that would otherwise not become visible.

I have several issues with this point of view on fixture management. In order to explain why is that, let’s recap the differences between how user exercises a system and how automated end-to-end tests do it.

What are the differences between usage by automated tests and by user?

There are three pretty big differences that I can think of:

An automated test has a scope, i.e. it has a beginning

When defining a test, we usually define the context in which a test should be executed, a series of steps and an expected outcome. For example, we might say:

Given there are two employees: Ken and Tom //the context
When I choose to remove Ken's data         //step 1
And list all my employees                  //step 2
Then I should only see Tom's data          //expected outcome

Here, the explicitly stated context is that we have two employees - if there is only one, executing the two steps would most probably create different results. Of course, there is also an implicit context that’s not written here, for example that the application is installed and turned on and probably that I’m logged into the application. The test relies on both kinds of context to be the same for every execution. In other words, in order to bring stable outcomes, a test requires a context to be defined. Users usually work in the context they are currently given and are able to adapt to it. For example, I want to create an e-mail account with an address "buziaczek@mymail.com", but this address already exists. No problem, I’ll just create "buziaczek-1984@mymail.com" and I’m ready to go. Most of the time, an automated test doesn’t make such decisions (although we can do something to bypass such situations, e.g. generating a portion of the e-mail address randomly, but note that it’s still not making a decision).

Thus, although it may happen for a user to follow a strict sequence of steps in a strictly defined context, this is not what their work with the application is about. If it was, we could probably automate these sequences in the software itself.

Anyway, a demand for strictly defined context for each automated test means that this context would usually have to be cleaned after or before each test (Meszaros writes about other ways, like generating distinct test data for each test, but I didn’t see them in practice all that often). This is somehow contradictory to wanting to achieve testing “as the user would” by executing tests in partially the same context (i.e. on the same running or installed application instance). The contradiction for me is that we seem to value test isolation, except that we don’t.

An automated test has a scope, i.e. it has an end

A user uses a product in a continuous manner, usually taking many, many steps during each session with an application. On the other hand, when reading on good practices for automated testing, one can spot that such tutorials hardly ever advise writing long, automated scenarios. It is more common to see several test cases for several different behaviors or variants of a single behavior rather than a single test case exercising all of these conditions. So, it looks like we deliberately try to run away from testing the system the way a user would in automated tests. Why is that?

This is for several reasons. One of them is that most of the time, a failed assertion ends an automated test, so the longer the test, the harder it is to get feedback from the later parts of that test, because there are more things that can fail along the way. Two other reasons that I’d like to talk about are: splitting work into manageable chunks and defect localization.

Splitting into manageable chunks means that we prefer shorter scenarios to longer ones because it is easier to maintain and modify each scenario independently when it’s smaller. This is because building up context becomes increasingly harder to grasp, understand and manage. Users, on the other hand, don’t need to manage scenarios - they dynamically choose their goals and steps based on the current situation.

Defect localization means that when a test fails, I need to search for the failure root cause only among what’s in the scope of this test. When a scope of a test is smaller, I have fewer places to search for such root cause. This is also one of the reasons why we set descriptive names for tests, i.e. we’d rather say “the user should be able to delete an employee” than “the user should be able to play with the system” - because in the first case, a failure points us closer to the root cause. The usual thing I do when I see an automated test failing is to try and run it in isolation (i.e. not as part of a suite) and in clean environment. This is because it gives me a better feedback on whether the failure is caused by something that’s in the scope. So, the broader the scope of a test, the lower defect localization.

In his wonderful book, Lean-Agile Acceptance Test-Driven Development, Ken Pugh outlines three conditions we strive to achieve for automated tests:

1. A test should fail for a well-defined reason.
2. No other test should fail for the same reason.
3. A test should not fail for any other reason.

And, while Ken admits this is often very hard (or impossible) to achieve, this is the ideal we should be targeting. Managing a persistent fixture introduces more complexity and possible root causes of test failures, driving us away from these three conditions.

An automated test has a scope, i.e. it has a purpose

As I said above, a test should fail for a well-defined reason. This well-defined reason for failure is a purpose of the test. In other words, every test failure should relate to its purpose. This purpose is usually conveyed in test name, for example “An added employee should appear on an employee report”. Note that very rarely do we see tests like “previous tests runs should not have led the application into an illegal or crashing state”. This is because the tests aren’t usually written to check whether an application crashes if we do some random things with it. Sharing parts of the fixture between tests doesn’t help them achieving their purpose.

I also said that it is probably impossible to get rid of all the things that are beyond the purpose of a test but still can impact the outcome of the test. For example, if a test loads anything from a disk, it can fail because of (unlikely but possible) disk drivers failure. These things are outside the scope of a test, but still influence its result.

The point is, the more things exist outside the scope of an automated test that can affect its outcome, the more the test becomes incompetent at testing what’s inside its scope. Do we want the tests to be incompetent in fulfilling their purpose? I hope not. Thus, my preference is to maximize the competency of a test by reducing to a minimum other things that can impact its outcome. This is how I believe I get the most value out of automated tests. This is in line with Meszaros’ write-up on the goals of test automation, where he stresses the ease of maintainability as one of such goals.

Automated tests teardown is often very error-prone and adds a lot of instability to the test suite, thus making it less competent in doing the thing it written for. For some time, For test execution time-related reasons, I’m afraid we’ll have to live with sharing some parts of persistent fixtures, however, If I could get rid of the necessity of teardowns in tests without sacrificing their run time, I’d do so in a blink of an eye.

Summary

My opinion on test isolation is that the more the better. For me, saying that partial lack of isolation is better (because it would uncover some defects only a user otherwise could) is really taking the biggest disadvantage of automated black-box tests and pretending it’s an advantage. It is relying on undeliberate potential effect of this disadvantage. Creating a test approach targeted towards uncovering such defects (i.e. relying on deliberate effort to uncover them instead of hoping they will come up from other tests) would in my opinion give a better return of investment.

Two new chapters on value obejcts

Hi!

It has been long since I last wrote about the progress of my book. Two chapters were added since, and both on value objects:

1. Value objects
2. Value object anatomy

Happy reading and, as always, I welcome any feedback!

I recorded a video: FizzBuzz To Chain of Responsibility to DSL using TDD in Java

Sorry for bad narration - I did very minor preparation to record this video. Also sorry for saying "dividable" instead of divisible.

This movie demonstrates implementation a variation of FizzBuzz kata without error checking, but with refactoring towards the Chain of Responsibility pattern. The first part is done using TDD and triangulation.

New chapter: Viewing object composition as a language

Hi!

New chapter on viewing object composition as a language was added. This chapter is about how the idea of having a system of composable objects leads to inventing a set of mini domain-specific languages.

Happy reading and I really welcome any feedback!