If you’ve been following my blog for a little while you’ll see that my tdd-kata has been progressing, I’m still not able to finish the whole thing but I’m definitely getting better at it. I have noticed that my tests seem to run slower than the demos even though I’m writing pretty simple tests that only run in memory and don’t touch anything external. In The Art of Agile Development James Shore states that you should be able to run about 100 unit tests in a second… as things were I was able to get 9 unit tests to run in around 2 seconds. Something isn’t right.

I’m using the current version of MbUnit 3 (3.2.5.17) really because when I started looking into unit testing I figured Mb (Much Better) Unit Testing… it’s gotta be better than NUnit. However it looks like the problem is Gallio trying to do too much.

At present Gallio can run tests from MbUnit versions 2 and 3, MSTest, NBehave, NUnit, xUnit.Net, csUnit, and RSpec.  Gallio provides tool support and integration with AutoCAD, CCNet, MSBuild, NAnt, NCover, Pex, Powershell, Resharper, TestDriven.Net, TypeMock, and Visual Studio Team System.

Which is great but because it’s doing so much I think it has more work to do (somehow it has to parse and understand the results from all those other frameworks and merge them into just 1 report) and that contributes to a slower experience. I had a quick look to see if I could set it so it only looks for MbUnit tests and ignores all the others, but alas… I couldn’t find such an option. I may be wrong here – I haven’t spent very long looking, and actually I’m hoping that someone reading this post can point me to a configuration option that does just that.

It seems rather odd to me that MbUnit / Gallio would be set, by default / ‘out of the box’ to be the slowest it can be. I would have thought (and again I’m no expert here) that most people would use just a single unit testing framework per project / solution and anyone that’s using more than one framework is likely to be relatively advanced and so able to make configuration changes to support their other frameworks. I would go so far as to say that it is a positive barrier to anyone looking to adopt MbUnit.

Fortunately MbUnit is really a super-set of NUnit so, by way of  a test, I ran my test suite for the tdd kata using MbUnit and then changed my references to NUnit and ran the tests again. MbUnit took 2.54 seconds, NUnit took 1.28 seconds.  Now I realise this is for a really small test sample but that’s significant to me, the longer I have to wait about for tests to run the less happy I am. Well, I say I ran the same tests… in actual fact there is a very useful feature of MbUnit that’s missing in NUnit – RowTests and I’ve come to love (and depend, for clarity’s sake) on RowTests. So I had to create separate tests for each row that I had in MbUnit so actually I ran a few more tests in NUnit than I ran in MbUnit but NUnit was still faster.

Ok so NUnit is faster but MbUnit has row tests… does NUnit support row tests. Well it seems that RowTests were in NUnit 2.4.7 but, as the comments in that post attest, they was removed in 2.5 but not really documented. Googling I quickly got to the the wiki answer for RowTest on NUnit which pointed me to the RowTestExtension for NUnit which I have just tried it and... I can't get it to work.
The version that's up on Google Code was built against 2.4.8 (and last changes were in 2008, so it doesn’t look like it is being developed or supported). When I dropped the dlls into the extensions folder for NUnit and tried to access the feature... nothing, it also wasn’t registering in Visual Studio. So then I downloaded the source files and tried to recompile them against 2.5.7, which is the version I've got. There were some build errors, methods not found and I'm putting that down to changes in NUnit between these versions. Well I'm not familiar enough with NUnit to know what the namespace changes would be or even where to start looking (plus I've not got the time), so it looks like RowTests are not possible with NUnit - which is too bad as they really kick ass in MbUnit (just a shame MbUnit/Gallio is so much slower)... maybe I'll look to xUnit.net in future?

Practicing the same thing over and over makes you better at doing it; this is true for anything, sports, martial arts, painting or even writing code.

That’s the aim behind the tdd kata that I’m doing every day (well every day at work ) and I’m seeing slow, but sure, progress. I posted the other day about my tdd kata progress and said that I was getting only so far and that I couldn’t complete step 3. Well today I managed to finish that step and with a few minutes to spare. Now I’ve unblocked myself I need to keep practicing to ensure it sticks. It reminds me of when I started doing chin-ups; at first I could barely manage to haul myself up, then after a while I could do 9 reps and I was stuck at 9 reps for 2-3 weeks – I just couldn’t get into double figures. Then one day I managed 13, from 9 to 13 in a day. Once I’d broken the (mental) barrier it seemed that I’d unlocked things and managed to make progress once again. The important thing here is that I kept at it, sure I spent a while doing the same thing and not seeing any progress but it all helped.

You may be able to tell from this post that I’m actually feeling pretty pleased with myself and I really have nothing more to say – I just wanted to tell someone about my progress!

A big part of the TDD kata I’m doing is running the tests (kinda obvious huh?) and as I’m practicing this mouselessly it is essential that I can invoke my test runner with a keyboard shortcut. My current runner is TestDriven.Net, I’ve been using it for a while and I’m quite happy with it – 3.0 dropped in June so it is being actively developed.

Because I like trying new things, and the tdd kata is a way for me to learn new things, I thought I’d see how the CodeRush Unit Test Runner compared. So after I’d finished with the kata and I had a few tests in my suite I ran them all with TestDriven.Net and the Unit Test Runner.

With TestDriven.Net the tests below total execution time was 1.604 seconds and took 2.24 seconds, not bad, not very fast but not bad. The Unit Test Runner reported a duration of 1.85948165430718 (pretty precise). I’m not sure (and it’s not clear) whether that’s the total execution time, in which case it is quicker than TestDriven.Net or if it’s the total time it was running for, in which case it’s slower than TestDriven.Net. But there is a difference, so giving it the benefit of the doubt I thought I’d try to use the Unit Test Runner instead of TestDriven.Net and see how I got on.

TestDriven.Net can be invoked by pressing the menu key + U, pretty quick though a little awkward to reach the menu key. Alternatively you can bind to any keystroke you want through the Tools > Options > Keyboard settings. Great, now I wanted to find the shortcut I could use to invoke the Unit Test Runner… uhm, nothing is mentioned in the menus and looking in the Keyboard settings I couldn’t figure out what I needed to bind, nothing jumped out at me as an option. So I Googled it for a bit and then went to the DevExpress support system to see what they said. Unfortunately the only post I found, that seemed relevant was Unit Test Runner Window Shortcut which was posted in June and confirmed that there is no shortcut available at the moment – though there are plans to create a shortcut in a future release.

To be honest this is pretty much a deal breaker for me. I’m really happy with just using the keyboard and lifting my hands off just to invoke a test, something I’m supposed to do every minute or so, is really slowing me down. I have to grab the mouse, work out where the cursor is, move the cursor over an icon then click the icon and make a selection (do I want to run the tests or debug the tests) – I know that sounds like a pathetic complaint, but really doing this as frequently as I need to in order to practice TDD is quite painful. Please DevExpress fix this in the very next release!

Update

Didn't think I'd need to update this post this quickly; it's been less than 3 hours and DevExpress have responded to my plea! In 10.1.6 (i.e. the next release) they've added the "ShowUnitTestRunner" action, so you're able to bind any shortcut for opening the Test Runner, which is good news, though I would need to request a daily build or wait for the official release to take advantage.

However Rory Becker has come up trumps with 2 options that you can use today! The first of which is to simply - doh! why didn't I think of that, and the other is a plug-in that shows the test runner

I bound Ctrl+R, Ctrl+T (run tests) to RunSolutionTests and it works great. I will also bind something for RunTestsAtCursor, which are the two I use the most frequently.

My test suite

using System;
using MbUnit.Framework;
using tddKata.Core;

namespace tddKata.Tests
{
    [TestFixture]
    public class StringCalculatorTests
    {
        [Test]
        public void Add_EmptyString_ReturnsZero()
        {
            int actual = StringCalculator.Add("");

            Assert.AreEqual(0, actual);
        }

        [Test]
        [Row("1", 1)]
        [Row("2", 2)]
        public void Add_SingleNumber_ReturnsNumber(string numbers, int expected)
        {
            int actual = StringCalculator.Add(numbers);

            Assert.AreEqual(expected, actual);
        }

        [Test]
        [Row("1,2", 3)]
        [Row("2,3", 5)]
        public void Add_TwoNumbers_ReturnsSumOfNumbers(string numbers, int expected)
        {
            int actual = StringCalculator.Add(numbers);

            Assert.AreEqual(expected, actual);
        }

        [Test]
        [Row("1,2,3", 6)]
        [Row("2,3,4", 9)]
        public void Add_MultipleNumbers_ReturnsSumOfAllNumbers(string numbers, int expected)
        {
            int actual = StringCalculator.Add(numbers);

            Assert.AreEqual(expected, actual);
        }

        [Test]
        [Row("1\n2,3", 6)]
        [Row("1\n2\n3", 6)]
        [Row("2\n3,4", 9)]
        public void Add_MultipleNumbersNewlineDelimiter_ReturnsSumOfAllNumbers(string numbers, int expected)
        {
            int actual = StringCalculator.Add(numbers);

            Assert.AreEqual(expected, actual);
        }
    }
}

It’s the start of a new week, so I thought I’d update my progress with the TDD Kata I mentioned last week.

By way of a quick recap the TDD Kata is a series of small programming tasks designed to help practice and develop skills in TDD, the full details can be found on the original post TDD-Kata-1 but I’m presenting the summary here:

1. Create a simple String calculator with a method int Add(string numbers)
   1. The method can take 0, 1 or 2 numbers, and will return their sum (for an empty string it will return 0) for example “” or “1” or “1,2”
   2. Start with the simplest test case of an empty string and move to 1 and two numbers
   3. Remember to solve things as simply as possible so that you force yourself to write tests you did not think about
   4. Remember to refactor after each passing test
2. Allow the Add method to handle an unknown amount of numbers
3. Allow the Add method to handle new lines between numbers (instead of commas).
   1. the following input is ok:  “1\n2,3”  (will equal 6)
   2. the following input is NOT ok:  “1,\n”
   3. Make sure you only test for correct inputs. there is no need to test for invalid inputs for these katas
4. Allow the Add method to handle a different delimiter:
   1. to change a delimiter, the beginning of the string will contain a separate line that looks like this:   “//[delimiter]\n[numbers…]” for example “//;\n1;2” should return three where the default delimiter is ‘;’ .
   2. the first line is optional. all existing scenarios should still be supported
5. Calling Add with a negative number will throw an exception “negatives not allowed” - and the negative that was passed. If there are multiple negatives, show all of them in the exception message

---

Stop here if you are a beginner. Continue if you can finish the steps so far in less than 30 minutes.

---

1. Numbers bigger than 1000 should be ignored, so adding 2 + 1001  = 2
2. Delimiters can be of any length with the following format:  “//[delimiter]\n” for example: “//***\n1***2***3” should return 6
3. Allow multiple delimiters like this:  “//[delim1][delim2]\n” for example “//[*][%]\n1*2%3” should return 6.
4. make sure you can also handle multiple delimiters with length longer than one char

The idea is that by practicing the same techniques over and over that you’ll ingrain them and they soon feel like second nature to you.

So how am I doing?

Well, without wanting to make too many excuses about Visual Studio taking forever to load up the Add Reference dialogue or struggling with adding projects (I have asked for a more powerful machine at work, but so far no joy), I always complete step 2 and with good refactoring – and getting this far seems to take me about 15-20 minutes. Step 3 however seems to be my nemesis. I haven’t, yet, found a way I’m happy with to allow a newline delimiter as well as a comma, it really is like trying to run in treacle – all progress stops. That said I am doing everything without using my mouse – I’m quite pleased with that; also as I said, I’m refactoring well, so nice, small, well named methods and I am doing TDD and it is starting to feel quite normal to write a test then follow the Red, Green, Refactor path.

What am I going to do about getting past my road block?

There are a number of video run throughs of the kata that I can learn from, going back to the martial arts metaphor this is like watching the senior students practicing – I can try to imitate their techniques and learn from their approaches. The two that appeal to me are Andrew Woodward doing the kata mouseless in .NET with NUnit and The kata in .NET with XUnit because they’re both done mouselessly and in .Net. I like Andrew’s approach because he is practicing Uncle Bob’s ‘refactor till you drop’ and I’ve just watched a good presentation by Uncle Bob on Bad Code, Craftsmanship, Engineering and Certification which mentioned it and it struck a chord with me.

Bobby Johnson’s demonstration (the kata in .NET with XUnit) is nice because he is using some nice new language features like LINQ that make his code nice and terse. Also he’s using XUnit and I’d like to give that a try soon.

Code Kata are small programming tasks that are repeated regularly to help developers learn and develop their skills and just the other day I discovered TDD Kata on Roy Osherove’s blog which he describes as “an exercise in code, refactoring and test-first” and recommends you practice daily.

The idea of kata is borrowed from martial arts in which patterns are practiced over and over until the techniques become ‘second nature’. When you start out the techniques are new and might feel strange, but over time you become more familiar with them and with continued practice refine your techniques becoming more efficient and more proficient in their use.

So today I started my practice; it is time-boxed to 30 minutes and one of my goals was to do the exercise without using my mouse to become more proficient in just using the keyboard. The exercise is to create a String Calculator using TDD; I won’t bore you with a blow-by-blow account of the steps I took on my first attempt. I didn’t complete the kata, but I will repeat it tomorrow and the day after and so on – each time I will aim to be just a bit better than before so I make small, incremental progress in learning and internalising the practice of TDD.

Incidentally there are some recorded sessions showing people doing the kata and in quite a few languages, so if you want to peek and see how someone else (in your favourite language) approaches this kata you can.