If you using a unit testing framework such as NUnit, you’re no doubt familiar with the [Ignore] attribute.  (Note:  I haven’t used MbUnit in a while, but I’m pretty sure that all this applies there as well.) For those who aren’t, when you adorn your [Test] with this attribute, then your test runner should pay no heed to this particular [Test].  Instead of going green or red, your test turns up as yellow in your test runner. At first blush, this seems handy – rather than deleting a failing or broken test you can have it [Ignore]d, much like your typical computer geek at a high school dance before the punch get’s spiked. Now you can deal with the wayward test on your own time with the pesky failure cluttering up your CI build (you do have a CI build box set up, right?).

Please bear with me while I submit to you: This isn’t a good idea. This is a bad idea. An ugly idea. The kind of bad, ugly idea than even Belcham-esque quantities of scotch couldn’t make pretty. Why? Well, the whole point to having a unit test is to test something. If your test is failing, that means something is broken.  You should be addressing the cause of the failure and dealing with the problem now. Covering up the failure with some attribute kind of defeats the purpose of having the unit test in the first place. I mean, really, if you had open, festering lesions on your skin would you just cover them up with a band-aid and pretend nothing was wrong?  Odds are you’d go to a doctor and get that checked out.

Alternately, perhaps the test isn’t valid anymore: your application has changed/evolved and the code that you were testing isn’t around anymore. Then, get rid of the test! Delete it from your test fixtures, and don’t look back.

Granted the world we live in is not black and while.  Sometimes we’re working on things, and we only want tests to run when it is explicitly asked to run.  To handle this scenario, NUnit provides the [Explicit] attribute.  This is a kinder, more gentle kind of ignore.  This instructs NUnit to only run the test when it is told to run the test.  If you add a comment to the attribute, then it helps those going through the build log understand why the test is being run.  However it is still possible to run the test if desired.

[Explicit("This test takes a really really really long time to run so we don't want to run it in this test suite every time")]

[Test]

public void My_very_important_test()

{

    // Test valuable things here

}

I know some people are going to protest my advice. So I will offer up this one last piece.  If you are going to [Ignore] a [Test] then please take advantage of the fact that you can add a comment to the Ignore which will show up when NUnit ignores the test:

[Ignore("This test is ignored as I wish to incur the wrath of Cthulhu (or his proxy)")]

[Test]

public void My_very_important_test()

{

    // Test valuable things here.

}

Now when your CI server runs your unit tests, you will have a nice friendly message that will inform others as to why the test is not running.

Usually when I develop, I write individual tests and then run individually with TestDriven.NET and collectively with Resharper.  In this case, as I was writing one unit test, it kept failing with TestDriven.NET.  At first I thought it was just me being stupid (always a good place to start), and then I was worried that perhaps the documentation for TypeMock was to blame.  So, I sent a email to Avi at TypeMock, mostly expecting confirmation of my stupidity.

Turns out the problem in this case was pretty obscure one (to me).  At home, I do my development in a VM running Windows 2008 64 bit.  When Resharper runs your unit tests, it does in a 32 bit process.  TD.NET, on a 64bit OS,  runs as a 64 bit process.  If you get TD.NET to run as a 32 bit process, then your tests will run fine under TD.NET.

To do so, from the Visual Studio 2008 Command Prompt, change to the directory that you have TD.NET installed in.  Then use corflags:

corflags /32bit+ ProcessInvocation.exe

Note that it’s probably best to do this when logged in as the Administrator.

I will say that I was happy with the support I got from the guys (Avi, Gil and Ohad) at TypeMock.  Given the fact that we’re about a half a world apart (western Canada vs. Isreal), they provided prompt, excellent support.  My thanks to them.

Usually when I develop, I write individual tests and then run individually with TestDriven.NET and collectively with Resharper.  In this case, as I was writing one unit test, it kept failing with TestDriven.NET.  At first I thought it was just me being stupid (always a good place to start), and then I was worried that perhaps the documentation for TypeMock was to blame.  So, I sent a email to Avi at TypeMock, mostly expecting confirmation of my stupidity.

Turns out the problem in this case was pretty obscure one (to me).  At home, I do my development in a VM running Windows 2008 64 bit.  When Resharper runs your unit tests, it does in a 32 bit process.  TD.NET, on a 64bit OS,  runs as a 64 bit process.  If you get TD.NET to run as a 32 bit process, then your tests will run fine under TD.NET.

To do so, from the Visual Studio 2008 Command Prompt, change to the directory that you have TD.NET installed in.  Then use corflags:

corflags /32bit+ ProcessInvocation.exe

Note that it’s probably best to do this when logged in as the Administrator.

I will say that I was happy with the support I got from the guys (Avi, Gil and Ohad) at TypeMock.  Given the fact that we’re about a half a world apart (western Canada vs. Isreal), they provided prompt, excellent support.  My thanks to them.

Chad Myers has a good, quick, post introducing QUnit, I’d strongly suggest checking it out if you do web development (which, IMHO, implies some use of JavaScript, right?).  Chad’s post covers the basics pretty good, and I think I would just be duplicating/plagiarizing Chad’s work if I were to blog the hour or two I spent getting to know QUnit here.

It is nice to see that JavaScript is slowly being recognized as first class citizen of the web programming world, and is gradually having it’s good name cleared of certain falsehoods.  It’s been my belief and observation for a while now that ASP.NET has done a pretty good job of stunting and retarding the adoption/use of JavaScript amongst .NET developers, at least in the Edmonton region.  Good to see that this attitude is starting to change.

Allow me to go out on a limb here, and state with absolute certainty that the correct answer is "It depends on your situation".  Allow me to elaborate.

1. The number sends a wrong message.  If your team decides that they need some arbitrarily value to shoot for, that is what they will concentrate on.  Developers will worry more about how many tests they need to write, rather than on the quality of their code.
2. Unit tests can suffer from the law of diminishing returns.  To achieve that mythical number of 100% coverage would require a large amount of effort.  I argue that in many cases, the high effort required drains away from the teams ability to keep delivering value in their software.  In particular, exception testing comes to mind.  You can spend a lot of time writing tests to cover parts of your code that, in theory, should never be executed.
3. Unit tests do not replace other forms of testing, so functionally speaking, you should still be covered.  Sure, you may have some code that isn’t covered with a unit test, but hopefully your crack team of functional testers will exercise that code as part of their efforts.  If your functional testers find a problem, and you don’t have a unit test that covers the code in question, then you will obviously write a test and fix the problem.  If your functional testers don’t find a problem, then you don’t worry about it. 
   • Remember that as a part of agile, you have short iterations with working software at the end of each iteration.  So, if there is some problem code that is not tested, your users will find out about it fairly quickly.  The power of rapid feedback at work.
4. The test part of TDD is a useful side-effect. The true value of TDD is that it helps you craft out a better design for your code.  If you ask me, TDD should stand for Test Driven Design, not Test Driven Development (I do believe credit for this observation belongs to Roy Osherove – at least that is where I heard from first).

Way back when I started out with TDD, I used to put the test fixtures in the same file as the class under test, sort of like:

public class ClassUnderTest

{

}


#if DEBUG

[TestFixture]

public class Tests_for_ClassUnderTest

{

}

#endif

My logic was basically this way, the test is right there – quick and easy to access.  Tests are, after all, supposed to serve as documentation, so by making the two classes share the same file, it was easy to find.  Plus, if you changed your class, you had quick access to your tests. When it came time to deploy, you could just did a RELEASE build, and the unit tests would not be included.  Or, if need be, send out the code with the unit tests included, and then your QA department could run your tests on their environment if desired.

However, I eventually moved away from this, for a couple of reasons:

I was still learning TDD, and I noticed I was doing a lot of thrashing.  This was because my test code is right next to my production code and I found myself falling into a "test last" pattern.  That is to say, I would write my code first, then my tests.  When my code didn’t work, I’d change it, and then "fix" my test so that my code would work.  I lacked the discipline and experience to write good, proper tests and I felt that part of the problem was it was to easy to fall back into the "Old Ways".

I found that by putting my test classes in a separate assembly, I would actually think about what I need, and that would result in better tests and (IMHO) better code.  After all, one of the real benefits about TDD is that it forces you to think about your design up front.  I found that by using a separate assembly to house my unit tests, I ended up changing how I approached TDD.  I spent a bit more time thinking about HOW my test would work, and HOW the code would be structured.  It sounds crazy, but it worked for me. 

In my experience, the good people in QA didn’t give two rips about our unit tests.  They added no value to them, as they were using their own tests.  So, if nobody cared to run the tests outside of the developers, there is no value in sending them out.  Think lean here:  no value, so no point to make the developer’s unit tests available.

Also, at the time, I didn’t like the idea that the code that was being deployed was the code that was being tested.  I’m not a big fan of compile time directives, and compile time directives change your code.  They always seemed to cause pain when you forgot one or misspelled it.  Of course, now that we’re older, and using Nant zealously, this isn’t really an issue.

So, for those reasons, I changed the structure of my solutions.  I briefly experimented with one test assembly per production assembly, but gave that up as to unwieldy.  If you had a project with six assemblies, then you’d double it up with six test assemblies.  Now when you wanted to run tests, you’d have 12 assemblies to compile and six to run the tests on.  Not a lot of fun.  Now imagine going onto a client site with 60 project solutions:  it gets nasty fast.

I eventually settled down to the conventional "…one test assembly to rule them all…" approach, and have been doing that for a while.  It seems to work well enough, and I never worried to much about the "urban sprawl" in my namespaces.  Resharper makes it pretty easy to find classes and such, so if the namespace of my testing assembly didn’t quite match up with that of my application, I never considered it to be a big deal.

I was never really concerned about making a class public, or a method public to aid in testability.  Most (all) applications I work on are basically in house line of business applications.  I’m not writing frameworks or API’s to be consumed to the general public.  The people that are using my classes can be trusted.  Besides, they all had access to the code anyway, so what is to stop them from just going in and change the class/method from internal to public anyway?

So, where should your tests live?  Really, I don’t think it matters.  It seems natural to me to have a separate test project, but that is only because I’ve been doing it that way for a long time.  I don’t really see anything wrong with putting your test fixtures in the same project as your production code.  I’d think it was great that you had unit tests (something one rarely sees here in Edmonton), so if housing tests and production code in the same assembly works for you, then why not?

In my Petshop2Monorail side project I’m working on in my spare time, I decided to see about using an in-memory database for my tests. I would get ActiveRecord to create the database each and every time a test fixture runs. There may (or may not) be some activity involved with populating the database, but at least I have tight control over the database schema and the data I’m working with.

I checked out the SQLite documentation, and in theory one should be able to use SQLite as an in-memory database. I couldn’t get it to work. I’m not to sure if it’s something I’m doing wrong or if it’s a limitation of the ADO.NET data provider I’m using. For now, I’m willing to turn a blind eye to this shortcoming, and use a file on disk. Yes, I am aware of some of SQLite’s other shortcomings (namely concurrency), but for the task at hand I don’t think that these shortcomings will negatively manifest themselves.  Yes, I’m aware that I’m having a unit test use a file on disk (something I say you shouldn’t do).

The one glitch I ran into was with ActiveRecord. I kept getting the error The IDbCommand and IDbConnection implementation in the assembly SQLite.NET could not be found. A quick bit of googling, and I had my answer. To summarize: There are two ADO.NET drivers for SQLite, and so therefore two NHibernate drivers: NHibernate.Driver.SQLiteDriver (old) and NHibernate.Driver.SQLite20Driver (new). NHibernate.Driver.SQLite20Driver is what I should have been using.

myGridView.RowDataBound +=

    delegate(object sender, GridRowEventArgs rowEventArgs)

    {

        InventoryPartVendorGridRowDecorator gridRowDecorator =

            new InventoryPartVendorGridRowDecorator(rowEventArgs.Row);

        gridRowDecorator.Decorate();

    }; 

Then, using the dynamic duo of NUnit and Rhino.Mocks, I would be able to test my decorator class independent of my web application.  I thought I could mock out the GridViewRow and a DataRowView, and be on my merry way.  Life would be good.  And it would have been, until the wretched DataRowView entered into the picture.

DataRowView sits pretty high up on the inheritance tree, doesn’t have any interfaces that are meaning full in this context, and it’s constructors are internal.  So, it proved to be pretty difficult to mock. 
To explain what I did, I will first show you my unit test:

[Test]

public void Decorate_InactiveVendor_TextDecorationLineThrough()

{

    MockRepository mockery = new MockRepository();

    GridViewRow mockGridViewRow = mockery.CreateMock<GridViewRow>(1, 1, DataControlRowType.DataRow, DataControlRowState.Normal);

    Hashtable dataRowViewDataItem = new Hashtable(1);

    dataRowViewDataItem.Add("Active", false);


    using (mockery.Record())

    {

        Expect.Call(mockGridViewRow.RowType)

            .Return(DataControlRowType.DataRow)

            .Repeat.Once();

        Expect.Call(mockGridViewRow.DataItem)

            .Return(dataRowViewDataItem)

            .Repeat.Once();

        mockGridViewRow.Style

            .Add(HtmlTextWriterStyle.TextDecoration, "line-through");

    }

    using (mockery.Playback())

    {

        InventoryPartVendorGridRowDecorator gridRowDecorator =

            new InventoryPartVendorGridRowDecorator(mockGridViewRow);

        gridRowDecorator.Decorate();

    }

} 

Notice what I did; I created a Hashtable and have my mockGridViewRow.DataItem return that.  In otherwords, I’m not mocking DataRowView at all, I just cut it out of the picture.  Then inside inside my InventoryPartVendorGridRowDecorator class I had to put this method:

private bool IsVendorActive()

{

    object val;

    object dataItem = _row.DataItem;

    // HACK [TO070508@1149] Because we can't mock DataRowView, we do this. 

    if (dataItem is DataRowView)

    {

        val = ((DataRowView) dataItem)[ActiveColumnName];

    }

    else if (dataItem is IDictionary)

    {

        val = ((IDictionary) dataItem)[ActiveColumnName];

    }

    else

    {

        val = null;

    }

    if (val == null)

    {

        throw new NullReferenceException("Could not find the field 'Active'.");

    }

    return TypeHelper.ConvertToBoolean(val, true);

} 

As you can see, what I ended up doing was sticking in a little "adapter" into my decorator.  Not pretty, but it works.

What would others do in this situation?