I have been using IronPython lately, and am having a good time testing code with its built-in unit test framework. Here is a brief example of how to write some unit tests for IronPython (or CPython if that's your flavor). These unit tests leverage the "unittest" module available in Python 2.2 and above.

Here is a short test class:

class TestCode:
    localInt = 1
    @classmethod
    def Method1(self):
        return 100
    def Method2(self, a, b):
        self.localInt = a * b

Here are some sample unit tests for it, using the unittest framework:

import unittest
class UnitTests(unittest.TestCase):
    def testMethod1(self):
        cut = TestCode()
        actual = cut.Method1()
        self.assertEqual(100, actual)

    def teststaticInt(self):
        self.assertEqual(1, TestCode.localInt)

    def testMethod2(self):
        cut = TestCode()
        cut.Method2(7, 4)
        self.assertEqual(28, cut.localInt)

if __name__ == '__main__':
    unittest.main()

This example should illustrate that we can write some unit tests for our Python code fairly easily and quickly. It's nice to have a built-in framework for unit testing. There is also a "doctest" module available for doing more like system testing. I am pretty new to Python, so I haven't got examples of this as yet.

If anyone has any suggestions on tools or experience with integration with Visual Studio those comments would be appreciated.

Shawn Neal has a great post (ok it might be a rant) on organizing test code. I agree with his article, on the separation of different kinds of tests into separate assemblies that perhaps can be run at different times (not always with the build). Only the true unit tests should probably be running with each build, and for each developer. If there are other tests that run very fast (by my definition, a fast test either passes or fails in under 1/10 second [100ms]) they too could be included in per-build and per-developer runs, provided that they don't bog down the total test time too much, and provided that they are in separate assemblies.

Someday, when technology allows for it, we should even be able to run our tests in parallel threads or parallel processes (see the 2.5 target features for NUnit). Speed is truly important. If your development team has 10 developers, every minute they wait for a build to complete or a test run to complete is 10 minutes a day of development time wasted. If they build and test 6 times a day each (that's not very much BTW) that is one team development hour per day. Realistically it's likely to be more like 5 or 6 development hours a day.

So if someone is paying attention to how time is spent, we could say perhaps that spending a couple of development hours on the tests to make them run faster, and save one minute per execution would more than pay for itself in a single day.

The moral of this story is keep the tests fast, and when they aren't, fix them or move them out of the main test path.

Acceptance criteria are the real key to being able to develop software in the agile model. Acceptance criteria is not really adequately represented by a use case, even with several alternate path scenarios... I have seen this and it just doesn't do the trick. A scenario is good, but it's really not acceptance criteria. We can however, analyze these scenarios and alternate paths, and usually come up with a pretty good set of criteria.

Beware of the incomplete criteria. It's darn hard to get it all. We know this. But, make an attempt anyway. Use the thinking that if it isn't in the criteria it won't be delivered, and that the developers can implement it any way they want... This thought should be scary enough for a customer that they will be forced to think hard about what they really want the software to do.

When we think we know what we are doing, we can write some tests, write some code. We think we have the idea, only to find out later that we had misunderstood or didn't have all the requirements. Acceptance criteria are absolutely the must-have thing for agile processes to succeed. We need the Customer or product owner [PO] to provide clear, concise, and specific criteria for our software.

I'm not saying that generating/gathering acceptance criteria is easy. It takes practice. It takes cooperation. It takes understanding on both the development team and the customer/PO that "if it isn't requested or specified in the acceptance criteria, it isn't going to be delivered." It's really kind of harsh actually to practice this, so I wouldn't advocate it. However, it might be a good idea to talk about it in those kind of terms while in the sprint planning or story negotiation phase.

Without the ability to know that we are doing the right thing, surely we will fail to deliver it. The customer knows exactly what they want. OK, most customers really don't know what they want. It is up to us as a development team to be able to know this, and help them out by asking lots of questions, and examining exactly what we propose to deliver. After all, that is why they hired us right?

Train your team. Let them know that the ancient role of "Business Analyst" is no place to be found in an agile team, and the whole team is really responsible for making sure to ask the right questions. I do like the whole user story concept, but it is far more practical in sprint planning to capture some of the actual acceptance criteria and store it attached to the story somehow, so that it is accessible by the team and can be implemented as automated tests.

Test automation is critical, mandatory, required, and not optional. Did I mention that test automation is required. Perhaps I should mention it again, in case my previous sentences weren't clear. Automate, Automate, Automate. Double-click run test suite, should be the demo in the sprint review and customer demonstration at the end of the sprint. This is really and truly the only way that agile teams can operate successfully.

If you have people doing any kind of manual testing (other than exploratory), this is a great opportunity to get the team to be far more productive by assisting the manual testers to get these tasks automated. Sometimes it is not practical to automate some manual tests. But in my experience - NOT OFTEN. If it looks hard to automate, someone should be asking why the code is so "untestable"... In most cases, the three or four dev or test hours it takes to automate the most difficult manual test case scenario is more than paid back in the number of times the automation can be run from then on. The number of bugs and breaks that it will prevent is a definite value add to the organization, and in my opinion, money in the bank.

The next Agile Beer Night is coming up on Tuesday September 30, at the Celtic Bayou pub in Redmond from 5PM to 7PM. Be there or be square...

I posted a simple sample of accepance test code in Selenium and WatiN along with a sample web site to test. You can download the zip file here.

I also have posted a Fitnesse fixture in that zip file that illustrates how we can create a simple test fixture for Fitnesse acceptance testing. The Fitnesse tests aren't in the set, but here is the page wiki code that makes use of the fixture:

!define COMMAND_PATTERN {%m %p}
!define TEST_RUNNER {dotnet\FitServer.exe}
!define PATH_SEPARATOR {;}
!path dotnet\*.dll

Here is an acceptance test using our BusinessObjectTestFixture test class:

!|FitnesseFixture.BusinessObjectTestFixture|
|UserId|Password|Authenticate()|
|administrator|secret0|ADMIN|
|admin|secret0|NONE|
|administrator|secret|NONE|
|user11|secret11|USER|
|user11|secret0|NONE|
|user|secret|NONE|


I also created some STIQ tests, here is the code for the tests and components. Extract this zip file under repository\ProjectRoot folder and it should be able to test the sample site also.

ATDDSTIQ.zip (3.57 KB)

I have been asked recently about tools I prefer to use in my every-day development. Here is a list of tools, and where to get them.

Visual Studio 2008      Development IDE                   Microsoft
TestDriven.net           Test Runner                         TestDriven.net
WinMerge                  Diff / Merge tool                   WinMerge.org integrates with VS and Tortoise too!
nUnit                        Unit Testing Framework          nUnit.org     Also see mbUnit and Gallio
Selenium                   UI test framework                 OpenQA       Also see WatiN
ReSharper                  Integrated toolkit for VS        JetBrains     (OK I don't actually use it but it's good.)
Tortoise SVN              Shell Integration with SVN      Tigris
Ankh SVN                  Visual Studio Plugin for SVN     CollabNet
Subversion (Server)     Version Control System          VisualSVN
Cruise Control.net       CI system                            ThoughtWorks
RhinoMocks                Mock Object System              Ayende
nAnt                         .NET Build Tool                     nAnt
Fitnesse                    Acceptance Test Tool            Fitnesse
STIQ                         Story Test Tool                    Solutions IQ
GIMP                         GNU Image Manipulation Prog. SourceForge
Notepad++                 Smart Text Editor                 SourceForge UK when Visual Studio just won't do...

Not development tools exactly, but extremely handy:
Process Explorer          Smarter Task Manager           SysInternals
FileZilla                      Upload/FTP client                  FileZilla
DivX                          Decoder                              DivX         Because sometimes we need to watch movies...

That's all I can think of at the moment, but am probably missing some things. I'm sure you'll all (please) chime in with what I forgot...

500 lines of code in one method? uh... U R DOING IT WRONG!

Long methods are a code smell. And a loud one at that. Methods should be focused on doing ONE thing. That thing can be simple or complex, but inner complexity should be refactored out into other (small) methods, keeping them each small and focused on a single task. A method with 4 nested levels of for/foreach/while statements is just so wrong in so many ways... Keep it simple. Keep it straightforward. Did I mention keep it simple?

Jeremy Miller posted a nice article a while back about short methods, and Ward Cunningham has another on C2 that Jeremy references.
We definitely need a Visual Studio plug-in to turn the background yellow when a method exceeds 20 lines, and turn red when it goes over 40 lines...

Here is a simple example. Code by intent means writing code by declaring what it does by well named (and usually long-named) methods. This method is only three lines long, but it conveys that there are two steps to be done, the second with the result of the first. This should be clear to anyone reading it. Further, the XML comment summary (missing from this example) should provide the reader with any information needed not conveyed with the title.

public void DoOneThing()
{
   int n = 0;
   n = DoStep1ReturnNumber(n);
   DoStep2WithResultOfStep1(n);
}

private void DoStep1ReturnNumber(int number)
{
   // simple method
}

private void DoStep2WithResultOfStep1(int number)
{
   // more simple stuff
}

Seriously, if there is something out there already that does this, please let me know! If not, I am going to have to write one...

We the practitioners of Agile and users of Beer, hereby proclaim Agile Beer Night.

The Agile Beer Users Group (Seattle Chapter) will be meeting on Thursday Aug 21, for information sharing, networking, agile discussions, a brief topic presentation on Automated Acceptance Testing, oh - and BEER. Come meet us for a discussion and brief presentation on Agile practices each month.

There is good food, good beer, and a good chance to meet some of the folks who do agile development in the Pacific Northwest, get to know their tricks, and share in their success. The Owl has a fine selection of food, drinks, and spirits for those who choose them.

ABN next meeting will be held in Seattle at the Owl and Thistle Irish Pub and Restaurant at 808 Post Avenue in downtown Seattle from 5PM to 8PM on Thursday August 21 2008.

The September ABN meeting will be on the East side in Redmond or Bellevue, but plans are not firm yet - stay tuned...


Please link this post, track it back, and let everyone know!

Next ABN on Thursday August 21 2008 5PM to 8PM at the Owl and Thistle Irish Pub and Restaurant at 808 Post Avenue in downtown Seattle