Test Driven Developer

Automated Tests are the Key to Scaling Agile

2011-12-27T11:53:06.555375-05:00

As Esther Derby's article discusses, it's the area of integration testing or cross-context testing where we most often discover the discrepancies in interfacing systems to each other. Automated acceptance testing, or automated integration tests are a key point for winning the bug battle in larger systems. If the teams and/or organizations negotiate a contract on test automation across these integration points, they can collaborate to create test frameworks and test automation that can be used to ensure proper functionality. These acceptance tests (or end-to-end tests as some call them) can and should be demonstrated to key stakeholders to make sure that the business understands what development is delivering, and that it meets the intended need.

As I often say, if we have automated acceptance tests, we can do ATDD: First, we automate the acceptance test, which fails because there is no code yet to test. Then, we write unit tests and code, more unit tests, and more code - striving ONLY to make the acceptance test pass. We should focus on just the one failing acceptance test, and not stray off into other functionality. Once we do, we can refactor to make sure we have the best solution, then we start all over again with writing more automated acceptance tests. That's ATDD!

Test automation is such a great thing to have on a project. It makes me smile each time an automated test catches a bug that would have otherwise gone unnoticed and perhaps even shipped to production.

This will be my last article for 2011, hope you had a great year! Best wishes for a fantastic 2012!

NUnit Tests for WinForms Application

2011-11-20T19:40:51.281-05:00

Lately I have been doing work on WinForms applications and have been looking at ways to test them. We use unit tests to test at the method level, and [ideally] our code is written to be able to separate the logic code from the UI code. However, not *all* applications are written this way (unfortunately). Today I am in a situation where I must be able to test an application at the UI level. I need to bring up forms, type in text boxes, poke buttons, and inspect results in labels, etc.

The answer is NUnitForms! This fabulous test framework is an open source project that extends the functionality of NUnit, and allows us to test applications at the UI level. We don't consider this to be a unit test or even integration test mechanism, but rather a functional test mechanism. Rather than having testers manually go through screen after screen in the application just to see if it all works, we'll write some NUnit code that will automate it all for us.

So How do I do it?

SourceForge bits: http://sourceforge.net/projects/nunitforms/files/nunitforms/

First, download the NUnitForms source code and add it to your solution. Then, write an NUnit test class, add a reference to the NUnitForms project and make your test class inherit from the NUnitFormsTest class. Add a reference to your application classes, and create a test. In the test, open your form with Form.Show() (so that it is visible). Then use a ButtonTester, a LabelTester, TextBoxTester, etc. to find each of the controls by name, and do the appropriate action to them. Then, after all of your actions are completed, call Application.DoEvents() to make sure to let everything have time to execute. You can then grab those text boxes, labels, etc. and Assert that the values are correct. That's it! Build it up step by step, and you have a fully automated end-to-end test!

If your forms are simple enough, you might even be able to use the Recorder application that comes with the codebase to record your keystroke and mouse actions as a C# code test. If your app has some startup code to accomplish, refactor it so that it's in simple methods, and include a hidden form in the app that your Recorder can open to accomplish the startup before opening other forms.

I have written and included sample code in a Visual Studio 2010 project in the zip file attached. This is a really great way to automate acceptance, functional, and end-to-end tests for a WinForms application project.

Sample code Here (5.5MB)

How to Sell Refactoring

2011-09-07T00:40:46.443875-04:00

Why would a stakeholder want to care about refactoring. It only makes development take longer, which means they get fewer features delivered. In some places it's a hard-sell. These are the situations where we find a lot of "Red, Green, Done" code written. Sometimes it's just Green, Done and even Done, Green. Let's examine why and how refactoring could be of interest to a stakeholder.

Stakeholders shouldn't know or care if a developer is writing unit tests, main-line code, or refactoring something.

It's not their job, it's the developer's job. There needs to be trust, and it goes both ways.
The stakeholders should be focused more on business strategy and how features and information can help the company make more money.
If they are that involved with development tasks they are either micromanaging or just nosey.
Not that there shouldn't be transparency - there should. But people should focus on their area of expertise and let everyone else do their jobs too.

Refactoring improves the code. Here are but a few reasons to do it.

More robust - fewer issues
Faster - more performant
Easier to change, requiring less training
Easier to understand the logic
Less easy to write bugs
Code operation is more stable
Cheaper/faster to maintain
Cleaner and simpler
Refactoring the design clarifies intent and makes testing easier
Clear code tracks more closely and obviously with business requirements
Refactoring sometimes makes bugs obvious and they can be eliminated in development rather than after test phases.

We can map these advantages to direct dollar savings in development

Most bugs can be tracked to code that is obtuse, complex, or convoluted.
Many studies have been done that show bugs are much less costly to eliminate in the earliest phases, design and development.
Not writing bugs is far less expensive than finding them and fixing them.

Truly informed stakeholders will not only accept refactoring, they will encourage it, because the practice actually saves time and money in the long AND short-run.

Lines of Code

2011-08-05T02:40:03.82175-04:00

Class size. So, how many lines of code is the right amount? Lines of Code has been a classic measurement of productivity, and code quality hasn't it? Today I saw a class [and PARTIAL Class for that matter] that was almost 120 thousand lines of code. Is this RIDONCULOUS or what. Forget for the moment that there are no tests for the entire project, and that it does not pass through a QA test cycle before getting sent to production. This seems to be beyond a code smell. This is a code gut-wretch... Anyway am hoping to make a positive contribution. Making suggestions for improvement. Suggested class size of 50 lines seems like a pipe goal... but will keep it in mind. And, you got a Test for That btw?? ;-)

Refactoring - A Practical Guide

2011-05-30T00:06:14.718-04:00

Refactor (re-fak'tr)
v.
1. (computing) To reformat and improve the internal structure of code without altering its behavior.

Here is a "Practical Guide" slide deck, to explain a bit about what refactoring is, and how we can use it to improve our overall code. Refactoring is a learned skill, and requires a bit of practice to do it well. I recommend perusing the links at the end of the deck, and reading up on some of the techniques to be familiar with them. The more we know about the patterns, we can do a better job of recognizing them and implementing them well.

Available in both PPTX and PDF
Refactoring - A Practical Guide.pptx (78.93 KB)
Refactoring - A Practical Guide.pdf (291.63 KB)

Practical Guide to Unit Testing and TDD

2011-04-06T22:20:28.9211356-04:00

Here is the updated presentation I did a few years ago. I updated the content with a few more relevant items and suggested material. Here are the PDF and PPTX format publications. Please present and distribute as needed.

A Practical Guide to Unit Testing12.pdf (2.86 MB)
A Practical Guide to Unit Testing.pptx (440.26 KB)

Do You Trust Your Tests?

2011-02-25T22:02:13.64175-05:00

Do we really trust our tests? If we have a test suite, and most run fine but sometimes, some tests fail for predictable or unpredictable reasons... should we still keep them in the suite? Here is a scenario. We have a test suite of almost 5000 tests, some of which are integration tests, some functional tests, and a lot of unit tests. They are organized into a set of lets say 25 assemblies total. Many teams (lets say 10) are working on the code base. Sometimes some of the tests that were written depending on data in the database fail, because the database is shared between all the 10 teams AND the CI build server. Sometimes a test or a few will fail, breaking the build. Most of the time, just re-initiating the CI build process will result in success the second time. So do we trust these tests? Do we believe in their results?

When using TDD, we first test the test itself, making sure it fails when we expect it to, and passes when we expect it to. Tests must set up and tear down anything and everything (including data in a database for integration tests) in order to be reliable. If we have legacy tests that weren't written this way, are they valid? They certainly don't meet the standard of TDD in my view.

Lets look at the reason we write tests at all...

To mitigate risk

that's it... in a nutshell.

If a test runs successfully sometimes but not others, does it really mitigate risk? Does it mitigate anything at all if we don't trust its result? I think NO. Are we better off leaving these tests as part of the suite? or are we better off just deleting them from the codebase - or... fixing them so that they are reliable. If they mitigate risk when they work, and they test a portion of the code that is valuable to ensure is working correctly, then they have to be fixed. Otherwise, they should just be removed from the test suite.

We depend on our test suites to tell us the state of the state for our working software. If they can't do this, then I would argue they are of no value to us in the overall big picture. Ensure that all tests are valuable, valid, and trustworthy. Make sure that things like time of day, data in a database (or lack thereof), or other transient factors aren't able to influence the outcome of the test. Only when we can rely on our test coverage to tell us the Real state of the code, can we deliver reliable, working software.

Tests as Documentation

2010-11-15T14:33:24.284625-05:00

I have long felt that one good reason for unit tests should be that of "Executable Documentation." Unit tests should illustrate the intent of the developers, as they wrote the code, so that we can make sure the code does what was intended. In contrast to other types of documentation, these tests are actually compilable and runable. There is very little chance that they can be out of sync with the actual product code, as long as there is a Continuous Integration [CI] system in place to execute the tests. This is in contract to documents, and other written forms. The only document that is kept in sync with source is one that is automatically extracted from the code.

All project documentation has a cost associated with it - both for creating it, as well as maintaining it. It's these ongoing costs that we on Agile project would like to keep to a minimum. We need only as much documentation as required by stakeholders, and no more. Unit code "documentation" is usually required by the "development management" stakeholder, usually implicitly. Anything else should probably be specified directly by the product owner, and should be a specific task on the sprint backlog to produce.

Documentation that is in sync with the product code, such as unit tests and automatically extracted documentation, or even release notes, has the lowest total cost of ownership [TCO] for the project long-term. See what you can do on your current or next project to lower your TCO for documentation with automation and Executable Documentation.

Unit Tests Should Be Fast.

2010-09-29T12:50:21.534375-04:00

These are some qualities of a good unit test: fast, small, focused (atomic), and isolated (order-independent).

A good unit test will execute in MILLISECONDS. NOT seconds. If a test takes more than a few hundred milliseconds to execute, it is probably too slow. This is not just an arbitrary performance metric...

Let me explain the philosophy behind this tenet.

A unit test suite should contain dozens if not hundreds of tests, organized into classes that have specific testing focuses. If a test runs in 1 SECOND (1000 ms) and there are 100 of these, that is almost a 2-minute wait for a developer to wait for the results of the suite. And we do this over and over and over... If we have to wait more than a few seconds (7 seconds is about the maximum time to keep the attention of the average developer) then the mind wanders, we get distracted, and it interrupts our workflow.

Not to mention the lost time... unit tests as a suite are (or SHOULD be) run frequently. Just imagine multiplying 2 minutes x 15 times per day (conservatively) x number of developers x developer rate per hour... This turns out to be quite an expense. Not to mention that the expense grows over time as new tests are added.

Tips:
- any unit tests that need to hit a database are not unit tests...
- any tests that need a network connection are not unit tests...
- some even say any tests that need a disk FILE are not unit tests...

Any of these issues can be resolved by the use of dependency injection and substituting mock objects in place of the database, service, or network resource. Using these objects will decouple the code from the service, and allow it to execute in isolation without requiring the actual resource. And, it will be FAST.

It's really worth doing, when we look at all the benefits. Take your unit tests to the next level and remove the integration with external resources. Make the tests fast, and everyone benefits.