As Esther Derby's article discusses, it's the area of integration testing or cross-context testing where we most often discover the discrepancies in interfacing systems to each other. Automated acceptance testing, or automated integration tests are a key point for winning the bug battle in larger systems. If the teams and/or organizations negotiate a contract on test automation across these integration points, they can collaborate to create test frameworks and test automation that can be used to ensure proper functionality. These acceptance tests (or end-to-end tests as some call them) can and should be demonstrated to key stakeholders to make sure that the business understands what development is delivering, and that it meets the intended need.

As I often say, if we have automated acceptance tests, we can do ATDD: First, we automate the acceptance test, which fails because there is no code yet to test. Then, we write unit tests and code, more unit tests, and more code - striving ONLY to make the acceptance test pass. We should focus on just the one failing acceptance test, and not stray off into other functionality. Once we do, we can refactor to make sure we have the best solution, then we start all over again with writing more automated acceptance tests. That's ATDD!

Test automation is such a great thing to have on a project. It makes me smile each time an automated test catches a bug that would have otherwise gone unnoticed and perhaps even shipped to production.

This will be my last article for 2011, hope you had a great year! Best wishes for a fantastic 2012!

Lately I have been doing work on WinForms applications and have been looking at ways to test them. We use unit tests to test at the method level, and [ideally] our code is written to be able to separate the logic code from the UI code. However, not *all* applications are written this way (unfortunately). Today I am in a situation where I must be able to test an application at the UI level. I need to bring up forms, type in text boxes, poke buttons, and inspect results in labels, etc.

The answer is NUnitForms! This fabulous test framework is an open source project that extends the functionality of NUnit, and allows us to test applications at the UI level. We don't consider this to be a unit test or even integration test mechanism, but rather a functional test mechanism. Rather than having testers manually go through screen after screen in the application just to see if it all works, we'll write some NUnit code that will automate it all for us.

So How do I do it?

SourceForge bits: http://sourceforge.net/projects/nunitforms/files/nunitforms/

First, download the NUnitForms source code and add it to your solution. Then, write an NUnit test class, add a reference to the NUnitForms project and make your test class inherit from the NUnitFormsTest class. Add a reference to your application classes, and create a test. In the test, open your form with Form.Show() (so that it is visible). Then use a ButtonTester, a LabelTester, TextBoxTester, etc. to find each of the controls by name, and do the appropriate action to them. Then, after all of your actions are completed, call Application.DoEvents() to make sure to let everything have time to execute. You can then grab those text boxes, labels, etc. and Assert that the values are correct. That's it! Build it up step by step, and you have a fully automated end-to-end test!

If your forms are simple enough, you might even be able to use the Recorder application that comes with the codebase to record your keystroke and mouse actions as a C# code test. If your app has some startup code to accomplish, refactor it so that it's in simple methods, and include a hidden form in the app that your Recorder can open to accomplish the startup before opening other forms.

I have written and included sample code in a Visual Studio 2010 project in the zip file attached. This is a really great way to automate acceptance, functional, and end-to-end tests for a WinForms application project.

Sample code Here (5.5MB)

Why would a stakeholder want to care about refactoring. It only makes development take longer, which means they get fewer features delivered. In some places it's a hard-sell. These are the situations where we find a lot of "Red, Green, Done" code written. Sometimes it's just Green, Done and even Done, Green. Let's examine why and how refactoring could be of interest to a stakeholder.

• Stakeholders shouldn't know or care if a developer is writing unit tests, main-line code, or refactoring something.
• It's not their job, it's the developer's job. There needs to be trust, and it goes both ways.
• The stakeholders should be focused more on business strategy and how features and information can help the company make more money.
• If they are that involved with development tasks they are either micromanaging or just nosey.
• Not that there shouldn't be transparency - there should. But people should focus on their area of expertise and let everyone else do their jobs too.
  
• Refactoring improves the code. Here are but a few reasons to do it.
  
• More robust - fewer issues
  
• Faster - more performant
  
• Easier to change, requiring less training
  
• Easier to understand the logic
  
• Less easy to write bugs
• Code operation is more stable
  
• Cheaper/faster to maintain
• Cleaner and simpler
• Refactoring the design clarifies intent and makes testing easier
• Clear code tracks more closely and obviously with business requirements
• Refactoring sometimes makes bugs obvious and they can be eliminated in development rather than after test phases.
  
• We can map these advantages to direct dollar savings in development
• Most bugs can be tracked to code that is obtuse, complex, or convoluted.
• Many studies have been done that show bugs are much less costly to eliminate in the earliest phases, design and development.
• Not writing bugs is far less expensive than finding them and fixing them.

Truly informed stakeholders will not only accept refactoring, they will encourage it, because the practice actually saves time and money in the long AND short-run.

Class size. So, how many lines of code is the right amount? Lines of Code has been a classic measurement of productivity, and code quality hasn't it? Today I saw a class [and PARTIAL Class for that matter] that was almost 120 thousand lines of code. Is this RIDONCULOUS or what. Forget for the moment that there are no tests for the entire project, and that it does not pass through a QA test cycle before getting sent to production. This seems to be beyond a code smell. This is a code gut-wretch... </bitchSession> Anyway am hoping to make a positive contribution. Making suggestions for improvement. Suggested class size of 50 lines seems like a pipe goal... but will keep it in mind. And, you got a Test for That btw??

Refactor (re-fak'tr)
v.
  1. (computing) To reformat and improve the internal structure of code without altering its behavior.

Here is a "Practical Guide" slide deck, to explain a bit about what refactoring is, and how we can use it to improve our overall code. Refactoring is a learned skill, and requires a bit of practice to do it well. I recommend perusing the links at the end of the deck, and reading up on some of the techniques to be familiar with them. The more we know about the patterns, we can do a better job of recognizing them and implementing them well.

Available in both PPTX and PDF
Refactoring - A Practical Guide.pptx (78.93 KB)
Refactoring - A Practical Guide.pdf (291.63 KB)

Here is the updated presentation I did a few years ago. I updated the content with a few more relevant items and suggested material. Here are the PDF and PPTX format publications. Please present and distribute as needed.

A Practical Guide to Unit Testing12.pdf (2.86 MB)
A Practical Guide to Unit Testing.pptx (440.26 KB)

Do we really trust our tests? If we have a test suite, and most run fine but sometimes, some tests fail for predictable or unpredictable reasons... should we still keep them in the suite? Here is a scenario. We have a test suite of almost 5000 tests, some of which are integration tests, some functional tests, and a lot of unit tests. They are organized into a set of lets say 25 assemblies total. Many teams (lets say 10) are working on the code base. Sometimes some of the tests that were written depending on data in the database fail, because the database is shared between all the 10 teams AND the CI build server. Sometimes a test or a few will fail, breaking the build. Most of the time, just re-initiating the CI build process will result in success the second time. So do we trust these tests? Do we believe in their results?

When using TDD, we first test the test itself, making sure it fails when we expect it to, and passes when we expect it to. Tests must set up and tear down anything and everything (including data in a database for integration tests) in order to be reliable. If we have legacy tests that weren't written this way, are they valid? They certainly don't meet the standard of TDD in my view.

Lets look at the reason we write tests at all...

• To mitigate risk

that's it... in a nutshell.

If a test runs successfully sometimes but not others, does it really mitigate risk? Does it mitigate anything at all if we don't trust its result? I think NO. Are we better off leaving these tests as part of the suite? or are we better off just deleting them from the codebase - or... fixing them so that they are reliable. If they mitigate risk when they work, and they test a portion of the code that is valuable to ensure is working correctly, then they have to be fixed. Otherwise, they should just be removed from the test suite.

We depend on our test suites to tell us the state of the state for our working software. If they can't do this, then I would argue they are of no value to us in the overall big picture. Ensure that all tests are valuable, valid, and trustworthy. Make sure that things like time of day, data in a database (or lack thereof), or other transient factors aren't able to influence the outcome of the test. Only when we can rely on our test coverage to tell us the Real state of the code, can we deliver reliable, working software.