Designed Testability

February 15, 2004 by
Filed under: Process, Project management 

The greatest opportunities for saving time and effort in any endeavour can usually be found in the activities that are most repetitive. In software projects this is often the testing stages, especially as the lifespan of systems is increasing, and more teams are adopting practices such as daily builds (that need to be exercised) or eXtreme Programming (where the goal is for the system to be live as much as possible). If a retrospective is the most powerful approach for improvement late in the project lifecycle and change management is a strong in-progress activity for control, designing testability into the system is one of the best activities to perform early on to reduce cost.

Like everything, focus on designed testability is not a silver bullet that applies to all projects, but if it makes sense and is done right, you will reap massive returns, both within the current project lifecycle and even more next time around. If you expect that the system you are developing will be around for a while, saving time every release can be significant. If it is a mission-critical system, testability is likely a mandatory requirement. If you are performing daily builds, a testable system will allow you to confirm that one build is stable before you forge ahead to the next build. On the other hand, if you are in a situation where none of these conditions exist, don’t put massive effort into testability, but take advantage of any solutions that already exist if they are available.

A critical part of any testable system is a strong test infrastructure. You need a lightweight shell to manage the tests that will build up over time (the magnitude of managed tests can often outweigh the deliverable code for the system). You need to be able to selectively run either all the tests or a reasonable subset as a regression suite. Just as important, you need to be able to quickly identify the issues rather than having to pore through a sea of data to identify whether any tests failed. The greatest leverage is gained if this infrastructure is introduced into the system early (which implies the need was identified early as well), but even if you have a legacy system there is always value in testing smarter.

While an infrastructure that supported these needs was not readily available 10 years ago and would cost time and money to construct, most of us no longer have that excuse – there are implementations in most popular languages (and many obscure ones) freely available on the Internet. The class of XUnit implementations is extremely powerful, straightforward to use, and best of all, open source.

The test infrastructure is not a complete solution to the testability challenge. Consistency of the approach to design and coding of the system is critical to prevent surprises in the test phases. There should be little variation in coding style or design-level implementations, both of which are best managed by putting standards in place early, and enforcing these standards with a well managed approach to peer reviews throughout the project.

Finally, the interfaces of any system are most susceptible to surprise and uncontrolled change and it is prudent to place particular effort in building harnesses to carefully exercise these interfaces, both the programmatic interfaces and the human interface (which is really just another external interface, but with virtually no control of how the user will interact with the system).

Testing isn’t an end of project activity. It starts with the early identification of the testability requirements of the system, and if done right usually involves the construction of a cradle of test support functionality that drastically reduces the risk of delivering a quality system on time. To wait until you start integrating to think about test activities is to court disaster. – JB

If you enjoyed this post, make sure you subscribe to my RSS feed!


Feel free to leave a comment...

  • What’s Happening

  • On The Road Again

    Jim frequently travels across Western Canada for engagements, and welcomes opportunities to meet, run a workshop, Diagnostic or Lunch and Learn session.

    Contact Jim if you would like to connect around any of the upcoming dates:

    • Blissfully at home in Vancouver, BC over the summer!
  • What People are Saying

    Jim is simply one of the smartest and most knowledgeable software development professionals I have ever met. His vast knowledge of the discipline is both leavened and enhanced by his acute awareness of and sensitivity to “the real world” of what actually occurs with real developers in actual practice. Jim is also a warm, open, honest person – rare gifts and an asset in any consultant.

    — Bonar Harris