Quality Taxonomies

August 5, 2007 by
Filed under: Process 

This is the first of a 4-part series on Software Quality Attributes, where we focus on the Taxonomy of Quality.

Software quality attributes, often called the non-functional requirements, are likely the most neglected category of overall project scope on software projects. This is due to a perfect storm of influences: we naturally think of requirements as the functional capabilities of our system, the discipline of extracting and refining these quality attributes is a difficult process requiring the collaboration of all the project stakeholders, and there is little guidance in the popular literature on how to break down this difficult problem.

When we discuss the scope of a product, we almost invariably discuss in terms of capabilities, features, or how this system needs to do the same things as that system that we are replacing. Whether this is done as a list of features on the back of a dinner napkin, a more detailed collection of use case, or the elusive detailed set of functional requirements, this is the traditional way of looking at things. This perspective is critical, to be sure (though I would suggest that use cases are relied on for more situations than they should be), but focusing solely on this view is a key reason that many software projects end up disappointing the client in the end.

Understanding the functionality of the system allows us to discern between a piece of tax software and the latest first-person shooter game, but we need to go beyond this to understand many of the distinctions between one tax application and another. The two systems will have roughly the same features (based on the current taxation business rules), but very often, one package will be far more attractive to the end user. This difference can and should be expressed in advance, so that we can actually build these distinctions proactively, rather than hoping for the best or trying to retrofit them in after we discover their absence in our first attempts to integrate the system.

These characteristics cover a broad range, which is one of the first challenges to overcome. Usability, performance and security are common examples of capabilities that are difficult to express as functionality, and there are many more areas to consider. What we need is a taxonomy that we can expect to cover the breadth of quality issues we may run into.

There are a number of taxonomies available. In Managing Software Requirements by Dean Leffingwell and Don Widrig, the authors suggest usability, reliability, performance and supportability (then include an ‘other’ catch-all category to pick up the pieces). Indeed, as published in Roger Pressman’s Software Engineering: A Practitioner’s Approach, some groups at HP have settled on an acronym, FURPS, that captures their perspective: functionality, usability, reliability, performance and supportability. Note that in recent years, FURPS has evolved into FLURPS+ for some teams, adding localization and another catch-all category.

I particularly like the idea of functionality, that which most of us consider to be the breadth of requirements, is merely one element in a broad landscape of considerations for overall product quality. This sentiment echoes the taxonomy that I was first exposed to for overall quality, from the Rome Air Development Center (RADC), when I was involved in avionics-based embedded systems development. Their overall taxonomy was thirteen items, correctness being one of them.

This number of elements seems to be roughly the amount required to gain reasonable coverage for overall quality. McCall, Richards and Walters suggest 11 different elements, Karl Wiegers in Software Requirements (second edition) suggested 12, RADC had 13, and based on experience, Wiegers has added 2 more to his list, and currently uses a total of 14. A comparison of these lists shows that there are many terms that are common to all of them, an indication that we are coming to a consensus.

These broad taxonomies can be broken into several schemes. Wiegers breaks the list into items that are of interest to the user, and items that are of interest to the developer. McCall’s list is broken down into the product’s operational characteristics, its ability to undergo change, and its adaptability to new environments. While this really doesn’t help in specifying software requirements for your project, it helps us understand the rationale for the breakdown, and to appreciate that no single stakeholder can adequately speak to all factors of quality for a product.

Given the similarity between these lists, it could almost be a matter of drawing straws to determine which one to use for your projects. My preference is to go with the current list of 14 from Wiegers, but to lean on an appreciation of the contributions of others that led to this point. I would expect that more people in the industry are familiar with Wiegers’ work than with the other sources (and besides, I train with his material!).

Pick a taxonomy, but be sure to start with one of these broad taxonomies! While it can be appealing to start with a shorter list, such as recommended by Leffingwell or used at HP, there is danger that you will miss a critical area for your project. As we will see, it is quite straightforward to start with a broad list and then remove those elements that clearly do not apply, but to start with a short list and expect to recall other areas as required is problematic. Think about it: there is a good chance that some of the techniques you apply for developing products today have been borrowed from successes on other projects. This propagation makes sense of the technique starts from a general starting point, but a narrow list that worked on one project will likely have gaps for your project. If anything, start with a broad list and feel free to expand it into other areas, based on experience, as Wiegers has done.

Remember, these taxonomies have been developed to provide reasonable coverage of all of the aspects of product quality. Few of these terms are expressed in a manner that makes them easily quantifiable (as good requirements statements should be). This is something that we will deal with in part 3 of the series, after we have dealt with the issue of how to narrow the scope of these attributes to consider for your specific project in part 2. We will look at the list of attributes in this taxonomy next week as well. – JB

If you enjoyed this post, make sure you subscribe to my RSS feed!


Feel free to leave a comment...

  • What’s Happening

  • On The Road Again

    Jim frequently travels across Western Canada for engagements, and welcomes opportunities to meet, run a workshop, Diagnostic or Lunch and Learn session.

    Contact Jim if you would like to connect around any of the upcoming dates:

    • Blissfully at home in Vancouver, BC over the summer!
  • What People are Saying

    Jim knows what he’s talking about and leverages his experience well. His goal is to make you better at what you do and has an uncanny ability to do this without being too intrusive! His weekly newsletter is just another example of how he educates as he assists.

    — Viktor Ohnjec, Founder, M2VP