QA, Test, CM Tips

This was a typical lab environment, with the two devices sitting on a bench, surrounded by oscilloscopes, logic analyzers, other test devices, and nervous managers. It seemed that no matter how rigorously we devised our tests, the system would seem to work well one minute, fail the next.

One of our standard sanity checks for the software was to use a loopback plug on one end of the makeshift cable to confirm that a specific set of tests worked. Unfortunately, what sometimes happened was that even when we removed the loopback connector, the loopback tests still worked! We pored over these tests and the operational code with a fine tooth comb.

Our cable was a bunch of loose wires, perhaps 4 or 5 feet long, so we could connect the two devices together. When it was time to try a loopback, we would pop the cable off the receiving device and attach the loopback to the end of the cable. How could anything possibly change in that situation?

We were making the rash assumption that these wires and connectors would work in a consistent manner for our tests.  It turns out that when we carefully ran the cable across the bench, the wires were close enough together so that the signal on one wire could be sensed on another wire right next door, effectively closing the loop. The software had actually been working as expected for quite some time, and when we introduced a properly built cable, our problems were resolved.

What had cost us all that time? We were making an assumption that what worked once would continue to work in the future, and there was no need to worry about it further.

As we work on a product for a period of time, we will make assumptions. Early discoveries or decisions become assumptions, which is a reasonable mechanism for us to have the capacity for new issues that we are dealing with. This works for most of the issues, which makes this a valuable mechanism to use, but it is not 100% effective.

Every time we add to an existing system, there is a finite chance that we will perturb some part of the system that is already in place. This is true for new work, and even more so when we are fixing defects in the system (studies have shown that the likelihood of injecting a new problem is 10x as likely when you are fixing bugs). When this happens, the system has regressed.

For that reason, we can’t assume that tests that have worked in the past will still work. This is one of the key drivers for building tests that can be maintained over time and managed in a regression suite.

Beyond a regression suite for software, though, it is important to capture your assumptions that have been made along the way. To some degree, a well managed set of requirements and design decisions can serve as a higher level regression suite. Without this information captured in a reasonable form, assumptions become implicit. Possibly internalized by those that were involved at the time, but invisible to everyone else. A dangerous way to proceed.

One great way to expose assumptions in your system is to get as many fresh sets of eyes looking at the problem as possible. Often, when I run inspections workshops, people will claim that new employees or people that are more junior will not add a great deal of value in an inspection or peer review, but I have found just the opposite to be true.

Without fresh perspectives into the problem, it can be easy to fall into a group-think trap. With new eyes, we get the naïve questions that will surface the assumptions that have been ingrained into the group. You may be surprised how often there is value in rethinking and correcting those old assumptions. What we get is a great deal more innovation in our solutions, while at the same time distributing the knowledge of the system, reducing the dependency on heroes.

Back to that old bug we had, it was someone just walking through the lab that looked over our shoulders and pointed out our problem. She didn’t have a strong engineering background, but she asked the questions we were all assuming to already answered.

Are you making any assumptions today that could use a sanity check? - JB [top]

Another Set of Eyes (Compendium 6.4)

There are many organizations, and plenty of pockets of resistance within organizations, that don’t leverage the value of peer reviews in any form. Individuals create their work products on their own, possibly taking the time to step back and have a look at them from a different perspective or running a few tests on them, then the product is essentially tossed over the wall to the next consumer.

Formal inspections are valuable, but more importantly we need to build a culture where it just doesn’t feel right to let anything go without another set of eyes going over our work. - JB [top]

Data Integrity (Compendium 5.38)

Generally, the suggestion that you should never argue with the data is a good one to follow, but there are clearly some caveats. There may be challenges with the measurer or the subject being measured, or both.

Quantification can sometimes be a poor argument for validity, or even reasonableness. Without a disciplined, active and evolving approach for managing the sanctity of the data being gathered, it can become just another bucket of numbers. - JB [top]

Outsourcing Quality (Compendium 5.24)

I grew up in Southern Ontario, where everyone had at least one relative or close friend that worked in the automotive industry, if they didn’t themselves. There was always a strong sense of brotherhood, and the oil crisis (the one a few decades ago) was not an easy pill to swallow. If someone worked at Ford or Chrysler and showed up driving a Nissan, there were times when that car could be flipped or torched just for being on the lot.

Even though there was a strong commitment to the local brands, there was also an understanding about the product and the business. Nobody would buy a car that was built on Monday morning, and partnership wasn’t a word you would use to describe the relationship between the unions and management.

Fast forward 30 years or so and the landscape has changed. While there can still be some animosity towards foreign car makers in areas that have been hit by the downturn that North American automakers are facing, there has been a steady growth of acceptance of the foreign cars here. Not only have these foreign automakers opened plants on our soil, they have grown to a point where most of their cars that are sold here are built here, with parts from this neck of the woods, and with North American workers.

I’m driving around in a Mazda that was built in Michigan. The engine block comes from a Ford foundry in Windsor. When I travel in a rental car, though, I am constantly reminded that there remains a significant difference in the quality of a car from an American manufacturer and one from a Japanese one.

Before going further, it is important to say that there are hits from North America, and duds from Japan. We had to replace the engine block in our Toyota minivan after only a couple of years. Not everyone that works in a North American factory is indifferent of the work they perform. There is a big difference across the genre, though, and it goes beyond the soil where the manufacturing plant is built. As we think about outsourcing our software products abroad, can we learn something when the shoe is on the other foot?

As the Japanese outsource the construction of their products to North America, they do so not because the labor or material rates are significantly lower than in Japan, but because of the ‘Built in America’ demands of the consumers here. They have identified a business challenge to manage, and they have done so very successfully. The domestic brands continue to lose market share, the foreign brands continue to rank higher in terms of quality and customer satisfaction.

One of the things that the Japanese have done very well is to outsource the attitudes around building their product, the same attitudes that made them successful initially in Japan. As a plant opened here, it was originally run by Japanese management, whose style is quite different than that which is traditionally North American. Everyone contributes to quality, all employees are respected and there is a strong sense of working together to build a strong product. Only when the attitudes are institutionalized and the quality approaches are ingrained are the reigns turned over to local management, and only those that are carefully groomed will get the job.

The big 3 automakers here would like to suggest that there is an unfair advantage, that without the higher wages driven by unions and the huge legacy of pensions, the newcomers are too agile to catch. One way to look at this is that while the workers get reasonable compensation and flexible pension plans, these were crafted as a partnership rather than out of being overpowered. Management has been able to succeed in building both a quality product and a sustainable business at the same time, and there is an ongoing conscious effort on both aspects.

It is interesting to compare this to my experience over the past couple of days as I moved to a Mac from a long line of Windows based machines. I speak about the transition in the past after only 3 days, even with the change of platform – previous transitions from one Windows machine to the next would often consume weeks before I felt settled in. The ease of learning is better, the ease of use is better. Far superior integration of the applications (and yes, I’m using Microsoft Office on the Mac), installations are cleaner, and I haven’t had to reboot yet (though I tried it once so far just to see how long it would take).

I understand why there is such a cult around Mac users, I’m already feeling the tug, the growing intense loyalty to the product. My perspective was always that Apple screwed up a long time ago by not opening up their hardware specs to the free market, but that viewpoint is changing. They may not own the lion’s share of the market, but they haven’t had the challenges resulting from the wide range of quality on their platform of evolving standards. My thinking now is that Apple may have recognized the risk of losing control of product quality.

The interesting thing about this comparison here is that the only words on the front of the box of documentation that came with the MacBook are “Designed by Apple in California”. The device was assembled in China, but Apple has done well to manage the outsourcing risks.

Outsourcing is not simply a function of geographic boundaries, though many will cling to that notion. Successful outsourcing involves the active management of the transfer of values and attitudes. If quality is on your radar, and if this is consciously transferred to your team, success will follow. - JB [top]

Measuring as a Necessary Evil (Compendium 5.18)

I can’t think of an industry where there is as narrow a perception of what constitutes a critical practice as the software industry. In most shops, and for more than enough ‘industry leaders’, developing and managing requirements is done when we have time. Estimation practice looks a lot more like target practice, and project management is akin to herding cats, with little or no semblance of pragmatic, proactive effort.

Measurement is something that few of us do well, if at all, in this industry. This unfortunately includes many of those that would like to suggest that measurement would be a good thing to do. There is little wonder that the industry continues to be plagued by many of the issues that were first raised decades ago, as most of us are left to our own devices to discover the same things that everyone else has experienced in the past.

Our anecdotal recollection of past performance is often way off the mark. We all have a tendency to be more generous with recollections of our own performance, and our perspective of others is strongly biased in one direction or another based on our past experience with them (whether or not that bias is justified). Anecdotal recollection is too far removed from reality to be used as a basis for rational decisions.

This is not news to people, but we still don’t measure.

In principle, most of us have a strong aversion to measurement. Taking the time to capture how big something was, how long it took for us to do something, or how much it cost is the kind of closure that few of us will take the time to do – we’re far too busy forging ahead with the next task on our list to deal with this triviality.

Layered on top of that is the fear that actual measurements will be used against us. I’ve seen timesheets used to insinuate that some people aren’t pulling their weight, or used to drive more work in less time as overtime is conveniently neglected. I’ve seen inspections decay into discussion of local sports teams because of the knowledge that any issues found would be reflected on annual performance appraisals. In some organizations, the fear of measurement is quite justified.

For these reasons, measurement is the last thing that most of us would want to spend our time on. With care and nurturing, though, we can build an appreciation for the value of appropriate management while holding the dysfunction at bay.

Take the time to consider where measures might add some value – identify the activities that you really don’t know how long they will take, or how much they will cost. Repetitive tasks are good; think of elements that you perform on every project or periodically, or that you could capture on a weekly basis to identify trends over time. Think about how you might capture some of these measures – what to count, where to store the results, when and how they might be used again.

There is a very strong chance that after a very brief period of time, on the order of 3-5 measurement cycles, you will gain some new insights that will surprise you. Over longer periods of time, you will be able to use this information for predicting future outcomes with much more credibility.

Done right, measurement is not evil at all, and it is an absolute necessity. There are a variety of data points that I capture on all aspects of the business to drive operations that I would be lost without. Yes, there are times that the data has not provided great news, and there have been strong urges to bias the results to sweeten the deal, even knowing that I would only be lying to myself.

I have learned, though, that even bad news in the form of unbiased, defendable data is extremely powerful in allowing me to make reasonable decisions in a clear manner. Measurement has become a core practice for driving a business that has spilled over into other areas (just ask my wife about the spreadsheet for timing her contractions), and for me, the loss of being able to effectively measure and base decisions on this insight would be a far greater evil. - JB [top]

Being Reasonably Self-Critical (Compendium 5.12)

In working with a large number of different organizations over the past 8 years, primarily focused on helping them become more effective at getting software out the door, there has been a continuing effort to crack that challenge of measuring the value of our improvement efforts. Many would argue it just can’t be done – most groups don’t have the benchmarks, there are so many factors that influence the overall results, it can be too easy to differ in the interpretations as to which factor contributed the most – or to interpret that the change is merely a function of noise.

While there has been some success in quantifying value for some organizations, there has also been a nagging problem. Some companies that would seem to score well on the quantified results continue to struggle to get their product out the door, while others that score relatively poorly – even some that will not show marked improvement quantifiably – show dramatic change and remarkable results with their efforts. Something is there beyond the data.

In retrospect, it appears that what is missing from the hard numbers gathered thus far is how well the organization can look inwardly and recognize that there is room for improvement. Call it openness for change, or an ability to be reasonably self-critical.

For some shops, improvement just won’t happen. It’s the equivalent of that unanswerable question “Do these pants make me look fat?” – the messenger will be damned for telling the truth, and will make no progress with a lie. Others will gladly lap up any feedback they can get and act on it accordingly.

It is quite easy to take a list of clients and place them somewhere on the continuum between totally closed and totally open to an external perspective of their performance. This is a phenomenon that starts at the top, and as it trickles down the chain it manifests itself in various ways.

When having an open discussion about practices and the resulting performance, some teams will be open and engaged while others can’t seem to wait until the meeting is over. Body language gives great insight (which is one reason why these sessions are most valuable face to face), and the same managed session can be as short as 45 minutes with some groups, and more than 3 times as long with others. Perhaps there is a new metric to be gathered here…

In particular, senior management engagement tells volumes. Some don’t bother participating or even attending at all, and others will argue that the information discussed is irrelevant for their organization. On the other end of the spectrum are those that are interested in the results, actively involved in the discussion, and solicit the input from the rest of the group in analyzing the information presented. Bravo!

To some degree, an assessment of an organization’s ability to be self-critical would be a key leading indicator of the success of any improvement engagement. It is one thing to change a few token practices for a team, it is another thing entirely to change habits and internalize appropriate practices so that they will be applied when the pressure is on. To be able to look inwards to identify potential changes that are within our own sphere of influence is essential. We all have room for improvement, all the time.

We need to find a reasonable balance in the continuum of self-criticalness. In working with a large number of people to improve their organizational skills, it has been fascinating to see those that are content to live in a physical environment where they can’t see their desktop and visitor’s chair for the piles of paper, or in the electronic equivalent where everything remains in their Inbox or their Desktop. Others are fastidious about staying on top of things, to the point where there’s angst when the stapler isn’t placed in the right direction. Somewhere in the middle lies the sweet spot where efficiency and control is maximized, busywork and stress is minimized, and the most real work gets done.

There is value in recognizing the degree to which we are open to acknowledging that we could improve, and the corresponding openness to hearing the feedback from the outside without defensiveness. While it may be one thing to maintain a public persona of strength that can border on puffery, it is another thing entirely to retain that boastful image internally, where it can often be interpreted as denial.

Being able to recognize our ongoing potential for change, and to reasonably act upon it, is an extremely valuable strength. - JB [top]

Rigid Agility? (Compendium 5.6)

The only thing that bothers me more than companies saying they’re in the Web 2.0 space is teams saying they are doing Agile – both bandwagons are getting pretty crowded these days…

Don’t get me wrong – I think that the latest Web technologies are way cool, and there is a lot of merit in the Agile approaches – both have their place. That stated, the rolling of my eyes is becoming involuntary – despite all the admonitions, neither of these movements is going to eliminate world hunger or cure cancer, or even completely supersede their predecessors - far from it. The newest approach is not necessarily the most appropriate for all situations.

The Agile approaches have been getting strong press for 5 or so years now, enough time for the major tool vendors to catch up and suggest that their suites fully support this game. What Rational did several years ago with the RUP, Microsoft is now suggesting they can do with their latest version of Visual Studio Team System. They provide a process tailoring environment that is admittedly version 1.0, and a process enforcement mechanism that can drive your artifacts through specific states and gates. First out of the gate are CMMI and Agile – I almost wonder if these were selected using a del.icio.us tag cloud to pick the approaches that generated the most buzz?

Problem is, building a generic CMMI process model for enforcing the development of software is like talking by building a sentence model around Webster’s Dictionary. Trying to run an Agile project around a state driven enforcement mechanism is simply an oxymoron – the whole point is to be able to adapt to the situation, and the last thing you want to do when the situation changes abruptly is to dive back in to change the defined process model. It seems that in our zeal to jump on the bandwagon, some of us seem to be missing the real point.

There appears to be a growing arrogance among the advocates in these camps, that the early demonstrated successes are proof positive that we are witnessing a new world order. Anyone can put together a mashup of public washrooms in San Francisco, and some companies are even starting to solve real business problems with new technologies. There is a wildly popular Waterfall 2006 conference website (including its own incomplete sections and bad links – nice touch, Mike…) that spoofs the early attempts at defining a standardized process, but after a few quick chuckles it all seems shallow (at least waterfall would be easy to model and enforce with Visual Studio Team System…). It’s a pretty safe bet that something will replace Web 2.0 in the future, and someday more people will begin to realize (maybe they’ll even teach it in schools, Oh My!) that a single branded Agile approach won’t solve all your problems.

It’s all got a deeply carnival feel about it – everyone is proudly displaying their affinity du jour, tool vendors are flogging their latest features and consultants are branding themselves accordingly. No doubt as soon as the luster wears off, people will flock to the Next Big Thing that comes along. I’m reminded of Monty Python’s spoof of religion in the Life of Brian:

• Brian:    You've got to think for yourself! You are all individuals!
• Crowd:  Yes, we are all individuals!
• Brian:    You are all different!
• Crowd:  Yes, we are all different!
• Single voice from within the crowd: I'm not.
• Crowd:  SHHHHH!

The technologies and the approaches we use are simply a means to an end, not the end in itself. Rather than saying "we’re Agile", or "we’re Web 2.0", wouldn’t it make more sense to brand yourself by saying "we deliver great products, and we do it in a way that is most effective"? While it is nice to have focused expertise, it is dangerous to pigeonhole yourself – be appropriately flexible.

Me, I’m content to watch this wild parade pass by – I’m sure there will be plenty of new floats on the way to keep me entertained. - JB [top]

Quality Circle Genealogy (Compendium 4.52)

It has been variously argued that the notion of Quality Circles has been around for perhaps half a century, and that Kaoru Ishikawa, he of the Fishbone or Cause and Effect Diagram fame, is known as the ‘Father of the Quality Circle Movement’. Common in manufacturing environments and ISO-based quality initiatives, Quality Circles bring a small group of people together voluntarily to solve their work related problems or improve their work environment. Handled appropriately, the shared participation fosters teamwork, cooperation and more broadly effective solutions.

All good stuff, to be sure, but it is safe to say that solutions through peer collaboration are a bit more than a 50 year old concept.

For hundreds of years, North American Aboriginals have been managing affairs through the use of circles of various kinds, such as healing circles as a means of supportive improvement and recovery and sentencing circles to develop consensus on fair and reasonable consequences to crimes. A little research shows that these long-standing practices have a number of strengths that would make them valuable to adopt as a means of managing change in software organizations.

One of the key elements of these circles is that everyone is involved in the decision, and the party to be changed (either healed or sentenced in the circles above) requests to participate in the circle process – it is not something that is mandated down. When participating within the circle, an environment is provided where it is safe to speak your mind without recrimination – openness and honesty are critical for success. All participants are empowered in the process, as they all have a voice and a shared responsibility in finding constructive resolutions for change.

Beyond the strengths of circles to address immediate problems, the shared decision-making process helps build a sense of community and capacity for resolving conflict, and it promotes these community values.

Indeed, it appears that sentencing circles include healing circles for the victim and offender, as well as follow-up circles to monitor the progress of the offender, providing what appears to be a much more comprehensive solution that the systems we are accustomed to, and can be effective at addressing the underlying causes of criminal behaviour, rather than just punishing that behaviour or simply masking the symptoms.

Many changes in software development would benefit from a similar holistic approach for change. In the small, as we address changes to the software product through Change Review Boards, broad participation is critical to ensure that impact is appropriately analyzed and disseminated, and significant changes can be effectively fostered through to the rest of the team. Maintaining an environment where any change can be voluntarily brought to the Board in a proactive basis will tend to reduce the need for the more severe form of ‘sentencing and recrimination’ that would be used to address situations where the build is broken and needs repair.

For more significant changes (to the product or the process), broad participation will promote buy-in across the team and dramatically improve the quality of the ideas that contribute to the change process. Collaboration and shared awareness improves the sense of community, which can only serve to improve or sustain team dynamics over time.

Whether you follow the lead of the Quality Movement or acknowledge that the principles of collaborative problem solving have much deeper roots, there is great value in leveraging the practices of circles within your teams. - JB [top]

Follow-through (Compendium 4.46)

In almost any sport, the first things a pro will talk to you about when you take lessons are about body position, focus, and the notion of strong and consistent follow-through. It’s a fundamental skill. While you can still play the game without a strong follow-through, you will never become world class – never. The pros and the better amateurs drill on the follow-through until it is second nature. In some sports such as archery and golf, many pros believe they can leverage the follow-through to continue to influence the trajectory after the arrow has been released or contact has been made with the ball – they aren’t done until the target has been hit.

In martial arts, we are taught to kick or punch through the target, not to simply meet the target. From the physics perspective, striving to simply meet the target drastically reduces the momentum and effectiveness of the punch. Without follow-through, we’re more likely to bruise our knuckles than impact the brick, but with proper focus beyond the immediate target we pass right through that target as though it wasn’t there – the first time you do it, it is surprisingly effective and painless.

Follow-through allows us to think beyond the immediate target and recognize the extended impact of our actions. When we think about the follow-through, that range of possibilities concerning what ‘done’ means quickly narrows, and we drastically reduce the likelihood that there will be a difference in expectations, and a corresponding disappointment with the results. We gain a deeper understanding of the breadth of impact of our practices, a wider appreciation for the true lifecycle of the influence and interdependence of our actions.

It’s like putting a new roll of toilet paper on when you’ve used the last piece. It’s like starting a fresh pot of coffee in the office when you’ve taken the last cup. It’s about managing the expectations beyond the context of ourselves.

We’re not really done requirements when the document is saved; we’re not done the project when the product is installed at the client site.

We don’t go through an analysis phase to simply produce a requirements specification; we capture the information needed for us to successfully complete the remainder of the project. We are developing the information that drives the schedule, the design and development effort, the validation effort, and we need to ensure that we have adequately addressed these goals. Having produced the document is merely making contact with the target, and all too often, that target is only superficially impacted.

We don’t produce code so that it compiles, we produce code so it solves a problem – it’s not about unit test, though that is a strong part of it. It’s not even about integration test, but that’s part of it, too. It’s about ensuring that the end user’s experience is complete and satisfying. It is the follow-through that we must focus on to ensure that we have accomplished our goal.

Spend the next day thinking about your follow-through in everything you do. It’s a safe bet that you will find a number of situations where you can complete the activity with simple things that will drastically affect the overall quality of the outcome. It can be as simple as saying ‘please’ and ‘thank you’ and ‘you’re welcome’ – and meaning it.

Don’t swing to simply make contact with the ball. Swing for the fence. Ensure your actions have the strongest impact possible. Follow-through. - JB [top]

You Are What You Eat (Compendium 4.40)

Even without going to the dietary extremes taken on in the documentary Super Size Me, we can certainly tell when we’ve had too much to eat, when we are hungry, or when we have eaten something that doesn’t agree with us. Despite my acknowledged cravings for sweets, I know that I’m not going to be happy after the fact if I indulge in something that’s not good for me. It’s not too different in software development, where the adage of ‘garbage in, garbage out’ is an unfortunate fact of life that few of us heed.

It would be too easy to beat the dietary metaphor to death, but the bottom line is that as a system that produces software, a software team needs to be efficient and effective. We need to recognize the quality of our downstream artifacts are going to be directly proportional to the quality of their predecessors. It is interesting to note that while most software teams obsess over their source code and a growing number of teams show interest and concern over quality design artifacts, still relatively few groups respect the value of the specification of their products (and the ongoing management of that specification). We are creating an environment where we will be lethargic and sloppy in producing the very thing we care about the most – our source code.

Here’s a quick litmus test for you.

Considering the overall lifecycle of products that are produced for your projects – the specification, the design, the source, the test artifacts – your order may vary given your chosen lifecycle, of course. To what degree do you manage these products from the following perspectives?

• There is planned time in the schedule to develop the products
• There is some form of peer review of the products, and validation that these products are adequate before advancing
• These artifacts are retained under a configuration management system
• These artifacts are baselined and controlled for change, ensuring that there is a system in place so everyone knows they have access to the latest version
• There is traceability (for completeness and consistency) between these products and the other products up and down the chain.

If you do all of these well for all the components of a project, it is a pretty safe bet that you are in good project health. If you aren’t doing some of these, or not doing them as thoroughly or consistently as you know you should, there is a good chance that these deficiencies involve the earlier stage products, such as your specs. If you weren’t even aware that you should be developing and managing all these products, then it is likely that software development isn’t nearly as fun as it could be. Some projects forge ahead with nothing more than a quick list of features and rarely meet expectations, some companies don’t explicitly design their products and suffer as the product evolves – both self-inflicted maladies.

Skimping on the early stage artifacts for a project essentially means that you are missing major portions of your diet. While you may feel you can make do without them, you will almost certainly suffer from a lack of efficiency, and there is a good chance that your results will be disappointing. Jumping straight to the dessert can seem appealing, but it is certainly not a sustainable approach - the guilty pleasure of something that tastes great but we know isn’t the best for us in the long run can be quite compelling and addictive.

How’s your software diet? Do you plan your meals or try to scarf down some techno-fast-food and skip your fruits and vegetables, only to suffer from ongoing bouts of indigestion? Would your development practices resemble a planned, balanced diet of healthy foods, or a mad scramble of Big Macs, pop, and a resulting expanding waistline? - JB [top]

Technological Advances? (Compendium 4.34)

It can be an interesting exercise to walk a group through a distillation of the key elements of effective projects. Most of the time, these elements consist of suggestions that the project team enjoys the experience, that reasonable planning takes place, and that communication is open and forthright. It is quite rare for a group to suggest that a key element for project success is strong adoption of technology. Perhaps the reason for this is that as we focus on bringing technology into our teams we lose sight of the real reasons for doing so.

Now, before labeling me a Luddite, let’s make it clear that I think technology can indeed be a good thing, but balance is required. If most groups tend to agree that clear communication is a must for successful teams, then this becomes one of the primary criteria for determining whether technology is a valuable thing to add. We need to remember that we’re best served not by simply adding technology to a project, but by carefully leveraging technology to improve our likelihood of success. Many tools can provide semantic and syntactic checks to eliminate one class of issues, for example, but they are certainly no replacement for good old gray matter.

Technology for its own sake rarely works, and most apparent advances will require some shaking out (and careful fostering by the early adopters) before they are effective for the masses. Remember the hype surrounding automatically generated code from the object models we produce in the design phase? While successful for a few disciplined teams that thoroughly understand their design and are comfortable living in that space, it requires vigilance to work within the design even when the product is being integrated. Despite the promise of CASE that has been around for 20 years, the full benefits will never hit the mainstream until the general population is as comfortable in the design space as they are in the implementation space. As it stands, many teams still don’t even explicitly emphasize design in their products.

While standing up in front of a group could be a prime opportunity for communication, technology has bogged us down here as well. A standard expectation from conferences these days is that the speakers provide their PowerPoint deck well in advance so that copies can be made for their participants, and more often than not, presenters lean on this deck as their sole source of information, many simply paraphrasing (or worse, reading) the text up on the big screen. I’ve been caught in this trap as well - a slide deck can be an addictive way of getting a presentation done rather than as a tool for effective communications. It can be interesting to stand up in front of a group without a presentation deck – walking without a safety net will force you to actually work to connect with your audience, and it will often be a refreshing experience for all.

Polished reports and clean drawings can easily provide the façade of professionalism and defendability. As computer usage grew in the home, it was the students that used technology that had a clear advantage with their projects – regardless of the content. We can easily get so caught up in the details of how a diagram looks or formatting issues that we can miss glaring holes in the information that we are actually presenting. While we originally had whole departments whose responsibility was to gather all the information in document form, that responsibility now lies with us – the ones that used to be responsible for the content only. With little consistent training in the presentation side, document formatting or even word processing, this polish side of the equation often consumes far too much of our focus.

The document has been the preferred solution to capturing and disseminating information for quite some time, but with the evolution of technology, perhaps the traditional document is too large-grain for our communication needs. Hypertext and collaboration spaces such as Groove and Share Point allow us to capture, disseminate and change manage more, smaller discrete items, which can foster improved communication and clarity. Unfortunately, most of the time we continue to focus on the document as a large entity that we must build, rather than working to ensure we build a shared understanding through a collection of discrete (but interconnected) ideas. Share Point can be far more effective than simply an online folder of documents, as most implementations seem to be.

These days, as we try to summarize more and more information into digestible summaries (especially as we push the information higher up the management chain) it becomes too easy to focus on form rather than function. This summarization can bring the important information to light, or can be used to obfuscate what is really happening – just ask any ‘victim’ of a dysfunctional management dashboard.

If we were to look at the quality attributes of the solution space for the communication problem we are trying to solve, we might perceive technology in an entirely different light, and the value that we gain from leveraging technology can come from unanticipated directions. We can improve clarity by using more, smaller pieces of information rather than building up huge, indigestible documents. We can focus on ensuring that key points are commonly understood through the use of feedback, which requires listening from both sides. We can leverage tools to facilitate common understanding, to automate the mundane tasks, but we must continue to ensure that we focus on getting our points across.

Be careful when trying to solve a problem with a technology solution – you may be muddying the waters rather than clarifying them. - JB [top]

Checklists and Signatures (Compendium 4.28)

The essence of consistency in a team environment is to ensure that everyone knows what the important things to do are, and to make sure they get done.

The standard operating procedure for driving your car usually does not include a thorough walk around and check of all essential features before getting in and heading down to the corner store – we simply hop in and head out. Technology has evolved to the point where cars are quite reliable, and the root cause of the vast majority of problems we have on the road is driver error. Take the same car in for service, though, or look through the owner’s manual and you will find a checklist of maintenance items, the things that should be done against a prescribed schedule to ensure that mechanical problems we don’t worry about on the road remain unlikely.

For aircraft, even though technology is at least as reliable as in cars, the magnitude of the consequences associated with a mechanical failure at altitude is generally much more severe. Despite maintenance schedules being more rigorously adhered to, all pilots, both commercial pilots and the private pilots I have flown with, go through a rigorous checklist and walk around before taking off. It would be foolhardy not to.

There are all kinds of situations where there are a list of things you need to remember to do, whether they are one-off situations like a grocery list, or repetitive situations such as flying a plane or ongoing automobile maintenance. Checklists are effective, concise memory joggers to help ensure that nothing falls off the plate.

In software development, there are many situations that lend themselves wonderfully to the use of checklists. Whether it is a list of items that need to be considered when initiating a project, design paradigms to be aware of, elements to watch out for when coding (or reviewing code), test cases to consider, or steps required to build or release the software, a checklist keeps the key information all in one place. When used as a supplement to a more comprehensive form of sharing understanding such as policies and procedures, checklists carry the essence that can be tacked on the wall and referred to repeatedly, and can also be maintained tactically to remain relevant.

Unfortunately, all these efforts of capturing and disseminating what is to be done are for naught if there is a risk that the tasks are going to be neglected anyways.

Watch closely before you take your next commercial flight, and you will find there are a number of signatures that are made. Up in the cockpit the pilot and co-pilot are signing off their pre-flight checklist, and the last thing the flight crew does before the door is closed is to sign off the manifest of the passengers that are aboard, then they go through their cross-checks and other pre-flight checks. Even most public washrooms will have a piece of paper by the door, initialed by the attendant that was in there to keep the place clean. The act of putting your name to a piece of paper attesting that you have done things can be quite compelling.

With software, though, we rarely see this. Document signoff has often become an administrative activity, and many organizations (even those with fairly weighty defined processes) have very little accountability associated with activities such as checking in code or even releasing software. While CM systems can be used to forensically identify the check-in culprit, few organizations have mechanisms in place to enforce a ‘standard’ list of pre-check-in activities such as peer review and unit and regression testing. This is interesting, as the ramifications of failing to follow the right steps in software development can often be closer to the results of an in-flight failure than to an empty toilet paper roll. The cost of putting simple accountability in place is far less than the cost of relaxed or uneven enforcement. An ounce of prevention.

If you can distill the intent of most comprehensive approaches for software development, especially those that are regulatory-based such as ISO and FDA, it is all about stating what you are going to do and showing that you have done it. Policies and procedures and templates can serve to support the letter of the defined approach, and can keep most auditors at bay as well. When you really get down to it, though, checklists and signatures can go a long way towards documenting and verifying that the intent has been properly achieved. - JB [top]

The End User's Advocate (Compendium 4.22)

Getting a great product out the door requires a great deal of initial activity to ensure that the right product will be built. There has to be a solid understanding of the market need to be addressed, which has to be thoroughly analyzed and specified to the point where the development team can go forth and build the desired solution. To deliver in a timely fashion, that specification needs to be the basis of a credible schedule, progress needs to be closely monitored and changes need to be managed through to project completion.

Before the product can go out the door, though, you need to validate that the product will truly meet the end user’s needs – that the product has been built right. Some companies will take it as a given that whatever they will build will satisfy the end user’s needs, and they are often surprised by the feedback they get from the market (which is sometimes no feedback at all, because the customer is off looking for someone else that will better resolve their problems). Others will exhaustively test against the specification to ensure that what was built is what was expected to be built – they generally will receive fewer surprises from the field related to features that don’t work properly, but they are still susceptible to omissions and errors related to addressing the original market need.

An important aspect of exercising the product that these companies have missed is the practice of ensuring that the delivered product actually meets the high level needs of the end user. In an ideal world, if project artifacts are derived from their predecessors properly, this would be a no brainer – the need maps to the specification, which maps through the design to the implementation. But in real life, there are just too many translations and implicit assumptions along the way. Has the spec actually been validated thoroughly against the needs or simply signed off based on its thickness or because the reviewer had far more pressing issues to worry about? Has the product evolved from a solid spec that addresses client needs, or has it been driven primarily by schedule pressures or the latest interesting technology choices?

There are plenty of side roads the building of a product can take, any of which can distract us from actually addressing the end user’s needs, so the final validation of the product before release must explicitly addresses this concern. While it is good and important to ensure that the product has been implemented as specified, it is critical to step back to the original problem and identify – in a quantified form if possible – whether or not the product solves the problem. If the intent was to optimize workflow in some manner, does the product measurably reduce time, improve accuracy or facilitate deeper insights? If the intent was to improve the user experience, can we unambiguously state that we have a more enjoyable product, or that it is more compelling to use?

There are two key ways of making this happen. The first, something that any organization could do, is to initially specify quantified goals for the system to meet that will determine whether the end user’s needs would be met. Identify tests that are constructed to validate whether the initial needs are indeed met, and use these tests to supplement the standard suite derived from the functional requirements and use cases.

The second approach is to ensure that the people performing the final checks on the product can accurately and reasonably act as an advocate for the end user. In many organizations, the product champion is leveraged initially to gather the voice of the customer in the analysis phase, but should also be used at the end of the day to ensure the user’s needs are met. I think a big part of what makes the titles from Electronic Arts successful is the fact that everyone in their testing teams, indeed most people in their development teams as well, are avid gamers themselves. They epitomize the best in end-user advocacy - they know what will work well in the market because they are actually part of the market themselves.

Do everything in your power to ensure that you know you have addressed the end user’s needs before the product goes out the door. - JB [top]

Tactical and Strategic Metrics (Compendium 4.16)

When putting together a practical collection of measures to track your project, product or process, it is important to recognize that you need a balance of short-term tactical measures as well as long-term strategic measures to gain a complete picture. While it is valuable to know whether you are going to make this week’s deliveries as promised, it is equally important to be sure the project is not headed for the proverbial cliff as well.

Start with some short-term measures first – they will generally be more localized, easier to measure, and provide the quick feedback that will make it easier to start and sustain a measurement program. You might find value in knowing the likelihood of completing your currently assigned tasks on time (or by how much you expect to blow out your initial estimates), or how the collection of these tasks roll up to determine whether the project will be completed on time.

Strategically, the change in tactical measures over time will reveal trends that can provide a different level of insight. Are we improving in our ability to estimate? Is the number of open defects diminishing as we approach our proposed delivery date? Do we see the definition of scope stabilizing over the life of the project?

Many strategic measures are derived from the change in tactical measures over time, implying the need to carry these measures forward in some sort of repository. Often this infrastructure is built into requirements management or issue tracking tools, but spreadsheets with pivot tables or simple databases will do in a pinch for tracking trends in measures such as estimates versus actuals. Remember to avoid going overboard with the tool support, focus on getting value out of the numbers themselves.

Spanning multiple projects, we can gather information about the effectiveness of the business as a whole. How much variability do we have in our project success? Can we correlate this variability with specific causes, such as new vs. maintenance projects, degree of technology innovation or – yipes - differences in who is running the project (which is more often a concern that the PM’s are missing adequate leadership, rather than some PM’s being worse than others)?

For any metrics program, it is important to take a disciplined approach to identifying what to measure, rather than simply measuring what is readily available or searching for numbers that justify your cause. In any case, no single measure is sufficient for a complete understanding of where you stand with your current approach – be sure that you know enough about the tactical and strategic issues to be able to make informed decisions. - JB [top]

The Ease of Obliviousness (Compendium 4.10)

Whether you are battling for a horizontal market with a dozen others or have captured a 100% of a vertical market, there is always room for improving your development team’s effectiveness. In fact, for some small organizations with a strong market position, it is quite possible to have an inefficient development organization and still maintain your revenue streams. How is this so? The quick answer is that there is no substitute for being the only product that solves a customer need. Still, whether it is known or not, this inefficiency manifests itself in reduced productivity.

For those unconsciously-incompetent organizations that are oblivious to their inefficiencies, their only hope is that competition will force them to violently extract their heads from the sand and examine the world around them. In the absence of this, they will continue dysfunctionally, their lack of pain keeping them oblivious to their opportunities.

To use the metaphor of fitness, these companies are unhealthy, while they could be working towards being healthy or fit. In cases like those above, they aren’t even aware that they are unhealthy or, more worrisome, sick. Healthy software teams are characterized by the existence of at least 10 key practices. These practices contain the appropriate amount of discipline in the following areas:

• Issue/bug/requirements change management,
• Source code and document control,
• Requirements specification,
• Pragmatic estimation
• Appropriate life cycle and project planning,
• Repeatable and frequent builds,
• Peer reviews and open sharing of work artifacts,
• Explicit allocating of dedicated testing resources,
• Existence of test plans and cases, and
• Project retrospectives.

Fit organizations also get excited about measurement, causal analysis, closed-loop improvements, requirements traceability, data collection and more.

Prior to investing, venture capitalists look for the integration of these known good practices into the everyday development project life cycle to significantly reduce corporate risk. Successful software development leaders educate teams on these key healthy practices and implement strategies for optimal adoption. Building this critical mass of knowledge will ease the burden of implementing change in established teams. You want to avoid, as insightful author Gerald Weinberg calls it, the 'Pickle Principle' where "cucumbers get more pickled than the brine gets cucumber-ed!"

How do you beat the "if it ain’t broke - don’t fix it" syndrome when you don’t know the definition of 'broke'? The quick answer is education - its pursuit and fostering. You also may want to consider getting a health check-up from the equivalent of your general practitioner, someone who will give you an objective assessment of where you stand or, in the case of many, where you painlessly lay. - GF [top]

Reading the Fine Print for Packaged Methodologies (Compendium 4.4)

Quite often, as we decide it is time to get our developmental ducks in a row, we find a need to coordinate a collection of material that will form the basis of documentation – both for our declared process, and for the products that this process is intended to produce. A quick search will generate a huge number of candidate packages to use as a basis: standards-based packages from industry associations such as the IEEE, loose smatterings of random offerings provided from community-based websites, individual elements from locations hither and yon throughout the internet, and comprehensive packages from many vendors.

There are pros and cons with each of these approaches to building a quality library, here are some of the questions that should be swirling around in your head as you imagine what the fine print should look like for any of these approaches.

Who’s behind this stuff? While some of the brightest minds in the field are contributing to the knowledge base overall, there is no minimum standard for self-promotion of templates. Individual templates or checklists may have some unique insights, but take nothing as guaranteed and use good judgment as you apply any information you find. Just because it comes from the internet, even from a well revered website, that is no guarantee it is appropriate information for your needs.

Will this fit our culture? Just as with any tool such as a bug-tracking system or project management software, packaged methodologies come as part (or all) of an assumed process, which may be agile and efficient or large and cumbersome. It should be relatively close to what you are doing now and for the most part support your current good practices (yes, every organization I’ve seen has at least some good practices that are worth focusing on and sustaining). Most established organizations don’t take kindly to major changes, and most new groups will likely have plenty of opinions about what the right approach should be. An external solution can serve to objectively mediate these different opinions, but in most cultures there will still be challenges to overcome.

What’s the real motive? Is the source of this packaged methodology really trying to improve your effectiveness while collecting a reasonable remuneration, or are they trying to lock you into their approach, their tools, their consulting. Is the methodology technology-centric (whether focused on a particular tool suite or not), quality-centric (focused on the definition and production of a well understood product), or team-centric (focused on maximizing the collaborative effectiveness of the group)? Do these motives align with yours?

How do the pieces fit together? In many approaches such as Extreme Programming, the overall methodology has been defined and evolved over time to work well as a complete system, where the pieces complement each other for reinforcement. Constant refactoring without a test-first approach, team ownership and coding standards is a recipe for disaster – and XP is one of the more concise ones around. Most comprehensive packages will eventually be tuned to fit together as a system, though there may be gaps and overlaps in version 1.0. If you mix and match, this should be one of your greatest concerns.

How easy is the fit? The broad, all encompassing methodologies usually started from what worked on a large project, then grew over time to be more generally applicable across the industry, to a point where they cover all the bases but need to be dramatically tailored down to be reasonably useful for even large projects. Is there any guidance on how to make it practical for your needs, and how much will that guidance cost? Remember that for the best fit, the methodology needs to be tailored down for the organization, then tuned specifically for each project’s needs. The more comprehensive the starting point, the more tailoring required.

What are the lifecycle costs? If you are buying a product, how are the licenses structured, are there maintenance fees (which would appear rather strange as there shouldn’t be a great deal of bugs or feature churn in a defined methodology – for the most part, the wheel has already been invented), and what will it cost to have someone come in and explain or help tailor this beast? On the other end of the spectrum, will you even be able to find the source of information for clarification if necessary?

Can we do it ourselves? There are plenty of reasons for tackling the challenges of creating a defined methodology yourself, with only light reliance on a packaged methodology:

• First and foremost, a methodology defined as a team for the needs of the team will be embraced by the team. Participation breeds ownership and acceptance, and you can’t buy that in a box anywhere.
• It is best to start small. Any wholesale change in approach will cause far greater culture shock than benefit for the group. The combined learning curves of many individual practices that comprise an overall methodology will be a major setback, one that some companies that I have worked with never did manage to recover from. Evolution as a gradual process has shown to work over millions of years.
• Grow organically. Methodology components such as coding or design standards, or review checklists are best grown based on the internal findings of the group. Find a problem once, fix it. Find it twice, identify how it should be remedied in a standard fashion and broadcast the news in a standard or checklist. They will become personal expressions of your best practices, rather than a generic set that will rarely apply to your situation.
• Play the field. Rather than locking into one source, learn from all that are available. Nobody has cornered the market on appropriate approaches for all software projects, and no one ever will. Take what works from wherever you can, and don’t be afraid to mix and match if you have considered the ramifications (remember the notion of a cohesive system above). Remember the bottom line is improved predictability and effectiveness for your projects, not adherence to someone else’s assumed approach. Best-practices are only situationally applicable.

For all of the reasons above, my leaning would be to take an internal approach to building up a defined methodology, at most using a broader overall methodology as a basis of defining the direction you are taking. Don’t take the sales pitch at face value, as with anything you need to understand what you are getting yourself into. Consider what the fine print would look like if it was available. - JB [top]

Touching the Void of Improvement (Compendium 3.50)

The movie “Touching the Void” tells of two British alpine climbers Simon Yates and Joe Simpson who deal miraculously with a seemingly impossible set of life-threatening challenges while climbing and descending the Peruvian Andes mountain of Siula Grande. Without giving away too much of the plot, one of the climbers shatters his leg while descending at the 20,000 foot level and falls hundreds of feet into a crevice. The story chronicles the amazing journey of Joe Simpson as he deals with overwhelming adversity and finds a way to get home amidst devastating obstacles.

There is a key moment in the movie when Simpson accepts that the big picture is just too much to digest and that dwelling on the immensity of it all is not a productive exercise. Progress for Simpson is picking a location 50 feet away and setting a goal to reach that goal in 20 minutes. As insignificant this may seem, remember Simpson is doing this on one-leg, without food and water, -40 temperatures, and navigating glaciered ice and rocks. Step by step and stumble by painful stumble, progress is made towards his impossible goal.

Developing software and, in particular, improving and changing the way teams develop software has a hint of this kind of challenge inherent in it. The exception is that, unlike Joe Simpson, teams tend to focus on the formidable view rather than on the 20 minute view. Huddling in fetal position in the crevice and freezing to death appears to be the chosen approach for many. We hear recurring mantras such as "we’ll never ship software with no known defects." or "we’ll never get enough testing resources" or "this organization will never change". For the many wishing to make improvements but feel overwhelmed, take some tiny steps – take one step or one stumble – but do move forward and leave the moaning masses.

Here are some things you can do:

• Bring awareness through measuring 2 or 3 aspects of development. Sometimes it just takes a comment like "gee, I never looked at it that way – thanks." to get change happening. A couple of measures we see on a regular basis are the Weekly Rates of Incoming Defects and Repair Rates of Defects. Another measure is the Percent of Total Test Cases executed, or Percent of Project Effort Allocated to Testing.
• Post the measures where people can view, debate, and generate ideas for potential solutions.
• Collect numbers on errors found during requirements, design, code or test review.
• Start sending emails recommending books, magazines, websites, articles – any insights and ideas from others.
• Start a Lunch-and-Learn session on a weekly basis and get developers to share favourite code snippets, personal tools, books read, or videos. Anything that gets the team sharing together will bring benefit to the project. And finally,
• Pass on the URL to the Clarrus Compendia – it’s a goldmine of challenging topics for your Lunch-and-Learns. There are so many, many more steps that can be taken with a minimal amount of effort. It is also irrational to believe that there is no wasted time in your development process and that improvement is not possible.

In the midst of crushing schedule pressures, massive disillusionment from poor management decisions, insufficient resources and challenging customers, you must find a way to make progress towards your goals. Whether you’re eating an elephant one bite at a time, descending from Siula Grande on one leg, or trying to reduce the number of delivered product defects, there is something small you can do – now! - GF [top]

What Were We Thinking? CM as a Memory Jogger (Compendium 3.44)

Have you ever run into the situation where, 6 months or a year after you’ve made a decision, you find yourself back in a similar situation, or even worse, in a real pickle because of the decision you had made? For the life of you it is impossible to remember what you were thinking in the first place. Been there, done that - if only I knew then what I know now. Hmmm…what did I know then, anyways? It seems like a lifetime ago...

Configuration management principles help the project out in many ways. Most of us recognize CM as a tool that helps us recover that file that mysteriously went missing, or back out of an implementation path that is even uglier than the one we hoped to fix. Living in the here and now, capturing and maintaining our information under CM gives us the confidence to move forward, knowing we have the safety net if we take a wrong step. It helps coordinate the team, and if used well, helps avoid the deadly integration stages that you may never dig yourself out of.

We have also come to depend on CM to give us assurance that we can pull together precisely what is needed to build the system – all the stuff we need, and nothing extra. Done right, we know precisely which versions of each file holds together to form a cohesive set. We can reproduce any set of information we have released without having to maintain a plethora of different versions. In some shops, CM has become a crutch that has allowed us to be very sloppy with our releases (…what’s one more branch?...), but that’s fodder for another article.

In many companies I have worked with, there is one capability in CM systems that is rarely used well – the capability of capturing the rationale behind any changes that are made to the system. Any CM tool worth its salt will allow you identify the reason for change – indeed, I don’t know of any commercial tools that don’t support this. Whether it is freeform text or a reference to a change request number, this audit trail is critical for the strategic evolution of the system. At any point in time, you should be able to go back to any change that has taken place and understand who made the change, why the change was made, and how it was made.

If you find that as you check in files you just click past that troublesome dialog box that pops up asking for a comment, you are performing a severe disservice to yourself and to the project. There are very few of us that can recall our rationale for all our actions even a week after the fact, let alone 6 months or more, and it is impossible for others to decipher the rationale from empty comment fields. Without the knowledge of why changes were made, you are left at best to relearn the rationale, or at worst to misinterpret the rationale and inject more problems into the system.

Once you have your project baselined, you have set up a commitment and expectations with all the stakeholders. Capture this baseline under a solid configuration, and manage change appropriately from that point on and your commitment will be preserved. Use your CM system to preserve the changes and the rationale behind those changes, and you have a built-in long-term memory for the myriad decisions that are made on any project. It is far easier to retrace your steps with a solid audit trail than to use forensics to try to reconstruct it after the fact. - JB [top]

Quality is Your Job (Compendium 3.38)

Software development groups often have a very twisted perspective of what quality means. There will be a QA group that beats up on the product after it has been thrown over the wall from the development team. These testers will poke at it from all sides and pass back their issues, whereupon the developers will argue from the position of "well, we wrote the stuff, so we certainly should know how it is supposed to work!" This internal bickering continues until:

• The project times out and the product is shipped anyways, or
• The QA group stops looking for bugs, so with ‘no new bugs’, the product can be shipped, or
• Someone in senior management repeatedly diverts the efforts to another direction, and few products ever get shipped.

I’ve seen all of these in action in the real world - usually it is not a pretty sight.

Generally in software, there is the notion that “quality is what other people are responsible for” on a project. We will put new hires into the QA/Test group so that they can familiarize themselves with the product in a place where they can’t do much harm (or much help, for that matter). The prima-donnas get the cushy design and architecting gigs (and all too often the relaxed standards of practice as well), those that don’t quite cut it end up elsewhere. We will appoint a Quality Director that sits with the management team only if we have to when quality standards are forced upon us because we want to sell to Europe, or work in Defense, or build Medical Devices. Then we’ll do our best to work around these pesky people that are trying to stop us from shipping our product anyways. Our goal is to get the job done, right?

I don’t think so. The goal should be to get the job right, because just ‘getting it done’ usually means you’ll be back at the same task later when someone else finds that it doesn’t really work. If you are lucky, it’s those newbie internal testers saving your bacon - the ones you have put there so they ‘can’t do much harm’. If you’re not so lucky, it’s a disgruntled customer – and don’t you dare try to lay the blame on the test team!

The process of developing a product, software or otherwise, involves a chain of people each doing their part to contribute to the whole, and the notion of quality should be on every person’s mind throughout that chain. As humans we all make mistakes, but it is unfair, inefficient and very risky to expect that the last person in the chain is responsible for all the mistakes injected throughout the process. Incorrectly naming these victims of our deeds the Quality Assurance group simply perpetuates the notion that it’s their job to clean up the mess while you move on to other things (and hope they don’t interrupt you with their trivial concerns).

Quality needs to be on everyone’s mind, at all times – it’s something that should be bottled and pumped through software development ventilation systems. We are all responsible for the quality of the components we contribute to the overall product, and we should be explicitly aware of what quality really means for our contribution. Quality Assurance isn’t the people at the tail end that has little time to find these nasty problems – a Quality Assurance team is best thought of as part facilitator, part mentor, part support team, part objective auditor. Someone that is really doing Quality Assurance is working to ensure that everyone else has the skills, knowledge and resources to do a quality job.

No matter where you are in your company’s food chain, if you think that quality is simply someone else’s responsibility, you are totally missing the point. It is your job, wherever your role, to ensure that your contribution does not degrade the overall system. Quality is your responsibility. - JB [top]

Meaningful Measures (Compendium 3.32)

Look at any car’s dashboard today, and you will find the same set of core indicators. The primary element is always a speedometer, followed by a less prominent fuel gauge and odometer, after which the information tapers off quickly. There may be a temperature gauge or just a couple of 'idiot lights', there might be a tachometer, there may be tire pressure indicators, possibly a few others. While there is a trend these days to increase the information available to the driver (not unlike the trend of adding features to already bloated desktop applications), the important elements of a dashboard really haven’t changed for years.

This consistency over time is, literally, no accident. The driver’s primary goal behind the wheel is to safely navigate from Point A to Point B, and aside from the feedback provided by the windows and mirrors, the dashboard provides the information that allows the driver to reach that destination. Am I moving at a reasonable speed, do I have enough fuel, is the engine working within acceptable operating parameters? The rest is secondary.

What’s meaningful to measure in a car is all a matter of perspective. Take that same car to the mechanic for a tune-up, and he will only glance at the odometer to see whether it’s close to the time for a new timing belt or other scheduled maintenance. The key indicators for the mechanic today come from the on-board computer that provides detailed information about the engine’s timing, fuel mixture, and a host of other measures that don’t even make sense to most drivers.

What’s meaningful to measure in software development is also a matter of perspective. As with a car, there are literally thousands of things that could be measured on a software project, and it is important to avoid the temptation to measure for the wrong reasons:

• We want to avoid simply looking at the measures that are at our fingertips and easy to gather in order to distill useful information ("our timesheets are telling us that we’re all working 40 hour weeks…"),
• Avoid manipulating information to tell us what we want to hear ("since we have stopped looking for defects, our defect count has gone way down...").

Dashboards for software projects, both commercial and home-grown applications, have been all the rage for the past few years - with all manner of information displayed. Rather than providing the useful information to appropriately drive the project, they often end up more like those activity centers that we place in our children’s playpens – they can serve to pacify senior management with all the charts and dials, but don’t do much else. Selecting appropriate measures is not that straightforward.

One person’s meaningful measures can be useless for someone else. A high Cyclomatic Complexity can raise alarms for an astute developer, while a low Schedule Performance Index might do the same for a project manager. Your perspective, consisting of the environment and culture you are working in, your role in the project, and whether you are focused on strategic or tactical issues will determine the goals you set. These in turn will help you ask the questions of the project to identify whether or not you are achieving these goals, and the questions will determine which are the most meaningful measures to take.

The GQM (Goals/Questions/Metrics) approach for building a metrics program, coined by Vic Basili, describes a disciplined approach to identifying measures that will be meaningful for your context. Done correctly, you will have a small handful of measures that make sense for you, and help you make the right decisions on your project. You will have the right dashboard to help you get to your destination with no accidents. - JB [top]

Quality of Life (Compendium 3.26)

Some of us live to work, others work to live. Unfortunately, there are too many stuck in the former category that would prefer to be in the latter. There is always pressure to find those nasty bugs that we just can’t seem to put enough band-aids on, or to meet that looming deadline. Or as Douglas Adams said, "I love deadlines. I love the whooshing sound they make as they fly by."

There are individuals (and indeed entire sectors) that relish the thought of putting in the long hours to get a seemingly impossible project out the door on time. It can become a badge of honour, a cultural norm that makes those that are interested in a different balance in life to appear as misfits. From what I have seen and experienced, though, this is not a sustainable approach. Individuals will eventually tire of the pace, and organizations will face ongoing turnover challenges. Brute force doesn’t appear to be the long-term answer to the quality question.

It is interesting to consider today, as the Cassini spacecraft settles in between the rings of Saturn, how quality applied appropriately in software development can make a significant difference.

Way back in the 6th issue of Fast Company, there was an article about the gang at t