The Audit Game
The first question I ask people in an organization that is ISO certified or has achieved a higher CMM rating is “what happens in this place a week before the auditor comes to town?” The response, either verbal or visually (such as nervous twitches or shifting around in their chair), tells me more than hours of additional queries ever could.
Unfortunately, the response is rarely good. It usually is some variation on the theme of the organization coming to a standstill – stop all work in progress, it’s time to get our documentation ducks in a row. Perhaps there needs to be a few practice sessions to make sure that those that haven’t been around this block before know how to properly answer any sensitive questions the auditor might have.
Preparing for an audit, any audit, is like preparing for a blood test – it’s ludicrous.
I’ve been involved with organizations where it was a regular routine to spend some time to polish a document in preparation for passing it over to QA for review. Being in QA at the time, it struck me as an absurd practice. If the primary value of any documentation on a project is to ensure consistent, shared understanding across the team so that there are fewer misunderstandings, the document isn’t for QA at all, but for the team. A reasonable QA person will not review for existence or even content of a document, but will check to ensure that documentation is properly used as a vehicle for team consistency, in-process. If that is not the case, all the polish for QA is wasted effort.
For external audits, whether it is for investors, clients, or an auditor with the intent of gaining a qualification from ISO or the SEI or similar (usually for sales or marketing purposes), that initial preparation in the audit dance is often premeditated setting of false expectations. While you may achieve your superficial goals of the audit, there will likely be some time in the future where you fail to meet those expectations. Credibility and any tactical advantage you may have gained will be lost with such a naïve strategy.
In all cases, a fair, unbiased external audit of current practices will provide you greater insight than any result you obtain after putting up a false front for the auditor. The primary goal for any audit should be an understanding of actual performance, not a superficial review of hastily prepared fluff. The notion of scheduled audits makes it difficult to achieve this goal, and often drives far more pomp and circumstance than effective assessment. It is more important that an organization demonstrates an ability to achieve stated goals through institutionalization of practices rather than satisfies all the simplistic checklist items that an auditor can bring to the table, regardless of the size of the checklist. Unfortunately, many auditors I have worked with (internal QA personnel and ‘certified’ external auditors) are ill-equipped to see past the easily simulated practices and inspect for true institutionalization and performance.
Any auditor worth his salt can quickly see through a façade. What do you do before the auditor comes to town? – JB