Art and Science
The March 2009 Edition of Harvard Business Review has an excellent article entitled ‘When Should a Process be Art, Not Science?’ Many great insights, but as with most things that we try to categorize, there is primarily a sense in the article that we need to choose, that the answer be one or the other: art or science. Relegated to a sidebar is the notion that science can be a platform for art, which I see as the primary insight in the article. This is something that anyone that works in that ‘process’ niche should think deeply about, even if you are like me and prefer to not utter that ‘p-word’ too loudly.
Most perspectives in the area of defining what we need to do (a.k.a. establishing process) tend to swing and stick to one end of the continuum.
There is the heavy process dogma, the exhausting translation of standards such as the CMMI or ISO or the PMBoK (though I would bet that none of these were intended to be interpreted in this manner) that ends up with stodgy standards, templates and forms to be completed, and all form of checks and balances that are to be completed for everything on the project. We generally produce a mountain of paper that is written once, never read and never updated, and a late, bloated product that doesn’t meet the end user’s needs. Pure science fails.
There is the other end of the spectrum, which is unfortunately where many weak agile interpretations end up. We race to get into the code as soon as possible, we assume that we are best served to discover along the way. What often happens is that technical debt builds up, and while the first few sprints appear to run smoothly, we often end up in a situation much like before, where we are missing deadlines (but this time there is no documentation to fall back on, clearly the worst case of whether it is up to date). Pure art fails.
In both cases, I have seen creativity, the art side of the equation, relegated to the same dysfunctional purpose. For iterations on an air traffic control system and for many sprints I have observed more recently, creativity is primarily used to figure out new excuses for why the product is late, and for new rationales for asking for more time. There is no creativity left to put into the product.
A balance is clearly needed between art and science.
In the HBR article, the example of science as a platform for art describes how Steinway builds pianos. The science of computer-controlled equipment and statistical process control for parts manufacturing supports the ‘voicers’ being able to use their judgment to customize the equipment for individual professional pianists. Wow.
The same approach is used in great software teams. There is an underlying science of process that makes sure that nothing falls through the cracks, that things that should be taken for granted actually can be. There is a configuration management system in place, and it is a given that everyone uses it correctly. There is a defect tracking system in place, where the same expectations hold. This is the absolute bare minimum for good teams: great teams extend both of these principles further up the chain (pre-code), and treat every decision they make in the same way – business needs, analysis, design and test artifacts, and everything else, too.
The reason that some agile teams have been able to demonstrate sustained success with apparently little ‘science-level’ process is that they are almost exclusively populated by seasoned veterans who have internalized the appreciation for the need for this science. They would shudder at the though of not having version control, or a current backlog of work, or the right tools to get the job done. The base science needs of their approach ‘go without thinking’, and hence, unfortunately, also ‘goes without stating’ as they profess their success with their artful approach.
Those that haven’t internalized this appreciation for the science then try the artful approach, and quickly build up their technical debt.
The right process for software development needs the balance between art and science, and this will vary based on the domain, the team’s experience and culture, and the product being built. We need to ask ourselves what level of science do we need to sustain our ability to be artful in the creation of our product. – JB
If you enjoyed this post, make sure you subscribe to my RSS feed!Comments
2 Responses to “Art and Science”
Feel free to leave a comment...
Jim, this has got to be one of the best posts I have ever read regarding the balance of art and science within software development. Your (extremely astute) observation of how teams populated with seasoned veterans *who have internalized the science* are most successful in artfully delivering using Agile approaches. Thanks for being so articulate and spot on!
[...] I read a blog post from a friend of mine from clarris consulting where he describes the art and science of software development. This is a good article for it is a pragmatic look at how successful software teams function and [...]