Inspection is not enough
Deming is world famous, especially in the Agile/Lean world. During his life Deming taught top management how to improve design (and thus service), product quality, testing and sales. His name is synonymous to quality improvement and his philosophies are still seen as visionary.
One of Deming’s principles is: “Cease dependence on inspection to achieve quality. Eliminate the need for massive inspection by building quality into the product in the first place.” Sounds good, right? In stead of testing every last inch of your product, you increase it’s intrinsic quality which is more sustainable, has a direct benefit for the customer and eventually lowers cost.
Now how does that relate to software development? In software development we do quite a bit of inspection, not only by peer reviews but mostly by testing. We test documents, units of code, modules of code, systems, user interfaces, services etc. We test functionality, speed, error handling, ease of use, fit for purpose etc. etc. My interpretation of what Deming would say to us: focus less on creating more tests, focus more on product and process quality.
Let’s start out with the obvious, tests are useful. Even when you’re 100% Deming compliant™ you’ll still have tests, especially automated tests. But I’m not arguing to stop testing, the point I’m making is that tests lure you into a false sense of security and are not a guarantee for improving quality. A symptom of that is in software development QA people (often called testers) seem mainly concerned with ‘checking’ and spend very little time on ‘mistake proofing’.
John Hunter says tests are a form of prevention. They are a form of Poke Yoke, Poke Yoke means “mistake-proofing”. His reasoning is: with a test in place, you can’t introduce a bug in the software. But this is not true, the test detects the bug, but doesn’t prevent the bug from entering the code. A better example of Poke Yoke in software development is using high level language constructs, these prevent mistakes that are easily made with low level languages. The difference is that the mistake-proofing is done at the level of human actions where the mistakes occur and prevents the mistake being made. A test doesn’t prevent a mistake being made, it just signals that the product contains (at least) one bug. So even if your test servers as a Poke Yoke, it’s only detecting, not preventing.
Prevent Construction Errors
Tests and other forms of inspection have one fundamental limitation. I’ll try to clarify this with an analogy:
Suppose you’re a gardener who has to keep a big lawn. The worst enemy of the gardener is the mole and its molehills. Now suppose you’ve discovered a molehill. You don’t want those on your lawn. So you can remove it and place a heavy object (a slab of stone) where the hole was dug. You’ve made sure the mole can’t create a molehill at that particular place, but you haven’t prevented the next molehill from appearing.
After talking to your software tester friend, he helps you to find a way to strategically put the slabs in place before the mole arrives, preventing a lot of the molehills from appearing. You’ll soon end up with a garden covered in stone slabs and it keeps your lawn fairly green. However, the obvious question is, why aren’t you trying to get rid of the mole?
In software development, I’ve seen too few team getting rid of the mole. Too few discussions on the real cause of bugs. I would expect a Deming compliant team™ asking: “What preceded writing that particular bit of code that led to the bug? What would make it impossible to create this bug again? How can we increase quality across the board to prevent these kind of bugs, not just this single one?”
Prevention vs. Inspection
Donald Reinertsen says “Inspection makes economic sense if it is more cost-effective than prevention.” Of course, this is a truism and automated tests are a relatively cheap way to do inspection. When inspection is your major instrument for assuring and improving quality, there’s a risk of test-addiction. Suppose you have lots of tests, but in production the customers finds a bug. What do you do? You create a test for it and that might not be the best way to handle the bug. If the actual cause is low software quality due to overly complex design or sloppy coding, it’s not enough to create a test.The opposite could even be true: the number of tests per quantity of software (LoC) should decrease over time because quality increases to such a level tests are no longer needed in large numbers.
Decrease the necessity of testing
In software development we need to fight the mole more often than patching the molehill. Do a root cause analysis with the team to uncover the cause of the bug and change your process, change they way you do design, introduce patterns, use pair programming, take a training course or do whatever prevents the mistake from occurring again, than will not only prevent that particular bug from re-occuring, it will prevent a whole slew of bugs because overall quality is higher. Your customer will thank you for it.