Software is NOT like 6-Sigma

Six-Sigma, ISO 9001, and the like are processes used in mass-production lines to ensure vanishingly small errors.

It makes sense, if you’re Dunkin Doughnuts producing 350,000 doughnuts per day. The problem is that some people insist that these processes should be applicable to software development.

“Why can’t software development be more like electronics development?” “Six-Sigma means almost zero defects on massive scales, so why can’t we use these techniques in software?” These are the kinds of questions you get.

And of course some people actually try. Defense contractors, CMMI devotees, and behemoth government-like organizations try.

But software development isn’t like making 350,000 doughnuts. With doughnuts the thing you’re making is completely understood and measurable, and the problems come in copying the Platonic form zillions of times without error. Difficult, I agree.

But in software you’re making ONE, massive, complex, changing, ill-understood thing, and making it ONCE. When we ship copies, they are perfect copies. Unless you’re burning DVD’s, in which case the act of burning might be best handled under Six-Sigma.

The problems are completely different.

For example, our software interfaces with other tools. When those tool vendors release new versions, it’s possible that they break our interface with them. Therefore, even with perfect copying and no existing bugs, our software will STILL BREAK!

For example, in software requirements are supposed to change. Possibly during development, definitely afterwards. Once it’s decided that a bridge is supposed to go from A to B, it’s done. Even if that decision turns out to be wrong, everyone accepts that either we should nuke the bridge or finish it anyway — there’s no concept that we should rebuild the bridge in mid-air and still make schedule. Not so in software — this happens all the time.

For example, software’s purpose changes over time. A doughnut is always supposed to be a doughnut. A home Wifi router gets small firmware updates, but no one expects it to do more than what it did on the they day bought it. Most software projects change radically over their lifecycle. Windows 1.0 barely solved any problems; Windows 95 changed everything, and Vista barely looks like it came from the same company that released XP. Even Google Search — the algorithms, advertising, and company goals are nothing like they were when it first became well known. But Dunkin still primarily makes doughnuts (and coffee), and we expect nothing now from 50-year-old bridges than what we originally expected.

There’s more. Software developers themselves are a different breed than workers on the doughnut assembly line floor. The world-view and skills required for software development turns over at least every 10 years (more often if you want to change jobs). And sometimes a new feature really is more useful to a customer than a bug-fix, though rarely does anyone admit it.

The point isn’t to say that it’s OK that we have bugs in software, or that we can’t do anything about, or that we shouldn’t try. It’s just that blindly applying process from other disciplines is not the answer.

  • Paul

    I completely agree!!! We currently have a NEW CIO who thinks 6 Sigma is the end-all be-all solution to software development.

    I’m not sure how to get him to change his mind. Any ideas?

  • Jason

    I find these sorts of debates turn religious. That is, someone believes 6-sigma is correct and someone else doesn’t and they will never convince each other.

    One approach might be to propose an experiment. One group "uses 6-sigma" (whatever that means for software development — another problem!) and one uses process X where "X" is what you’re trying to convince him of.

    You need objective measure of "success." Number of bugs of certain types and/or severities, defect density, … I don’t know, but something applicable to both groups. There also needs to be an "equal amount of work," meaning similar codebases (size, age, maturity), languages, spec availability, etc..

    The most accurate would be two groups doing exactly the same thing, but this might be a luxury you can’t afford.

    Hard evidence is hard to argue against; that’s the good news. The bad news is that I’ve found "executives" to be unwilling to empirically test their beliefs (lest they be disproved and humiliated by a subordinate). I hope yours is more reasonable, but if he’s stubborn on this front you might be out of luck.

  • Jason

    Does this CIO have previous software development experience?

    If yes, and he still likes 6-s, then I’m running out of ideas.

    If no, perhaps he could be persuaded to consult other CTO/CIOs of similar companies to ask their opinion. Or a software process consultant?

  • Alex

    I was an engineer in 1985 when 6-sigma was introduced to our company. Most people who spoke about it knew nothing about statistics. Most organizations are in 3-4 sigma range and CANNOT AFFORD 6 sigma, it is very expensive and not really necessary. Where do you see 6 sigma which is equated to 3 ppm defect rate? Car industry? aircraft industry? C’mon people. I can understand when you manufacture product in millions (like ICs and microcomputers) then you can even trend 6sigma events. If you do not get production in millions you will not be able to prove 3 ppm defect rate!
    Also the ICs manufactured in sterile environment fail after 12 month quite often, just look at your warranty. So even if 6 sigma is implemented you may have no superior quality of the product just because of the design issues.

    Often 6-Sigma is a lure to replace all kind of IT equipment and infrastructure to accomplish a desirable flowchart structure of 6 Sigma process.
    Defects always will exist unless your process has no variables. Just use common sense, analyze your process using your good college education and do not try to get 3 ppm defect rate, save your company money. Remember, Apollo missions were accomplished before 6-sigma era and after 6-sigma was introduced we had all kind of problems in all industries. The main effort should be placed on the design.