The Pursuit of Quality: Chasing Tornadoes or Just Hot Air?

Many thanks to Helmut Pichler and Manfred Baumgartner of Anecon who invited me to speak at their joint seminar with Microsoft at the Microsoft office in Vienna, Austria in May. Thanks also to Andreas Pollak of Microsoft who organised the event and who acted as my able assistant when my remote control did not work.

The event agenda and sessions are described here. Andreas assembled the Powerpoint and a voice recording into a movie which is reproduced below. Apologies for the first minute of the talk, as my remote control gadget didn't work. Andreas kindly offered assistance :O)

The talk, ah yes, the talk. Essentially, it's an attempt to discuss the meaning of quality and how testers use test models. Abstract below.

I hope I don't upset too many British Royal Watchers, Apple Product devotees or McDonalds lovers with this talk. I'm not one of you, I'm afraid.

If your browser doesn't show the embedded video, follow this link.

Abstract:
Rain is great for farmers and their crops, but terrible for tourists. Wind is essential for sailors and windmills but bad for the rest of us. Quality, like weather, is good or bad and that depends on who you are. Just like beauty, comfort, facility, flavour, intuitiveness, excitement and risk, quality is a concept that most people understand, but few can explain. It’s worse. Quality is an all-encompassing, collective term for these and many other difficult concepts.

Quality is not an attribute of a system – it is a relationship between systems and stakeholders who take different views and the model of Quality that prevails has more to do with stakeholders than the system itself. Measurable quality attributes make techies feel good, but they don’t help stakeholders if they can’t be related to experience. If statistics don’t inform the stakeholders’ vision or model of quality, we think we do a good job. They think we waste their time and money.
Whether documented or not, testers need and use models to identify what is important and what to test. A control flow graph has meaning (and value) to a programmer but not to a user. An equivalence partition has meaning to users but not the CEO. Control flow, equivalence partitions are models with value in some, but never all, contexts.
If we want to help stakeholders to make better-informed decisions then we need test models that do more than identify tests. We need models that take account of the stakeholders’ perspective and have meaning in the context of their decision-making. If we measure quality using technical models (quality attributes, test techniques) we delude both our stakeholders and ourselves into thinking we are in control of Quality.

We’re not.

In this talk, Paul uses famous, funny and tragic examples of system failures to illustrate ways in which test models and (therefore testing) failed. He argues strongly that the pursuit of Quality requires that testers need better test models and how to create them, fast.