Testing is in a mess

Got your attention? I’m triggered into posting what follows because Rob Lambert has published an ‘ebook’ The Problems of Testing. Now it reminded me of a blog I wrote 18 months or so ago – but never published. I never pressed the ‘publish’ button because I thought it was rather too negative. But Rob’s frustration is shared and here’s my take on it. I don’t agree with all he says, but I think there’s a certain amount of complacency, self-delusion and over capacity in our business. Here what I wrote … I was going to call this post “Do you engage your brain before you test?” but as I wrote it, I got more and more upset by the state of our discipline. (I can hardly bring myself to call it a discipline any more). It seems to me that the major ‘trends’ occurring in our discipline are being promoted and adopted at a terrifying rate. I’m not worried about the rate of change, or being left behind. I am worried that the people who are being sold these approaches and implementing them are taking on so-called ‘solutions’ without understanding what the underlying problem is, what they are trying to achieve, or how to evaluate ‘success’. People have no time and no frame of reference in which to think or consider the pros and cons of available courses of action. Often, they don’t even know who they are testing for. For example:

  • tools are used to automate tests that have unknown or limited value
  • ‘crowds’ might help us test but what is the objective, where is the control or accountability?
  • test design techniques are promoted as ‘good practice’ without people understanding the concept of test models, their value and limitations
  • coverage is discussed endlessly without any understanding of it’s meaning, subjectivity and interpretation
  • the language and terminology we use is riddled with duplication, inconsistency and ambiguity
  • outsourcing/offshoring are promoted as being effective and economic – without any discussion of its value
  • the exploratory/ad-hoc v planned/documented testing debate is a yah-boo-sucks schoolyard shouting match – how is an interested observer to understand the issues?
  • most debate is about software testing, but we test systems, don’t we? Why don’t we adopt a Systems Approach?
  • certification schemes are embedding many of these flawed ideas and are strangling competitive, viable and often better alternatives.

I could go on. If I suggested that 50% of the testers currently in our business shouldn’t be – how would you argue against that? Why are YOU in this business? For example – can you string two words together? One of the reasons I wrote the TESTER’S POCKETBOOK was to drill down to the fundamentals that underpin all testing (if I could find them). It’s been a struggle, but I’ve come up with some and they are useful. I call these fundamentals TEST AXIOMS and you can see them here. They make sense to me as a starting point for discussion on testing and test practices, but they also underpin much of the thinking about test strategy and improvement. Ask them of yourself or your organisation or the next person you interview. They work for me. Am I too pessimistic about the state of our industry? Is anything going well out there? If/as/when the economies of our various countries squeezes testers out of businesses – will you still be in a job? How will you justify your role? Developers, analysts and users combined can do most of what you do. Can you do their job? Who is indispensable now? Think about it.