DevOps is killing QA. Really?

I read a fairly, let’s say, challenging, article on the website here:

It’s a rather poor, but sadly typical, misrepresentation or let’s be generous miunderstanding of the “state of the industry”. The opening comment gives you the gist.

“If you work in software quality assurance (QA), it’s time to find a new job.”

Apparently DevOps is the ‘next generation of agile development … and eliminates the need for QA as a separate entity’. OK maybe DevOps doesn’t mandate or even require independent test teams or testers so much. But it does not say testing is not required. Whatever.

There then follows a rather ‘common argument’ – I’d say eccentric – view of DevOps at the centre of a Venn diagram. He then references somene elses’ view that suggests DevOps QA is meant to prevent defects rather than find them but with all due respect(!) both are wrong. Ah, now we get to the meat. Nearly.

The next paragraph conflates Continuous Delivery (CD), Continuous Integration and the ‘measurement of quality’. Whatever that is.

“You cannot have any human interaction if you want to run CD.”


“The developers now own the responsibility rather than a separate entity within the organization”

Right. (Nods sagely)

“DevOps entails the use of vendors and tools such as BUGtrackJIRA and GitHub …”

“To run a proper DevOps operation, you cannot have QA at all”

That’s that then. But there’s more!

“So, what will happen to all of the people who work in QA? One of the happiest jobs in the United States might not be happy for long as more and more organizations move to DevOps and they become more and more redundant.”

Happy? Er, what? (Oh by the way, redundant is a boolean IMHO).

Then we have some interesting statistics from a website I can’t say I know the site or the source of data well. But it is entirely clear that the range of activities of ISTQB qualified testers have healthy futures. In the nomenclature of the labels for each activitiy the outlook is ‘Bright’ or ‘Green’. I would have said, at least in a DevOps context that their prospects were less than optimal, but according to the author’s source, prospects are blooming. Hey ho. Quote a source that contradicts one’s main thesis. Way to go!

But, hold on – there really is bad news …

“However, the BLS numbers are likely too generous because the bureau does not yet recognize “DevOps” as a separate profession at all

So stats from an obviously spurious source have no relevance at all. That’s all right then.

And now we have the killer blow. Google job search statistics. Da dah dahhhhh!

“As evidence, just look at how the relative number of Google searches in Google Trends for “sqa jobs” is slowly declining while the number for “devops jobs” is rapidly increasing:”

And here we have it. The definitive statistics that prove DevOps is on the rise and QA jobs are collapsing!

qa jobs vs devops jobs

“For QA, the trend definitely does not look good.”

So. That’s that. The end of QA. Of Testing. Of our voice of reason in a world of madness.

Or is it? Really?

I followed the link to the Google stats. I suggest you do the same. I entered ‘Software Testing Jobs’ as a search term to be added and compared on the graph and… voila! REDEMPTION!!!

Try it yourself, add a search term to the analysis. Here is the graph I obtained. I suggest you do the same. Here’s is my graph:

Now, our American cousins tend to call testers and testing – QA. We can forgive them, I’m sure. But I know the term testers is more than popular in IT circles over there. So think on this:

The ratio of Testers v DevOps jobs is around five to one. Thats testers to ALL  JOBS IN DEVOPS IS FIVE TO ONE.


So. A conclusion.

  1. Don’t pay attention to blogs of people with agendas or who are clearly stupid.
  2. Think carefully about the apparent sense but clear nonsense that people put on blogs.
  3. Be confident that testing, QA or whatever you call it is as important now as it was forty years ago and always will be.

It’s just that the people who do testing might not be called testers. Forever.

Over and out.



Introducing the Tools Knowledge Base

The Software Tools Market is Changing

  • The Digital revolution means software must be aligned much more precisely with business
  • Customers get almost daily mobile app updates; they want the same for their business applications
  • Continuous delivery, DevOps, Shift-Left and pervasive automation are the means to succeed
  • New processes, disciplines and the tumbling walls between silos mean tools are essential
  • The automation challenge has moved from selection and implementation of 1-3 tools for each discipline to selection and implementation of 20-30 tools for a team in a DevOps regime
  • Tools need to be integrated technically, but the integration of team disciplines is just as important.

The Need for a Tools Directory

For as long as the web has existed, there have been websites that provide lists of references to tools that support, for example, test automation. These web pages and sites have been set up by individuals, wishing to share their knowledge of software tools for their own communities.

It’s a burdensome task to create and maintain these lists. Vendors move webpages around, they rename tools, they merge and split tool functionality, they add new tools and new vendors and tools are popping up all the time. It’s really hard to maintain the accuracy of lists like these. If you look around the various websites that provide such lists, this is what you tend to find:

  • Listings do not provide much detail beyond simple categorization, e.g. ‘Web’ or ‘Mobile’ test tools
  • Invariably, the lists are incomplete. Common tools are listed; less well-known tools are often missing
  • Most listings are dominated by proprietary tools. Open source tools are less well-represented, although some ‘free tools’ listings do exist, they are still incomplete
  • Many tools have functionality that spans multiple categories. Some are available in proprietary, some are open source and deployed on workstations, servers or SaaS platforms. Tools might be listed in multiple categories, but usually not
  • Tools listings often only provide a link to a vendor web page for the tool and little else. Forums, training, supporting service companies or contractors are not usually listed and cannot be searched
  • Tools cannot be compared with respect to functionality, licensing, platforms or integrations
  • There are no tools usage statistics available; we have to rely on vendor marketing to gauge their popularity
  • Not enough information, too much advertising.

The Tools Knowledge Base (

The Tools Knowledge Base is a free-to-use service providing information on tools, vendors and the consultants and service companies that support them.

  • A searchable directory of tools: Our focus is (broadly) DevOps, SDET, Test and collaboration. Each tool record stores limited data but links to the vendor or developer web page. This basic information and the content of the tools web page are downloaded and indexed nightly by our search engine
  • We don’t replicate tool web pages: We only collect the minimal amount of information that allows us to index the tools information for searching
  • A sophisticated search engine: The Whoosh! search facility can be used to find tools matching your search criteria. It’s not Google but has most of the features that Google search provides
  • A hierarchical tool type/features list: Every tool can be properly profiled and compared (currently 313 features)
  • Resources: Content such as images, videos, scripts or training content or links to other web content can be uploaded and associated with every tool or company
  • Questions & Answers: Every tool and every resource can be discussed. Registered users can post questions and answers; the notification system will send messages to users/owners of that tool
  • 18,664 (as of 17 March 2016) searchable posts: We download the content of blogs from 302 bloggerseach night. These posts are indexed and searchable. We do not store the blog posts, we only index
  • Embeddable Content: We offer a range of APIs allowing conferences, service companies and consultants to access and share our data on their own websites
  • Partnerships: Conferences, trade magazines, websites and domain experts who support us.

What do I do Now?

If you are tools user: we’d like you to register, and identify the tools you use. your tools chain will appear on your profile. If a tool you use is not in TKB, then we invite you to create it. (If you want to embed the tools you use as a list in your website – we have an API for that).

If you are a tools expert or tools service provider: please see above, plus… We are looking for people who are knowledgeable enough to review or possibly edit the features listings for the tools you know well. The features hierarchy will grow and evolve over time – help us to perfect it. We will list you as a service provider on the tools you know best. It’s the least we can do.

If you are a tool vendor: we ask you to search for the tools you offer and check they are in the system and properly described. If your tools exist in the system and you want to manage the information we hold, that’s fine – let us know and we can make you the administrator (after a couple of quick checks). If you want, we can maintain the data on your behalf, for a small fee. Do get in touch. Alternatively, nominate a tools expert (see above) and we’ll invite them to keep your details correct.

If You Own/Contribute to an Open Source Project: We make exactly the same offer as we make to the vendors. you are free to edit the information for your tools in the same way. you might already be represented on a site like GitHub – we’re just offering an extra publicity channel – it might help you reach a broader audience. People looking for tools often start their search with proprietary products but rarely see free tools listed side by side. Now is your chance.

If you are a blogger: Search for one of your recent blog posts and if you find it – your blog is already indexed in the system. If not, you can suggest the blog and register it yourself. Please note we are looking for blogs that cover our scope (DevOps, Testing, Collaboration). Company blogs are acceptable if they focus on these topic areas. Some general technology blogs might be acceptable, at our discretion. Offensive or blatantly commercial posts are not acceptable.

If your blog is indexed, let us know, we will give you credit for it on your profile.

If you maintain your own online tools listing: please get in touch. We believe we already have a more comprehensive list (in our scope) than anyone else. We offer to provide you with ad free, embeddable tool type listings for your existing site. Join us as a partner and tools expert and help us to improve the data in our system to improve the listings on your site. We currently have 183 tool type listings. All are available for free. We’ll give you your own lists and track the usage data and share that with you too. Drop me a line and I can explain what we can do for you.

If you want a tools listing on your own site: Get in touch. We make the same offer as above.

Can I advertise my tool or service on your site? Of course you can. We offer competitive CPM (cost per thousand impressions) rates. Our users are tools-focused so your ads will be seen by the right people. We will of course provide monthly statistics for your perusal.

Where is the Knowledge Base?

Please take a look and let me know what you think.

No Tester is an Island

No tester is an island,
Entire of himself,
Every tester a piece of the team,
A part of the main.
If a case be washed away by the test,
Acceptance is the less.
As well as if a phase were.
As well as if a test of thy friend’s
Or of thine own were:
Any test failure diminishes me,
Because I am involved in testing,
And therefore never send to know for whom the test tolls; 
It tolls for thee. 

Huge apologies to John Donne