Why ISO 29119 standard is bad for software testing

Testers’ Arena: Contest #9 of 12

Problem statement:

Read the links at http://www.huibschoots.nl/wordpress/?page_id=1771 and come up with your blogpost highlighting why ISO 29119 standard is bad for software testing.

Participants:

Bipin Nagpure

Bipin_nagpure@mindtree.com

Organization: Mindtree Ltd.

Why ISO 29119 standard is bad for software testing???

To understand above question we MUST understand first what ISO 29119 is?

Let’s in short understand of it,

What is ISO 29119 Software Testing !!!

ISO 29119 Software Testing is an internationally agreed set of standards for software testing that can be used within any software development life cycle or organisation. By implementing these standards, you will be adopting the only internationally-recognised and agreed standards for software testing, which will provide your organisation with a high-quality approach to testing that can be communicated throughout the world.

Understand key points of it: (Please click on each of the link to see them in details on softwaretestingstandard website)

Currently parts 1-3 are published, and parts 4-5 are in draft.

It replaces a number of existing software testing standards as follows:

So now let’s check following point which says,

What is Wrong With ISO 29119? by James Christie blog

Well, it embodies a dated, flawed and discredited approach to testing. It requires a commitment to heavy, advanced documentation. In practice, this documentation effort is largely wasted and serves as a distraction from useful preparation for testing.

Such an approach blithely ignores developments in both testing and management thinking over the last couple of decades. ISO 29119 attempts to update a mid-20th century worldview by smothering it in a veneer of 21st century terminology. It pays lip service to iteration, context and Agile, but the beast beneath is unchanged.

Couple of attributes of valid ISO standards that ISO 29119 violates is that an ISO standard is not supposed to:

Distort the market, or 2. Have adverse effects on fair competition.

By misrepresenting the challenges of software testing, and reducing it to little more than a glossary of terminology and prescriptive document definitions it certainly does its best to distort the market. Will it have adverse effects on fair competition? It’s too early to say definitively, but as many senior test experts around the world can testify: It usually doesn’t take very long until “optional” standards get accompanied by certification schemes which in turn quickly finds their way into frame agreements and business contract templates as “must haves”. And before you know it, you have to have an ISO 29119 certification to work with regulated software or to get a government contract or job as tester.

Then what about ISO 29119? By James Bach’s Blog

The ISO organization claims to have a new standard for software testing. But ISO 29119 is not a standard for testing. It cannot be a standard for testing.

A standard for testing would have to reflect the values and practices of the world community of testers. Yet, the concerns of the Context-Driven School of thought, which has been in development for at least 15 years have been ignored and our values shredded by this so-called standard and the process used to create it. They have done this by excluding us. There are two organizations explicitly devoted to Context-Driven values (AST and ISST) and our community holds several major conferences a year. Members of our community speak at all the major practitioners conferences, and our ideas are widely cited. Some of the most famous testers in the the world, including me, are Context-Driven testers. We exist, and together with the Agilists, we are the source of nearly every new idea in testing in the last decade.

The reason they have excluded us is that they know we won’t agree to any simplistic standard based on templates or simple formulae. We know those things look pretty but they don’t help. If ISO doesn’t exclude us, they worry they will never finish. They know we will challenge their evidence, and even their ethics and basic competence. This is why I say the craft is not ready for standards. It will be years before all the recognized experts in testing can come together and agree on anything substantial.

The people running the ISO effort know exactly who we are. I personally have had multiple public debates with Stuart Reid, on stage. He cannot pretend we don’t exist. He cannot pretend we are some sort of lunatic fringe. Tens of thousands of testers have watched my video lectures or bought my books. This is not a case where ISO can simply declare us to be outsiders.

The Burden of Proof

The Context-Driven community stands for excellence in testing. This is why we must reject this depraved attempt by ISO to grab power and assert control over our craft. Our craft is still an open marketplace of ideas, and it is full of strong debates. We must protect that marketplace and allow it to evolve. I want the fair chance to put my competitors out of business (or get them to change their business) with the high quality of my work. Context-Driven testing has been growing in strength and numbers over the years. Whereas this ISO effort appears to be a job protection program for people who can’t stomach debate. They can’t win the debate so they want to remake the rules.

The burden of proof is not on me or any of us to show that the standard is wrong, nor is it our job to make it right. The burden is on those who claim that the craft can be standardized to study the craft and recognize and resolve the deep differences among us. Failing that, there can be no ethical or rational basis for standardization.

This blog post puts me on record as opposing the ISO 29119 standard. Together with my colleagues, we constitute a determined and sustained and principled opposition.

A Flawed Approach by Iain McCowatt – Stop 29119

Standards in manufacturing make sense: the variability between two different widgets of the same type should be minimal, so acting in the same way each time a widget is produced is desirable. This does not apply to services, where demand is highly variable, or indeed in software, where every instance of demand is unique.

Attempting to act in a standardized manner in the face of variable demand is an act of insanity: it’s akin to being asked to solve a number of different problems yet merrily reciting the same answer over and over. Sometimes you’ll be right, sometimes wrong, sometimes you’ll score a partial hit. In this way, applying the processes and techniques of ISO 29119 will result in effort being expended on activities that do nothing to aid the cause of testing.

And in testing, that’s a major problem. When we test, we do so with a purpose: to discover and share information related to the quality. Any activity, any effort that doesn’t contribute to doing so is waste. As “complete” testing is impossible, all testing is a sample. Any such waste results in a reduction in sample size, it equates to opportunity cost: an opportunity lost to perform certain tests. For a project constrained by quality, this translates into increased time and cost. For a project constrained by time or money, this translates into a reduction in the information available to stakeholders, and a corresponding increase in risks to quality.

The 29119 crowd might tell you that the new standard takes this into account, that it encourages you to tailor your application of the standard to each project, that it is sufficiently comprehensive that you need only select the processes and techniques that apply. This is the Swiss Army Knife fallacy: if you have one you’ll never need another tool. One of the problems with a Swiss Army knife is that it’s not much use if you need a pneumatic drill, or an ocean liner.

Some experiences which people faced

For several years when I was working as a test manager, I interviewed and hired numerous people. Once during an interview, a candidate pulled out a certificate and held it up for me, telling me he should be hired instantly due to his certification. I asked that he put the certificate away and talk with me. After discussion, it was my assessment that the candidate had little working understanding of how to test a product. Perhaps he had memorized material or had someone else take the exam for him – whatever had been the case; there was no evidence that the candidate would be able to perform well. As this event took place some years ago, I will refrain from trying to recall more specifics.
Given this same scenario, if I did not have a testing background but needed to hire someone, perhaps I would have been convinced based on the certificate that the candidate was equipped for the job.

This is where certification is dangerous – at first glance, it sounds good, it seems like it should provide some assurance that a person is qualified. But as I have found, having a certificate does not provide evidence of a person’s knowledge or skill.

In my work, my own personal experience as an employee and consultant, I have not worked at a company requiring certification. I have heard from several colleagues in Europe that companies do require certification and I find this concerning. Further I have heard colleagues say they have paid the money, taken the exams and become certified just so they can find work or keep employment even though they do not believe the information gathered from certification has helped them.

Key points for Why ISO 29119 standard is bad

The knowledge needed for a tester varies based on what type of software a tester is working with.
Materials within the certification are NOT developed or continue to evolve in a timely enough manner to address the every changing software/hardware/technology world in which software testers work.

Collected data from

http://www.softwaretestingstandard.org/

http://www.houseoftest.se

http://www.ministryoftesting.com

http://qualityremarks.com/the-petition-to-stop-iso-29119/

http://karennicolejohnson.com/

http://www.satisfice.com/blog/archives/1464

http://www.developsense.com/blog/2014/08/rising-against-the-rent-seekers/

– End of the Blog –