Some test managers and test consultants are very busy pointing out the right processes, organisational structures and methods to use in software testing.
But no methods, processes and structures can assure great testing. Great testing is created by people.
This quote by Neil Armstrong, which I came across a couple of years ago, is worth remembering whenever we lead people in testing:
“The way […] that made [the Apollo project] different from other sectors of the government to which some people are sometimes properly critical is that this was a project in which everybody involved was, (1) interested, (2) dedicated, and, (3) fascinated by the job they were doing. And whenever you have those ingredients, whether it be government or private industry or a retail store, you’re going to win.”
Some drinking glasses are very fragile, but fragility does not have to be a bad thing. I think most people will prefer a fragile but thin and beautiful champagne glass for over a heavy, robust one. Thin glass just suits champagne better. A fat, fruity Barolo will go better in a thicker, more robust glass.
But both the thick, more robust glass, and the thin fragile glass share a property: There is no way you can make them stronger once they have been cast or blown. In fact, since glass is an amorphous solid material, it tends to become more fragile over time. Vibrations of the molecules in the material will eventually distort the structure, making it weaker.
The diagram below illustrate how a glass subjected to daily use over a period of time can eventually break when force it is subjected to exceeds the ”breakting threshold” of the weakest point in the glass. The threshold is the maximum force the glass can take before it breaks, and in the case with the fragile champagne glass, the threshold decreases over time: The glass is becoming more fragile with age.
Now, let’s imagine that science discovers a way to strengthen molecular bindings in amorphous materials by vibration. Lets say, we can somehow convert the kenetic energy in vibrations of molecules into potential energy in the intramolecular bindings. The effect would be that we could make a type of glass, which gets stronger with use.
The breaking threshold will now increase over time: The more the glass is bumped around during normal use, the stronger it will get. The diagram will look like this:
Note that this does not mean that the glass has become unbreakable. The only thing that has changed is that the curve illustrating the strength of the glass has changed from going downwards to going upwards and the forces the glass is subjected to during daily use no longer crosses that curve.
The term ‘antifragility’ is new, invented by Nassim Taleb and first used in a publication in the appendix to the second edition of his book The Black Swan.
Taleb says antifragility is a property of the oganic and the complex. Human bones are antifragile: Children’s bones are in fact quite fragile and grow stronger with use. But if you sit down (or live in weightlessness), they become fragile. That does not mean that bones can’t break – of course they can break. But like the imaginary antifragile glass above, bones of a well trained person are just able to take a lot more “beating” before they do so.
Dead things like champagne glasses, washing machines, computers and their software are inherently fragile and unless someone incorporates something which can implement the feedback of use into strength. Similarly, adding complex, but still ordinary software to a system cannot make it antifragile, nor will any special kinds of tools used in the development of it make neither the project, nor the product, antifragile. Thanks to Jesper L. Ottosen for very reviewing this blog post.
”There are two classes of things […] One class of things that gain from disorder, and one class of things that are harmed by disorder.”
Nassim Nicholas Taleb, author of the best seller ”Black Swan” is out with a new book: ”Antifragile: How to live in a world we don’t understand”. He gave a lecture at the London School of Economics on December 6th 2012 during his book tour. The lecture is available as a podcast here.
”Technology is inherently fragile.”
The words are Nassim Taleb’s , and the statement should not surprise any testers: Testers can find bugs in even the best pieces of computer software: It is only a question of having useful testing heuristics, how much effort we’re using, and about observation skills of course.
”In psychology, people talk about post traumatic disorder. But very few talk about post traumatic growth.”
I am a big fan of Nassim Taleb for his original philosophical thinking and his ability to think and speak clearly about subjects which are very complex and sometimes even counter intuitive.
Taleb has a lot to teach us in testing, and it is very obvious to me that fragility is something that we should start looking for.
”The difference between the cat and the washing machine […] is you need to fix [the washing machine] on every occasion. The organic self-heals.”
Computer systems do not self-heal – they are inherently fragile.
But let’s step back for a moment, taking a broader look. Let’s look at the systems incorporating the computers and the software: Organizations using information technology to run their business, communities using IT to stay connected, factories with machinery, workers, managers and computers to run them. Can any of these systems be described as antifragile?
”You should never let an error go to waste.”
My question is ”Can testing be applied in such a way that it not only detects fragility, but instead facilitates the development of anti-fragility?”
I believe that the answer is yes. And yes; there are antifragile systems out there incorporating computers and IT.
Please consider a recent test activity you participated in. Now think about the system which was building the product you were testing (my apologies for using manufacturingterms here): The people, the project teams, the organization. Such a system is organized into layers, and while the bottom layer (where the technology is) is usually inherently fragile, some of the higher level layers wereperhaps antifragile?
This is where I see the role of testing coming in:
”It’s not about trial and error – it’s about trial and small error.”
In this very statement, Nassim Taleb, in my humble opinion, speaks clearly about what testing is about. The antifragilesystem for developing products grow stronger when testers find problems, since not only will the system learn from experience; no the antifragile system will prepare itself for things that are worse than what was experienced.
Put in another way: The antifragile software project does not just fix the bugs it encounters. The antifragile software project fundamentally eliminates problems based on knowledge from the small problems testers find.
So my message to project and and program managers is this: Don’t hire testers to find the defects. You should hire great testers to ensure your projects experience many small problems, allowing them to grow stronger and build better products: If your project systems are anti-fragile by structure, leadership and management, not only will bugs found by testers not be found in production: The overall quality of the product will be better!
And that, to me, is where the real business value of testing is!
Thanks to Jesper L. Ottosen for very constructive reviewing and commenting of drafts of this blog post.