A Stage Performance

I did a talk at TestBash Germany last week that sparked lots of positive response, but also some critique. Critique is fair: It was a 30 minute inspirational talk in which I wanted to explain why Immanuel Kant’s work “Critique of Pure Reason” matters to testers. Quite a few people found me afterwards, asked me questions, and commented: Critique. Job well done (I’m padding my own shoulder here).

But, as it sometimes happens in these times of SoMe, a tweet showing one of my slides, followed up by a tweet of another, both included to provoke thinking about the state of software testing, sparked a discussion which has taken a direction I feel I need to relate explicitly to. Click the tweet to read the thread:

I was laughing on stage, and it came from my heart as what I intend to say with the slide in the talk is that all current ways to think about testing are bad, but I have the the answer. An arrogant message? Well, I’m on stage for 30 minutes, honest, but performing the gig. I mean what I say, but I also asked people to judge for themselves (I’m sure they did), and it was in the context of a talk intended to entertain and get people thinking at the same time.
I think criticizing the status quo is central if you propose something new. “If it aint broke, don’t fix it,” and I think there are broken things in testing. I dot it publicly regularly when I blog on social issues in Denmark on DenOffentlige.DK, where it’s welcomed. Danish prime minister Lars Løkke Rasmussen thanked me for critique on one occasion when I complained about a reform he proposed to help young people out of the educational system. I made him think, he said.
So I can’t claim ignorance. I feel I hadn’t thought enough about this. It was a pair of good slides in the context of the conference, the stage and the talk, but definitely bad on twitter just so.
When the whole talk is published in a few weeks, I hope people will go and listen to both context and intention, and perhaps even offer me feedback on the whole thing.
However, the cat is out of the box now, so I need to explain my critique more thoroughly. I criticized the three testing “philosophies” I see out there: ISTQB, Context Driven Testing, and Agile testing. Here we go:

ISTQB

I haven’t seen anything yet from these people, but that doesn’t surprise me. ISTQB is pretty robust to critique. That’s actually my problem with it: It’s a castle. As long as it stays inside the castle, that’s ok, I have no problem with it. Problem is, these people don’t stay in the castle: They dominate the market for software testing.

Context Driven Testing

James Bach cares about critique. I love him for that! But, James, please don’t take it personally. You’re one of the greatest thinkers about testing. I hope you’ll dig your way though Kant’s first Critique as I think you’ll find it enlightening. Kant is a genius with his own pedantic style, but his philosophy is consistent.
With that in place, here’s my answer to the thread above: I have a philosophical (logical) problem with the principles of context driven testing, that – when I read them literally–confuse me about whether testing can critique quality. Here’s the argument:
Testing is a thing that critiques things, including things about things, e.g. quality. Critique is not just critique, it involves judgement. CDT has principles and they can be used to self-evaluate whether I’m context driven, so I can judge my testing, but the principles are so crafted that when read literally, they don’t allow me to critique and judge my testing. Here’s why: The first and second principles says this: “(1) The value of any practice depends on its context. (2) There are good practices in context, but there are no best practices.” The first one exclude the possibility that a practice can be valued before it’s practiced in context. Since testing in context is practice, it means the value of testing can’t be known a priori. The second says there are no best practices, which implies there are no worst ones either. No best or worst, even in theory? Well, if that principle is to be taken literally, then no, there are no best or worst practices, not even in theory. But if there is no best and worst, not even in theory, it means that value is not a thing: Value is, at best, a point, or an empty void, perhaps even everything. The implication of this is that every practice, even type of practice, is of unknown value (everything is a thing, but it’s a thing we cannot know), even when attempted valued in context, that is a posteori. So the snake seems to be eating its own tail here: No testing in CDT can have value. This is paradoxical because testing must have value to critique as things without value have no ontology and only things with ontology can create bodies of critique, i.e. critique with ontology. The reason for this has to do with the philosophical concept of ontology, which things must have in order to critique. This gives the context driven tester a problem: When context driven (according to the principles), I can’t assign a value to my work, and if I can’t do that, my meaningful existence as a working tester ends. Second, it gives my stake holder a problem: If my testing is literally valueless, then the outcomes of my testing must be valueless too as the critique is without body, ontology, value. Valueless things can exist, and quantum physics shows that things happen out of nothing, but things that happen out of nothing cannot represent knowledge as knowledge depends on ontology and value. So, what is that which is something, but not knowledge? I think data is a good word for that, potentially not noise, but still not knowledge, probably not even information. In a provocative tone I’d say data is that which discriminates no testing from context driven testing. Mind blown? I tell you, I was when I realized.
Of course, I’m overly pedantic and attacking a single principle on purely logical grounds, which is in many ways unfair since there’s much more to CDT than it’s first two principles, e.g. great people doing awesome work training important skills to testers with the aim om maximizing the outcomes of testing (my interpretation), thereby actually working to maximize the value of testing, making testing a sustainable activity, and saving the IT-world from the politics of ISTQB), CDT will survive the paradox, I’m sure. But the principles are still around and read, and even though the argument above above is long and annoying (I would probably get lost in it myself, if I was reading it from someone else) it is my experience that people actually read the principles, and that the paradox ‘gets’ them intuitively, even if they can’t express it.

Agile Testing

I actually have only noticed a single reaction to my slides from agilists, but I’d like to share my problem with agile, as it is actually much, much bigger than any of the two above, and doesn’t only extend to testing. My problem with ISTQB is mostly political, whereas that of CDT is philosophical. Agile’s problem is that it doesn’t appreciate testing because it doesn’t fundamentally desire to create value. It’s secret desire is power for the developers. That’s my own diagnosis, you are welcome to disagree, but here’s why: There This time my problem starts with the values, for example the one that values working software over comprehensive documentation. The old rule, “no work is done before the paperwork is done” still applies, but this principle in combination with Agile’s idea of continuous delivery combined with budgetary constraints (resources are always scarce in the real world) turns the principle into a choice between documentation OR software. In a resource constrained reality, the net result is therefore that there’s always more important work to do delivering working software than to document what was done, and there is therefore never good documentation coming out of Agile. This is just one example of why Agile projects are never done, as the other value statements tend to collapse in the same way as soon as they are subjected to resource constraints. The result is developers working long hours, economics out of control, technical debt growth, and lots of wasted work. Agile is on steroids creating legacy, I think. And I’m not done yet: Who really knows quality? At the core of Agile is not quality for customers, but for developers since they are the ones writing the software and that has been forgotten by the ones writing the principles, that contains no model for judging the core values of quality, innovation, collaboration, truth. There’s a fundamentally unfair situation: Developers own the power to make the software, and the users’ only power is reduced to influence in the daily collaboration. Oh yes, there’s potentially also the possibility of rejecting deliverables to production – but then always at the cost of precious and expensive time, delays and hence suffering in production because of still unsolved problems: At the cost of quality. Notice, I’m not looking for a silver bullet to solve this forever, because one such does not exist, but for a fairer relationship between developers and customers. Market mechanisms will not fix this, not even over iterations, retrospectives, and team learning. Why? Well Adam Smith wasn’t wrong about market dynamics, but his ideas have proven themselves insufficient by far. Agile is far from demonstrating the value of self-organizing organizations. I think it is probably taking us further away from just that.

There. Those are my professional opinions about the three. Meanwhile, I work in Agile, performing CDT and traditional testing. Is there a solution? I think so. For now, I work by the principle that testing is by its very intention and nature practical/empirical and rational, rooted in knowledge a priori and drives human experience (knowledge) a posteori. But that’s old news, the Stoics knew that already some 2000 years ago.
And there are great people in Agile and CDT actually trying to make things valuable for those that matter, and I listen to them a lot.
But I think its time look for new models for value than those that are implied in the above testing philosophies. If you disagree, and find there’s no problem, that’s ok with me, but please accept at least an honest attempt looking for a new philosophy in testing by consulting one of history’s greatest and most acknowledged philosophers, and that such work should nok be without a critique of the status quo.

Feedback welcome

I’m living my life by principles like this: We don’t know who we are until we see ourselves in the face of the other.
Even though I had the stage light in my face, I felt connected to the audience. Performing a talk starts in the speaker between the gut, heart and the intellect: Patos, ethos, and logos. But without connection to the audience, the talk may be perfect, but without effect. From my gut feeling and the feedback I received, what I said had effect.
Feel free to use the comments below. I’ll try to be quick approving comments, but please accept my apologies up front that replies may take a little while as I’m facing a busy time right now both privately and professionally.

#Testfest done.