Tag: business value of testing

Categories
Blog posts in English

Core Values in Testing

There’s something about life that you won’t find anywhere else.
– Ole Brunsbjerg, headmaster.

The Copenhagen Context Driven Testing meetups are becoming a tradition thanks to the work of Carsten Feilberg and Agniezka Loza. In June, I chaired a workshop in Ballerup near Copenhagen during one of the meetups. 16 testers shared ideas about values in software testing.
There are four or five basic human values which everyone shares. The good, the beautiful, the true and the just. Freedom relates to these four. We express and rate them differently and they are intrisic to us, subjective, but still shared among humans.
My personal human values shape my actions, words and thoughts, and thus also the words and expressions I use in my daily language. My language can tell you about my values and therefore something about who I am.
Workshop and procastination
In the workshop I chaired in June, I asked the participants to pick picture cards to illustrate thoughts about testing. Then they spoke about the picture and about testing. We shared our words and statements on post-it’s and I collected them.
I was busy at work after the workshop, and the box with the words ended up on my desk. Summer and vacation came, and I procastrinated opening it. One of the last days of vacation, I finally read the words on the post-it’s.
Here are the words:

Knowledge; Information; Curiosity; Exploration; Investigation; Fight :-); Courage; Confidence; Balance; Collaboration; Evolvement; Surprise; Order; Performance; Discovering stuff, that others have not (yet) discovered; beautiness. Usability (easy/better ways of using stuff). Universatility. User experience design; Good (better) end user experience; user needs; user satisfaction; Sustainability. Creativity. Responsibility. Curiosity; Easeing somebody else’s job; Striving; Alertness; Communication; Added communication; added collaboration; information sharing; Building bridges; (Make it) fun; Excellence; Any word / anything; Getting a kick; Covering / exploring; Contradictions / paradoxes; Building trust; Finding (new) ways; Getting to know; Helping; Revealing; Avoiding losses; Whole solutions; Support descisions; Transparancy; Quality; Assessing quality; Avoid scandals; Improvement; Business needs; Filling gaps; People; To spoil illusions (own and others’); Digging for something deeper; Truth; Structure; Growth; Responsibility; Team work; Exploration; Progress; Seeing/finding possibilities; Erkendelse/Erkenntnis/realisation; Business value; Honesty.

DSC_1239
Truth and testing
It’s interesting to note that many of the words above relate to the value ‘truth’. Testing implies couriosity, gives a kick, spoils illusions, happens through exploration etc.
I consider ‘truth’ to be the fundamental core value in testing. Truth as a term is a complex thing, but when we use words that relate to the value ‘truth’, it’s easier to see.
As a tester, I prefer things that are true and don’t accept stories that can’t be verified. I rate things that are more true than other things. For example, I tend to dislike reducing truth to numbers, and prefer a more nuanced understanding of subjects.
I do have beleifs, hyphotesis, and test ideas, but at the end of the day, the ideas only prove their worth when they have been evaluated.
More than truth?
But look again.
Many (most?) of the words deal with things that are not related to ‘truth’: Reponsibility, easening other peoples jobs, evolvement, user experiences, whole solutions, improvement, business value etc.
This reminds us that testers are not just concerned with ‘truth’, i.e. testing, but also value how testing is used and the results that the whole team or company achieves.
What does this tell me as a testing leader?
It tells me that in my leadership, I cannot only focus on testing ideas, spoiling illusions, and telling the truth if I wish to motivate and encourage our teams to work efficiently and independently doing their testing.
I have to consider how the testing contributes to achieving other goals and higher goals.
I have to consider that coorporation with colleagues work well. That the product we somehow help with is something that makes users happy. That there are bottom line results because of our testing. That disasters are prevented.
These things are not just ‘context issues’. They are core to testing leadership.
Word play
I have played with the words on the cards and come up with a mission statement for a hypothetical testing team. The mission statement somehow expresses this.
Feel free to play with the words yourself.

We are testers. We are ready to spoil illusions, both our own and others’. We have courage to do so and generally like to be surprised. So we always dig for something deeper, a deeper understanding, a realization, an ‘erkenntnis’. We get a kick when that happens. Through testing, we seek truth, but we also feel a responsibility to make our testing useful to create user friendly and whole solutions, support growth and improvement, and sustainability. Our testing thus aims to assist the creation of pleasing and aestethic solutions, to serve other peoples needs and hopes, and in general to do good.

PS: The quote from my uncle Ole Brunsbjerg at the top of this article is to remind us that there is more to life than testing. Or anything else. Life is very rich and as humans, we value all of it.

Categories
Blog posts in English

Speaking to Management: Coverage Reporting

Test coverage is important. In this post, I will reflect about communication issues with test coverage.
The word coverage has a different meaning in testing than in daily language. In daily language, it’s referring to something that can be covered and hidden completely, and if you hide under a cover, it will usually mean that we can’t see you. If you put a cover on something, the cover will keep things out.
Test coverage works more like a fishing net. Testing will catch bugs if used properly, but some (small) fish, water, plankton etc. will always pass through. Some nets have holes through which large fish can escape.
What’s so interesting about coverage?
When your manager asks you about test coverage, she probably does so because she seeks confidence that the software works sufficiently well to proceed to the next iteration or phase in the project.
Seeking confidence about something is a good project management principle. After all: If you’re confident about something, you are so because you don’t need to worry about it. Not having to worry about something means that you don’t have to spend your time on it, and project managers always have a gazillion other things that need their attention.
The word is the bug
So if confidence comes out of test coverage, then why is it that it managers often misunderstand us when we talk about coverage?
Well, the word actually means something else in daily language than it does when we use it in testing. So the word causes a communication “bug” when it’s misunderstood or misused.
We need to fix that bug, but how? Should we teach project managers the ”right” meaning of the word? We could send them to a testing conferences, ask them to take a testing course, or give them books to read.
That might work, but it wouldn’t solve the fundamental communication problem. It will move higher up in the organisational hierarchy.
An educated manager will have the same problem, not being able to make her peers and managers understand what ”test coverage” means. After all, not everyone in the organisation can be testing experts!
STOP mentioning coverage
A good rule of thumb in communication is: When your communication is likely to be misinterpreted, don’t communicate.
I, as a tester knows what test coverage means and more importantly what it does not mean, but I cannot expect others to understand it. Thus, if I use the word, I will probably be misunderstood. A simple solution to this is to stop using the word. So I won’t say sentences like: Our testing has covered some functionality.
The thing I can say is: We have carried out these tests and we found that.
This will work well until someone asks you to relate your testing to the business critical functionality: Ok, then then tell me, how much of this important functionality do your tests cover?
Uh oh!
Stay in the Testing Arena – or be careful
American circuses have enormous tents and two, three or even four arenas with different acts happening at the same time. A project is always going on in different arenas as well: For example we might have a product owner arena, a development arena, a test arena, and a business implementation arena.
Some people play in several arenas: I think most testers have at some point in the career made the mistake of telling a developer how to code. Likewise, we can probably all agree that there’s nothing more annoying than a developer telling a tester how to test.
Confidence belongs in the product owner arena, not in testing. This is because testing is about qualifying and identifying business risks, and since confidence does not equal absence of risks, it’s very hard for us to talk about confidence. And coverage.
This doesn’t mean you can’t move to another arena.
You can indeed look at things from the product owners perspective, that’s perfectly ok! Just make sure you know that you are doing it and why you are doing it: You are leaving your testing arena to help your product owner make a decision. Use safe-language, when you do.
Talk facts and feelings
Confidence is fundamentally a feeling, not a measurable artefact. It’s something that you can develop, but it can also be communicated: Look confident, express confidence, talk about the good stuff, and people around you will start feeling confident.
Look in-confident, express worry, talk about problems, and people around you will start feeling worried.
Testers always develop feelings about the product we’re testing, and we can communicate these feelings.
I know two basic strategies in any type of test result communication:

  • Suggest a conclusion first, then tell’m what you’ve done
  • Give them all the dirty details first, then help your manager conclude

Which communication strategy you pick should depend on the context, e.g. your relation with the manager. If everything looks pretty much as-expected (whether that’s good or bad), your manager has trust in you, and you have good knowledge of the business risks, then I wouldn’t worry too much about serving the conclusion first, and then offer details later mostly to make sure you and your manager doesn’t misunderstand each other. And that nobody will later be able to claim that you kept silent about something.
But if something is way off, or your manager doesn’t trust you (or you don’t trust her), peoples lives may be at stake, or you just have no idear what’s happening, then stick to the details – do not conclude. And that, I think, implies not using the term ”testing coverage”.

Categories
Blog posts in English

Antifragility by Testing?

”There are two classes of things […] One class of things that gain from disorder, and one class of things that are harmed by disorder.”

Nassim Nicholas Taleb, author of the best seller ”Black Swan” is out with a new book: ”Antifragile: How to live in a world we don’t understand”. He gave a lecture at the London School of Economics on December 6th 2012 during his book tour. The lecture is available as a podcast here.

”Technology is inherently fragile.”

The words are Nassim Taleb’s , and the statement should not surprise any testers: Testers can find bugs in even the best pieces of computer software: It is only a question of having useful testing heuristics, how much effort we’re using, and about observation skills of course.

”In psychology, people talk about post traumatic disorder. But very few talk about post traumatic growth.”

I am a big fan of Nassim Taleb for his original philosophical thinking and his ability to think and speak clearly about subjects which are very complex and sometimes even counter intuitive.

Taleb has a lot to teach us in testing, and it is very obvious to me that fragility is something that we should start looking for.

”The difference between the cat and the washing machine […] is you need to fix [the washing machine] on every occasion. The organic self-heals.”

Computer systems do not self-heal – they are inherently fragile.

Photo of Nassim Nicolas Taleb giving a lecture
Nassim Nicholas Taleb (photo: Bloomberg)

But let’s step back for a moment, taking a broader look. Let’s look at the systems incorporating the computers and the software: Organizations using information technology to run their business, communities using IT to stay connected, factories with machinery, workers, managers and computers to run them. Can any of these systems be described as antifragile?

”You should never let an error go to waste.”

My question is ”Can testing be applied in such a way that it not only detects fragility, but instead facilitates the development of anti-fragility?”

I believe that the answer is yes. And yes; there are antifragile systems out there incorporating computers and IT.

Please consider a recent test activity you participated in. Now think about the system which was building the product you were testing (my apologies for using manufacturing terms here): The people, the project teams, the organization. Such a system is organized into layers, and while the bottom layer (where the technology is) is usually inherently fragile, some of the higher level layers were perhaps antifragile?

This is where I see the role of testing coming in:

”It’s not about trial and error – it’s about trial and small error.”

In this very statement, Nassim Taleb, in my humble opinion, speaks clearly about what testing is about. The antifragile system for developing products grow stronger when testers find problems, since not only will the system learn from experience; no the antifragile system will prepare itself for things that are worse than what was experienced.

Put in another way: The antifragile software project does not just fix the bugs it encounters. The antifragile software project fundamentally eliminates problems based on knowledge from the small problems testers find.

So my message to project and and program managers is this: Don’t hire testers to find the defects. You should hire great testers to ensure your projects experience many small problems, allowing them to grow stronger and build better products: If your project systems are anti-fragile by structure, leadership and management, not only will bugs found by testers not be found in production: The overall quality of the product will be better!

And that, to me, is where the real business value of testing is!

Thanks to Jesper L. Ottosen for very constructive reviewing and commenting of drafts of this blog post.