Category: Blog posts in English

Categories
Blog posts in English

Core Values in Testing

There’s something about life that you won’t find anywhere else.
– Ole Brunsbjerg, headmaster.

The Copenhagen Context Driven Testing meetups are becoming a tradition thanks to the work of Carsten Feilberg and Agniezka Loza. In June, I chaired a workshop in Ballerup near Copenhagen during one of the meetups. 16 testers shared ideas about values in software testing.
There are four or five basic human values which everyone shares. The good, the beautiful, the true and the just. Freedom relates to these four. We express and rate them differently and they are intrisic to us, subjective, but still shared among humans.
My personal human values shape my actions, words and thoughts, and thus also the words and expressions I use in my daily language. My language can tell you about my values and therefore something about who I am.
Workshop and procastination
In the workshop I chaired in June, I asked the participants to pick picture cards to illustrate thoughts about testing. Then they spoke about the picture and about testing. We shared our words and statements on post-it’s and I collected them.
I was busy at work after the workshop, and the box with the words ended up on my desk. Summer and vacation came, and I procastrinated opening it. One of the last days of vacation, I finally read the words on the post-it’s.
Here are the words:

Knowledge; Information; Curiosity; Exploration; Investigation; Fight :-); Courage; Confidence; Balance; Collaboration; Evolvement; Surprise; Order; Performance; Discovering stuff, that others have not (yet) discovered; beautiness. Usability (easy/better ways of using stuff). Universatility. User experience design; Good (better) end user experience; user needs; user satisfaction; Sustainability. Creativity. Responsibility. Curiosity; Easeing somebody else’s job; Striving; Alertness; Communication; Added communication; added collaboration; information sharing; Building bridges; (Make it) fun; Excellence; Any word / anything; Getting a kick; Covering / exploring; Contradictions / paradoxes; Building trust; Finding (new) ways; Getting to know; Helping; Revealing; Avoiding losses; Whole solutions; Support descisions; Transparancy; Quality; Assessing quality; Avoid scandals; Improvement; Business needs; Filling gaps; People; To spoil illusions (own and others’); Digging for something deeper; Truth; Structure; Growth; Responsibility; Team work; Exploration; Progress; Seeing/finding possibilities; Erkendelse/Erkenntnis/realisation; Business value; Honesty.

DSC_1239
Truth and testing
It’s interesting to note that many of the words above relate to the value ‘truth’. Testing implies couriosity, gives a kick, spoils illusions, happens through exploration etc.
I consider ‘truth’ to be the fundamental core value in testing. Truth as a term is a complex thing, but when we use words that relate to the value ‘truth’, it’s easier to see.
As a tester, I prefer things that are true and don’t accept stories that can’t be verified. I rate things that are more true than other things. For example, I tend to dislike reducing truth to numbers, and prefer a more nuanced understanding of subjects.
I do have beleifs, hyphotesis, and test ideas, but at the end of the day, the ideas only prove their worth when they have been evaluated.
More than truth?
But look again.
Many (most?) of the words deal with things that are not related to ‘truth’: Reponsibility, easening other peoples jobs, evolvement, user experiences, whole solutions, improvement, business value etc.
This reminds us that testers are not just concerned with ‘truth’, i.e. testing, but also value how testing is used and the results that the whole team or company achieves.
What does this tell me as a testing leader?
It tells me that in my leadership, I cannot only focus on testing ideas, spoiling illusions, and telling the truth if I wish to motivate and encourage our teams to work efficiently and independently doing their testing.
I have to consider how the testing contributes to achieving other goals and higher goals.
I have to consider that coorporation with colleagues work well. That the product we somehow help with is something that makes users happy. That there are bottom line results because of our testing. That disasters are prevented.
These things are not just ‘context issues’. They are core to testing leadership.
Word play
I have played with the words on the cards and come up with a mission statement for a hypothetical testing team. The mission statement somehow expresses this.
Feel free to play with the words yourself.

We are testers. We are ready to spoil illusions, both our own and others’. We have courage to do so and generally like to be surprised. So we always dig for something deeper, a deeper understanding, a realization, an ‘erkenntnis’. We get a kick when that happens. Through testing, we seek truth, but we also feel a responsibility to make our testing useful to create user friendly and whole solutions, support growth and improvement, and sustainability. Our testing thus aims to assist the creation of pleasing and aestethic solutions, to serve other peoples needs and hopes, and in general to do good.

PS: The quote from my uncle Ole Brunsbjerg at the top of this article is to remind us that there is more to life than testing. Or anything else. Life is very rich and as humans, we value all of it.

Categories
Blog posts in English

Are you playing the Russian roulette? Learning from failure

I think most (if not all?) testers have witnessed situations like this: A new feature of the system put into production, only to crash weeks, days or just hours later.
”Why didn’t anybody think of that?!”
Truth is, quite often, somebody did actually think about the problem, but the issue was not realised, communicated or accepted.
Below is the story about the space shuttle Challenger accident in 1986.
Disaster…
Twentynine years ago, space shuttle Challenger exploded seven minutes into the flight killing the seven astronauts aboard.
Theoretical physicist Richard Feynman was a member of the accident commision. During the hearings he commented that the whole decision making in the shuttle project was ”a kind of Russian roulette”.
The analogy is striking. Russian roulette is only played by someone willing to take the risk to die.
I don’t know anyone who deliberately want to play the Russion roulette, so why did they play that game?
Feynman explains: [The Shuttle] flies [with O-ring erosion] and nothing happens. Then it is suggested, therefore, that the risk is no longer so high for the next flights. We can lower our standards a little bit because we got away with it last time…. You got away with it but it shouldn’t be done over and over again like that.
The problem that caused the explosion was traced down to leaking seals in one of the booster rockets. On this particular launch ambient temperatures were lower than usual and for that reason the seals all failed. The failed seals allowed very hot exhaust gasses to leak out of the rocket combustion chamber, and eventually, these hot gasses ignigted the many thusand litres of higly explosive rocket fuel.
Challenger blew up in a split second. The seven astronauts probably didn’t realise they were dying before their bodies were torn in pieces.
It was a horrible tragedy.
Chapter 6 of the official investigation report is titled: ”An accident rooted in history.”
The accident was made possible because of consistent misjudgements and systematically ignored issues, poor post flight investigations, and ignored technical reports. The accident was caused because three seals failed on this particular launch, but the problem was known and the failure was made possible because it was systematically ignored.
The tester’s fundamental responsibilites
As a tester, I have three fundamental responsibilities:

  1. Perfom the best possible testing in the context
  2. Do the best possible evaluation of what I’ve found and learnt during testing.  Identify and qualify bugs and product risks.
  3. Do my best to communicate and advocate these bugs and product risks in the organisation.

The Challenger accident was not caused by a single individual who failed detecting or reporting a problem.
The accident was made possible by systemic factors, i.e. factors outside the control of any individual in the programme. Eventually, everyone fell into the trap of relying on what seemed to be “good experience”. The facts should have been taken seriously.
A root cause analysis should never only identify individual and concrete factors, but also systemic factors which enabled the problem to survive into production.
Chapter 6 of the Challenger report reminds me that, when something goes wrong in production, performing a root cause analysis is a bigger task than just finding out the chain of events that lead to problem.
Many thanks to Chi Lieu @SomnaRev for taking time to comment early drafts of this post.

Photo of the space shuttle Challenger accident Jan. 28, 1986. Photo credit: NASA
Photo of the space shuttle Challenger accident Jan. 28, 1986. Photo credit: NASA

Categories
Blog posts in English

The Context is Copenhagen #CPHContext

#CPHContext 2015 opened today at the Tivoli Hotel & Congress Center. I couldn’t make it today, but judging from tweets, it has been good. I’ll be there tomorrow morning at 8.00 for the main day of the conference.
I’m really looking forward to it!
Frankly I’ve had very little time to study and prepare myself for the programme, but I know I’ll be meeting some of the world’s greatest testers there. I expect to listen, share and learn.
Kudos to Morten from PrettyGoodTesting for setting up this conference – great job done!
PS: I’ll bring my camera, so expect photos on my flickr photo stream after the event.

Categories
Blog posts in English

Let's Test Oz changed me

It’s Friday afternoon, September 19th 2014 and the Boeing 747 from Thai is full to the limit with children, teenagers, parents and businesspeople.
I am sitting in a middle seat in the window row on the right side of the airplane just in front of the wing. I have my Bose noise cancelling headphones on and I’m thinking.
I am leaving Sydney, flying home from the first Let’s Test Oz.
The Jumbo accelerates along the runway, then takes off. With engines whining, babies crying, people chatting and teenagers cheering, we head for the skies. The sounds are damped inside my Bose’s.
I’m alone with my thoughts, but not alone in the World.
I catch a glimpse of the pacific and the Sydney skyline. Then, the plane turns in over the Australian continent.
Australia and Let’s Test Oz has changed me.

Picture form Let's Test Oz workshop with Fiona Charles on test leadership.
Picture form Let’s Test Oz workshop with Fiona Charles on test leadership.

In democracy, debate changes politics

I participated in a political workshop a few days ago. It was organised by the Danish minister for social affairs, who is seeking input to his political campaign. We’re facing a parlamential election in 2015 and the danish democracy is waking up. I’m not a candidate or even a member of a political party, but I’m going to work hard to put social politics on the agenda.

Participating in the workshop was interesting. I shared some of my knowledge, learnt a bit and even got a few new contacts.

Let’s Test Oz was like that, except it was three days of sharing, learning and connecting about my profession: Software testing.

Innovation in testing

I think most people in the Context Driven Testing communicty percieve themselves as innovators. But not everything going on in the projects, companies and organisations we’re working in, is innovative. A lot is standardized, dull and repetitive work controled by machine bureaucracies.

Some testing is even political.

Politics is about power, and knowledge is power.

Testing, being a knowledge producing performance, is therefore a political instrument. Testing is more than that of course, but it’s always potentially a political instrument, and sometimes it’s even used as one.

Politics is applied psychology, and whenever there are humans, there is politics.

In public politics, the political processes are more or less visible and open. In projects, politics is often hidden or kept away from testers and tech people.

I’d like to see more testers (and tech people) getting into the political games. We shouldn’t just trust managers who know little about testing to control us. The game is just too important to miss.

That was, more or less, the message of my presentation at Let’s Test. James Bach, who was presenting at the same time as I was, saved me from having a large audience, but the room was well filled and we had a good discussion afterwards. I shared and learnt.

I’m in love with democracy and I have always and will always vote. Yet, voting is just the smallest part of our political democracy: It’s the debates that really change the world.

And to me, the most fantastic thing about the Context Driven Testing community is the debates we’re having. Through these debates, we’re changing ourselves, each other, and the world. To the better.

Leaving home for a conference

My family depends on me. That’s the short story. The slightly longer story is that I have four children and a wife, and neither of them fit well in our society. We struggle with a daily fear of stress, anxiety, depression, paranoia and even psycosis forcing it’s way back into our lives.
The Danish welfare system supports us, but I’m indispensable to my family. I’m not that stress sensitive.
Before I went to Sydney, I had decided not to feel guilty that I was leaving them for a week. I knew that, my wife had deliberately not thought about MH 17 having been shot down over the Ukraine a few months earlier carrying people going to a conference in Austrailay, and while I was away, she had also deliberately ignored the terror threats in Sydney.
So had I.
I enjoyed myself enormously at Let’s Test Oz and in Sydney.
The problems in our family are symptoms of autism but they’re also symptoms of society’s reaction to the collective anxiety over the post cold-war world:
Everything is possible if you dream it, yet diffuse threats surround us everywhere.
Fears and anxiety
I was a teenager in the 1980’s and back then we knew that the civilised world would cease to exist if war broke out. We were living with the fear.
Compared to anxiety, fear is easy: You can talk about it.
Fear and conflict unites us against something or someone.
Anxiety is a demon that slowly sucks energy out of you. Anxiety seperates people from each other. Collective anxiety kills humanity.
No doubt, there’s anxiety in Austrailia, but there’s a human side to the culture, which I sensed very strongly. I wrote a long essay about it in Danish which has been very well received. Returning home, I have found hope that Denmark could be changing to the better: A less anxious and more humane culture.

Struggeles for testers

However, the problems testers in Austrailia and New Zealand are struggeling with seem to be the same as here. Managers require enless rows of silly estimates, defects are counted, information is kept away from people needing it, and software quality is something everyone is talking about, yet few are doing anything seriously about it.
It’s not surprising that workers in our industry suffer. Some are working in hell-machines, and while we can have a civilised discussion about it over a steak and a sip of quality Austrailian redwine, or even a row of beers, many in the instry still need to calm themselves on a daily basis with hard exercise, alcohol or just lots of coffee. Or medicine.

The Context Driven Testing community is trying to change all this.

We debate. And we are becoming political performers.

Thank you

Let’s Test Oz had a track of ”testing influencing management”, which I shared my thoughts on, but also gave me lots of inspiration. The whole conference was a blast!

I’m very grateful that the programme commitee accepted my proposal and challenged me with the invitation to Sydney.

The trip around the globe changed me and gave me a deeper understanding of our testing community and humanity as such.

Thank you Let’s Test Oz and everyone there!

Sydney Skyline
Sydney Skyline
Categories
Blog posts in English

Welcome to Oz

It’s fun to arrive in a new country and get a sense of the culture. This is my first time in Austrailia.
”What do you want, mate?”, said the guy in a kebab bar in Sydney. And he looked me directly in my eyes. That blew a fuse in my brain!
If someone in Denmark looks me in my eyes and say: ”What do you want?”, I know I’m in trouble and should get away quick. But I was in Sydney and the kebab guy was just a helpful Australian. He made me a great kebab with fries, which was just what I needed for my jetlagged body.
DSC_0258
Australians are wonderfully helpful… and very direct. A woman asked me if I needed help with my bag when I was walking down stairs. A guy approached me to offer directions for me. ”Thank you”, ”No worries, mate!”
I’m down under, what on earth should I worry about?
I’m here to attend and speak at Let’s Test Oz 2014 taking place in Blue Mountains outside Sydney. What a setting! What a conference! I’ll have to come back to that in a blog post after the conference. For now, I’ll just share my immediate impression of Australian culture.
DSC_0294
Words make a difference. Americans love words of latin origin, I think it makes them feel important. They have ”view points” in the landscape. Aussies call things what they are: Lookouts.
Europeans have ”colleagues”. It makes us feel smart. Aussies are ”workmates”.
“Mate” is a funny word. In the animal world, mates are sexual partners. Dictionary.com lists ”partner in marriage” as the first of seven definitions of “mate”. The word is of German origin, coming from ”gemate” which just means someone eating on the same table.
A porter in Australia will say ”After you, mate” and look you in the eyes, whereas a porter in the UK will say ”after you, sir” and look down.
Walking down to the Opera House on Sunday morning, I saw the sign in the photo below. Note that the someone changed the text, but even if it hadn’t, I’ve never before seen a public sign telling people what kind of language to use:
”The following is prohibited… Use of obscene or indecent language… Penalties apply”
What if someone takes a megafone and starts shouting obscene and indecent language (it’s ok to use your imagination here) from the coast? Would that be ok? I should try, shouldn’t I? After all, I am a tester.
DSC_0261 (1)
In Denmark, we would never put up such a sign. We’d just silently push people off the wharf and leave them to drown in the water if they don’t talk nicely and behave according to our unwritten rules. Yes, we might be the happiest people in the world, but that’s only because we’re ready to exclude anyone who isn’t.
Is there a flipside? I haven’t seen it yet, but I’ve seen and heard enough to notice that testers have the same problems in Australia as in the rest of the world. Also, I’ve been told that organisations are strictly hierachical, according to colonial tradition. Coming from a culture in which organisations are flat and everyone usually has very direct access to managers on all levels, and where colleagues appraise each other for speaking against the manager, that always surprise me.
There may be more to it, however. I’m not sure.
Enough for now. Enjoy Let’s Test Oz if you’re here! I know I do.

Categories
Blog posts in English

Do it right: A value in Context Driven Testing

The problem with me is that I’m really bad at following instructions. When people tell me to do something in a certain way, I do it differently.
It’s a problem when I cook, because I‘m not particularly good at cooking. So I have to follow recipies, and I often mess it up slightly. I’m improving, learning strategies to remember, but this is a fundamental personality trait for me.
And not one I’m sorry about. Because it’s not a problem when I test!
I always wanted to be a great tester.
I tend to become really annoyed with myself when a bug turns up in production in something I have tested. ”Why did I miss that?!” I feel guilty. You probably recognise the feeling.
The feeling of guilt is ok. The fact that we can feel guilt proves that we have consciousness, empathy and do the best we can. People who don’t care, don’t feel guilt.
But in testing, finding every bug is fundamentally impossible, so we have to get over it and keep testing. Keep exploring!
Even before I learnt about Context Driven Testing, I knew that great testing could never be about following test scripts and instructions. I noticed that I got bored and lost attention when I executed the same test scripts over and over again, but I also noticed that I missed more bugs when I only followed the instructions.
So I stopped following the instructions. This gave me an explanation problem, however: “Uhh, well I didn’t do quite what I was asked to do…. but hey you know, I found these interesting bugs that I can show you!”

Can you hear it? That won’t impress old-school project managers with spreadsheets to check and plans to follow.

Context Driven Testing has helped me greatly in becoming a better tester. The thing is that CDT teaches me to do a great testing job without instructing me exactly what to do. Instead, the community shares a lot of heuristics I can use to help me do a great testing job, and through thinking-training and inspiration from others, it’ll help me develop my capacity to do great testing in whatever contexts in which I’m working.
It may be a bit difficult to grasp at first. A little worrying, perhaps.. But CDT is a really, really powerful testing approach.

And CDT has helped me explain what I do to my project managers. Even the old-school types!

Social services and testing

A few days ago, I read an article about quality in social services in which the following statement from the vice president of the Danish social workers union caught my attention: ”It’s about doing it right, not about doing the right things.” He was speaking of how the authorities try to help vulnerable children and their families.
The statement resonated with me, and a bit later it occurred to me that it even sums up what CDT is about:
Context Driven Testing is about doing it right, not about doing all the right things.
Note that I’ve added the word ‘all’ here.
There’s more to CDT of course, but this is the core of CDT – to me. Some readers may lift their eyebrows over the ”doing it right”-thing: ”Does he mean that there is ONE right way to do it?” Read on…
The past 10 years, I’ve worked in contexts where CDT is not really the thing we do, and if I was to be as context driven as managers in my context had asked me to be, I would not really have been context driven. You get the picture, I’m sure.
But with CDT in my briefcase, I can work to make changes and improving things in the specific context in which I’m working. As a consultant and contractor, I’m usually hired to fix a problem, not to revolutionize things, and I’m expected to do a “good job” fixing it.
”Doing it right” is of course about doing a good job, and ”doing a good job” should really not be taken lightly. CDT helps me do a good job, even when I’m not working in contexts that actively supports CDT. That’s because CDT is flexible: It empahsizes that testing should never be driven by manuals, standards or instructions, but by the actual context in which it takes place, and that’s actually quite difficult to disagree with, even for old school project managers!

Further, if my context (i.e. project manager) ask me to do something, I do it – even if it’s in a standard. Sometimes there’s a good reason to use a standard.

Context driven testing is not defined by any methods or even by a certain set of heuristics. Nor is it defined by a book, standard or manual. Neither is there any formal education, certification or ”club” that you have to be a member of in order to call yourself Context Driven.
Instead, Context Driven Testing is about committing oneself to some core values, and to me the most important value in CDT is contained in the sentence:
It’s about doing it right, not about doing all the right things.

Why would anyone wan’t anything else?

(Thanks to @jlottossen for reviewing this post and for suggesting several changes to the original version.)

Categories
Blog posts in English

"A demain!" – story about dealing with a sales-focused vendor

Have you worked with a service vendor which consistently did not meet dealines, yet you still kept buying services from them?
This is a story about a car manufacturers’ service organisation being geared towards sales – not service. I’m sharing it here because it has analogies to what’s happening in the software industry. So I hope you’ll stay with me:
My Renault Espace broke down on a vacation in France. It’s a great car, usually very reliable, but like anything else, it has its weak points. The gearbox is one of them: I’ve seen numerous reports on failing gearboxes. Mine did 277,000 km before failing. Not bad, considering…
I delivered the car to a large Renault operation in the city of Frejus on the Côte d’Azur. I was well received, there was even a girl speaking English. I felt comfortable, and got a chance to look at the new cars.
Repairing an automatic gearbox is a specialist task, but replacement gearboxes are available. I started looking at the various options for repair – and for getting the family home to Denmark before our vacation ended.
So I asked the manager at the garage two questions:

  • How much will a repair be?
  • When will the car be ready?

Now, I’ve done a bit of work on my cars over the years, and I know that unplanned things can happen, so I was happy that he gave me a conservative time estimate that would allow for delays in the process. I was most worried whether a new gearbox was in store somewhere in the Renault network, but there was one in Paris, so I accepted.
And with the replacement gearbox arriving from Paris just a few days later, I was happey. When I checked with the garage, they told me they’d start working the same day, and I could have the car two days later. That would be one day earlier than promised.
”Sounds good, we’re ahead of plan,” I thought, and since buffers are always good to have, and I felt I could accept the optimistic estimate. I still trusted them.
But I was in for a surprise when I called two days later to ask about pick up. The answer was ”a demain!” – tomorrow. Any problems?, I asked. No, no problems, they said, ”a demain”.
I went to see the car later the same day, and the picture here shows what found: Note that the old gearbox is still in the car. The mechanic had obviously not started working on my car the day they said he would do it.
IMG_8259-3-SMALL
”C’est possible,” they claimed next day when the mechanic was still busy reassembling the car. I didn’t trust that, as the car obviously needed more work than was available on that day only. A sound test drive, for example!
As I’m writing this, I’m waiting for my car, sitting next to the service counter, where cars are registered for repairs. There’s a large poster showing the people working in the garage – and then there are all the new cars here. I like the electric vehicles Renault is offering, and everyone here is smiling and polite, even the service people. I feel comfortable here.
Renault’s business model is 100% sales oriented: They want me to buy services, buy a new or newer car instead. They smile and tell me all sorts of apparantly good reasons why the repair was delayed – they have even apologized to my family.
But there’s one thing they haven’t told me yet: They prioritized someone else’s car over mine, and they didn’t start working on it until there was no slack in the plan anymore.
This could be an example of french ”laissez faire” attitude, but I don’t think so. I’m not at all worried about the quality of the repair itself, as I saw the mechanic several times while he was working on the car. I know how Renault train their mechanics, and it was obvious that he was doing a really proper job.
No, It’s the planning that sucks. they didn’t know when they’d be done, so they didn’t call me to let me know the plan was in jeopardy. They just hoped. Even today they said: ”Dix heures”, and it’s now 09:59.
The interesting thig is that this is supporting their sales! It’s obvious that Renault has tuned even their flawed, but kind service organisation towards new sales.
How is YOUR vendor tuned?
PS: I’ve got the car now, and it’s as great as ever. Ready for at least another 150,000 km!

Categories
Blog posts in English

Speaking to Management: Coverage Reporting

Test coverage is important. In this post, I will reflect about communication issues with test coverage.
The word coverage has a different meaning in testing than in daily language. In daily language, it’s referring to something that can be covered and hidden completely, and if you hide under a cover, it will usually mean that we can’t see you. If you put a cover on something, the cover will keep things out.
Test coverage works more like a fishing net. Testing will catch bugs if used properly, but some (small) fish, water, plankton etc. will always pass through. Some nets have holes through which large fish can escape.
What’s so interesting about coverage?
When your manager asks you about test coverage, she probably does so because she seeks confidence that the software works sufficiently well to proceed to the next iteration or phase in the project.
Seeking confidence about something is a good project management principle. After all: If you’re confident about something, you are so because you don’t need to worry about it. Not having to worry about something means that you don’t have to spend your time on it, and project managers always have a gazillion other things that need their attention.
The word is the bug
So if confidence comes out of test coverage, then why is it that it managers often misunderstand us when we talk about coverage?
Well, the word actually means something else in daily language than it does when we use it in testing. So the word causes a communication “bug” when it’s misunderstood or misused.
We need to fix that bug, but how? Should we teach project managers the ”right” meaning of the word? We could send them to a testing conferences, ask them to take a testing course, or give them books to read.
That might work, but it wouldn’t solve the fundamental communication problem. It will move higher up in the organisational hierarchy.
An educated manager will have the same problem, not being able to make her peers and managers understand what ”test coverage” means. After all, not everyone in the organisation can be testing experts!
STOP mentioning coverage
A good rule of thumb in communication is: When your communication is likely to be misinterpreted, don’t communicate.
I, as a tester knows what test coverage means and more importantly what it does not mean, but I cannot expect others to understand it. Thus, if I use the word, I will probably be misunderstood. A simple solution to this is to stop using the word. So I won’t say sentences like: Our testing has covered some functionality.
The thing I can say is: We have carried out these tests and we found that.
This will work well until someone asks you to relate your testing to the business critical functionality: Ok, then then tell me, how much of this important functionality do your tests cover?
Uh oh!
Stay in the Testing Arena – or be careful
American circuses have enormous tents and two, three or even four arenas with different acts happening at the same time. A project is always going on in different arenas as well: For example we might have a product owner arena, a development arena, a test arena, and a business implementation arena.
Some people play in several arenas: I think most testers have at some point in the career made the mistake of telling a developer how to code. Likewise, we can probably all agree that there’s nothing more annoying than a developer telling a tester how to test.
Confidence belongs in the product owner arena, not in testing. This is because testing is about qualifying and identifying business risks, and since confidence does not equal absence of risks, it’s very hard for us to talk about confidence. And coverage.
This doesn’t mean you can’t move to another arena.
You can indeed look at things from the product owners perspective, that’s perfectly ok! Just make sure you know that you are doing it and why you are doing it: You are leaving your testing arena to help your product owner make a decision. Use safe-language, when you do.
Talk facts and feelings
Confidence is fundamentally a feeling, not a measurable artefact. It’s something that you can develop, but it can also be communicated: Look confident, express confidence, talk about the good stuff, and people around you will start feeling confident.
Look in-confident, express worry, talk about problems, and people around you will start feeling worried.
Testers always develop feelings about the product we’re testing, and we can communicate these feelings.
I know two basic strategies in any type of test result communication:

  • Suggest a conclusion first, then tell’m what you’ve done
  • Give them all the dirty details first, then help your manager conclude

Which communication strategy you pick should depend on the context, e.g. your relation with the manager. If everything looks pretty much as-expected (whether that’s good or bad), your manager has trust in you, and you have good knowledge of the business risks, then I wouldn’t worry too much about serving the conclusion first, and then offer details later mostly to make sure you and your manager doesn’t misunderstand each other. And that nobody will later be able to claim that you kept silent about something.
But if something is way off, or your manager doesn’t trust you (or you don’t trust her), peoples lives may be at stake, or you just have no idear what’s happening, then stick to the details – do not conclude. And that, I think, implies not using the term ”testing coverage”.

Categories
Blog posts in English

A Sustainable Mission for Context Driven Testing?

This image changed the world. It was taken from Apollo 9 in 1968 and shows the blue Earth rise over the grey and deserted Moon. Our world seems fragile.
This image changed the world. It was taken from Apollo 9 in 1968: “The vast loneliness is awe-inspiring and it makes you realize just what you have back there on Earth,” astronaut Command Module Pilot Jim Lovell said. Image credit: NASA.

I have lately become worried about certain developments in society.
For years, scientists, politicians and others have warned us that we’re responsible for irreverisble changes to our planet: Climate changes, most notably. They’re telling us we need to change to sustainable energy sources.
Sustainability is about more than energy, and I’m worried that in the society changes imposed upon us by the combined effects of globalization and the need for serious resource conservation, we are at the same time becoming increasingly indifferent about the lives of certain groups of people. I remember how many used to be develop deep feelings of indignation when pictures of hungry or poor children were shown on tv. It has changed and such pictures don’t have much effect any more. And worse: We genrally don’t even care about poverty close to ourselves.
I feel this may be linked to a macroeconomic pattern we’re seeing almost everywhere in the world: The rich are getting richer, but the poor are still as poor as they used to be. In Southern Europe, we have enormous unemployment among young people. Economists are raising a warning that we are about to loose a whole generation.
Does this affect testers too? After all we’re safe, working in IT, technology of the future?
Well inequalities in income and life conditions are growing on our planet, and this is worrying, since inequality has historically been a trigger of wars and revolutions, and has always been damaging to democracy and society as a whole. So yes, I think we have very good reasons to be worried about the future for ourselves, our families and for our societies.
James Bach recently published a blog post which has inspired me. Testing is a performance, not an artifact, he says. It made me think about how I differentiate the great testing performance from the poor performance. Is it only a subjective measure (aka ”the music performance was good”), or could there be some objective measures in play?
I think we should judge the testing performance by the artefacts it produces: Knowledge artefacts which are valuable in the business context in which we’re testing, income artefacts to me as a tester, and entertainment artefacts (testing is fun).
But I’ve realised that there is something missing: The performance should also be judged by its contribution to society as a whole. Testing should somehow contribute to sustainability in order to be a meaningful profession for me, social sustainability as well as energy and materialistic sustainability.
This can be taken as a strictly political point of view, and I could choose to execute it by only accepting jobs in socially responsible companies and in organisations and comapnies which are making sustainable products.
But it can also be seen as a mission for our craft as a whole. Like science itself has had to face the fact that it is not just a knowledge producing activity, but has to face the fact that it is changing society by the knowledge it is producing, we as testers also have to face the fact that the knowledge we are producing is applied by certain ways. Being a responsible tester does not mean that I’m only responsible for testing.
Therefore, I think that we should take on the endavour to develop our craft from being just a knowledge producing performance, to be a wisdom producing performance.
Philosopher Nicholas Maxwell is the author of ”From Knowledge to Wisdom” in which he outlines a revolution in science. In the introduction to the second edition he writes (p 14, second ed. 2007):

There is thus, I claim, a major intellectual disaster at the heart of western science, technology, scolarship and education – at the heart of western thought; and this long-standing intellectual disaster has much to do with the himan disasters of our age, our incapacity to tackle more himanely and successfully our present world-wide problems. In order to develop a saner, happier, more just and humane world it is certainly not a sufficient condition that we have an influential tradition of rational inquiry devoted to helping us achieve such ends. It is, however, I shall argue, a necessary condition. In the absense of such a tradition of thought, rationally devoted to helping us solve our problems of living, we are not likely to resolve these problems very successfully in the real world. It is this which makes it a matter of such proound intellectual, moral and social urgency, for all those in any way concerned with the academic enterprise, to develop a kind of inquiry more rationally devoted to helping us resolve our problems of living than that which we have at present.

Should this apply testing, as well as “science, technology, scolarship and education”? Yes, it certainly should. Will it be easy to adopt this thinking in testing? No, not at all.
First of all, we shouldn’t start throwing away any of the good things we’ge learnt and developed. Like the ”scientific method” is still a necessary but not suffucient condition for the progress of science, our values and ideas about great testing are still all-important in testing. They are just not sufficient.
I think we who belong to the Context Driven Testing school are far better equipped than other testing schools to accept the sustainability point of view. After all, we’re already successful developing testing into a sustainable performance. Other testing schools still struggle with their explicit or implicit underlying short-term profit-making ambitions.
And although we’re obviously playing a polyfonic music piece, speaking many voices, not saying or meaning exactly the same about testing or CDT, it seems to me that everyone in the CDT school share the mission of developing testing as a craft as a creative, value producing performance, where value is what matters to stakeholders of the product under test. Let me call this our shared mission.
This is a wonderful mission, but in the new context, it has to give way for a better one: We’re only percieving the craft of testing in isolation or in its immediate context, and we have to raise our heads and relate our craft to the greater context of society.
So I propose that we in the Context Driven School adopt the mission to develop testing towards being a wisdom enhancing performance, where wisdom is knowledge that helps build a sustainable society.
What do you think?

Categories
Blog posts in English

Testing Hopes for 2014

DSC_0540A
Christmas is a ”Lichtfest” for us in the North. Daytime, at this time of year, only lasts a few hours and the sun never really rise on the horizon. Christmas reminds us that light days will return and it’s time to look ahead on the year to come.
I have two hopes for software testing for 2014:

  1. I hope we will stop looking for simple explanaitions why something failed: The product, the testing, the development.
  2. We cannot expect all managers to be testing experts, so we need better documented and qualified testing practices (in various contexts) in order to support better top management software testing decisions.

Looking back on 2013…

I had a busy 2013, privately as well as professionally. Let’s Test in May was fantastic! A few weeks later, I gave a successful lecture on Context Driven Testing in IDA-IT.
I have for long wanted to link my favorite philosopher Niels Bohr to testing. Denmark celebrated the 100 year anniversary of Niels Bohrs articles on the atom model this year. Niels Bohr was a Nobel Prize wining physicist, but more than anything, he was a philosopher – my favorite philosopher by far.
My second favorite is Nassim Taleb. Taleb published his new book Antifragile in late 2012, and I read it this year. But it was his previous book Black Swan that made me a fan.
In chapter 12 of Black Swan, Taleb criticizes historicism: Always wanting to find causes of why things happen. That happens a lot in testing too:

  • ”Why was that bug in the system!?”
  • ”Why didn’t test find it!?”
  • ”Who blundered!?”

Taleb points out that explaining an event is just as difficult as predicting the future. He argues that any logical deductions and computations involved in analysing an event will yield random results.
Good managers knows that appreciating and handling a team’s frustration over something not going as planned is important, but we are too often committing the error of turning a psychological healing process into a development system, mindlessly making up apparently deterministic explanations for the unexpected – the random.
Randomness and historicism
Randomness is actually two different things: (1) Indeterministic mathematical radomness. (2) Something that is acting chaotically, but still according to deterministic laws.
The ”butterfly in india” is an example of a chaotic, but deterministic chain of events: It is said that the beating of a butterfly’s wings in Delhi can cause a thunderstorm in North Carolina.
According to Newtonian and relativistic physics, determinism is a fundamental property of nature, but since most of the events involved in the forming of the thunderstorm are outside our reach, we won’t be able to reconstruct the event completely anyway.
This is perhaps where Taleb and Bohr might disagree, since Bohr did not believe in determinism as a fundamental property of nature.
With quantum physics, Bohr, Heisenberg, Pauli and other pioneers were able to show that events on the nuclear level do not follow rules of causality. An electron, for example, moves from one energy level to a lower, releasing a photon, spontaneously.
”So what? We’re not living in microcosomos. Butterflies don’t move electrons, they set complete molecules in motion. Causality should still apply on any observable level.”
This is a valid counter argument, but Bohr, in several of his philosophical writngs, points out that the lack of casualty on the subatomic level does in fact affect the macroscopic level: There are many amplification systems in nature, which amplify single quanta of energy into macroscopic effects. One such is the human eye, which can detect single photons and amplify it as a stream of information sent to the brain, where it can trigger actions. Obviously there are lots of such amplification systems in the brain and our bodies, so maybe there’s no such thing as determinism in people? And in nature in general, for that matter.
Does having a bad childhood make someone bad?
Se we’re essentially left with a world of repeatable patterns. Statisticians know that children of poor parents will usually be poor themselves. That is a well known pattern, but does it work the other  way too? Does a bad childhood make you a bad person?
Obviously no. The pattern cannot be linked to the individual, per se.
But that doesn’t mean patterns aren’t useful: Patterns simplify reality, and simplification is necessary in all planning and management.
Many projects have contracts which are negotiated several years before the testers start. Such contracts often specify which kinds of testing should take place e.g. how acceptance testing should be carried out.
Now, we can’t expect all IT contract managers to be testing experts, but if we can document research evidence of the usefulness of e.g. exploratory testing, we’re much more likely to be able to convince them to use it constructively, even when they’re working on the early planning phases.
Happy 2014!