Simon Morley posted a very interesting post about Challenges with communicating models about two weeks ago. Mental models are what we unconscously use to understand a situation, and communicating models to others is an interesting challenge: “[…] models do not transmit themselves – they are not necessarily understood on their own – they need the necessary “synch’ing” to align both parties to achieve comprehension, communication and dialogue”, as Simon summed it up in a comment on the blog post.
Simons post and the very good discussion he and I had about it started me thinking on the psychological perspective and how important empathy is: The “synch’ing” relies on empathy.
I really liked Simons blog post because above all it highlights the subjectivity of mental models. Models are not something you can implement in an organisaton just by e.g. e-mailing them to all employees. If you want someone to ‘get’ your model, you need to actively communicate it. Which is not possible without empathy.
Empathy is something that we associate with friendship and love, but it plays part in all communication processes between human beings, including those we as testers engage with at work.
From time to time we come across people who seems to have a total lack of understanding of what we’re doing: Colleagues, managers, customers. Most of the time, people who don’t understand need only a good explanation of the situation and our point of view. But sometimes, an explanation isn’t enough: Some people just don’t seem to want to understand.
People under pressure or in stress can be like that, and we often associate this with aggressive behaviour, rudeness. Or maybe we just see the rudeness, and only later realise that maybe there was a problem with understanding.
Empathy seems to exclude this behaviour. Empathy relies on a cognitive ‘feature’ of our brain which attempts to copy thoughts and feelings of other people: It tries to decode what those who you interact with are thinking based on verbal as well as unconscious ques, e.g. body language. It’s quite obvious that having ‘a notion’ of what someone else thinks and feels can make communication much more successful – if you feel sympathy for the other persons feelings and thoughts.
This can work both ways: Loss of empathy in a situation can mean that you think everybody else thinks and feels the same as you, and it can cause quite a lot of confusion and frustration when you realise that other’s are’nt thinking the same as you.
It can happens to all of us: The brain is not a perfect and consistently operating machine, but rather a very adaptable and flexible organ. For example in situations of crisis, empathy is one of the first things to go. A person in a crisis shift from being a social creature to goal oriented and focused, typically on survival – at any cost.
There are people who don’t want to understand, e.g. due to politics. But there are some people who involuntarily just aren’t able to get to ”the other side” of the argument, for example because they’re having ”a bad day”.
(Some people with autism and ADHD can be characterised by having problems with empathy. This is a quite severe handicap for them, since not only do they have problems decoding what other people think og feel, they can also have problems seperating their own thoughts and feelings from what other people are thinking and feeling. The sad situation for empathy impaired people is that they often don’t have a choice: Even when everything is good is it extremely difficult for them to decode other peoples thoughts, feelings and intentions – and therefore extremely difficult for them to communicate and interact successfully. Noticing how successfully others interact often just makes them feel plain stupid. This can lead to severe depression.)
Author: Anders Dinsen
Categories
Turning up the Heat
– What are you doing?
– I’m turning up the heat. To see if anything catches fire. It’s an old CID trick.
Detective Chief Inspector Jack Frost in the TV series “A Touch of Frost”
In the follow up post after my presentation at Let’s Test, I concluded that the next step in my work on black swan testing should be on operationalisation. This post introduces a a testing heuristic which I’ve successfully used myself. I call it “Turning up the Heat”
The basic idea is that odd things happen when a system is put under pressure, and that we can learn stuff about the system by doing so.
There are basically three ways to do it:
- Load testing, i.e. putting the system under exceptionally heavy load for a period
- Soak testing, i.e. loading the system lightly for a long time
- Destructive testing, e.g. crippling subsystems by disabling or forcing malfunctioning
They can of course be combined.
In the quote above, heat is a metaphor for the psychological pressure Jack Frost is subjecting suspected murderers to, and here ‘load’ is also a metaphor for any environment changes that can have an impact on the way the system works. It could be a high data load, e.g. 10 or 100 times the normal rate of requests to a service, but it could also be something quite different, e.g. a 10 degrees higher than normal temperature in the server room. The black swan domain is the systems domain, so any component in the complete system is a valid target for putting load on.
Likewise, ‘crippling’ is a metaphor here for doing something to a subsystem that will cause the subsystem to work differently from normal: It could be as simple as just removing a component, or bugs could be deliberately introduced. In fact the target doesn’t have to be one single subsystem: Changing the same thing in several subsystems, e.g. compile all software modules with a buggy version of some widely used library, can be a simple and efficient apporach.
As testers, we often don’t have direct access to the tools needed to load and cripple subsystems and complete systems. I find that in order to practically “turn up the heat”, I often have to rely on the help of others, e.g. developers and system administrators. This leaves me with a communication and cooperation challenge which should not be taken lightly.
There are no right or wrong approaces in testing, but there’s a risk of wasting time: I.e. spending time on preparations and never getting down to the actual testing – thereby not learning. That’s one of the reasons I usually prefer simple techniques rather than planned approaches.
In one project I’ve worked on, the testing tool we used had a simple load testing function. I managed to crash the test environment completely by just running the tool off my own pc, and this eventually gave us some important information about a vulnerable subsystem (the root cause of the crash was not what anyone expected when the system stopped responeding). I spent less than an hour on this test – though getting the problem diagnosed and the environment recovered involved somewhat more work afterwards by the system admins, I’m afraid.
This actually points me to another point related to cooperation: While load testing or soak testing is normally non-destructive, only do it if the project can afford repairing what might be affected by possible malfunctions of the system. This could include other testers not being able to complete other testing activities!
The illustration below is taken from an old book, I’m reading (*):
Fig. 3 shows a desert land which will be cultivated by irrigation, i.e. the artificial application of water. Fig. 4 shows the amount of fertile soil available. Now, the farmer can decide to spread the soil all over the area, by which the layer of fertile soil will be so thin that nothing will grow in any part of the land. That is not a good plan and all the work involved will be fruitless.
But there’s an alternative: The soil can be spread over a section of the land, for example the area marked in fig. 3. This way the layer of soil will be thick enough to ensure that there will be exuberant growth and good utilization in the smaller area. This is obviously a much better plan.
This scenario not only applies to farming: It illustrates a problem we often face in testing, where the amount of functionality being developed is much larger than the what we can cover in a decent way. It is my experience that it is always better to focus testing on sections of the system than to try to check everything: There will be areas of the system which will be left untested, but what you test, you will cover well.
As a decision maker, I’d much rather have in depth knowledge about parts of the system, than to know very little about everyting. It will give me much better foundation for making good business decisions.
*) The book is Holger Paaskesen: “Vi lærer for livet?” from 1968. It’s English title would be “We learn for life?” and it’s a book about school education.
The pervasive, but intangible nature of Black Swans means that that practical testing with the aim of demonstrating actual problems is probably either not going to give any useful evidence, or so much resources will be spent proving a point, that one might be missing the point itself completely.
This was the fundamental problem I was facing when I finished reading Talebs book and wanted to apply his philosophy into actual testing. I realised that I needed a model, and the Skype incident of december 2010 led me in the right direction.
The model that I’ve come up with is that black swans are system errors. This may not be true in all circumstances, but it’s a good model and it’s helping me come up with solutions to the testing problems.
Unfortunately, treating black swans as system errors also mean that instead seeing Black Swan Testing as a practical testing activity, I’m moving it to a meta level, where the ‘root causes’ are of a more abstract nature and often not directly observable.
In my speach here at Let’s Test yesterday, I introduced three classes of system attributes and suggested that practical testing, with the aim of learning about potential black swan incidents in a system, should focus on these attributes:
- Complex versus Linear Interactions
- Tight versus Loose Couplings
- Barriers
The two first come from the work of sociologist Charles Perrow, in particular his book Normal Accidents, the third on I owe to psychologist James Reason, author of Human Error. I’ll come back to these attributes in later blog posts, but for now you just have to accept them as system attributes that play parts in system errors and Black Swans.
But we’re at a conference with all sorts of things going on: My presentation was well received, the discussions afterwards were great, but Let’s not just talk… let’s do it, Let’s Test.
I think James Lyndsay and I got the idea at about the same time yesterday: Let’s take Black Swan Testing into The Test Lab.
So I did, and it was great. I had a great team of very brave testers, and the mission was clear: Find indications of Black Swans, look for tight couplings, complex interactions, and barriers.
Did we succeed? Not really. But it was loads of fun and we learned a lot!
In particular, I learned that while I think I have a very good idea of what Black Swan Testing is, I need to work on the practical aspects: Making useful charters, coaching and teaching testers efficiently on the subject, reporting… Black Swan testing must be communicated and operationalized.
That’s the next problem, I’m going to address.
Gosh, I’m feeling a bit used… The schedule is really tight here and the party yesterday was great. Lots of input! Plus, I was really focused on my presentation today, I spent a bit extra “cognitive energy” on that ‘output’ process… Here’s what I’ve done today:
6-7: Wake up
7-8: Walk in nature (found some freesbee throwing creatures…)
8-9: Breakfast
9.30: Rob Sabourin keynote (really good!)
11.30-12.30: Presenting Testing in the Black Swan domain
12.30-13-30: Lunch, having very interesting talk with Rob Sabourin and others
13.30-14.30: Curing our Binary Disease with Rikard Edgren (inspiring, we’re all growing older and maturing, I guess)
14.40-15.40: You are a scientist with Christin Wiedemann (I loved it – I’m a scientist too)
15.40-16: Coffee, cookies, smoothies (Runö is such a great venue!)
16-17: Coaching Testers with Anne-Marie Charret (inspiring!)
My presentation was really well received and we had a great discussion afterwards. I love the discussions we are having here! What better place could I have found to first present my ideas about Black Swan testing than here among people who share the passion for testing, trying things, forming hypothesis and learning?
We’ll be doing experiments in the Test Lab tonight at 20, so discussions are’nt over yet. I love it!
Here are a few selected photos from today. I may do a writeup about the day later today or tomorrow. Maybe.
My head feels a little dizzy, but outside my hotel window, the sun is again shining from a clear blue sky, little birds are singing, and it’s looking to be a wonderful day today. Runö, the conference venue, is a really nice place- and so is Let’s Test! If you’re here, you will know what I mean, if not you just have to believe me: Never have I been with such a friendly and bright bunch of people! I love it!
After the opening keynote yesterday, I went to on my presentation for today. I was almost done, but then the team (that’s me!) decided to refactor everything! Oh dear! But it had too many slides, and they just didn’t work well together, and that’s a showstopper, right?
The good news is that the I got the presentation fixed: The refactoring succeeded! Thanks to friends and colleagues for allowing me to reflect with you on the subject (there’s the friendly thing again!).
The bad news is that I had to skip the tutorial I planned to go to.
I didn’t skip tutorials completely, though, as Iwas very kindly allowed to jump in on Henrik Andersson and Leo Hepis’ tutorial “Now, what’s your plan?”, which started at 3 pm.
I’m sure all the workshops were excellent, but this was really, really cool: During the 3 hours, we got to develop test strategies for our testing team, incorporate really challenging context changes, learn about what context is, and discuss buth our own and other team’s approaches and solutions to the challenges we were put through.
Normally, context is something which is “just there”. As a team member, I’m often not given all the needed knowledge about context, but still I have to relate to it anyway and develop my own test strategies, or when I’m given management responsibilities, the strategy of the whole team. Still, the context is shaping my strategy and it does so in so many ways. And then we have context change: Things aren’t static, right? Although we all prefer working in stable environments, things do often change: Sometimes to the better, sometimes to the worse, sometimes just to something different, but the point is that we cannot disregard context changes, since they affect us whether we want it or not.
How do teams react to context changes? I observed at least four different “reaction modes” within the teams during the workshop:
- “Ok, what’s this?”: It’s a completely new situation and there are no prejudices or previous context to take into account. This is fun and generally feels good.
- Resistance, chaos, integration, new status quo – i.e. all of the phases of the Satir Change Model. This can be a difficult process, especially it the team resides in the “chaos” phase for long.
- Relief: A context change clears everything up, and the project can go from “problem fixing” to “solution mode”. This feels very good too.
- Panic: The context change is sudden and feels like a bomb had been dropped in the middle of the project and is now threatening to blow everything up: The team panics. Hopefully, the bomb can be defused and the panic can be cleared.
So what is context? A couple of definitions surfaced – I didn’t get them all, but here’s a couple:
- The variables which significantly influence the task
- Those aspects of the total environment that seem important/relevant
- Context is anything that changes my model
What’s your definition?
Categories
Let's Test is underway!
As we approached Arlanda, I noticed something I recognised: I had checked out the conference venue on Google Eearth at home, and here was something that looked familiar:
“Can I get off here, please?”, I asked.
“Sorry no, we’re out of parachutes”, the air hostess replied.
Ok, I could live with that: Esko Arajärvi was flying in from Finland, and thanks to the excellent travelling page of the conference, he had offered me a ride in the car he rented, and he was waiting for me. I love when things go as planned!
Runö is a very beautiful place. “Is this heaven?” Michael Bolton asked in his opening keynote, but then said no, since his family wasn’t here. Good point!
Michael was as good as ever, telling us that while everyone has a desire for certainity, we’re fundamentally incompetent whenever we do something new. Testing is fundamentally about acceptance, acceptance that we’re fallible, incompetent, etc, but we can learn from failure.
Mature people don’t try to get rid of failure, they manage it. So do we. We know that failure is an important part of the process and the trigger for us is to find out what’s wrong and make it better.
In fact, certainty can be damaging, and “a key part of our service is to reduce unwarranted and potentially damaging certanity about the product.”
I’m taking a break now. I’m still working my presentation tomorrow; I work best under pressure, and the pressure is building up! Besides, Michael gave me more good inspiration which I have to process!
One thing you don’t have to worry about if you are NOT going to Let’s Test is how to get there. For family reasons, I had to wait until Monday morning to travel, and getting there from Copenhagen means getting up very early. It was 4 in the morning when I got up. 5 hours later, I’ll be listening to Michael Bolton’s opening keynote 🙂
So what am I expecting?
First of all: Good discussions about issues in testing. Testing requires reflection and thinking, and I need the input to stay razor sharp as a tester 😉
Second: Get great feedback on my ideas and thoughts about how we work with Black Swans in testing. We need better heuristics to sharpen our fucus, and I’m hoping – no: expecting that my ideas coupled with the feedback from conference participants will teach me and everyone else more on the subject.
SK400 is leaving in 30 mins now. I think it’s time for a cup of coffee!
It’s time to try something new for me: I have decided to do live blogging from Let’s Test conference on context driven testing next week. I’m mentally almost 100% focused on my own presentation about on Tuesday morning, but I’ll try to keep up with as much as possible to share it here with those who can’t be there in person – yes, I’m thinking first and foremost on my friend Jesper Ottosen @jlottosen