Tag: context driven testing

Categories
Blog posts in English

Let's Test Oz changed me

It’s Friday afternoon, September 19th 2014 and the Boeing 747 from Thai is full to the limit with children, teenagers, parents and businesspeople.
I am sitting in a middle seat in the window row on the right side of the airplane just in front of the wing. I have my Bose noise cancelling headphones on and I’m thinking.
I am leaving Sydney, flying home from the first Let’s Test Oz.
The Jumbo accelerates along the runway, then takes off. With engines whining, babies crying, people chatting and teenagers cheering, we head for the skies. The sounds are damped inside my Bose’s.
I’m alone with my thoughts, but not alone in the World.
I catch a glimpse of the pacific and the Sydney skyline. Then, the plane turns in over the Australian continent.
Australia and Let’s Test Oz has changed me.

Picture form Let's Test Oz workshop with Fiona Charles on test leadership.
Picture form Let’s Test Oz workshop with Fiona Charles on test leadership.

In democracy, debate changes politics

I participated in a political workshop a few days ago. It was organised by the Danish minister for social affairs, who is seeking input to his political campaign. We’re facing a parlamential election in 2015 and the danish democracy is waking up. I’m not a candidate or even a member of a political party, but I’m going to work hard to put social politics on the agenda.

Participating in the workshop was interesting. I shared some of my knowledge, learnt a bit and even got a few new contacts.

Let’s Test Oz was like that, except it was three days of sharing, learning and connecting about my profession: Software testing.

Innovation in testing

I think most people in the Context Driven Testing communicty percieve themselves as innovators. But not everything going on in the projects, companies and organisations we’re working in, is innovative. A lot is standardized, dull and repetitive work controled by machine bureaucracies.

Some testing is even political.

Politics is about power, and knowledge is power.

Testing, being a knowledge producing performance, is therefore a political instrument. Testing is more than that of course, but it’s always potentially a political instrument, and sometimes it’s even used as one.

Politics is applied psychology, and whenever there are humans, there is politics.

In public politics, the political processes are more or less visible and open. In projects, politics is often hidden or kept away from testers and tech people.

I’d like to see more testers (and tech people) getting into the political games. We shouldn’t just trust managers who know little about testing to control us. The game is just too important to miss.

That was, more or less, the message of my presentation at Let’s Test. James Bach, who was presenting at the same time as I was, saved me from having a large audience, but the room was well filled and we had a good discussion afterwards. I shared and learnt.

I’m in love with democracy and I have always and will always vote. Yet, voting is just the smallest part of our political democracy: It’s the debates that really change the world.

And to me, the most fantastic thing about the Context Driven Testing community is the debates we’re having. Through these debates, we’re changing ourselves, each other, and the world. To the better.

Leaving home for a conference

My family depends on me. That’s the short story. The slightly longer story is that I have four children and a wife, and neither of them fit well in our society. We struggle with a daily fear of stress, anxiety, depression, paranoia and even psycosis forcing it’s way back into our lives.
The Danish welfare system supports us, but I’m indispensable to my family. I’m not that stress sensitive.
Before I went to Sydney, I had decided not to feel guilty that I was leaving them for a week. I knew that, my wife had deliberately not thought about MH 17 having been shot down over the Ukraine a few months earlier carrying people going to a conference in Austrailay, and while I was away, she had also deliberately ignored the terror threats in Sydney.
So had I.
I enjoyed myself enormously at Let’s Test Oz and in Sydney.
The problems in our family are symptoms of autism but they’re also symptoms of society’s reaction to the collective anxiety over the post cold-war world:
Everything is possible if you dream it, yet diffuse threats surround us everywhere.
Fears and anxiety
I was a teenager in the 1980’s and back then we knew that the civilised world would cease to exist if war broke out. We were living with the fear.
Compared to anxiety, fear is easy: You can talk about it.
Fear and conflict unites us against something or someone.
Anxiety is a demon that slowly sucks energy out of you. Anxiety seperates people from each other. Collective anxiety kills humanity.
No doubt, there’s anxiety in Austrailia, but there’s a human side to the culture, which I sensed very strongly. I wrote a long essay about it in Danish which has been very well received. Returning home, I have found hope that Denmark could be changing to the better: A less anxious and more humane culture.

Struggeles for testers

However, the problems testers in Austrailia and New Zealand are struggeling with seem to be the same as here. Managers require enless rows of silly estimates, defects are counted, information is kept away from people needing it, and software quality is something everyone is talking about, yet few are doing anything seriously about it.
It’s not surprising that workers in our industry suffer. Some are working in hell-machines, and while we can have a civilised discussion about it over a steak and a sip of quality Austrailian redwine, or even a row of beers, many in the instry still need to calm themselves on a daily basis with hard exercise, alcohol or just lots of coffee. Or medicine.

The Context Driven Testing community is trying to change all this.

We debate. And we are becoming political performers.

Thank you

Let’s Test Oz had a track of ”testing influencing management”, which I shared my thoughts on, but also gave me lots of inspiration. The whole conference was a blast!

I’m very grateful that the programme commitee accepted my proposal and challenged me with the invitation to Sydney.

The trip around the globe changed me and gave me a deeper understanding of our testing community and humanity as such.

Thank you Let’s Test Oz and everyone there!

Sydney Skyline
Sydney Skyline
Categories
Blog posts in English

Do it right: A value in Context Driven Testing

The problem with me is that I’m really bad at following instructions. When people tell me to do something in a certain way, I do it differently.
It’s a problem when I cook, because I‘m not particularly good at cooking. So I have to follow recipies, and I often mess it up slightly. I’m improving, learning strategies to remember, but this is a fundamental personality trait for me.
And not one I’m sorry about. Because it’s not a problem when I test!
I always wanted to be a great tester.
I tend to become really annoyed with myself when a bug turns up in production in something I have tested. ”Why did I miss that?!” I feel guilty. You probably recognise the feeling.
The feeling of guilt is ok. The fact that we can feel guilt proves that we have consciousness, empathy and do the best we can. People who don’t care, don’t feel guilt.
But in testing, finding every bug is fundamentally impossible, so we have to get over it and keep testing. Keep exploring!
Even before I learnt about Context Driven Testing, I knew that great testing could never be about following test scripts and instructions. I noticed that I got bored and lost attention when I executed the same test scripts over and over again, but I also noticed that I missed more bugs when I only followed the instructions.
So I stopped following the instructions. This gave me an explanation problem, however: “Uhh, well I didn’t do quite what I was asked to do…. but hey you know, I found these interesting bugs that I can show you!”

Can you hear it? That won’t impress old-school project managers with spreadsheets to check and plans to follow.

Context Driven Testing has helped me greatly in becoming a better tester. The thing is that CDT teaches me to do a great testing job without instructing me exactly what to do. Instead, the community shares a lot of heuristics I can use to help me do a great testing job, and through thinking-training and inspiration from others, it’ll help me develop my capacity to do great testing in whatever contexts in which I’m working.
It may be a bit difficult to grasp at first. A little worrying, perhaps.. But CDT is a really, really powerful testing approach.

And CDT has helped me explain what I do to my project managers. Even the old-school types!

Social services and testing

A few days ago, I read an article about quality in social services in which the following statement from the vice president of the Danish social workers union caught my attention: ”It’s about doing it right, not about doing the right things.” He was speaking of how the authorities try to help vulnerable children and their families.
The statement resonated with me, and a bit later it occurred to me that it even sums up what CDT is about:
Context Driven Testing is about doing it right, not about doing all the right things.
Note that I’ve added the word ‘all’ here.
There’s more to CDT of course, but this is the core of CDT – to me. Some readers may lift their eyebrows over the ”doing it right”-thing: ”Does he mean that there is ONE right way to do it?” Read on…
The past 10 years, I’ve worked in contexts where CDT is not really the thing we do, and if I was to be as context driven as managers in my context had asked me to be, I would not really have been context driven. You get the picture, I’m sure.
But with CDT in my briefcase, I can work to make changes and improving things in the specific context in which I’m working. As a consultant and contractor, I’m usually hired to fix a problem, not to revolutionize things, and I’m expected to do a “good job” fixing it.
”Doing it right” is of course about doing a good job, and ”doing a good job” should really not be taken lightly. CDT helps me do a good job, even when I’m not working in contexts that actively supports CDT. That’s because CDT is flexible: It empahsizes that testing should never be driven by manuals, standards or instructions, but by the actual context in which it takes place, and that’s actually quite difficult to disagree with, even for old school project managers!

Further, if my context (i.e. project manager) ask me to do something, I do it – even if it’s in a standard. Sometimes there’s a good reason to use a standard.

Context driven testing is not defined by any methods or even by a certain set of heuristics. Nor is it defined by a book, standard or manual. Neither is there any formal education, certification or ”club” that you have to be a member of in order to call yourself Context Driven.
Instead, Context Driven Testing is about committing oneself to some core values, and to me the most important value in CDT is contained in the sentence:
It’s about doing it right, not about doing all the right things.

Why would anyone wan’t anything else?

(Thanks to @jlottossen for reviewing this post and for suggesting several changes to the original version.)

Categories
News in English Nyheder

I'll be at Let's Test Oz in Sydney in September

DSC_0540AI’ll take a 22 hour flight Copenhagen to Sydney in September, where the fourth Let’s Test conference and the first Let’s Test Oz will be held at a resort in Blue Mountains, New South Wales, Austrailia.

The conference programme was announced a while ago, and it’s pretty awesome. I’ve attended and spoken twice at Let’s Test in Stockholm. I’m sure the Oz-edition will be as fantastic as its Stockholm counterpart.

I haven’t yet decided on my program, but I’m looking forward to Fiona Charles’ keynote by Fiona Charles and her workshop on leadership
But there’s a lot of promising stuff in the programme

I insist that testing is and should be a value adding activity. As testers, we’re not just finding other people’s mistakes; we make a positive contribution to the project with the knowledge we are collecting in our testing.

To do great testing takes clever thinking, and clever thoughts never live in isolation. They’re shared, bounced and developed into great ideas. Let’s Test is an inspirering conference, a place where great ideas develop, and that’s why I like the conference so much.

My own contribution to the programme this year is a session about politics called “All is fair in love and war”. I commit to context driven testing, but testing can be a driver for change as well. I find that a key to do it is to do clever politics on top of the knowledge we have and collect in testing. Some testers have grown to hate politics, but politics can help us, if we use it wisely: With an ethical standpoint, and with a sound vision of what we want to achieve.

Twice has Let’s Test refuelled my capabilities as a tester, a test leader, a test manager, and a test analyst. Twice has Let’s Test inspired me and given me new friends and acquaintances. I’m looking forward to my third Let’s Test, this time down under, where I expect to meet some great testers from the southern hemisphere, take time for a good talk, do some testing, have a beer or two, take a walk in nature…

You can register for Let’s Test Oz here.

Categories
Blog posts in English

Speaking to Management: Coverage Reporting

Test coverage is important. In this post, I will reflect about communication issues with test coverage.
The word coverage has a different meaning in testing than in daily language. In daily language, it’s referring to something that can be covered and hidden completely, and if you hide under a cover, it will usually mean that we can’t see you. If you put a cover on something, the cover will keep things out.
Test coverage works more like a fishing net. Testing will catch bugs if used properly, but some (small) fish, water, plankton etc. will always pass through. Some nets have holes through which large fish can escape.
What’s so interesting about coverage?
When your manager asks you about test coverage, she probably does so because she seeks confidence that the software works sufficiently well to proceed to the next iteration or phase in the project.
Seeking confidence about something is a good project management principle. After all: If you’re confident about something, you are so because you don’t need to worry about it. Not having to worry about something means that you don’t have to spend your time on it, and project managers always have a gazillion other things that need their attention.
The word is the bug
So if confidence comes out of test coverage, then why is it that it managers often misunderstand us when we talk about coverage?
Well, the word actually means something else in daily language than it does when we use it in testing. So the word causes a communication “bug” when it’s misunderstood or misused.
We need to fix that bug, but how? Should we teach project managers the ”right” meaning of the word? We could send them to a testing conferences, ask them to take a testing course, or give them books to read.
That might work, but it wouldn’t solve the fundamental communication problem. It will move higher up in the organisational hierarchy.
An educated manager will have the same problem, not being able to make her peers and managers understand what ”test coverage” means. After all, not everyone in the organisation can be testing experts!
STOP mentioning coverage
A good rule of thumb in communication is: When your communication is likely to be misinterpreted, don’t communicate.
I, as a tester knows what test coverage means and more importantly what it does not mean, but I cannot expect others to understand it. Thus, if I use the word, I will probably be misunderstood. A simple solution to this is to stop using the word. So I won’t say sentences like: Our testing has covered some functionality.
The thing I can say is: We have carried out these tests and we found that.
This will work well until someone asks you to relate your testing to the business critical functionality: Ok, then then tell me, how much of this important functionality do your tests cover?
Uh oh!
Stay in the Testing Arena – or be careful
American circuses have enormous tents and two, three or even four arenas with different acts happening at the same time. A project is always going on in different arenas as well: For example we might have a product owner arena, a development arena, a test arena, and a business implementation arena.
Some people play in several arenas: I think most testers have at some point in the career made the mistake of telling a developer how to code. Likewise, we can probably all agree that there’s nothing more annoying than a developer telling a tester how to test.
Confidence belongs in the product owner arena, not in testing. This is because testing is about qualifying and identifying business risks, and since confidence does not equal absence of risks, it’s very hard for us to talk about confidence. And coverage.
This doesn’t mean you can’t move to another arena.
You can indeed look at things from the product owners perspective, that’s perfectly ok! Just make sure you know that you are doing it and why you are doing it: You are leaving your testing arena to help your product owner make a decision. Use safe-language, when you do.
Talk facts and feelings
Confidence is fundamentally a feeling, not a measurable artefact. It’s something that you can develop, but it can also be communicated: Look confident, express confidence, talk about the good stuff, and people around you will start feeling confident.
Look in-confident, express worry, talk about problems, and people around you will start feeling worried.
Testers always develop feelings about the product we’re testing, and we can communicate these feelings.
I know two basic strategies in any type of test result communication:

  • Suggest a conclusion first, then tell’m what you’ve done
  • Give them all the dirty details first, then help your manager conclude

Which communication strategy you pick should depend on the context, e.g. your relation with the manager. If everything looks pretty much as-expected (whether that’s good or bad), your manager has trust in you, and you have good knowledge of the business risks, then I wouldn’t worry too much about serving the conclusion first, and then offer details later mostly to make sure you and your manager doesn’t misunderstand each other. And that nobody will later be able to claim that you kept silent about something.
But if something is way off, or your manager doesn’t trust you (or you don’t trust her), peoples lives may be at stake, or you just have no idear what’s happening, then stick to the details – do not conclude. And that, I think, implies not using the term ”testing coverage”.

Categories
Blog posts in English

A Sustainable Mission for Context Driven Testing?

This image changed the world. It was taken from Apollo 9 in 1968 and shows the blue Earth rise over the grey and deserted Moon. Our world seems fragile.
This image changed the world. It was taken from Apollo 9 in 1968: “The vast loneliness is awe-inspiring and it makes you realize just what you have back there on Earth,” astronaut Command Module Pilot Jim Lovell said. Image credit: NASA.

I have lately become worried about certain developments in society.
For years, scientists, politicians and others have warned us that we’re responsible for irreverisble changes to our planet: Climate changes, most notably. They’re telling us we need to change to sustainable energy sources.
Sustainability is about more than energy, and I’m worried that in the society changes imposed upon us by the combined effects of globalization and the need for serious resource conservation, we are at the same time becoming increasingly indifferent about the lives of certain groups of people. I remember how many used to be develop deep feelings of indignation when pictures of hungry or poor children were shown on tv. It has changed and such pictures don’t have much effect any more. And worse: We genrally don’t even care about poverty close to ourselves.
I feel this may be linked to a macroeconomic pattern we’re seeing almost everywhere in the world: The rich are getting richer, but the poor are still as poor as they used to be. In Southern Europe, we have enormous unemployment among young people. Economists are raising a warning that we are about to loose a whole generation.
Does this affect testers too? After all we’re safe, working in IT, technology of the future?
Well inequalities in income and life conditions are growing on our planet, and this is worrying, since inequality has historically been a trigger of wars and revolutions, and has always been damaging to democracy and society as a whole. So yes, I think we have very good reasons to be worried about the future for ourselves, our families and for our societies.
James Bach recently published a blog post which has inspired me. Testing is a performance, not an artifact, he says. It made me think about how I differentiate the great testing performance from the poor performance. Is it only a subjective measure (aka ”the music performance was good”), or could there be some objective measures in play?
I think we should judge the testing performance by the artefacts it produces: Knowledge artefacts which are valuable in the business context in which we’re testing, income artefacts to me as a tester, and entertainment artefacts (testing is fun).
But I’ve realised that there is something missing: The performance should also be judged by its contribution to society as a whole. Testing should somehow contribute to sustainability in order to be a meaningful profession for me, social sustainability as well as energy and materialistic sustainability.
This can be taken as a strictly political point of view, and I could choose to execute it by only accepting jobs in socially responsible companies and in organisations and comapnies which are making sustainable products.
But it can also be seen as a mission for our craft as a whole. Like science itself has had to face the fact that it is not just a knowledge producing activity, but has to face the fact that it is changing society by the knowledge it is producing, we as testers also have to face the fact that the knowledge we are producing is applied by certain ways. Being a responsible tester does not mean that I’m only responsible for testing.
Therefore, I think that we should take on the endavour to develop our craft from being just a knowledge producing performance, to be a wisdom producing performance.
Philosopher Nicholas Maxwell is the author of ”From Knowledge to Wisdom” in which he outlines a revolution in science. In the introduction to the second edition he writes (p 14, second ed. 2007):

There is thus, I claim, a major intellectual disaster at the heart of western science, technology, scolarship and education – at the heart of western thought; and this long-standing intellectual disaster has much to do with the himan disasters of our age, our incapacity to tackle more himanely and successfully our present world-wide problems. In order to develop a saner, happier, more just and humane world it is certainly not a sufficient condition that we have an influential tradition of rational inquiry devoted to helping us achieve such ends. It is, however, I shall argue, a necessary condition. In the absense of such a tradition of thought, rationally devoted to helping us solve our problems of living, we are not likely to resolve these problems very successfully in the real world. It is this which makes it a matter of such proound intellectual, moral and social urgency, for all those in any way concerned with the academic enterprise, to develop a kind of inquiry more rationally devoted to helping us resolve our problems of living than that which we have at present.

Should this apply testing, as well as “science, technology, scolarship and education”? Yes, it certainly should. Will it be easy to adopt this thinking in testing? No, not at all.
First of all, we shouldn’t start throwing away any of the good things we’ge learnt and developed. Like the ”scientific method” is still a necessary but not suffucient condition for the progress of science, our values and ideas about great testing are still all-important in testing. They are just not sufficient.
I think we who belong to the Context Driven Testing school are far better equipped than other testing schools to accept the sustainability point of view. After all, we’re already successful developing testing into a sustainable performance. Other testing schools still struggle with their explicit or implicit underlying short-term profit-making ambitions.
And although we’re obviously playing a polyfonic music piece, speaking many voices, not saying or meaning exactly the same about testing or CDT, it seems to me that everyone in the CDT school share the mission of developing testing as a craft as a creative, value producing performance, where value is what matters to stakeholders of the product under test. Let me call this our shared mission.
This is a wonderful mission, but in the new context, it has to give way for a better one: We’re only percieving the craft of testing in isolation or in its immediate context, and we have to raise our heads and relate our craft to the greater context of society.
So I propose that we in the Context Driven School adopt the mission to develop testing towards being a wisdom enhancing performance, where wisdom is knowledge that helps build a sustainable society.
What do you think?

Categories
Blog posts in English

Testing Hopes for 2014

DSC_0540A
Christmas is a ”Lichtfest” for us in the North. Daytime, at this time of year, only lasts a few hours and the sun never really rise on the horizon. Christmas reminds us that light days will return and it’s time to look ahead on the year to come.
I have two hopes for software testing for 2014:

  1. I hope we will stop looking for simple explanaitions why something failed: The product, the testing, the development.
  2. We cannot expect all managers to be testing experts, so we need better documented and qualified testing practices (in various contexts) in order to support better top management software testing decisions.

Looking back on 2013…

I had a busy 2013, privately as well as professionally. Let’s Test in May was fantastic! A few weeks later, I gave a successful lecture on Context Driven Testing in IDA-IT.
I have for long wanted to link my favorite philosopher Niels Bohr to testing. Denmark celebrated the 100 year anniversary of Niels Bohrs articles on the atom model this year. Niels Bohr was a Nobel Prize wining physicist, but more than anything, he was a philosopher – my favorite philosopher by far.
My second favorite is Nassim Taleb. Taleb published his new book Antifragile in late 2012, and I read it this year. But it was his previous book Black Swan that made me a fan.
In chapter 12 of Black Swan, Taleb criticizes historicism: Always wanting to find causes of why things happen. That happens a lot in testing too:

  • ”Why was that bug in the system!?”
  • ”Why didn’t test find it!?”
  • ”Who blundered!?”

Taleb points out that explaining an event is just as difficult as predicting the future. He argues that any logical deductions and computations involved in analysing an event will yield random results.
Good managers knows that appreciating and handling a team’s frustration over something not going as planned is important, but we are too often committing the error of turning a psychological healing process into a development system, mindlessly making up apparently deterministic explanations for the unexpected – the random.
Randomness and historicism
Randomness is actually two different things: (1) Indeterministic mathematical radomness. (2) Something that is acting chaotically, but still according to deterministic laws.
The ”butterfly in india” is an example of a chaotic, but deterministic chain of events: It is said that the beating of a butterfly’s wings in Delhi can cause a thunderstorm in North Carolina.
According to Newtonian and relativistic physics, determinism is a fundamental property of nature, but since most of the events involved in the forming of the thunderstorm are outside our reach, we won’t be able to reconstruct the event completely anyway.
This is perhaps where Taleb and Bohr might disagree, since Bohr did not believe in determinism as a fundamental property of nature.
With quantum physics, Bohr, Heisenberg, Pauli and other pioneers were able to show that events on the nuclear level do not follow rules of causality. An electron, for example, moves from one energy level to a lower, releasing a photon, spontaneously.
”So what? We’re not living in microcosomos. Butterflies don’t move electrons, they set complete molecules in motion. Causality should still apply on any observable level.”
This is a valid counter argument, but Bohr, in several of his philosophical writngs, points out that the lack of casualty on the subatomic level does in fact affect the macroscopic level: There are many amplification systems in nature, which amplify single quanta of energy into macroscopic effects. One such is the human eye, which can detect single photons and amplify it as a stream of information sent to the brain, where it can trigger actions. Obviously there are lots of such amplification systems in the brain and our bodies, so maybe there’s no such thing as determinism in people? And in nature in general, for that matter.
Does having a bad childhood make someone bad?
Se we’re essentially left with a world of repeatable patterns. Statisticians know that children of poor parents will usually be poor themselves. That is a well known pattern, but does it work the other  way too? Does a bad childhood make you a bad person?
Obviously no. The pattern cannot be linked to the individual, per se.
But that doesn’t mean patterns aren’t useful: Patterns simplify reality, and simplification is necessary in all planning and management.
Many projects have contracts which are negotiated several years before the testers start. Such contracts often specify which kinds of testing should take place e.g. how acceptance testing should be carried out.
Now, we can’t expect all IT contract managers to be testing experts, but if we can document research evidence of the usefulness of e.g. exploratory testing, we’re much more likely to be able to convince them to use it constructively, even when they’re working on the early planning phases.
Happy 2014!

Categories
Blog posts in English

An illustration of the resource vs coverage problem

The illustration below is taken from an old book, I’m reading (*):
Illustration from: Holger Paaskesen: Vi lærer for livet?
Fig. 3 shows a desert land which will be cultivated by irrigation, i.e. the artificial application of water. Fig. 4 shows the amount of fertile soil available. Now, the farmer can decide to spread the soil all over the area, by which the layer of fertile soil will be so thin that nothing will grow in any part of the land. That is not a good plan and all the work involved will be fruitless.
But there’s an alternative: The soil can be spread over a section of the land, for example the area marked in fig. 3. This way the layer of soil will be thick enough to ensure that there will be exuberant growth and good utilization in the smaller area. This is obviously a much better plan.
This scenario not only applies to farming: It illustrates a problem we often face in testing, where the amount of functionality being developed is much larger than the what we can cover in a decent way. It is my experience that it is always better to focus testing on sections of the system than to try to check everything: There will be areas of the system which will be left untested, but what you test, you will cover well.
As a decision maker, I’d much rather have in depth knowledge about parts of the system, than to know very little about everyting. It will give me much better foundation for making good business decisions.
*) The book is Holger Paaskesen: “Vi lærer for livet?”  from 1968. It’s English title would be “We learn for life?” and it’s a book about school education.

Categories
Blog posts in English

Gooooood morning, Rrrrrrunö!

My head feels a little dizzy, but outside my hotel window, the sun is again shining from a clear blue sky, little birds are singing, and it’s looking to be a wonderful day today. Runö, the conference venue, is a really nice place- and so is Let’s Test! If you’re here, you will know what I mean, if not you just have to believe me: Never have I been with such a friendly and bright bunch of people! I love it!
After the opening keynote yesterday, I went to on my presentation for today. I was almost done, but then the team (that’s me!) decided to refactor everything! Oh dear! But it had too many slides, and they just didn’t work well together, and that’s a showstopper, right?
The good news is that the I got the presentation fixed: The refactoring succeeded! Thanks to friends and colleagues for allowing me to reflect with you on the subject (there’s the friendly thing again!).
The bad news is that I had to skip the tutorial I planned to go to.
I didn’t skip tutorials completely, though, as Iwas very kindly allowed to jump in on Henrik Andersson and Leo Hepis’ tutorial “Now, what’s your plan?”, which started at 3 pm.

I’m sure all the workshops were excellent, but this was really, really cool: During the 3 hours, we got to develop test strategies for our testing team, incorporate really challenging context changes, learn about what context is, and discuss buth our own and other team’s approaches and solutions to the challenges we were put through.

Normally, context is something which is “just there”. As a team member, I’m often not given all the needed knowledge about context, but still I have to relate to it anyway and develop my own test strategies, or when I’m given management responsibilities, the strategy of the whole team. Still, the context is shaping my strategy and it does so in so many ways. And then we have context change: Things aren’t static, right? Although we all prefer working in stable environments, things do often change: Sometimes to the better, sometimes to the worse, sometimes just to something different, but the point is that we cannot disregard context changes, since they affect us whether we want it or not.
How do teams react to context changes? I observed at least four different “reaction modes” within the teams during the workshop:

  • “Ok, what’s this?”: It’s a completely new situation and there are no prejudices or previous context to take into account. This is fun and generally feels good.
  • Resistance, chaos, integration, new status quo – i.e. all of the phases of the Satir Change Model. This can be a difficult process, especially it the team resides in the “chaos” phase for long.
  • Relief: A context change clears everything up, and the project can go from “problem fixing” to “solution mode”. This feels very good too.
  • Panic: The context change is sudden and feels like a bomb had been dropped in the middle of the project and is now threatening to blow everything up: The team panics. Hopefully, the bomb can be defused and the panic can be cleared.


So what is context? A couple of definitions surfaced – I didn’t get them all, but here’s a couple:

  • The variables which significantly influence the task
  • Those aspects of the total environment that seem important/relevant
  • Context is anything that changes my model

What’s your definition?

Categories
Blog posts in English

I'll be live blogging from Let's Test

It’s time to try something new for me: I have decided to do live blogging from Let’s Test conference on context driven testing next week. I’m mentally almost 100% focused on my own presentation about on Tuesday morning, but I’ll try to keep up with as much as possible to share it here with those who can’t be there in person – yes, I’m thinking first and foremost on my friend Jesper Ottosen @jlottosen

Categories
Blog posts in English

The problem with test documentation

The agile manifesto explicitly values ”working software over comprehensive documentation.” In test, this means that actual testing is valued over test documentation. I would have put it this way: Focus is on the quality of the product, not on the quality of the documentation of the product.
We can probably all agree that it’s more fun to test and explore software than writing documentation. But it would be too radical, if we skipped writing documentation at all!
I think, however, that we testers should be more critical about the documentation we actually do produce, and that we should look for ways to improve the documentation we’re making.
The problem with documentation is not that too much time is spent writing it instead of actually testing the product. The problem is that the quality of the documentation is often just not good enough.
Most organizations have standards for test documentation and require their testers to write specific types of documents based on mandatory templates.
Templates are generally a good thing, but they can be problematic if you limit your writing process to ”filling in the gaps.” A good document contains useful information to the reader, so the most important step in the writing process is finding out what information is actually needed.
Not all sections of a template can be equally useful in all contexts (or useful at all), and very often you need to document things which there has not been left space for in the template.
But, finding out what to document is not trivial. Basically, it requires that you know the context of the project you are working on. Determining context can be difficult if you are new on a project, but asking questions can give you answers which will help you define the context.
Here are some questions, which I find useful:

  • Who will be reading the document? How will they be reading it? The challenge is to include content which the actual readers will find useful and to structure the content in a way so that they can find that information.
  • What information are they looking for? Try to get people to answer in concrete terms rather than using abstract vocabulary. The problem with written documentation is that stake holders will often not read it – most testers seem to prefer to explore systems on their own rather than reading documents, and managers will often not have time to read everything, but will only check the headline and certain details. But if readers get what they look for, chances are that they will read the document.
  • What kind of analysis do I need to carry out on the system to better understand it and test it? Writing a document about the system can often help me understanding it. I will discover missing knowledge and knowledge which I didn’t see initially. The writing process is part of the analysis.
  • Are there regulatory requirements which specify that testing should be documented to a certain detail and in a particular way? In this case, the test documentation is actually a product of the project.
  • Should the document assist knowledge transfer once I’m no longer on the project? In that case, the big question is what kind of knowledge should be ”transferred.”

I checked the section about documentation in Kaner, Bach and Pettichord’s Lessons Learned in Software Testing a few days ago. They have a better and longer list of context-free questions to ask about documentation which will help you find out what kind of documentation you need. They also list a few references to other lists of useful questions, so I suggest you take a look at that too.
Are there any specific types of documentation which is particularly useful? Indeed, there is:

  • Mind maps are becoming increasingly popular with testers, but ‘old style’ software diagrams like swimlane diagrams, state diagrams, and flow charts etc are still working well. The problem with mind maps is that they are often personal to the author of the map and not very easy to read.
  • Test scripts can be very useful in initial stages of knowledge transfer: A script can help another tester finding out how a certain procedure is performed in the system. However, a script will by itself tell the tester anything about the context of the script, and this is something which is often missed: Knowledge about a system under test is much more than knowing how to do things.
  • Check lists are actually much more useful than the name implies: A check list will list things to verify, but unlike a script will not specify in detail how to verify them. That information has to be available elsewhere e.g. in user manuals.
  • I always look for a document describing the system context in a top-down manner: What is the system doing, for who, and how? If it isn’t there, I don’t mind writing that document myself.
  • A catalogue of tools used in testing is often also very useful. Test tools are often not well documented (or not documented at all), and that can be a hurdle for new testers when they come aboard a project. A well written ”tips and tricks for testing system X” will get them up to speed faster and can act as a platform for sharing knowledge about testing specific parts of the system. I like Word documents for their self-contained’ness, but a Wiki could be better in many situations – the important thing is that such a document is actively maintained.

What are your preferred test documents?