test education – ASYM APS – IT Quality and Safety

Teaching Testing to Teenagers

I had the opportunity of being teacher for my son Frederik’s 7^th grade class today at Kvikmarken. The school had decided to send the ”real” teachers on a workshop together, so the job of teaching the children was left to volounteering parents. I’m enthusiastic about learning, so felt obliged to volunteer.
I gave them a crash course in software testing.

Software on punched paper tape from the mid 60's — Software on punched paper tape from the mid 1960's

These boys and girls are really smart. They quickly grasped an understanding of what software testing is about: Exploring and learning. Remember, they are brought up with laptops, mobile phones, and the internet, so they’ve learned to cope with all the shortcomings and bugs that come with today’s technology. I could have expected them to be blind of bugs. But no, they do see them – and they are splendid explorers and learners.
I started my lesson with a flash back 40-50 years ago when computers were as big as a classroom and software was written on note paper and stored on punched tape. I compared this with today’s state of the art commodity computer: An iPad, smaller than a school book – and a lot more powerful than computers then.
Complexity has increased by a factor of at least one million – and it shows! (Though I should add that the evolution of software development tools and methods has also improved a good deal.)
They now know what a bug is, why there are bugs in software, what testing is about and how testing is a learning and discovery process. They also have an idea of why we’re testing – they particularly liked this one: We test because it feels good to break stuff!
Finally, the had a chance to prove their collective exploration abilities in a testing exercise. They did splendidly!
The last slide of my presentation contained a quote from James Bach on Twitter yesterday, words of a true tester on bug hunt using his senses:

Say “it looks bad” and I hear you.
Say “it smells bad” and I taste BLOOD.

I’m happy that Frederik’s classmatetes liked my lesson: ”Great presentation, thanks!”, ”Wow, testing is fun!”, ”You’ve got a really cool job!”
I think there’s good reason to expect software engineering to improve a lot when these boys and girls get to be responsible for software engineering!

When Niels Bohr and his team of geniuses at his institute in Copenhagen developed quantum physics, they fundamentally changed science by proving wrong an assumption dating very far back in science: That everything happens for a reason. In science, building and verifying models to predict events based on knowledge of variables is an important activity, and while this is still meaningful in certain situations, quantum mechanics proved that on the microscopic level of atoms and particles, things don’t happen for a reason. Put in another way: You can have effects without cause. In fact, effects don’t have causes as such at this level.
This means that in particle physics, you cannot predict precisely what’s going to happen, even if you know all variables of a system. Well in fact you can, but only if you’re prepared to give up knowing anything about when and where the effect will happen.
This is counterintuitive to our daily understanding of how the world works. But there’s more: According to quantum physics, it is impossible to seperate knowledge of variables of a system from the system itself. The observation of the system is always part of the system, and thus changes the system in an unpredictable way.
If you find this to be confusing, don’t be embarrassed. Even Einstein never accepted this lack of causality.
Bohr was a great scientist, but he was also a great philosopher. He did a lot of thinking about what this lack of causaility and the inseperability of observation from events would teach us about our understanding of nature. On several occasions he pointed out that even on the macroscopic level, we cannot ignore what is happening on the atomic and particle level. First of all because quantum physics did away with causality as a fundamental principle, but also because quantum effects are in fact visible to us in our daily macroscopic life: He used the example of the eye being able to react on stimuli as small as those of a single photon and argued that it is very likely that the entire organism contains other such amplification systems where microscopic events have macroscopic effects. In some of his philosophical essays he points out how psychology and quantum mechanics follow similar patterns of logic.
So does testing. In software testing we are working to find out how a compuster system is working. Computers are algorithmic machines designed in such a way that randomness is eliminated and data can be measured (read) without affecting the data, but the programs are written by humans and are used by humans, so the system in which the computer program is used is both complex and inherently unpredictable.
We’re also affecting what we’re testing. Not by affecting the computer system itself, but by affecting the development of the software by discovering facts about the system and how it works in relation to other systems and the users.
In some schools of software testing, the activity is reduced to a predictable one: Some advocate having “a single point of truth” about what is going to be developed in an iteration, and that tests should verify that implementation is correct – nothing more. They beleive that it is possible to assemble “all information” about a system before development starts, and that any information not present is not a requirement and as such should not be part of the delivery.
That is an incorrect approach to software engineering and to testing in particular. Testing is much more than verification of implementation, and the results of testing are as unpredictable as the development process itself is. We must also remember that it is fundamentally impossible to collect all requirements about a product: We can increase the probability of getting a well working system by collecting as much information as possible about how the system should work and how it is actually working (by testing), and comparing the two, but the information will always be fundamentally incomplete.
Fortunately we’re not stupid. It is consistent with quantum physics.
Studying the fundamental mechanisms of nature can lead to a better understanding of what we are working with as software engineers and as software testers in particular.