{"id":736,"date":"2015-05-21T21:09:51","date_gmt":"2015-05-21T19:09:51","guid":{"rendered":"http:\/\/blog.asym.dk\/?p=736"},"modified":"2015-05-21T21:09:51","modified_gmt":"2015-05-21T19:09:51","slug":"are-you-playing-the-russian-roulette-learning-from-failure","status":"publish","type":"post","link":"https:\/\/www.asym.dk\/index.php\/2015\/05\/21\/are-you-playing-the-russian-roulette-learning-from-failure\/","title":{"rendered":"Are you playing the Russian roulette? Learning from failure"},"content":{"rendered":"

I think most (if not all?) testers have witnessed situations like this: A new feature of the system put into production, only to crash weeks, days or just hours later.
\n\u201dWhy didn’t anybody think of that?!”
\nTruth is, quite often, somebody did actually think about the problem, but the issue was not realised, communicated or accepted.
\nBelow is the story about the space shuttle Challenger accident in 1986.
\nDisaster…<\/b>
\nTwentynine years ago, space shuttle Challenger exploded seven minutes into the flight killing the seven astronauts aboard.
\nTheoretical physicist Richard Feynman was a member of the accident commision. During the hearings he commented that the whole decision making in the shuttle project was \u201da kind of Russian roulette\u201d.
\nThe analogy is striking. Russian roulette is only played by someone willing to take the risk to die.
\nI don\u2019t know anyone who deliberately want to play the Russion roulette, so why did they play that game?
\nFeynman explains: [The Shuttle] flies [with O-ring erosion] and nothing happens. Then it is suggested, therefore, that the risk is no longer so high for the next flights. We can lower our standards a little bit because we got away with it last time…. You got away with it but it shouldn’t be done over and over again like that.<\/i>
\nThe problem that caused the explosion was traced down to leaking seals in one of the booster rockets. On this particular launch ambient temperatures were lower than usual and for that reason the seals all failed. The failed seals allowed very hot exhaust gasses to leak out of the rocket combustion chamber, and eventually, these hot gasses ignigted the many thusand litres of higly explosive rocket fuel.
\nChallenger blew up in a split second. The seven astronauts probably didn’t realise they were dying before their bodies were torn in pieces.
\nIt was a horrible tragedy.
\nChapter 6 of the official investigation report<\/a> is titled: \u201dAn accident rooted in history.\u201d<\/b>
\nThe accident was made possible because of consistent misjudgements and systematically ignored issues, poor post flight investigations, and ignored technical reports. The accident was caused because three seals failed on this particular launch, but the problem was<\/strong> known and the failure was made possible because it was systematically ignored<\/strong>.
\nThe tester’s fundamental responsibilites<\/b>
\nAs a tester, I have three fundamental responsibilities:<\/p>\n

    \n
  1. Perfom the best possible testing in the context<\/li>\n
  2. Do the best possible evaluation of what I’ve found and learnt during testing.\u00a0\u00a0Identify and qualify bugs and product risks.<\/li>\n
  3. Do my best to communicate and advocate these bugs and product risks in the\u00a0organisation.<\/li>\n<\/ol>\n

    The Challenger accident was not caused by a single individual who failed detecting or reporting a problem.
    \nThe accident was made possible by systemic factors, i.e. factors outside the control of any individual in the programme. Eventually, everyone fell into the trap of relying on what seemed to be “good experience”. The facts should have been taken seriously.
    \nA root cause analysis should never only identify individual and concrete factors, but also systemic factors which enabled the problem to survive into production.
    \nChapter 6 of the Challenger report reminds me that, when something goes wrong in production, performing a root cause analysis is a bigger task than just finding out the chain of events that lead to problem.<\/b>
    \nMany thanks to Chi Lieu     @SomnaRev<\/a> for taking time to comment early drafts of this post.<\/em>
    \n

    \"Photo<\/a>
    Photo of the space shuttle Challenger accident Jan. 28, 1986. Photo credit: NASA<\/figcaption><\/figure><\/p>\n","protected":false},"excerpt":{"rendered":"

    I think most (if not all?) testers have witnessed situations like this: A new feature of the system put into production, only to crash weeks, days or just hours later. \u201dWhy didn’t anybody think of that?!” Truth is, quite often, somebody did actually think about the problem, but the issue was not realised, communicated or […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[14,26,63],"_links":{"self":[{"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/posts\/736"}],"collection":[{"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/comments?post=736"}],"version-history":[{"count":0,"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/posts\/736\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/media?parent=736"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/categories?post=736"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/tags?post=736"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}