{"id":277,"date":"2011-04-02T11:10:35","date_gmt":"2011-04-02T09:10:35","guid":{"rendered":"http:\/\/blog.asym.dk\/?p=277"},"modified":"2011-04-02T11:10:35","modified_gmt":"2011-04-02T09:10:35","slug":"acceptance-tests-are-not-enough","status":"publish","type":"post","link":"https:\/\/www.asym.dk\/index.php\/2011\/04\/02\/acceptance-tests-are-not-enough\/","title":{"rendered":"Acceptance tests are not enough!"},"content":{"rendered":"

Acceptance testing is a key method in Agile. One way of defining acceptance tests are Gojko Adzic<\/a>‘s \u201dSpecification by example\u201d paradigm which has gained quite a bit of momentum lately. I personally found it to be both refreshing and nice when I heard him present it at Agile Testing Days 2009, and I also found his book Bridging the Communication Gap<\/a> a nice read.
\n

\"Photo<\/a>
Gojko Adzic demonstrating communication gaps at Agile Testing Days 2009<\/figcaption><\/figure>
\nI’m sceptical of the concept of acceptance testing. Not because verification of agreed functionality is not a good thing, but because it tends to shift attention to verification instead of exploration.
\nThis will shift attention from testing to problem prevention. Is that bad, you ask? Isn’t it better to prevent problems than to discover them?
\nWell, most people think \u201dwhy didn’t I prevent this from happening?\u201d when problems do happen. Feelings of regret are natural in that situation and that feeling can lead you into thinking you should improve your problem prevention. And maybe you should, but more examples aren’t going to do it!
\nReal testing is still necessary.
\nTo explain why, I’ll consult one of the great early 20’th century mathematical philosophers: Kurt G\u00f6del<\/a>. In particular his first incompleteness theorem<\/a>. It says that no consistent system of axioms whose theorems can be listed by an “effective procedure” is capable of proving all facts about the natural numbers.
\nWhat does this mean to us?
\nIt means that we will never be able to list all things that can be done with this particular set of data.
\nA specification is a kind of listing of \u201dvalid things to do\u201d with data, thus G\u00f6del’s theorem teaches us that there are infinitely more things to a system than any long list of requirements. This also applies when the requirements are listed as examples.
\nIf you’re in the business to deliver products of only \u201dagreed quality\u201d to a customer, you can be all right only verifying things which are explicity agreed. If something goes wrong you can always claim: \u201dIt wasn’t in the specification!\u201d
\nBut if you’re striving for quality in a broader sense, verifying that the system works according to specifications is never going to be enough.
\nGojko has made a good contribution to agile. Examples can be useful and efficient communication tools, and if they are used correctly they can help making users and other interested parties better aware of what’s going on on the other side. His contribution can help bridge a communication gap<\/em>. It can also produce excellent input for automated unit tests.
\nJust don’t let it consume your precious testing time. The real testing goes far beyond verification of documented requirements!
\nIf you want to learn more about this, I recommend you sign up for one of the Rapid Software Testing<\/a> courses offered by James Bach<\/a> and Michael Bolton<\/a>.
\n
\"Photo<\/a>
Michael Bolton with one of the many interesting and challenging test exercises at the RST course<\/figcaption><\/figure><\/p>\n","protected":false},"excerpt":{"rendered":"

Acceptance testing is a key method in Agile. One way of defining acceptance tests are Gojko Adzic‘s \u201dSpecification by example\u201d paradigm which has gained quite a bit of momentum lately. I personally found it to be both refreshing and nice when I heard him present it at Agile Testing Days 2009, and I also found […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[9,10,12,26],"_links":{"self":[{"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/posts\/277"}],"collection":[{"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/comments?post=277"}],"version-history":[{"count":0,"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/posts\/277\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/media?parent=277"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/categories?post=277"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/tags?post=277"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}