{"id":310,"date":"2011-04-13T10:03:33","date_gmt":"2011-04-13T08:03:33","guid":{"rendered":"http:\/\/blog.asym.dk\/?p=310"},"modified":"2011-04-13T10:03:33","modified_gmt":"2011-04-13T08:03:33","slug":"the-problem-with-test-documentation","status":"publish","type":"post","link":"https:\/\/www.asym.dk\/index.php\/2011\/04\/13\/the-problem-with-test-documentation\/","title":{"rendered":"The problem with test documentation"},"content":{"rendered":"<p>The <a title=\"Link to the Agile Manifesto\" href=\"http:\/\/agilemanifesto.org\/\" target=\"_blank\" rel=\"noopener noreferrer\">agile manifesto<\/a> explicitly values \u201dworking software over comprehensive documentation.\u201d In test, this means that actual testing is valued over test documentation. I would have put it this way: Focus is on the quality of the <em>product<\/em>, not on the quality of the <em>documentation <\/em>of the product.<br \/>\nWe can probably all agree that it&#8217;s more fun to test and <em>explore <\/em>software than writing documentation. But it would be <em>too <\/em>radical, if we skipped writing documentation at all!<br \/>\nI think, however, that we testers should be more critical about the documentation we actually do produce, and that we should look for ways to <em>improve <\/em>the documentation we&#8217;re making.<br \/>\nThe problem with documentation is not that too much time is spent writing it instead of actually testing the product. The problem is that the quality of the documentation is often just not good enough.<br \/>\nMost organizations have standards for test documentation and require their testers to write specific types of documents based on mandatory templates.<br \/>\nTemplates are generally a good thing, but they can be problematic if you limit your writing process to \u201dfilling in the gaps.\u201d A good document contains useful information to the reader, so the most important step in the writing process is finding out what information is actually needed.<br \/>\nNot all sections of a template can be equally useful in all contexts (or useful at all), and very often you need to document things which there has not been left space for in the template.<br \/>\nBut, finding out what to document is not trivial. Basically, it requires that you know the context of the project you are working on. Determining context can be difficult if you are new on a project, but asking questions can give you answers which will help you <em>define <\/em>the context.<br \/>\nHere are some questions, which I find useful:<\/p>\n<ul>\n<li><em>Who<\/em> will be reading the \tdocument? <em>How<\/em> will they be reading it? The challenge is to \tinclude content which the actual readers will find useful and to \tstructure the content in a way so that they can find that \tinformation.<\/li>\n<li><em>What <\/em>information are they \tlooking for? Try to get people to answer in concrete terms rather \tthan using abstract vocabulary. The problem with written \tdocumentation is that stake holders will often not read it \u2013 most \ttesters seem to prefer to explore systems on their own rather than \treading documents, and managers will often not have \ttime to read everything, but will only check the headline and \tcertain details. But if readers get what they look for, chances are that they <em>will<\/em> read the document.<\/li>\n<li>What kind of analysis do I need to \tcarry out on the system to better understand it and test it? Writing \ta document about the system can often help me understanding it. I \twill discover missing knowledge and knowledge which I didn&#8217;t see \tinitially. The writing process is part of the analysis.<\/li>\n<li>Are there regulatory requirements \twhich specify that testing should be documented to a certain detail \tand in a particular way? In this case, the test documentation is \tactually a product of the project.<\/li>\n<li>Should the document assist \tknowledge transfer once I&#8217;m no longer on the project? In that case, \tthe big question is what kind of knowledge should be \u201dtransferred.\u201d<\/li>\n<\/ul>\n<p>I checked the section about documentation in Kaner, Bach and Pettichord&#8217;s <a title=\"Link to the page about the book on Amazon.com\" href=\"http:\/\/www.amazon.com\/Lessons-Learned-Software-Testing-Kaner\/dp\/0471081124\" target=\"_blank\" rel=\"noopener noreferrer\">Lessons Learned in Software Testing<\/a> a few days ago. They have a better and longer list of context-free questions to ask about documentation which will help you find out what kind of documentation you need. They also list a few references to other lists of useful questions, so I suggest you take a look at that too.<br \/>\nAre there any specific types of documentation which is particularly useful? Indeed, there is:<\/p>\n<ul>\n<li>Mind maps are becoming increasingly popular with testers, but &#8216;old style&#8217; software diagrams like swimlane diagrams, state diagrams, and flow charts etc are still working well. The problem with mind maps is that they are often personal to the author of the map and not very easy to <em>read<\/em>.<\/li>\n<li>Test scripts can be very useful in initial stages of <em>knowledge transfer<\/em>: A script can help another tester finding out how a certain procedure is performed in the system. However, a script will by itself tell the tester anything about the context of the script, and this is something which is often missed: Knowledge about a system under test is much more than knowing <em>how <\/em>to do things.<em><\/em><\/li>\n<li><em>Check lists<\/em> are actually much more useful than the name implies: A check list will list things to verify, but unlike a script will not specify in detail how to verify them. That information has to be available elsewhere e.g. in user manuals.<\/li>\n<li>I always look for a document describing the system context in a top-down manner: What is the system doing, for who, and how? If it isn&#8217;t there, I don&#8217;t mind writing that document myself.<\/li>\n<li>A catalogue of <em>tools<\/em> used in testing is often also very useful. Test tools are often not well documented (or not documented at all), and that can be a hurdle for new testers when they come aboard a project. A well written \u201dtips and tricks for testing system X\u201d will get them up to speed faster and can act as a platform for sharing knowledge about testing specific parts of the system. I like Word documents for their self-contained&#8217;ness, but a Wiki could be better in many situations \u2013 the important thing is that such a document is <em>actively maintained<\/em>.<\/li>\n<\/ul>\n<p>What are your preferred test documents?<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The agile manifesto explicitly values \u201dworking software over comprehensive documentation.\u201d In test, this means that actual testing is valued over test documentation. I would have put it this way: Focus is on the quality of the product, not on the quality of the documentation of the product. We can probably all agree that it&#8217;s more [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[10,26,34,51,98],"_links":{"self":[{"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/posts\/310"}],"collection":[{"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/comments?post=310"}],"version-history":[{"count":0,"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/posts\/310\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/media?parent=310"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/categories?post=310"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.asym.dk\/index.php\/wp-json\/wp\/v2\/tags?post=310"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}