Usability testing is done for roughly the same reason as other kinds of testing: To discover new knowledge about the system under test. In this case knowledge about how users work with the system.
But Usability Testing is a very different discipline from ordinary system testing.
In my opinion, the one thing that makes usability testing different from system testing is, that it never discovers absolute facts about the system.
Instead, a usability test will only say something about how the system works in relation to someone – and this someone is a person – or persons. And as you have probably experienced several times in your life, real persons aren’t absolute, predictable, or static – they’re quite dynamic and you never really know what to expect. Usability testing is a bit like integration testing against a remote system, which keeps changing, even while you test it!
Another aspect is that it’s much more important, yet also more difficult, to describe the context in which the system is to be used and the test is executed in.
I’ll illustrate that with a basic example: Many corporations implement systems for internal use which are not really user friendly, but since the employees get paid for using them, they don’t mind this – and once they’ve used the system for a few weeks, the system may have become an efficient work tool. An opposite example is most computer games, which are deliberately designed to be inefficient, but are usually easy to learn and use. These two systems or applications may be used on the same day by the same person, but in different situations, of course.
Setting the context is not always possible, though. For example is that most people will only respond to things which they understand: Chances are that if you had tested Facebook on users in 1995, they wouldn’t like it at all because they would not understand what it was to be useful for.
Essentially, I feel that the tester must be much more conscious about what he is doing and how it is affeting the test results. I actually believe that the description of the context itself and the way you describe it is as influential on the results you get, as the system and the users themselves.
Yes, testing usability can be very challenging!
Categories
0 replies on “Usability testing is different”
I liked your ‘poetry’ piece as well. You might enjoy these articles of mine, I have been thinking along the same lines:
http://www.stickyminds.com/sitewide.asp?ObjectId=16433&Function=DETAILBROWSE&ObjectType=COL&sqry=*Z%28SM%29*J%28MIXED%29*R%28relevance%29*K%28simplesite%29*F%28chris+mcmahon%29*&sidx=1&sopp=10&sitewide.asp?sid=1&sqry=*Z%28SM%29*J%28MIXED%29*R%28relevance%29*K%28simplesite%29*F%28chris+mcmahon%29*&sidx=1&sopp=10
http://pragprog.com/magazines/2010-11/writing-and-performing
(and I’m @chris_mcmahon on Twitter, if you use that)
Thanks a lot for the reference to your articles. That’s a interesting debate you’re opening there.
Engineering is an applied art, there’s little doubt about that, and we’re always deeply buried in the fundamental problem that we’re inventing the problems we’re solving as we go along.
[…] Very good blog post about Usability. As a tester, do you care about it? #testing news: Usability testing is different http://bit.ly/dEx4Fs […]
Nicely put.
Regarding this statement: “Usability Testing is a very different discipline from ordinary system testing”
I wonder if a word like “skill” or “task” would be a better substitute for “discipline”. On one hand, other non-functional areas like Performance, Reliability, or Security could be treated the same way, so it’s really a testing skill…
But on the other hand, I think I see software testing in general growing into specialized roles, where usability tester could be one.
Anyway – nice post. Look forward to reading more.
Thanks for your comment 🙂
I think usability testing has traditionally been done mostly outside of the testing community and that it’s a new trend to combine the two. It probably depends on the project which is the best approach.
I see your point about using the words “skill” or “task”, I think I used “discipline” to highlight that I tend to see usability testing to be more of a CHI-engineering task, than a testing task.
I agree fully regarding usability that “it never discovers absolute facts about the system.”
But I think the same goes for other system testing as well.
A system is reliable to someone, the functionality is adequate to someone etc.
Usability testing is different because it is about usability, and also because it is more obviously subjective.
Very good point, Rikard.
I think I meant “objective”, not “absolute”, thanks for clarifying that!