During the first three years of my testing career, the bulk of my time was spent writing test scripts, planning each step out individually, stating what I would do and what I expected to see. I would then run through these scripts, usually finding a couple small bugs here or there. Only when I had finished all of this, could I get on and actually do some testing. To explain what i mean, the test scripts, both writing and running, isn’t actually software testing; it’s validation.
Although validation is a massively important part of software testing (and also the easiest to do), it also happens to be the part that generally uncovers the least bugs. During those first three years, the most bugs and the most valuable, were discovered after I’d run through and ‘passed’ the test script, but had returned to explore the software, just playing with it, using my imagination.
Most of my time since then has been spent on exploratory testing, I’m happy to say. Sure, I do some validation along the way, but now it’s just 5% of my job. The question that often comes with exploratory testing is how to document what’s been done. The truth is that most exploratory testing isn’t really documented at all. It’s very easy to get carried away with the testing itself and let the test notes become an afterthought.
Who reads them?
The reason people don’t tend to prioritize writing their test notes is because, unless you leave the job and your replacement needs to step into your shoes, nobody else is going to read them. This may sound like a dramatic statement here, but it’s not. The project managers who demand that every User Story comes with test notes after it has been marked as done, don’t read them, neither does the Test Manager who preaches about their importance. The fact is, no one does.
The reason I started that paragraph with ‘nobody else’ is because although the people that say they want you to produce test notes won’t actually read them, the person who does, or at least should be reading them, is you.
The only way an organisation can actually benefit from notes is if the tester who carried out the tests can use them to refresh their memory, such as if they need to pick up a session again, or try to find out what had previously been done, if they need to recreate an issue found in the session, and of course, if your potential replacement needs to find out what’s been going on.
I say ‘only’, but they are extremely valuable if done in a useful way. Beyond the commitment to writing the notes themselves, the most important part of writing effective test notes is writing them in a format that works for you, which might be one of the four ways listed below that I’ve seen people use, and find effective for documenting their test notes.
- Text Editor
- Spreadsheet
- Mind Map
- Pen & Paper
Text editors are the simplest and most obviously note-taking tools, and Mind Maps will be a familiar concept to all except those who have been living under a rock for the past decade, or simply working in the public sector.
Spreadsheets fill me with dread because Excel was where I spend three years writing test scripts, but they are in fact a very powerful and useful way to keep track of what you’ve been doing. To some, pen and paper may seem somewhat antiquated, but that’s the point I’m leading into.
Do whatever works for you.
Some people may find a physical notepad is the best way to make their test notes. Some people even draw doodles to accompany their text to help them to understand what’s going on. It doesn’t matter how new or old the method is, or what’s in vogue at that moment, your test notes method has to be one that suits your way of doing things.
Prescribed Format
Now we come to one of the biggest problems I’ve seen recently across the industry, that of the prescribed test note format. Managers and team leaders are declaring that one format should be used by all, normally because it’s what works for said manager or team leader.
This makes zero sense.
As an example, you might be the manager of a team of testers, and you find spreadsheets to be the most effective way to write your test notes. Because it suits you, you decide that all of your team should also use spreadsheets to log their testing. Sure, it might not suit everyone perfectly, but most people will probably get on fine with it.
But then there are a couple of testers in the team that it just doesn’t click with. It isn’t a natural fit with the way they test. The result of this will likely be poor test notes, and a tester who resents the process. Nothing can damage the motivation of a tester more than forcing an unnecessary process on them. As a manager, you could make test notes compulsory, sure, but you’ll only damage the morale of the team and the quality of testing if you enforce one format for all.
As an industry, we’ve made so many forward steps away from the over-structured, creativity-devoid days of test scripts. Let’s not now take a backward step by hamstringing our testers with valueless constraints like a prescribed format for their test notes.
