March 2012 - Phases of Testing

As it becomes more instinctive for me to decide what part of our application to test next, e.g. smoke testing a new kit, what bug fixes to concentrate on, or new story features to explore; I can start to focus more attention on exactly how I'm going about my testing.  This includes the preparation/planning, the main phase of testing itself, and the reporting of my findings afterwards.


Pre-Testing
I have started to get into the habit of using Mind Maps (XMind) to get the main topics down in black and white.  Although it's a very straight forward premise I find it very useful as I can easily forget even the brightest idea pretty quickly if I don't write it down.  Mind Maps also double as useful information for future reference.  So anyone who wishes to test a feature they haven't seen before can view the Mind Map for test ideas to get them started. I find them of particular use when testing a brand new feature or area of the product with which I am less familiar.


Testing
When I first started testing I was paying more attention to what was going on in front of me on the screen rather than making notes of what I saw.  This was very bad practice as memory tends to be a lot less reliable than a written word.  The smallest changes in steps to reproduce a bug or unusual behaviour will mean you can no longer produce the desired result.  However memorable it seems at the time there is no substititute for recording your findings.  I have now started to try and make an effort to make notes when I test now (using Notepad++), especially when testing something new in an exploratory way as it could prove invaluable when trying to recall steps in producing any bugs found.
I also tend to use Firefox a little more than other browsers so that I can have firebug running in the background.  Then I can include the firebug error output along with my description when I make a bug report, which can sometimes be useful for developers in pinpointing the origin or cause of the defect.
Screen grab tools and videos are also very useful to back up a description and can often make things a lot clearer than a written description alone in visualising what's gone on.


Re-Testing
Under this heading I include retesting in the sense of Regression testing and also Re-Testing bugs which are believed to have been fixed.
When I run regression tests I do not make as many notes since the paths followed are already well trodden and the steps taken are already recorded step by step.  However I use the same methods for recording findings if a bug is found whilst running regression tests.
When Re-Testing bug fixes it can be very easy to fall into the trap of only testing the 'happy path', or the scenario in which the bug was originally found.  It's also important to test around the edges of the happy path and test the bug scenario with similar but not identical variations in order to be confident the bug(s) have been fully fixed and will not resurface later on.  The number of different paths you can test is obviously time dependent but the more you have time for the better.


Reporting Test Findings
Once testing is complete it is important to quantify what areas of the product have been tested and in what way.  This is especially important when describing how to reproduce a bug as if this is not accurate then the developers may not be able to reproduce it for themselves.  The steps to reproduce are also important when the bug is fixed and you need to repeat the original steps used to create the bug.  You must also make sure you include all relevant setup information such as the browser which was used, the kit version, user credentials, scenario, and any accompanying event logs.  This will give the developers the best chance of identifying and reproducing the bug and reduces the chance they will need to come back to you with further questions.  It should also minimise the chance of the hearing the well known 'It works on my machine' response.