Website testing and feedback

I was recently asked by a company to give them some post-launch feedback on their new website.

This kind of website testing is an important step before a product is unleashed upon the unsuspecting public, and whenever I can I am happy to supply my expert advice.

As I was juggling between the questionnaire and the website I was testing, it occurred to me how similar this process was to the old website testing process on LiteTest (where you had to look at the website and then answer the questions), and how unwieldy this process is for doing just that, website testing.

What was even more tricky in the whole process was being able to provide good feedback in the questionnaire. As I was going through the testing process (see How To Test for some hints on doing website testing and giving feedback in this "black box" fashion) I found my self completely avoiding the feedback form and writing all the feedback in Notepad and then pasting it in to the text boxes in the questionnaire.

I quickly answered the ratings from memory, and then pasted all my site testing feedback into the tiny little text box at the bottom of the questionnaire.

The part that was most important in this process was left as a tiny text box at the end of the page. This leads me to the point I want to make with todays post:

When creating a site test questionnaire like this, it's important to analyse what you value the most and what is most important to your website at this stage in it's life cycle.

It's not finding out if 200 people thought the first page had a good design or not, it's more important to know what exactly they thought about the design, by providing textual feedback.

Ponder this scenario: you ask 200 people to rate your websites navigation and everyone gives it a score of 2 / 5 (where 5 is the best and 1 is the worst). This will tell you that 200 people think you should definitely work on your navigation. This is valuable when trending the data over time and releases, so it is really valuable data!

But what if only 2 people gave you feedback, and said "All your links in the menu are broken and all the other links in your website takes the user away from the website without opening in a new window". Now this is valuable site test feedback!

In the first scenario, you know that the website needs to improve it's navigation (I used a quite obvious case here so perhaps the author would have found this quite quickly, but imagine if this was buried on a page that didn't get much attention from the developer but would get lots of hits from the general public). This tells you that frankly, the navigation is "knackered".

In the second scenario, two users (even one would have done it in this case) told you that your navigation was bad/broken, as well as letting you know what the problem was!

This is more important than gaging the number of users who likes your navigation and gives it a rating.

So to conclude this post:

It's more important with individual feedback when looking for initial improvements and in the beginning of integrating a website test system (or just collecting feedback).

Ratings and other variable data is more important in the long run, when you want to do trending and compare your website release by release.

So when you start looking for feedback for your website, and you do your site testing, remember to put more importance on textual feedback, and do these tests often!

It's better to have a few tests done now and then than put all the horsepower into one test (as can be understood from this article here about the rule of 5 testers)