Website testing and feedback

I was recently asked by a company to give them some post-launch feedback on their new website.

This kind of website testing is an important step before a product is unleashed upon the unsuspecting public, and whenever I can I am happy to supply my expert advice.

As I was juggling between the questionnaire and the website I was testing, it occurred to me how similar this process was to the old website testing process on LiteTest (where you had to look at the website and then answer the questions), and how unwieldy this process is for doing just that, website testing.

What was even more tricky in the whole process was being able to provide good feedback in the questionnaire. As I was going through the testing process (see How To Test for some hints on doing website testing and giving feedback in this "black box" fashion) I found my self completely avoiding the feedback form and writing all the feedback in Notepad and then pasting it in to the text boxes in the questionnaire.

I quickly answered the ratings from memory, and then pasted all my site testing feedback into the tiny little text box at the bottom of the questionnaire.

The part that was most important in this process was left as a tiny text box at the end of the page. This leads me to the point I want to make with todays post:

When creating a site test questionnaire like this, it's important to analyse what you value the most and what is most important to your website at this stage in it's life cycle.

It's not finding out if 200 people thought the first page had a good design or not, it's more important to know what exactly they thought about the design, by providing textual feedback.

Ponder this scenario: you ask 200 people to rate your websites navigation and everyone gives it a score of 2 / 5 (where 5 is the best and 1 is the worst). This will tell you that 200 people think you should definitely work on your navigation. This is valuable when trending the data over time and releases, so it is really valuable data!

But what if only 2 people gave you feedback, and said "All your links in the menu are broken and all the other links in your website takes the user away from the website without opening in a new window". Now this is valuable site test feedback!

In the first scenario, you know that the website needs to improve it's navigation (I used a quite obvious case here so perhaps the author would have found this quite quickly, but imagine if this was buried on a page that didn't get much attention from the developer but would get lots of hits from the general public). This tells you that frankly, the navigation is "knackered".

In the second scenario, two users (even one would have done it in this case) told you that your navigation was bad/broken, as well as letting you know what the problem was!

This is more important than gaging the number of users who likes your navigation and gives it a rating.

So to conclude this post:

It's more important with individual feedback when looking for initial improvements and in the beginning of integrating a website test system (or just collecting feedback).

Ratings and other variable data is more important in the long run, when you want to do trending and compare your website release by release.

So when you start looking for feedback for your website, and you do your site testing, remember to put more importance on textual feedback, and do these tests often!

It's better to have a few tests done now and then than put all the horsepower into one test (as can be understood from this article here about the rule of 5 testers)

The importance of feedback

Do I need feedback?
Every website that wants to improve needs feedback.
A quick analysis of your website by a few users can give you a good insight and even scope out a few problems you did not know you had.

You only need as little as 5 users or less to get a good grasp of where you might need to improve, or to emphasise the positive sides of your product (source: http://www.useit.com/alertbox/20000319.html)

Feedback doesn't have to be negative. It's very common to worry about feedback being too harsh and that scares a few people into denial.

Feedback lets us improve, and the more we improve the site, the better it will be. The better the website is the more sales it will make!

How can feedback help me and how do I work with it?
What can we do with all the feedback we receive?
Organise the feedback into groups and give it weight.
As an example:
3 users think that your navigation is lacking and your website could improve its main navigation menu.
1 user thinks that your overall colour scheme is disrupting.
2 users think that your registration process is too long and difficult.

Organise the feedback into work packages.
We start with the 3 users’ opinion on the navigation. This is not a lot of work for one of the developers, and with a little bit of thought and work this can be reworked almost instantly. We put this on top of the priority list because a) it is the most common issue and b) it's quick to fix.
The second one down the list will be the one user who thought the overall colour scheme was bad. After some brief research we find that the colours we picked for the website is not ideal so we put this as the second work package.

Why did we not put the two user feedback (the registration issue) as the second work package? Because the registration process (in this hypothetical scenario) is tied in to a large portion of the website and the cost of development in time and money would be too high to give enough return on the investment. We put this as a "nice to have" issue. We save this, because we want to think about this issue for the next big development of the website.

In short: by making use of feedback we can improve our website. A simple analysis now, and some clever handling of the feedback, can lead to long term savings and improved business.

New version of LiteTest

A new version has finally been released. A comlpete facelift but things have changed under the hood as well.

We have moved to MVC amongst other things, but let's talk about some new features first:

Some of the bigger changes and aditions we have now, are the section we made for design feedback.

Users can now upload design concepts and get them rated as well as get notes on them before they go into production. This step is essential. It's all well and good to get feedback on your website when it's already created, but what if you want to know more before you spenou thed time and money on creating it?

This will give you a good starting ground to catch the most prominent design errors before you create your website.

Another big change we have made was to the actual test process. Many of our users found it tedious to move back between the test page and the actual website they were testing.

We found that the best way to accomodate this, was to move the testing process to the actual website they were testing. Now users can browse the website they are giving feedback to, while being able to provide their analysis.