Photo by: everyones idle

Recently, VisualWebsiteOptimizer.com became a Feedback Army customer. They make a product to aid the A|B testing process and use the data you have to increase sales on your site. Great stuff.

You can read more about their experience using Feedback Army over at “How a $10 investment gives website improvement ideas worth thousands“.

Their post is interesting given the nature of their business and there is an insight worth taking from it:

Usability Testing provides you with hypothesis about what to fix on your site and sometimes why (e.g., users are frustrated because they can’t do [blank]). Split-testing and other data-driven techniques are a means for you to experiment with these hypothesis.

If you’re not sure where to start when improving your site, get it looked at. Take some of the ideas and come up with an experiment to test them. Rinse, lather, and repeat.

Photo by: Martin Lysholt Nielsen

Recently I came across a post, from David Jones at i-proving.ca, discussing Feedback Army and UserTesting.com applied to a project. David’s first experience with Feedback Army was disappointed but hopeful. Don’t worry, this story has a happy ending and it shows how to get the most out of Feedback Army.

Do Users Understand My Site?

Here is a snapshot of the post:

I tried this out on http://www.devcreek.com. While developing it we were concerned about usability but there was no process in place to test and refine this beyond gut reaction, so there is lots of changes that could be recommended. On the down side, it is not really a site for the general public but rather for development teams or customers working with a development team on a project, so how would a common web user provide much useful feedback?

David used the default Feedback Army questions. These questions cast a wide net and give the reviewers a chance to point out glaring issues they find. They’re useful as a first run to learn how outsiders perceive a site but they’re not good for zeroing in on an issue and getting depth.

David was happy with three of the ten responses and they brought out issues his team had debated before. He also noted he received his responses within four hours. That said, he was disappointed in the others as they didn’t lead to a helpful insight.

They haven’t gotten very far browsing the site, and their comments seem quite generic. No magic here, still we have talked about the video tutorial before, someone else recommended centering the content, and so on. So it has some merit.

David made the feedback he received available on his blog.

A Usability Test

I read the comments from David’s post and I was delighted to see David tried Feedback Army again. He read through the Usability Testing HOWTO article on this blog and tailored the “Is my Site Easy to Use” template to create a navigation test for his site. Here are the questions he asked:

Please visit my site and try to find the Deployment Summary report for the Panacea project. Get as far as you can and then answer these questions:

1. How far did you get? What slowed you down?
2. What would you change about this process?
3. What would you keep the same about this process?

After running this test David noted he found the results from these questions much more promising. He generously provides the responses he received here.

Here are David’s tips for getting the most out of Feedback Army:

• Provide an introduction to the project and your goals for the feedback at the beginning of the request.
• Frame the questions as a series of tasks that users should follow
• Create an initial question asking the user to state how far they got and where they gave up

This feedback was much more useful to David and allowed him to extract some quantitative information his team could act on.

Final Thoughts

This story is interesting because it shows two different ways you can use Feedback Army. You can go for a broad approach and learn how people see your site or you can get very specific. Either way, Feedback Army is a tool to bring people to your site and to get them to speak to you directly. What will you ask?