Usability Testing Tips Feedback Army blog Wed, 05 Oct 2011 16:13:51 +0000 en-US hourly 1 Website Reviews from United States, Canada, Australia, and the United Kingdom Wed, 05 Oct 2011 16:13:32 +0000 rsmudge November 2011 will mark three years of Feedback Army. In the three years I’ve operated this service, I’ve received consistently positive feedback about the price point and quality of the reviews.

I have heard one repeated request though. Many of you have asked for the ability to get reviews from the United States or other countries where English is the primary language.

For a long time, I debated making this an option, as I know the price point is one of the advantages of Feedback Army. However, after running an experiment, I’ve confirmed that I can offer you reviews from reviewers from either the United States, Canada, Australia, and the United Kingdom at a price point close to what Feedback Army offers today.

To provide the same fast review turn-around that I’ve always offered, I’m raising the prices slightly to pay more per review.

Also, each review will now show you the city and country the review came from as well. This information is automatically determined.

Here are the new prices:

Package Total New Cost/Review Old Cost/Review
10 responses $20 $2.00 $1.50
25 responses $40 $1.60 $1.20
50 responses $75 $1.50 $1.10

As you can see, this slight price increase isn’t too far from the old model. The new 50 response package is the same price as the old 10 response package. And, remember that all reviews will now come from the United States, Canada, Australia, and the United Kingdom.

Enjoy and thanks for using Feedback Army!

– Raphael

]]> 0
A little blog downtime — oops Mon, 03 Oct 2011 20:41:05 +0000 rsmudge An automatic update to WordPress 3.3 recently broke the Feedback Army blog. The new WordPress required a PHP update and refused to display any pages until I updated. One of you was kind enough to drop an anonymous note letting me know about the error. Thank you. It was resolved within about three hours after receiving the note. — Raphael

]]> 0
The server broke! Ok, it’s fixed now. Tue, 14 Sep 2010 00:41:41 +0000 rsmudge Recently, Feedback Army started acting unstable. The service was going up and down like a yo-yo. I’ve finally narrowed the cause down to the metaturk feature. Metaturk is a collection of statistics about the “army” providing feedback. This data set has become a little unwieldly and loading it was causing the server to run out of memory.

I’ve turned the metaturk feature off and the Feedback Army server is stable again. No feedback was lost due to these issues. My apologies for any concerns this may have caused. Feedback Army is actively loved and here to stay.

If you have any questions, feel free to contact me.

– Raphael

Update 2200h: the metaturk feature is now restored. On further investigation, the problem was caused by a corrupt metaturk file. Deserializing it caused the server to go into an infinite loop. I’ve also verified (with a script) that no other files are corrupt.

]]> 0
Feedback Army Live: Presentation + Q&A Tue, 18 May 2010 13:21:25 +0000 rsmudge Last week I spoke about Feedback Army at the Startup Rockstars event in Arlington, VA. This was my first time speaking about Feedback Army in over a year, it was a lot of fun to make some points and see how folks responded to them. Here are the videos from the event.

The first video is a five-minute pitch about Feedback Army. You’ll learn what the service is (hopefully you know if you’re here), how customers use it, and what they think of it.

The second video features ten minutes of questions and answers. My favorite part is the surprise appearance of a Feedback Army reviewer / Mechanical Turk user. This definitely added spice to the event

Here are the questions fielded:

  1. Why does Feedback Army use a third-party service to recruit reviewers? (0:00)
  2. What is the target market for Feedback Army? (1:00)
  3. One nice thing about Feedback Army as a business (2:25)
  4. How does Feedback Army ensure quality of reviews? (3:00)
  5. Does Feedback Army let you pick reviewers by demographics? (4:30)
  6. How does Feedback Army recruit the reviewers? (5:45)
  7. I’m an anonymous mturker and I want to say… (6:50)
  8. Is Feedback Army for sale? (9:40)

If you want to see what Feedback Army is about live, you’ll want to peep this Q&A.

]]> 0
How a Simple Change Increased Revenue by Millions Sat, 24 Apr 2010 16:04:25 +0000 rsmudge When developing a project, some of us have many masters to answer to. The marketing people want a mailing list and data on the customers, the CEO wants flash and pizzazz, and users want something that lets them carry out their transaction with you as painlessly as possible. Sometimes, these wants are in conflict with each other.

I’d like you to head over to the $300 Million Button. It’s one of my favorite usability testing success stories. A large e-commerce site used usability testing to learn how users really felt about some of these requirements. This hard data led to changes that made the site better for users and revenue better for the business (doesn’t everyone win in this case?).

You can also use Feedback Army to get feedback of this type. If there is a controversial feature getting built, ask several people to try it and see how they respond. In this case, I’d write:

We’re testing the checkout process of our e-commerce site. We don’t expect you to pay but would like to know how you feel about it. Select an item from our store and get as far as you can during checkout. Then answer the following questions:

1. On a scale of 1-10, how likely are you to complete the checkout process? Why?
2. What slowed you down?
3. What could make it faster for you?
4. What part of the process did you like?
5. What part of the process annoyed you?

With these questions, you can find out how users feel about that particular process and take an appropriate action.

]]> 0
Give Users a Simple Exit Strategy Tue, 13 Apr 2010 23:27:14 +0000 rsmudge 2008-07-18-004

Photo by: Alex //Berlin _ as+photography

This past weekend I had a phone call from an employee of a company that connects travelers with those who have couches and want to earn extra income. This employee, we’ll call him Jordan, mentioned that someone had made a request on my Syracuse, NY apartment (one I haven’t lived in for over a year) and asked if I was going to respond to it.

The phone call caught me by surprise because two days ago, I visited Jordan’s website to disable my listing permanently. When I got there, I found myself a little lost. I couldn’t find a help feature from the account management screen. I then went to the homepage and saw an FAQ link at the bottom. Finding this I tried a few search queries until I learned that I needed to visit my calendar and select an option to disable my listing.

Boy did I feel dumb, the calendar is the first place I should have looked!

This story highlights what I want to write about today. When you develop a service, you should take into account what happens when a user stops using your service. For some services this is easy, nothing happens. The user’s content and account stays.

For other services, this detail is critical. In the case of Jordan’s service, when a user likes me stops using the service, it hurts. I’m the scourge of his service. He looks good when travelers get quick and courteous responses from those with couches. I no longer have a claim to the property with the couch I was renting. For Jordan, he may want to consider an active strategy to force people like me to exit without polluting his community. Maybe an email ping after a year of no account activity from someone with an active listing.

Sometimes having an exit strategy is not necessary but it builds good will. I particularly like what does. lets you export your data into an XML file that you can then import to another service. Knowing that I can sign up for an account, invest into making a blog on, and later get that data back is nice.

Now, ask yourself: “when a user stops using my service, what is their exit strategy?” If you can, help them out. If a user’s lack of exiting negatively affects your community, make sure you’re doing something to protect your business.

]]> 0
How I Market Feedback Army Wed, 07 Apr 2010 12:12:17 +0000 rsmudge Sell Video Games (FSOD)

Photo by: PinkMoose

Recently I was asked via email for my methods marketing my online business. I’ve tried a lot of things to promote Feedback Army. Here is a quick list of the methods I’ve tried and how well they worked for me.

Cold Emails

I’ve had a lot of success with cold emailing. To find prospects, I visit and search for tags related to usability testing and startups to look for blogs that might take interest in Feedback Army. I then visit the blog, view the about page, and see what the blogger has written about usability testing. Once I have this information, I contact the blog owner modifying the first sentence or two of a template email to say something about their site. I also include a coupon for the blog owner to try Feedback Army. I usually see a response from 3/10 emails, which isn’t too bad.

When cold emailing, ask for an opinion or feedback. Do not ask for a review. If the blogger wants to, they’ll review your site. If they give you feedback that makes your business better, that’s helpful too.

I usually target small sites, but even popular bloggers have helped me out in unexpected ways. I once contacted Andrew Lock, the producer of the excellent Help My Business Sucks show. He never wrote about Feedback Army but he did mention it at a conference and this led to new customers.

If you plan to use this tactic, I recommend you read Startup Marketing Advice from Balsamiq Studio. Peldi’s advice help put me on the right path here.

Start a Blog

Now that people are aware of Feedback Army, I’m trying to market it through my blog (hey, you’re here). You’ll notice that I post about once a week. My goal is to be consistent and open in my blogging. I find posts where I’m most vulnerable (read: open and honest) generate the most attention. I seed relevant posts on Hacker News. It’s hit or miss but when it hits, it’s worth it. A hit blog posts also leads to a sales spike for the current day and the day after. Sometimes it also leads to other blogs writing about Feedback Army.

Advertising and Analytics

I’ve tried ads on It took little effort on my part and it made me feel like I was doing something to market my business. However the party ended once I installed google analytics and started tracking “goals”. I noticed my ads weren’t converting to sales. After A|B testing variations of my ads, I decided to abandon and Google AdWords.

Before you buy any advertising, I recommend that you install Google analytics and set it up to track whatever you consider a conversion or goal. It’s easy to do and will help you decide if you’re seeing an ROI from your advertising.

When exploring advertising opportunities, make sure you set a time limit, a budget, and a have a means to measure the effectiveness of the ad.

Cold Calling

I’ve also tried cold calling.  To do this I decided on a target (small web development shops). I chose these smaller businesses as I felt I would be able to build rapport with them easily. I used google to put together a list of names and numbers. I also wrote a script to guide me through the beginning of the conversation. Before my first call I practiced this script with my cell phone held up to my ear until I felt confident and automatic saying it.

When cold calling, it’s important to know what your goal is so you can steer the conversation there. My goal was to get an email address so I could send a Feedback Army coupon worth ten responses. The callee wins and I win because I’m able to track how many coupons were redeemed and how many sales came from it.

The people I spoke with were pretty cool and the conversations weren’t awkward like I initially feared. Despite the favorable chit chats, the conversion rate from this experiment was terrible. For a startup with a low-cost product like Feedback Army, I do not recommend cold calling.

Affiliate Marketing

I tried setting up an affiliate sales program for my mechanical turk worker base. The idea was to give each worker a URL that they could refer people to. My software would pay a bonus to the worker when someone they referred bought feedback. I was very excited when I put this program in place. Unfortunately it ran into a few difficulties:

1) The profit margin on Feedback Army was too low to support an effective affiliate marketing program. I had several complaints that I wasn’t offering enough of a reward.
2) I believe recruiting the workers on Mechanical Turk was a mismatch as the workers didn’t entirely understand the program.

If you’re thinking of going this route, I recommend designing your price packages to accommodate this beforehand.

Search Engine Optimization

I’ve tried some white-hat SEO stuff. I still only get 10% of my traffic from search engines and it’s not a big money-maker for me. Someone once said if people aren’t searching for your business by name then it’s more important to focus on the product. Make sure some people are happy with what you’re doing before you focus on SEO. No organic growth is a bad sign.

That said, you should check your basics to make sure you’re not doing anything that will hurt you. If your business can accommodate it, look into finding a way users can share what they bought or build out the content on your page. For example, I’m planning to create a badge to let clients share their Feedback Army results on their sites. I don’t know how well it will do, but as you can see from this post–marketing is about trying a lot of things.

Make sure you visit and look for events in your area. I have tiny cards I printed via that I hand out to the folks I meet. In the past I haven’t seen a lot of conversions through this approach (although I’m not very aggressive about going to events to solely market feedback army)–but it is a good way to get your idea in front of people and get instant feedback. This is easy to do and if you’re an extrovert (like me), it’s a lot of fun. If you live in an area with a startup community, look for events where you can pitch your business to a crowd. This is also a great way to get exposure.

Customer Service

Feedback Army’s sales continue to grow each month. To keep this up, I focus on the most important marketing tactic of all: delivering a good product and providing good customer service. When my software detects an error or a customer uses my contact form, it goes straight to my cellphone. This way I’m able to address issues quickly. Good customer service has led to these customers coming back. I believe this is responsible for the growth of Feedback Army.

Final Thoughts

The world of marketing is filled with options. As you can see, it’s also a lot of work. Don’t let yourself get lulled into believing there is one magical thing that will build your business for you. An off-shore team offering to “increase your SERPs” for $30 is not going to make your business. Once you get going your efforts will pay off. Remember that you may have to try a few things and don’t expect every campaign to succeed. Ultimately, your business will be made by delivering a good service and acting like a responsible netizen. Don’t forget that.

]]> 5
Prices Updated Thu, 01 Apr 2010 03:03:28 +0000 rsmudge As promised, I’ve updated the Feedback Army prices. Here are the packages:

Package Price Cost / Response
10 responses $15.00 $1.50
25 responses $30.00 $1.20
50 responses $55.00 $1.10

Value will continue to stay part of the Feedback Army equation. One response is still around what you might pay for a click or a thousand impressions on an advertising network. The difference? Our reviewers give you actionable feedback to improve your web presence.

We look forward to continuing to serve you and make the web a better place.

]]> 0
Confidence and Entrepreneurship Tue, 30 Mar 2010 16:40:24 +0000 rsmudge Barack Obama - CONFIDENCE

Photo by: springhill2008

Recently, I was looking for an accountant using Yelp. I came across a review where the reviewer asked simple questions and the accountant kept avoiding straight answers. Worse, the accountant was non-committal and kept exclaiming “I’m an honest accountant” as a reply to every question. This woman’s behavior didn’t give her customers confidence in her as an accountant.

Part of being in business is dealing with the responsibility of dealing with things that can go wrong. It’s important to realize that fear of these bad outcomes exists and to realize you can deal with it when it happens.

I once bought eye glasses from a nice boutique place in Syracuse, NY. The glasses took an extra few weeks to get to me. Why? It turns out the frames I selected were a poor fit for the prescription I had. While making the final pair of glasses, three pairs of lenses were broken. Someone had to eat that cost. It wasn’t me.

Even though these bad situations are a part of business, they’re rare. When they do happen, they have a cost and that cost is usually something that the overall profit margins can absorb.

I used to work as a consultant. One thing I really disliked is I behaved much like the accountant in the first story. I was always thinking about what could go wrong and working with a fear of what would happen if I couldn’t deliver what I had promised. Fortunately the clients knew my work and put up with my quirks.

From that experience I’ve learned. If you’re in business, it’s ok to plant your flag and exclaim “this is what I’m offering”. Own it and be ready and willing to tackle what goes wrong.

I run a website service that gives customers (some big, some small) paid feedback on their site. I have a control freak’s nightmare. I work with a completely anonymous work force. Imagine the level of control I have in this situation. When I started, I had tested the concept and was satisfied it provided value. Yet, I was still pretty scared of it so I priced it way low to compensate. Fast forward over a year later, this service is growing and my anonymous work force has serviced many happy customers. There is a lot of opportunity for things to go wrong.

Once, I had a worker write a very vulgar review filled with swear words. Imagining this situation in the beginning of my service may have been enough to stop me from pursuing it. The customer brought it to my attention and was happy to accept my apology and a few extra responses as compensation. I also added a feature to let me ban misbehaving workers to my system. I learned that bad can happen but when the bad happens, it’s also a chance to shine.

Part of being in business is having the courage to own some task and tell the world you can do it. Don’t worry about what can go wrong–it will. When it happens, you’ll find customers are happy to work with you so long as you’re fair and treat them with respect. When starting, focus on inspiring confidence and realize the rest will work itself out.

]]> 0
Price Updates are Coming Fri, 26 Mar 2010 02:54:17 +0000 rsmudge Next week, on 31 Mar 10, I will be updating the Feedback Army price packages. These new packages will allow me to experiment with creative ways to further reward the reviewers and build new features to benefit you.

Here are the new packages:

Package Price Cost / Response
10 responses $15.00 $1.50
25 responses $30.00 $1.20
50 responses $55.00 $1.10

With these prices, one review is still around the cost of a click on an online advertisement. Not a bad deal.

]]> 0