Subscribe via: ( Email / RSS )

Saturday, December 03, 2011


As someone who always dreaded "market research" and data, I have to say that over the years I've become much more appreciative of marketing metrics.  Most importantly, measuring consumer behavior, online and offline, has radically improved my marketing results.  Enter the concept of user testing.  In today's guest post Jason Thai explains user testing and the more common pitfalls associated with it.

User-Testing Overview
Anytime a web-based business releases a new feature, several issues pop up, keeping product managers tossing and turning at night:

·         1. Are there any glaring bugs with the new feature?
·         2. How will customers view the new feature? 
·         3. Did we design the feature in the most customer-friendly way?
·         4. Does the new feature impact other unidentified areas in the site?
·         5. And most importantly, how will the feature impact overall conversion?

If only there was a way to test your products before launch…  Luckily, several companies have built fantastic user-testing platforms, providing companies with a pool of unbiased respondents to provide feedback as if they were real customers.  A lot of the sites offer great features such as video walk-throughs with step-by-step user interaction.  Another great aspect of user testing is that there are few constraints on what you can test.  A few of the tests we’ve run include:

·         Testing preferences for  before and after web page designs
·         Testing the ease of our order flow process
·         Testing the usability of new customer features

Also, user-testing is relatively inexpensive, making it a no-brainer for identifying pitfalls prior to product launch.  

What Happens When User-Testing Results Go Against the Grain?
However, are the results of user-testing always gospel?   We, at FoxTranslate, a certified translation service ran into that exact issue.  A few months ago, we redesigned our homepage.  We felt that the redesign more effectively conveyed our key service features and more professionally segmented the content.

Current Homepage
Redesigned Homepage



To validate our intuition, we ran user-testing to compare the two sites, expecting our redesigned homepage to be the clear champion.  Unfortunately, the results of user-testing suggested otherwise.

User-Testing Results

If you needed a document translated,
which page would you more likely use?
Common Themes for Picking the Current Page
Here is the split between the "preferred" current page and the "preferred" redesign page

          “The current homepage feels more credible and has a more professional look”

          “The current homepage feels more secure”

          “The current homepage is much easier to navigate and gather information”

          “The current homepage has a much better layout”



The results showed that users clearly preferred the current homepage, meaning it might be time to head back to the drawing board.  

Call it stubbornness, but we proceeded to test the redesign on our actual customers using Google website optimizer.  Google website optimizer tested the effectiveness of our new homepage by sending half our customers to the old homepage and half to the new homepage.  From there, website optimizer calculated conversion rate (# of transaction made / site visitors) and revenue generated, allowing us to gauge which page lead to more transactions.

Results of Current vs. Redesigned Homepage Test

Redesigned Homepage Conversion Rate and
Transaction Size Improvement


Over the course of a month, our new homepage generated 17% more revenue vs. the current homepage on similar customer visits.   6% of the increase was due to more converting visitors and 11% of the increase was due to people spending more per transactions.

Our Learnings
So, why didn’t the actual results align with the results from user testing?  Sometimes, it’s hard to replicate your customers – Regardless of the results; we are still advocates of user-testing.  However, sometimes, it’s hard to find everyday people to replicate your customers.  We’re not selling laundry detergent, so generic user panels can only tell us so much because they don’t exactly know what your customers care about.

About the Author: Jason Thai is a marketing manager for FoxTranslate, specializing in document translation of business, legal, immigration and academic documents in over 30 different languages.

8 comments :

Bradford Duncan said...

I got a lot out of this Blog post. I really stress doing market research. Your comment about "How will customers view the new feature.", rings true. I've been sending a SurveyMonkey by e-mail with marginal results.

Custom Lanyards said...

What a lovely post.

Custom Lanyards said...

What a lovely post.

SEO marketing said...

A really nice topic picked and explain here by Jason Thai in this guest post.

Thanks,

Anonymous said...

As a business owner, recognizing your audience and appealing to their needs is essential if you hope to ultimately be successful. Knowing what your customers want and what appeals to them has the potential to make or break a sale and even your entire business. I agree completely with you that it is extremely important to do adequate research about your customers in order to best serve their needs and make the most our of your company.

phuket hotel said...

Great Work dude.

Chelseayoulater said...

You're blog is amazing! I have gathered so much information from this, specifically about video marketing http://www.mojovideomarketing.com/. I hope you can share more of your interesting blogs soon. I'll be waiting for it. Thanks.

articles on internet said...

Thanks for this informative post! Market research is really important before launching a certain product. Consumers point of views are important. :)

Post a Comment

Please note that we reject all posts that clearly are leaving a comment simply to acquire a back link. Only comment if you have something of value to share with other readers.

Thanks!




Copyright 2018. The Marketing Blog.