SmartWool Sells More Socks With A/B Testing

It’s not too difficult for an experienced conversion optimization consultant to look at a horrid web design and have a strong feeling as to what changes will result in higher conversion rates. But when the original design is beautiful and already nails best practices, even expert intuition is good for nothing more than a hypothesis. Testing is essential.

This was the case when SmartWool came to us at Blue Acorn with the desire to carefully and systematically redesign various pages on their site. As it stood, their site was already beautiful, but that wasn’t enough. If you’ve ever had the pleasure of wearing any of their products, you know they value form AND function. The SmartWool team is very open to the idea of testing different page layouts to see which one works best for their visitors. One of the first pages we tested was the category page. Due to its high position in the conversion funnel, it receives a lot of traffic, meaning we would see results from a test quickly.

We collaborated with SmartWool’s team and came up with a brand new design that looked great–it was unique and aesthetically pleasing. The new design showcased product images in varying sizes. It broke up the repetition of images aligned in equal size rows—and we all loved it. Despite how fantastic we thought it was though, both companies wanted to test the new category page against a variation that was less unique and more in line with the e-commerce best practice of repetition. Using repetitive image attributes enables better eye tracking when scanning a number of products.

Our goal for this test was increasing average revenue per visitor. We hypothesized that changing the category page layout to feature all of its product images in a repetitive fashion and in the same size will result in a higher average revenue per visitor.

The results were quite a surprise. Despite its more ordinary aesthetic, the variation category page (B) yielded a 17.1% increase in average revenue per visitor—after testing 25,000 visitors—with a statistical significance of 95%. (Learn how to add revenue tracking to your experiments in Optimizely.)

So what do these findings this mean? Making a product image bigger may lead to more clicks on the product, but that doesn’t always guarantee checkout. More visitors might end up on product pages for products they don’t necessarily want to purchase, which means getting to the product page of a product they do want will require  extra clicks and extra time, a.k.a. added friction. The key word, though, is might. It was always possible for A to test better, which is why testing is essential.

Read more details! Download the full case study from Blue Acorn, and visit this Optimizely Learning Center article to add revenue tracking to your experiments.