Posted April 01, 2014

3 Unexpected A/B Testing Lessons: A Small Business’s Story

In this post, you will discover why A/B testing is an important part of your small business’s marketing strategy and how easy it is to build a thoughtful optimization engine. We’ll walk you through the story of FoxyCart, a 5-person, bootstrapped software company that helps developers create custom e-commerce stores.

shape

How did Spreadshirt, a custom design and apparel marketplace, increase clicks to their sales page by 606%?

How did subtle copywriting adjustments on Insound’s checkout page increase conversions by 54%?

That’s the power of website optimization. Relatively small changes to your website can generate significant boosts to your company’s user engagement, sales, and revenue goals.

In this post, you will discover what A/B testing is and why it is an important part of your small business’s online marketing strategy and how easy it is to build a thoughtful optimization engine.

We’ll walk you through the story of FoxyCart, a 5-person, bootstrapped software company that helps developers create custom e-commerce stores.

FoxyCart’s Story

Brett Florio and Luke Stokes started FoxyCart almost eight years ago to solve a major pain point among small businesses.foxycart-logo-8in

“We had a web development company and were doing client work,” explains Brett. “Every time we worked with an e-commerce client, we’d spend hours and hours on each one. We realized that there had to be a better way—so we started building FoxyCart.”

Brett Florio, Co-Founder, CEO, Product Manager of FoxyCart

Brett Florio, Co-Founder, CEO, Product Manager of FoxyCart

FoxyCart is a small business that caters to small businesses selling one or two products online. This niche is one that has never-before been tackled—FoxyCart’s competitors are solutions for larger online storefronts.

In 2013, after seven years in business, FoxyCart launched its first online marketing initiatives. Recognizing a need to fully optimize their marketing spend, Brett and his team started working with Optimizely.

“Testing helps us make better decisions about our marketing,” says Brett.

Luke Stokes, Co-Founder, CTO and Lead Developer at FoxyCart

Luke Stokes, Co-Founder, CTO and Lead Developer at FoxyCart

Here are the key (unexpected) lessons that Brett and his team would teach fellow business owners about A/B testing.

Lesson #1: Testing Neutralizes Debates

Brett and Luke are passionate, opinionated co-founders. They’ve been known to engage in long, heated debates over ‘what’s best for the product and customers.’

As you can imagine, this process is a time-sink.

Brett and Luke started using Optimizely with the original goal of becoming data driven. They used split testing as a core part of every new product launch, website enhancement, and marketing campaign.

Along the way, something surprising happened. The two opinionated co-founders started arguing less and less.

“One of the best things about testing culture is that we don’t argue anymore,” says Brett. “We’ve gotten into the habit of saying, ‘stop, let’s test it.’ We’ve stopped fighting over the swing at the playground.”

Opinionated co-founders always think they’re right. Testing will help you rely less on intuition and more on objective truths.

Lesson #2: Testing Will Help You Avoid ‘Bad Advice’

Every homepage needs a big, bold sign-up button. That’s marketing 101, so challenging this ‘best practice’ must be crazy. Or is it?

Recently, FoxyCart was working with a design agency who recommended implementing a big ‘sign up’ button. Brett was open minded but hesitant.Something in his gut said “don’t do it!”

He tested his hypothesis that the large sign-up button would actually hurt conversions. Here’s what he found:

“We did generate more sign-ups,” explains Brett, “But many of these users were not engaged over the long-term. Why? Because these users were probably looking for something different and signed up for FoxyCart too quickly.”

We were able to confirm our suspicions in a way that we had solid data. As much as everyone says you might need one, we didn’t.

Lesson #3: Testing Isn’t Enough

Testing is one part of a bigger CRO story—one that involves analytics, user segmentation, and continuous experimentation.

That’s why FoxyCart connects Optimizely to the team’s favorite analytics tools: KISSmetrics and Segment.io. They use KISSmetrics to define user segments and monitor customer behavior over the long term and Segment.io, to export data to any dashboard that Brett and his team could want.

“It’s really surprising to me that more small businesses don’t take advantage of these extremely important integrations,” says Brett. “Optimizely seamlessly connects to other platforms, which is immensely valuable. In KISSmetrics, we monitor eight different user funnels. We look at the performance of our A/B tests over the long-term, to see how users are progressing through the funnel.”

Brett and his team have made these integrations extremely straightforward.

“The technology is amazing, and we were able to get everything set up without doing additional work,” he says.

It’s About The Mindset

Using an A/B testing tool is about the journey as much as it is about the data and has the potential to help your small business develop a strong sense of direction.

“The value of the testing for me is not just in the results, but in the mindset to get to the results,” says Brett. “It’s less about running a particular test and more about being able to move in the right direction and know what we’re doing — and even know what the right direction should be.”

With A/B testing, small businesses have the power to compete with industry giants to:

  • Optimize customer experiences
  • Test business hypotheses
  • Increase sales and leads

What surprising lessons have you (or your small business) learned from multivariate testing? Share your thoughts in the comments section below.