These A/B Tests Increased Revenue Per Visitor for VeggieTales by 38%

VeggieTales is an animated children’s TV series, hugely popular with pre-schoolers in the United States. Since 1993, they have sold over 70 million videos, 15 million books, 7 million CDs, and all sorts of accessories touting the iconic vegetable characters. 

They wanted to systematically optimize the design of their entire e-commerce site, store.veggietales.com, so they got in touch with my optimization strategy team at Blue Acorn for direction and results.

Our initial research drove us to start with some subtle design changes that, after reaching statistical significance through A/B testing, now receive 100% of the traffic as permanent design changes. All together, these subtle changes led to significant changes in revenue and growth:

• 38% increase year-over-year in sitewide revenue per visitor
• 42% increase year-over-year in sitewide conversion rate

By first examining their analytics, we were able to determine where to start testing and what to test.

At a glance, here are the results we produced for the VeggieTales store:

• 17.4% increase in revenue per category page visitor
• 13.9% increase in revenue per product page visitor
• 14.3% increase in revenue per checkout page visitor
• 36.8% increase in revenue per homepage visitor with social media changes
• 28.1% increase in revenue per mobile visitor

Let’s look at the hypotheses and variations that produced these results.

Primary conversion metric: RPV

We feel the truest metric for measuring the success of optimization testing for an e-commerce site is Revenue Per Visitor (RPV). Increasing our clients’ conversion rates is easy if they agree to start to sell everything at 90% off, but that would kill their revenue. Constantly attempting to improving your conversion rate is still a smart idea. However, e-commerce merchants should focus on RPV their primary Key Performance Indicator (KPI).

The Tests

Category Page

Original 

VeggieTales Original Category Page

VeggieTales’ category pages had featured very large, bright, distracting banners. Customers landing on the page didn’t have general guidance on where to go, and they were getting distracted from finding products they wanted and continuing to the product page.

Hypothesis: Dropping the banners would result in a higher RPV.

Variation

VeggieTales Variation Category Page

Results: Without the banner, VeggieTales’ category pages produced a 17.4% increase in RPV.

Statistical Confidence Level: 95.3%

Product Page

Original

VeggieTales Original Product Page

Using custom heat mapping tools, we confirmed that VeggieTales’ current product pages had very poor spacing, distracting attributes, and lacked organization. They weren’t awful by any means, but they weren’t optimal either.

Hypothesis: Improving the spacing and increasing the prominence of the primary call-to-action will result in a higher RPV.

Variation

VeggieTales Variation Product Page

Results: The product page variation produced a 13.9% increase in RPV.

Statistical Confidence Level: 96.1%

Checkout Page

Original

This is where eCommerce conversions happen. However, VeggieTales’ checkout page included all of the same header and footer elements that were found on every other page of the site. They’re good elements, but they serve as unnecessary distractions to a visitor trying to convert to a customer, and can be eliminated using a website optimization tool like Optimizely.

VeggieTales Original Cart Page

Hypothesis: Simplifying the checkout by removing all of the distractions in the header and footer – except those designed to act as confidence builders—will result in a higher RPV.

Variation

VeggieTales Variation Cart Page

Results: The more streamlined checkout page variation produced a 14.3% increase in RPV.

Statistical Confidence Level: 98.1%

Homepage

The original homepage had a strong callout for free shipping, which is never a bad idea for converting visitors, but it overshadowed the site’s social icons.

However, we didn’t just want to move the social icons to the header, we wanted visitors to be aware of the immense size of the VeggieTales community, which serves as a confidence builder.

VeggieTales Original Store Homepage

Hypothesis: By adding social icons to the header and making visitors aware of the size of the VeggieTales community, visitors will be more likely to have social interactions within the community and consumer confidence will increase.

Variation

VeggieTales Variation Store Homepage

Results: Not only has VeggieTales gained more fans and followers, but the variation homepage produced an astounding 36.8% increase in RPV.

Statistical Confidence Level: 95.7%

Mobile Site

Original Mobile UI

There’s actually nothing mobile about this UI. It is simply the desktop UI viewed on a mobile device. It’s nice in that it doesn’t rely on Flash, which is still not supported on iOS, but it requires a tremendous amount of effort to navigate the site on a smartphone.

VeggieTales Store Original Mobile Experience

Hypothesis: Sending mobile traffic to a mobile-themed UI will increase the mobile conversion rate and revenue per visitor.

Variation

VeggieTales Store Mobile-Optimized User Experience

Results: Thanks to testing, VeggieTales now knows they can be generating a lot more revenue from their mobile visitors, as this quick, non-customized, mobile-themed UI saw a 28.1% increase in revenue per visitor.

Statistical Confidence Level: 99.1%

Just the beginning

The improvements VeggieTales has experienced thus far, which have resulted in 38% sitewide increase YOY in RPV, are just the beginning. Once the changes to the homepage variation are applied to the rest of the site, we expect that number to grow even higher. We focused on the lowest of the low-hanging fruit—or should we say vegetable—in order to produce the quickest gains for our client. Now it’s time to pick the rest.