Best A/B testing examples and use cases

When it comes to an experimentation framework, don't put your ideas on a pedestal, and don't get side-tracked by someone else's.  

Ideas are a dime a dozen. 

The only thing that matters is understanding user behavior and how it affects the outcome of every digital experience you build for your customers.  

• What kind of digital experiences are you building?  
• How is every experience leading your visitors to the buying moment?  
• What's the impact?  

So, ignore everything else—and START. The best experiments reach statistical significance and can help you:  

• Improve your website's user experience  
• Increase your website's conversion rates  
• Personalize your marketing campaigns to increase engagement  
• Optimize your digital marketing strategy to drive revenue growth    

Whether you're a startup or an established digital business, this guide will help you understand how data-driven experiments can help you achieve your business goals. From A/B testing to personalization, we'll dive into some of the best experimentation use cases and examples.  

Retail  

1. Increase products viewed  

Overall hypothesis: Increase the navigation options for all of your subcategories. It will increase the number of products viewed and allow visitors to find what they are looking for in fewer clicks.    

Solution: Increase the navigation options to all subcategories    

Rationale: By providing more navigation and functionality options to all subcategories, ecommerce visitors will have a broader range of choices and easier access to specific products. This will reduce the time and effort required to find desired items, resulting in an increased number of products viewed.  

2. Increase add-to-cart rate  

Overall hypothesis: Use lifestyle images versus your standard studio images for products. This template will increase the add-to-cart rate and create a more relevant experience for your visitors.  

Solution: Use lifestyle images instead of standard studio images for products  

Rationale: By using lifestyle images that depict products in real-world contexts and situations, you can provide visitors with a more relatable and engaging checkout experience. This visual representation can help them envision themselves using or benefiting from the product, thereby increasing their likelihood of adding it to their cart.  

3. Higher conversions  

Overall hypothesis: Use testimonials as social proof on your key product or web pages (for example, “X” people purchased in the last 24 hours). These a/b test results increase purchase conversion by creating urgency for your visitors.    

Solution: Use social proof on key products, such as displaying the number of recent purchases  

Rationale: Incorporating social proof elements, such as showcasing the number of people who have purchased a particular product within a specific time frame, can create a sense of urgency and influence visitors to make a purchase. This psychological trigger taps into the fear of missing out and encourages potential buyers to convert.  

4. Increase average order value  

Overall hypothesis: Test various thresholds to qualify for free shipping. It will increase purchase AOV by enticing visitors to add more items.   

Solution: Test different thresholds for qualifying for free shipping  

Rationale: By experimenting with various thresholds for free shipping, you can incentivize visitors to increase their order value to reach the minimum requirement for free shipping. This approach encourages customers to focus on the cost-saving opportunity by adding more items to their cart.  

5. Decrease bounce rate    

Overall hypothesis: Personalize the landing page with the offer a visitor saw in an ad. It will decrease the bounce rate as that first visit will be more relevant.  

Solution: Personalize the landing page based on the offer seen in an ad  

Rationale: By showing different versions and tailoring the landing page content to align with the specific offer that a visitor saw in an advertisement, you can create a seamless and consistent user experience. This personalization enhances relevancy, minimizing the disconnect between the ad and the versions (original version, new version, etc.) of a landing page, and reducing the likelihood of visitors bouncing off the site without taking any further action.   

Travel  

1. Increase loyalty program participation  

Overall hypothesis: Indicate flights that would move a customer into a higher loyalty tier. It will increase bookings and loyalty program participation by incentivizing your visitors to act.  

Solution: Show flights that would enable customers to move into a higher loyalty tier.  

Rationale: By highlighting flights that would contribute to a customer's progression to a higher loyalty tier, you can create a sense of achievement and reward. This incentive encourages visitors to book those specific flights, leading to an increase in both bookings and participation in your loyalty program.  

2. Increase bookings   

Overall hypothesis: Present an interactive map with timely low rates. It will increase bookings by capturing the “act in the moment” traveler.     

Solution: Present an interactive map with timely low rates.    

Rationale: By providing an interactive map displaying current low rates, you can cater to the spontaneous and impulsive nature of travelers looking to book immediately. This visual representation of enticing rates in real-time encourages visitors to take action and make a booking, increasing bookings.  

3. Increase search rate  

Overall hypothesis: Show the price on the date selector. It will increase the search rate by showing visitors the best options for them earlier.    

Solution: Present the price on the date selector.     

Rationale: By displaying the price directly on the date selector, visitors can quickly assess the cost associated with different travel dates. This upfront pricing information empowers users to make more informed decisions and identify the best options for them earlier in the search process, ultimately increasing the search rate.     

4. Increase bookings via conversion rate optimization (CRO)   

Overall hypothesis: Highlight the seat types available on the search results (for example, “aisle seats available!”, as it will increase bookings and entice visitors to act against their preferences.   

Solution: Highlight seat types available on the search results.   

Rationale: By drawing attention to specific seat types, such as aisle seats, on the search results page, you can create a sense of urgency and FOMO (fear of missing out) for visitors. This enticement to act against their initial seating preferences can lead to an increased conversion rate, resulting in more bookings.  

5. Increase average order value     

Overall hypothesis: Move the location of ancillary options later in the funnel to increase the average order value, because it will be less distracting to the earlier steps in the funnel.   

Solution: Move the location of ancillary options later in the funnel.  

Rationale: By shifting the placement of ancillary options further along in the booking funnel, you allow visitors to focus on the primary steps of selecting flights and entering necessary details without distractions. This streamlined approach minimizes cognitive load and decision fatigue early on, enabling customers to make more informed and value-driven choices, thereby increasing the average order value.    

B2B  

1. Increase participation and relevance  

Overall hypothesis: Contextualize your chat widget based on the content a visitor has consumed. It will increase participation and be more relevant to visitors’ preferences for a better customer experience.   

Solution: Contextualize the chat widget based on the content consumed by the visitor.     

Rationale: By tailoring the chat widget's context to align with the specific content a visitor has consumed, you can create a personalized and relevant experience. This customization enhances visitor engagement and encourages participation by addressing their specific interests and preferences.  

2. Increase leads generated   

Overall hypothesis: Offer a time-based free trial to improve lead generation and make your value props specific for visitors.   

Solution: Offer a time-based free trial and tailor value propositions to visitors.     

Rationale: By providing a time-based free trial, you offer visitors an opportunity to experience the value of your product or service without committing upfront. This incentive can significantly increase leads generated as visitors are more likely to provide their contact information to take advantage of the trial. Additionally, by making your value propositions specific to each visitor, you can enhance the relevance and further motivate them to convert into leads.  

3. Increase relevance   

Overall hypothesis: Make your solutions pages specific to departments and industries. It will increase the leads generated and make the use case more relevant for every visitor.    

Solution: Customize solutions pages for your target audience.  

Rationale: By tailoring your solutions pages to cater specifically to different departments and industries, you can enhance the relevance of your offerings for every visitor. This customization demonstrates a deep understanding of their unique needs and challenges, increasing the likelihood of generating more leads. Visitors will feel that the use case aligns closely with their requirements, making them more inclined to engage and convert.  

4. More completed forms  

Overall hypothesis: Reduce the number of required fields on your acquisition form. It will increase the leads generated by creating a lower barrier to entry.     

Solution: Reduce the number of required fields on the acquisition form.     

Rationale: By minimizing the number of mandatory fields on your acquisition form, you can decrease the perceived effort and time required for website visitors to complete the form. This reduction in the barrier to entry significantly increases the click-through rate (CTR), as visitors are more willing to provide their contact information with a simplified form.  

5. Highly relevant communication   

Overall hypothesis: Use Account-Based Marketing (ABM) to speak to specific value props of a single product. It will be more relevant to those visitors.     

Solution: Utilize Account-Based Marketing (ABM) to target specific value propositions of a single product.  

Rationale: By leveraging Account-Based Marketing (ABM) strategies, you can customize your marketing efforts to directly address the specific value propositions that are most relevant to individual visitors. This approach ensures that the messaging and content they receive align closely with their needs and interests, making your communication highly relevant and increasing the likelihood of engaging and converting those visitors.    

Media  

1. Increase article per visit    

Overall hypothesis: Present related articles as the call to action (CTA) on a story when a visitor hovers or highlights certain text. It will increase the article per visit as it directly relates to a visitor’s interest in their current story.   

Solution: Present related articles when a visitor hovers or highlights certain text.  

Rationale: By providing related articles that are contextually relevant to the content a visitor is engaging with, you can enhance their reading experience and encourage them to explore additional articles. This targeted approach leverages the visitor's current interest and curiosity.  

2. Increase engagement    

Overall hypothesis: Cycle the primary stories upon a return visit. It will increase articles per visit by increasing the likelihood that visitors see more of what is interesting to them.     

Solution: Cycle the primary stories upon a return visit.     

Rationale: By cycling the primary stories displayed to visitors upon their return, use the cta or call to action button option to provide them with fresh and varied content. This approach increases the chances of showcasing articles that align with their interests, resulting in higher engagement and more articles viewed per visit.  

3. Decrease bounce rate  

Overall hypothesis: Highlight trending comments on articles. It will decrease the bounce rate and encourage conversation on our stories.    

Solution: Test variations and highlight trending comments on articles.    

Rationale: By tweaking and highlighting trending comments on articles on the homepage, you can create a sense of social validation and community engagement. This strategy not only reduces the bounce rate by capturing visitors' attention but encourages them to stay and interact on the page. 

4. Increase content per visit  

Overall hypothesis: Rotate your primary navigation categories to those trending. It will increase articles per visit by bringing what a visitor is interested in more forward.     

Solution: Rotate primary navigation categories to those trending.  

Rationale: By aligning the primary navigation categories with the trending topics or content, you can ensure that visitors can easily access the articles that pique their interest.  

5. Increase subscription rate  

Overall hypothesis: Test the threshold of the number of articles consumed in your paid model. It will increase the subscription rate by encouraging visitors to read more.   

Solution: Run tests around the threshold of the number of articles consumed in the paid model.  

Rationale: By experimenting with different thresholds for the number of articles visitors can access in the paid model, you can optimize the subscription rate. Incentivize visitors to read more articles by offering a taste of the content and value they can access through a subscription.  

-----------------------------------------------------------------

To see real-life a/b testing examples and how digital businesses have used these experimentation techniques and more to achieve their goals, as well as tips and best practices for implementing experimentation into your business strategy, check out some of the best a/b testing case studies on the world's leading digital brands.