Contextual bandits: The next step in personalization

When it comes to personalization, we're in competitive times. We're in hard-to-please times. We're in attention-span-of-a-goldfish times.

Everyone knows personalization matters, that's not news. But how do you deliver truly relevant experiences that drive conversions without wasting resources? That's where contextual bandits come in.

Why smarter personalization matters...

Creating truly personalized experiences at scale presents unique challenges. When you have multiple products, audience segments, and user attributes to consider, traditional approaches require significant manual effort.

Rules-based personalization demands extensive setup time to configure the right conditions and targeting for each segment. Meanwhile, determining which experiences resonate with specific user attributes often involves considerable guesswork and testing.

Enter contextual bandits. By selecting relevant user attributes, contextual bandits automatically learn which experiences work best for different audiences, providing valuable insights while maximizing conversions from day one.

What is a contextual bandit?

The term "multi-armed bandit" comes from the classic slot machine analogy (the "one-armed bandit").

Imagine a casino with multiple slot machines. Which one do you play to maximize your winnings? That's the basicchallenge.

Contextual bandits take this to the next level by factoring in who's pulling the lever. They leverage user data to make better algorithmic decisions and deliver 1:1 personalization. The machine learning model balances the impact on your primary metric with the data it has about each visitor (the context).

A contextual multi-arm bandit serves the best-performing variation for every visitor based on their unique profile at that specific moment. This varies for different visitor profiles as the goal is to drive maximum impact for every visitor in each session.

Instead of tediously mind-mapping every variation to different user archetypes (a.k.a manually configuring static rule-based targeting), you can rely on the contextual bandit to make those decisions more accurately for you.

From multi-arm bandits to contextual bandits...

What makes contextual bandits different from multi-arm bandits? Context.

Traditional MABs look for a single best-performing variation for all users, while contextual bandits identify winning variations based on user profiles such as device type, location, behaviors, purchase history, and more.

Let's compare:

1. A/B testing: Fixed traffic allocation where visitors are randomly assigned to different variations, with each person seeing only one experience while waiting for statistical significance.
2. Multi-arm bandits: Optimizes for a single best-performing variation. Shifts traffic dynamically but seeks one"winner."
3. Contextual bandits: Personalizes for individual users based on context. Different users get different experiences based on what's most likely to convert for their profile.

Every missed optimal experience is a lost conversion opportunity. With A/B testing, winners are generalized from a limited segment. MABs improve this but still seek one "best" variation for everyone.

Contextual bandits serve each visitor the best variation for them at that moment. When profiles change, the relevant variation changes too. If a visitor converts on a product, they'll see a related product on their next visit, not the same one, thus increasing the chances of converting again.

How contextual bandits work

Contextual bandits balance the impact on your primary metric and user attributes to dynamically distribute the most relevant variation to each visitor at that specific moment.

Here's a simplified explanation:

• Learning period: The model starts with 100% exploration, randomly assigning variations to visitors to gather diverse data for predictions.
• Balancing exploration and exploitation: Once enough visitor behavior data is collected, the model begins exploiting (serving personalized variations). It dynamically adjusts exploration/exploitation rates as it receives more events.
• Continuous adaptation: The model maintains some exploration (maximum 95% exploitation) to ensure continuous learning and avoid missing opportunities.

Selecting the right primary metric is critical as the impact on it influences the model distribution. Therefore, it’s suggestedto be tracked as close to where the contextual bandit is running, ideally on the same page.

User attributes are equally crucial. The more complete your set of attributes (products purchased, viewed, categories browsed, etc.), the better your model will perform. Optimizely's model supports unlimited attributes from standard (client-side), custom (API), and external (third-party) sources.

Contextual bandits use cases

Here are examples of wider industry applications:

• Retail: Homepage product carousels personalized by shopping frequency and purchase history.
• Media: Homepage content suggestions (sports, series, movies) based on viewing habits and devices.
• Software: Dashboard feature highlights tailored to user role and usage patterns.

However, any real examples, you ask?

Our beta participants are already implementing and seeing results:

• A financial services customer is using homepage contextual bandits to deliver relevant banking products based on customer history.
• A pizza restaurant chain is using checkout page contextual bandits to suggest add-on items based on cart contents.
• A telecommunications company is using profile page contextual bandits to present upsell offers based on current subscriptions.

The digital team at Optimizely is also using contextual bandits. They're using CMABs on our homepage to match visitors with products based on their company, role, industry, and location.

Here are some initial results:

• 13.62% higher engagement with targeted content
• 3.37% improvement in marketing planning
• 20.79% improvement in validation with testing

The team says it's working well across the board.

Further, here’s what a results page looks like in the dashboard.

Disclaimer: This is an early preview of Optimizely contextual bandits results page

Image source: Optimizely

Benefits of implementing contextual bandits

CMABs deliver substantial business value by:

1. Providing truly personalized experiences for every user: Instead of one-size-fits-all approaches, CMABs deliver the right content to the right person at the right time.
2. Increasing conversion rates on primary metrics: By showing users what they're most likely to respond to, CMABs drive higher engagement and conversion.
3. Adapting dynamically to changes in visitor behavior: The system serves the best variation in every session, even as user preferences evolve.
4. Eliminating opportunity costs from traditional testing: Unlike A/B tests that require weeks or months to reach statistical significance, contextual bandits start optimizing immediately, reducing exposure to underperforming variations in real time.
5. Requiring minimal maintenance: CMABs are ideal for pages where content doesn't change too frequently. As time goes on, the ML model gets sharper with the data it collects, making this a "set it and forget it" optimization that can be left running continuously.

CMABs increase the likelihood of conversion, positively impacting ROI and eliminating the opportunity costs that A/B testing or traditional bandits incur.

Optimizely's contextual bandit implementation: What makes it different

Here's how we're doing things differently:

• Advanced tree-based models: We've developed models for both binary classification and regression tasks, making our system flexible and adaptable to different types of data and experiment setups.
• Feature importance insights: Our system measures attribute impact and displays feature importance, providing insights on which attributes drive conversions.
• Dual-model & incremental learning: We handle all prediction types with specialized models that continue learning from new data without starting from scratch.
• Dynamic feature processing: Our preprocessing automatically converts features and handles data issues. Using XGBoost, we build multiple simple trees that learn from mistakes instead of one complex tree, preventing overfitting through regularization and other techniques.
• Integration with the broader ecosystem: Our CMAB implementation works seamlessly with Optimizely'sexperimentation and personalization suite, making it easy to elevate your strategy without additional tools or complexity.

The future of personalization is contextual

In a no-one-size-fits-all world, context is king.

As competition for attention intensifies, static approaches to personalization simply aren't working anymore. The brands that win will be those that can deliver truly relevant experiences in the moment, adapting continuously to changing customer behaviors and preferences.

Ready to explore how contextual bandits can help you drive higher engagement, conversion rates, and customer satisfaction?

Check out this 2 minute Navattic tour to see what contextual bandits look like in the platform.