Virgin Media O2 is dialing up innovation with Experimentation + AI

Virgin Media O2, a leading telecommunications provider in the UK, is on a mission to enhance digital experiences through AI and experimentation. In this interview, Doychin Sakutov, Digital Director of Product, AI, and Experimentation at Virgin Media O2, shares insights on how the company is leveraging Optimizely’s solutions to drive faster decision-making, optimize customer journeys, and foster a culture of continuous learning. From utilizing AI-driven experimentation techniques like multi-arm bandits to accelerating insights with Stats Accelerator, Sakutov discusses how his team is reshaping digital strategy to maximize customer engagement and business impact.

Integrating Product, AI, and Experimentation for better customer experiences

0:00

/

0:00

Key Points:

Doychin Sakutov, is the Digital Director of Product, AI, and Experimentation at Virgin Media O2.
He's recently added 'AI' and 'Experimentation' to his title alongside 'Product', recognizing the need for all three of these functions to work hand-in-hand for program success.
The business is in the early phases of AI integration but has a clear vision for leveraging AI to enhance customer experiences. The company is starting with a tangible approach, addressing real customer pain points, and iterating based on results.

A digital strategy transformation

0:00

/

0:00

Key Points:

Virgin Media’s digital team faced challenges in quickly responding to customer demands due to a rigid platform and dead-end user journeys.
This lack of data into customer behavior on their site limited the team’s ability to make informed decisions.
The company aimed to break away from its traditional mindset and embrace continuous testing and learning, ensuring every website change had a measurable impact.
By implementing Optimizely Web Experimentation, Virgin Media gained the ability to track user behavior and test new customer journeys, making results measurable and ensuring every change had a clear outcome.

We wanted to make a statement with our change of mindset. We wanted to onboard the best available tool for experimentation, which will allow us to push the boundaries of what's possible.

Doychin Sakutov- Virgin Media O2

Rethinking Experimentation to drive innovation

0:00

/

0:00

Key Points:

Virgin Media believes in continuously evolving goals through experimentation, allowing them to uncover unexpected opportunities.
The company embraces a wide testing approach, pushing their teams towards insightful customer discoveries that go beyond the scope of predefined solutions.
This broadened mindset resulted in a major win. They transformed an outdated direct debit payment method by implementing open banking, ultimately improving order conversion rates.

I think everybody talks about [the] North Star, everybody talks about the set of metrics that the teams need to chase, but in reality, you need to evolve. You need to let grow and experiment...

Doychin Sakutov- Virgin Media O2

Maximizing sales with Multi-Armed Bandit Experimentation

0:00

/

0:00

Key Points:

Virgin Media leverages multi-armed bandit (MAB) experimentation to maximize rewards while minimizing risk.
MAB experimentation is particularly valuable during high-stakes events like Black Friday. The algorithm dynamically shifts traffic to the best-performing variants, ensuring optimal performance without sacrificing order conversion rates.
MAB helps to fine-tune offers and placements in real time, leading to higher sales volume.

The idea of learning what works and what doesn’t work for customers in a short period of time is gold. We don’t want to have something live for weeks, even months, without knowing the impact on customers.

Doychin Sakutov- Virgin Media O2

Using Stats Accelerator for Rapid Experimentation

0:00

/

0:00

Key Points:

Stats Accelerator helps the team to quickly identify what works and what doesn’t. It greatly reduces the time spent running experiments, allowing the team to set winning variants into motion sooner.
During Black Friday, the tool helped pinpoint the most effective customer messaging and placements, enabling rapid iteration based on customer preferences.

I'm a big fan of Multi-armed Bandit experimentation because it's a great opportunity to maximize reward while minimizing any regrets.

Doychin Sakutov- Virgin Media O2

Built a scalable and impactful experimentation culture

0:00

/

0:00

Key Points:

Virgin Media sees experimentation as the key to refining customer experiences, treating personalization as an extension of testing and learning.
It's all about the culture of experimentation. The company’s experimentation team started with one person and grew into a dedicated department by continuously demonstrating business value.
They foster a flexible collaboration style, allowing teams to determine the best ways to work together. This adaptability has been key to scaling their experimentation program effectively.

Shaping the future of customer experience

0:00

/

0:00

Key Points:

Virgin Media sees AI as an evolving tool that has the power to enhance customer experiences.
They prioritize approachable, data-driven AI applications, such as predictive recommendations, to reduce customer decision fatigue.
Virgin Media aligns its KPIs with customer journey stage. For new customers, they look to sales, while for existing customers, service reliability, upsell, and contract renewals measure success.

It’s very important to understand the different types of journeys in the context of the customer. If we adopt one success metric across the board, we’ll fail.

Doychin Sakutov- Virgin Media O2

Conclusion

Virgin Media O2’s digital transformation journey with Optimizely showcases a robust strategy centered on experimentation, customer-centricity, and data-driven innovation. By tackling challenges such as slow release cycles and limited customer journey insights, the company has significantly enhanced its agility and responsiveness. Optimizely's experimentation tools have enabled Virgin Media O2 to boost customer acquisition and uptick order conversion rates. Personalization and experimentation efforts are seamlessly integrated to ensure tailored customer journeys that are impactful. AI has also been implemented into their approach to tangibly improve customer experience by use case. With a growing team and a flexible, collaborative approach, Virgin Media O2’s partnership with Optimizely is not just driving immediate wins but laying the foundation for sustained digital growth.

Industry

Telecom

Products used