SeaWorld delivers step change in customer experience through relentless experimentation

North America Optimizer Award Winner - 2022

BEST EXPERIMENTATION PRACTICE

This award is presented to the Optimizely customer who runs stellar experimentation programs, demonstrating strong data-driven approaches to solve real customer problems and drive impactful business decisions.

SeaWorld and the digital opportunity

SeaWorld Parks & Entertainment Inc. is a leading theme park and entertainment company that connects guests of all ages to the natural world through up-close experiences with animals, educational attractions, thrilling rides, and spectacular shows. 

As one of the world’s foremost zoological organizations, SeaWorld is committed to the conservation of wildlife and natural habitats worldwide, and have rescued over 40,000 animals to date. Its portfolio of brands includes well-loved parks such as SeaWorld®, Busch Gardens®, Aquatica®, and Sesame Place®. 

As its customers have increasingly turned to digital channels to purchase tickets and add-on experiences, SeaWorld, as part of a larger digital transformation initiative, took on the challenge of modernizing their 12 branded websites, with a relentless focus on improving usability and increasing conversion. 

Identifying opportunities through a comprehensive data framework

Historical website data was sparse, so the newly organized Digital team established a baseline of tools, methods, and KPIs to guide the optimization effort: 

• Granular conversion funnel reporting to identify significant drop-off points 
• Customer feedback study to understand top detractors and promoters 
• Page-level interaction analysis and heatmaps to recognize on-page frustrations and engagement 

• Live usability studies and session replays to gather qualitative insights and identify root causes 
• Transaction data analysis to determine impactful segments to focus on 
• A/B testing to validate hypotheses independent of other business changes 

Over 100 hours of research and analysis led to several actionable insights: 

• Over 75% of site traffic was on a mobile device, and because the sites were not designed or optimized for mobile, user frustration was much higher there than on desktop. To have the greatest impact, they’d need to take a mobile-first approach.  
• User research showed that customers lacked clarity at key moments in the purchase flow. Questions like “Do younger children get in free?” and “What if I need to reschedule or cancel a planned visit?'', without an easy way to find an answer, led to uncertainty, support calls, and lost revenue. It was important to learn what customers need to know, and when, and then placing that information where it mattered most.  
• Session replays illuminated user behaviors and expectations throughout their journeys. It became clear that users wanted to click on images to get to more details, poor form design caused friction between the cart and checkout, and checkout errors became blockers due to unclear error messaging. 

Relentless pursuit for optimized experiences through rapid experimentation

SeaWorld identified Optimizely as a powerful experimentation partner due to the simplicity of using a visual editor, robust post-experiment analytics, and ability to scale support for multiple websites and platforms including native mobile apps. To aid in SeaWorld’s success, Optimizely’s Experimentation Services team leaned in to validate some of the early metrics, allowing SeaWorld to double down on a solid foundation to scale rapidly. SeaWorld also integrated Optimizely with Quantum Metric to sync real-time customer behavior with their existing testing strategy which proved critical to provide direction and context. This integration allowed the group to focus experimentation on top customer needs, improved the testing success rate, and ultimately validated the impact of optimization on the customer experience.

The SeaWorld team wanted to maximize the ROI on early experiments, so their experimentation journey started at the bottom of the funnel, where conversions occur. Risks were mitigated by testing hypotheses on their lower traffic sites, and porting validated solutions to higher traffic sites. 

Over the course of a few months, Optimizely’s high-velocity testing capabilities enabled SeaWorld to run over 50 experiments from homepage to checkout. Successful experiments informed SeaWorld about what worked best for their customers. As is often seen with experimentation, some findings were contrary to prior beliefs, reinforcing the need for frequent and consistent A/B testing. 

SeaWorld’s team was new to the company, but they brought meaningful previous e-commerce experience and relied on best practices during experimentation: 

• Every brand and park website had different customers, products, and associated behaviors. As a result, the team adapted to new findings for each website without assuming winning experiments were perfectly transferable to other sites. 
• With the goal to drive impact quickly, the team started with compound changes on a single page. If there was an additional need to understand the impact of individual changes, they would be decoupled and validated separately via multi-variant testing. 
• Because the business runs various sales/promotions at different times of the year, winning experiments were re-tested multiple times to ensure results were consistent during promotional events. 
• Optimizely data was tied to other sources of input including customer feedback, session replays, transactional product mix – this led to holistic data-driven decision making that went deeper into why some experiments worked while others did not. 
• In some cases where trade-offs were required, in-session experimentation data was supplemented with offline/lifetime value data to understand the downstream impact of a change and determine if it was net-positive for the company. 

An example of a cart optimization experiment that drove impact:

Analysis and research

• Funnel data showed cart-to-checkout progression rate was lower than benchmarks, especially on mobile devices
• Session replays and heatmaps showed users weren’t scrolling past 60% of the screen, therefore missing important components such as cross-sells. Additionally, there were several unintentional clicks: eg, on the header, on the CTA to go to checkout before the login form was filled.
• Transactional data showed only some users were purchasing products that required sign in; signed-in users converted lower vs. guest checkouts.

Hypotheses

• Separating the cart flow into two distinct steps (view cart, login), and redesigning the form will reduce cognitive overload and correct user errors
• Simplifying the order summary and swapping its placement with the list of items will allow users to manipulate cart items but also see cross-sells
• Defaulting to guest checkout will reduce the friction for majority of users
• Displaying a guest-friendly cancellation policy will increase confidence
• Increasing the size of buttons and adopting a modern design system will bring the experience closer to other websites that users are familiar with

Results (vary based on website)

• 5-10% increase in cross-sells added to cart
• 3-4% increase in visits from cart to checkout
• 1% lift in conversion

 

Before (Control)	After (Test)

Testing & learning pay off for SeaWorld

SeaWorld Parks & Entertainment Inc. had record business performance in 2021 because of several revenue initiatives, including the digital transformation work. During this time, SeaWorld’s websites saw tangible conversion lifts by improving the pre-visit customer journey, while its mobile apps achieved industry-leading app store ratings after a full, in-park customer journey focused rebuild. 

Key objectives for website optimization were to create modern pre-visit experiences, significantly reduce page load times, and allow seamless recovery from any errors. To this end: 

• Learnings from the experimentation phase were supplemented with a modern design system, and conversion funnel pages were rebuilt to reduce friction and confusion. Newly redesigned pages were re-tested using Optimizely to ensure gains are consistent with prior learnings, and the components are being rolled out in batches to the 12 websites over the course of 2021-2022. 
• Bottom of funnel failure rates were cut in half, through thoughtful design and service improvements, eliminating redundant errors, and improving success rates by providing actionable error messages. 
• Latency on key user actions (e.g., add-to-carts) was improved by 40%, and further levers like optimizing payload will significantly improve page load times.  

Overall, these improvements have led to millions of dollars of additional revenue, and a significant improvement in website usability scores.  

Taking it to the next level with Optimizely

SeaWorld looks to Optimizely not only as an experimentation partner, but as a source of truth, and a platform to guide decision making on a variety of digital initiatives. As SeaWorld continues to unlock new capabilities and further improve the customer experience, it will use Optimizely to facilitate a fail-fast, continuous, product development cycle and plan to explore even more Optimizely capabilities to achieve faster and deeper levels of experimentation.