How EA turned a big win into a metropolis

SimCity 5 sold 1.1 million copies in the first two weeks of its launch in March 2013. 50% of sales were digital downloads.

For EA, A/B testing was an integral to the release’s digital success. Here is the story of how the product team at Maxis, who managed SimCity’s release, used Optimizely to amplify sales.

“Optimizely helped us learn a lot about our users – what’s working and what’s not – so we could make changes on our site to optimize our conversion percentages. In the end, this translated to higher revenue for us.”

-Mike Burk, Senior Online Product Manager, Maxis

As EA prepared to release the new version of SimCity — a launch they knew would attract significant hype — the digital marketing team managing the campaign worked on ways to optimize the SimCity microsite where visitors would purchase or download the game.

The Maxis digital marketing team opted to extend a promotional offer in order to drive more pre-orders of the game on SimCity.com. The offer was displayed as a banner across the top of the pre-order page and as part of the ad creative in paid media buys. This prominent placement put the promotion in front of many potential buyers, but according to the Maxis team, was not driving the increase in SimCity pre-orders they had expected.

While many site visitors were seeing the promotional banner, it’s large size moved the actual call-to-action button down on the page.

Hypothesis

The Maxis team hypothesized that simply moving the call-to-action higher on the page by changing the way the promotional offer was displayed could drive more purchases and increase revenue generated from SimCity.com.

The Test

The Maxis team set up an A/B test pitting the original SimCity pre-order landing page against several variations that displayed the promotional offer in different ways. Each variation corresponded with a hypothesis the team had set beforehand regarding placement, color, and display of the promotional offer. One variation removed the promotional offer from the page altogether.

After building each variation in Optimizely’s visual editor, the team set their experiment goal — clicks on the “Pre-Order” button — to track purchases generated from the test. They allocated 100% of site traffic to the experiment in order to find an optimized solution as quickly as possible during the high-traffic SimCity launch period.

Original

Variation

Results

The Maxis digital marketing team quickly discovered shocking results – the variation with no offer messaging whatsoever drove 43.4% more purchases.

“We found that the variations with the promotional offer actually performed worse than just a clean version of the page. This was really surprising to us, but it had a pretty significant impact,” Burk said.

Now knowing that the promo banner actually hurt conversions, the Maxis product team decided to apply this information across a larger scope. A follow-up test on EA’s paid media creatives that mirrored the initial test (removing promotional text from each ad) also increased conversions from highly-qualified ad traffic.

The Takeaway

• Question assumptions. An A/B testing tool provides real, actionable data to help you better understand how your web properties are performing. Even if think you’ve optimized your site, test it out. In EA’s case, the common belief was that promotional offers drive more purchases — this turned out to be false. Testing gave them the information needed to maximize revenue in a way that would not have been otherwise possible.
• Apply lessons across channels. Testing arms you with information that can increase conversions beyond just your website. EA applied the lessons from the experiment on the SimCity pre-order page to their paid advertising campaigns, yielding exponential results.

Turns out people just wanted to buy the game, no extra promo incentive necessary.

06/22/2021 Editors note: Updated links. To see more recent posts, go here.
09/05/2023 Editors note: references in the article updated.