Move, Inc. (operator of Realtor.com), a News Corp company, is a provider of online real estate services for consumers and real estate professionals to help navigate all stages of the home-buying process. The company is heavily invested in understanding its target audience, with teams devoted to both analytics and user research.
The company takes action on these insights and measures the impact of changes via a robust optimization program. Suma Warrier leads digital optimization for Move, Inc., and has spent her time at the company building out a robust team and process to enable high-velocity testing.
At the end of 2013 after Suma joined the company, the revenue goal was set and communicated across the company. 2014 would become a year where A/B testing and optimization contributed to a portion of the company’s revenue. The goal was a stretch, which drove strong focus and prioritization. They chose a new optimization platform (Optimizely), and began to staff a new team focused on optimization.
At Move, the Optimization team is housed within the Analytics department. For the team to be successful, they needed cross-functional support from Research, Design, Product, QA and Engineering. The core Optimization team included Suma, Roger, and Hima to manage the test prioritization, building and running tests, and post-experiment analysis. Two key technical assets that they focused on assigning immediately were QA and developer resources.
As the optimization team became comfortable with their communication and process, speed and cadence became a priority. In order to take advantage of the benefits of optimization, Suma wanted to ensure that the team could rapidly test many ideas, quickly make decisions, and incorporate customer data into their process. By laying a strong foundation, the team was able to reach a pace of launching 1 new experiment every 2 weeks.
Suma credits the Move team’s success with rapid testing to two factors. First, the optimization tool (Optimizely.) A key criteria in choosing the team’s tool was finding one that would enable rapid testing and empower the end-users. This would become very important to ensuring the team would meet their optimization goals, but also showcased the value of switching to a new tool.
The second factor was the team’s focus, which allowed them to ask more questions about their customer behavior and find answers faster, while moving the product as a whole in the right direction. As Suma puts it, the risk of not moving quickly is that ideas will be deliberated, but never executed. Their team took the opposite approach, and was able to reach very tangible, quantifiable goals for their business.
In this experiment, Suma and her team built an experiment hypothesis with intelligence collected from their web analytics.
On the search results page, conversions on right-rail content were relatively low. The Move team wanted to find a way to engage visitors who might not be ready to click on listings, but are looking for other information that will help them progress towards choosing a home. They hypothesized that surfacing links to intermediate steps in the home-buying process would improve search engagement with the right-rail and make conversions in later stages of the conversion funnel more seamless.
Suma and her team added a module (outlined in red) that highlighted getting pre-qualified and checking a credit score as logical next steps for visitors.
The Result: The change in right-rail content didn’t improve conversions, but it did provide an opportunity for further testing. The team was able to move on from the type of content surfaced to exploring other elements of the right rail functionality, and how it could improve the visitor experience.
Suma Warrier
Senior Optimization Manager, Move, Inc., Move Inc
In usability studies, Move researchers found that when viewing listing pages for home, it was generally simple to complete an action on the page (saving, requesting details, etc.). When it came time to navigate away from the page, visitors were getting stuck. They wanted to see other houses and view other results from their searches.
The optimization team hypothesized that adding a “Back to search” link would help users understand where to navigate once they were ready to leave a listing page.
A “Back to search” link was added to the navigation elements, next to a breadcrumb. This experiment focused on helping visitors to engage with multiple listings during their sessions, increasing the likelihood that they would advance further down the conversion funnel.
The Result: The change positively affected the engagement of visitors on listing and search pages, demonstrating that the researched change made a quantifiable impact on the visitor experience.
By maintaining close alignment and open communication with their analytics and user research peers, the Move, Inc. team is able to consistently produce high-quality experiments that help each group to accomplish and measure their goals while improving the usability of their site and overall customer experience.
As Move’s optimization program has become more visible and successful, many other teams have started to brainstorm areas where they wanted to test and optimize. Suma hopes that in 2015, the team will successfully scale best practices for ideating, prioritizing, and executing experiments.
In the meantime, Suma and her team are now focused on ramping up new channels where they want to be optimizing, including optimizing for mobile visitors.