As the Lead Strategy Consultant at Optimizely, I’m lucky to work with customers of differing levels of Experimentation program maturity, from customers who are just starting their program and getting their team in place to customers that never make a decision without experimenting first. My work focuses on building Experimentation methodologies with a company’s teams while setting the structure and measurement practices to grow the program.

I’ve also been on the other side! Up until joining Optimizely, I ran the Experimentation program at the American Medical Association (AMA). Growing Experimentation at a non-profit brought its own challenges, but working with an Executive Sponsor who helped me drive the program forward was the first step towards building a true culture of Experimentation.

At the AMA I started from the same point I see a lot of Experimentation programs start. We had to be scrappy and find any opportunity to run an A/B test. These were often limited by where we could test (typically lower traffic pages) and what we could test (typically messaging and imagery).

But as we found some wins that impacted important site metrics and understanding of our important segments, leadership took notice. There was a conscious effort to put more resources behind Experimentation and grow it from just a side practice of our Digital Analytics team. We had the buy-in to create a fully dedicated Experimentation program.

What is our definition of an Experimentation program?

An Experimentation program goes beyond running some simple A/B tests to optimize conversions. It’s much bigger than that. An Experimentation program is the process of integrating Experimentation into your company’s workflow and its culture. The entire company embraces Experimentation methodology, and multiple departments conduct experiments, learn from them, and surface those insights to the rest of the organization.

A true Experimentation program has set workflows, governance, resources, and is directly connected to your company’s KPIs. There are goals set about how many tests you will run and the positive impact those should have on the company’s KPIs. Most programs start at a Level 1, like we did at the AMA, but there is plenty of room for growth along Optimizely’s Maturity Model.  As an Experimentation program reaches maturity, there is typically technology dedicated to conducting all Experimentation and the metrics are tied to company success.

Why does an Experimentation program need an Executive Sponsor?

Creating an Experimentation culture is hard. It requires an organizational mindshift — and is likely one of the biggest digital initiatives a company will undertake in a year. When I was being scrappy at AMA I realized I didn’t have a seat at every table when it came to planning digital initiatives for the organization. I had to push this forward at those tables by having my Executive Sponsor vouch for our successes and impact.

An Executive Sponsor helps ensure success by:

• Participating visibly and publicly
• Communicating support for Experimentation
• Building coalition within the organization
  

A program owner drives the change, but an Executive Sponsor can advocate for this change at the executive level, help connect Experimentation to the core business, and allocate for what is ultimately a highly cross-functional/cross-departmental effort. You need a quarterback on the field (the program owner), but you also need a coach on the sideline to make calls when change is needed.

Your Executive Sponsor will also need to connect Experimentation to those important company KPIs (Key Performance Indicators) and make calls on your behalf with those other coaches (executives) in the company. For example, if you work in e-commerce, what are the targets to improve incremental revenue and purchases from Experimentation? On the flip side, how can Experimentation help reduce costs associated with returns or your call center? Setting these targets in your annual planning for overall digital is something only an Executive Sponsor can effectively do.

How do I choose the right Executive Sponsor?

There’s no universal “right” person, although you will achieve the best results through someone who wields influence and can galvanize support within your company. Explore what makes the most sense for your situation. Your sponsor may lead the group closest to your digital measurement – in many organizations, this is the head of analytics or consumer insight. Or they might work within the group that ships your products — the head of product development. (It may be out of your hands completely and just the group who pays for Optimizely — that’s okay!)

Whatever the department, keep two primary things in mind as you identify your sponsor:

1. Who is tying their personal and team results (e.g. OKRs, or objectives and key results) to Experimentation?
2. Who can quickly unblock some of the primary challenges when starting your program (more on that in the next section)?

#1 is the most important. To truly shift to an Experimentation-first company, you have to actively set goals and performance measures for the individuals participating. Which executive is willing to put pen to paper on this? At Optimizely, our Executive Sponsor is the SVP of Product, Claire Vo,  who ensures that each Product Manager has goals — in our case, OKRs — for the number of experiments run, number of features with experiments, etc. Find someone who will set this direction from the top.

What are the primary responsibilities of a program’s Executive Sponsor?

Once you’ve selected your sponsor, now it’s your turn to coach them. Here are the main things to ask for:

1. Resources, resources, resources. Your Experimentation program needs support from many cross-functional teams to thrive. So, when there’s a gap, you’ll need your sponsor to find solutions to fill it.
2. Engineering support is key. When there’s a need for integrations or engineering work, your sponsor should help gain the support required to unlock that cross-department work.
3. Communicate the value of Experimentation. Asking, “Where does Experimentation fit in here?” when company initiatives / campaigns are planned and in executive levels meetings they attend. Your sponsor should always remind others how Experimentation can contribute to solving to all customer problems.
4. Stay top of mind. Sharing results and learnings greases the wheels of an Experimentation program. Get your sponsor to share what you’re uncovering in front of the company as often as you can.
5. Present Experimentation results regularly. Set up quarterly reviews of the program where your sponsor can provide feedback on progress against KPIs and OKRs. As an active participant in our weekly and quarterly reviews, Claire reminds everyone of the importance of Experimentation to the company.

How do I keep the excitement up with my Executive Sponsor?

Keep in mind what matters most to your sponsor: what work makes them look good to their boss? Look to attach experiments to these key focus areas for your executive stakeholder.

• If their top priority in a given quarter is successfully launching a new feature on the site, how can Optimizely provide more data around customer behaviors?
• If it’s a new marketing campaign, can targeted experiments for our most important channels continue to the messaging and value props of the campaign? Would symmetric messaging move the needle for conversion even more?

Don’t walk out of a meeting (Experimentation-focused or not) when your sponsor is present without discussing how Experimentation could support the initiatives being discussed. Everything your team is doing contributes to solving the same customer problems and experiments can unlock a lot. I’ve seen this flip with one of my customers after the sponsor was “trained” to ask the question of how Experimentation can help solve the customer problem. Now he won’t let anyone out of the room without an experiment idea.  

Finally, run an experiment that you know will catch their eye: one that is the sponsor’s idea. Even if it jumps your prioritization practice, it will be worth it to keep them energized. Who doesn’t like to talk about their own ideas?

When I was working at the AMA, this was my tipping point. I also got my sponsor to put his name to his ideas as we shared results. This transparency illustrated that it was okay to be wrong and that we should really solved our customer problems through Experimentation.  

—————

So what are your immediate next steps?

1. Identify your Executive Sponsor (if you haven’t already).
2. Work with them to define individual performance measures (OKRs, V2MOM, etc) tied to Experimentation from the top down.
3. Get your Executive Sponsor to commit to participating in your weekly meeting.

These small steps can go a long way, but will set you down the path to a successful Experimentation program.

Also, read more about how to build out your program with you Executive Sponsor by downloading the Building an Experimentation Program ebook.

What other questions do you have about an Executive Sponsor for your program? Leave them as a comment below and let’s chat!