Just for context – this is the 3rd and final chapter of where we tell you EXACTLY how we ate our own dog food. Meaning: How we at Optimove have been using Optimove to do precisely what we’re preaching for: a) going from tens to hundreds of customers segments, and b) building a super-personalized, multichannel, orchestrated CRM Marketing strategy based on all these segments and the different customer lifecycle stages.

Why did we do it?

We are using Optimove for our own needs because it is such a unique, powerful tool.

We are telling you all about it in detail because marketing leaders often hear things like “hundreds of customer segments” and freak out. This series of blogs breaks it down in the simplest terms, showcasing just how easily doable it is, with the right technology.

And now, we’re getting to chapter C, where we show you how you can measure and optimize this whole thing regularly.

Because having the proper infrastructure and even the right strategy on top of it, is just not enough. As data-driven marketers, we always must make sure we’re directly contributing to the company’s revenue. Measurement and optimization are the way to do it.

So, here, we’ll go over how I measured the effectiveness of our CRM strategy, including:

– Why measuring the bottom-line impact of your CRM strategy is necessary?

– What can you do when you can’t use the traditional Test/Control methodology?

– The advantages and disadvantages of my testing strategy

Let’s go.

Measuring the bottom-line impact of your CRM strategy

First, you must accept that in order to prove that your CRM strategy drives revenue, you have to run tests that include control groups (that are excluded from your communications). And no, it really does not mean leaving money on the table.

While Optimove’s Test/Control testing solution is exceptionally robust compared to most solutions, there are cases where other methodologies are required, and Optimove’s flexibility comes into play.

So, What Can I Do if I Can’t Run the Classic A/B/n Test?

In my previous posts in the series, I described how most of Optimove’s customer marketing is sequenced. That means that in most of our core content sequence/franchises, each campaign is only sent to customers that received the previous campaign. While this concept serves us well for our own strategy, it presents a new challenge for testing.

Activating a Test/Control campaign for each of these sequences will allow control group customers to “exit” the chain since they have not received the previous campaign. For example, a customer can receive our weekly tip #1 and then fall in the control group of weekly tip #2, breaking the chain and not receiving #3.

That would not be ideal, of course. To help with that, we needed to create a top-level measurement system for the entire sequence/franchise – a gigantic, journey-level A/B test.

So, we did.

Instead of looking at a single touchpoint within the customer’s journey, we analyzed our customers based on two groups – one that received our communications and one that did not receive ANY.

Let’s dig deeper and see how this is done.

Meet Super Control: A Way to Measure your Entire Strategy as a Whole.

Optimove offers two ways to move from campaign measurement to journey-level measurement:

• Super Control Group – where we measure the entire communications strategy vs. No communications at all.
• Streams – where we measure two different (entire) journeys.

In my case, I decided to go with the super control method.

Why Super Control?

In Super Control, for each campaign along the treatment, we exclude the same customers, which gives us a benchmark that shows what results will look like if we avoid customer communication altogether.

Since, in my case, I was launching a completely new treatment/journey/franchise/CRM Marketing initiative, I needed to measure the impact of this strategy against a no-communications strategy.

We may have chosen Streams as our testing method if I wanted to compare two different communication strategies. But that’s, in my case, for perhaps a later stage – when optimizations come into play.

But now, I needed first to validate the mere necessity of this whole new treatment. And if results showed the control group did better, it most likely meant my treatment required not just a tweak but a re-think.

Of course, Optimove makes it easy to execute such a testing solution, as it is built into the platform. It independently splits your customer base into slices, each accounting for a pre-set percentage of your customer base. This is called the “random customer slice” attribute. Using this attribute, we can choose a random slice, for example, slice #10, and use it as our control group all throughout the treatment.

Super-Controlling Our Multichannel Communications

Since the CRM strategy here is a multichannel one, we had also to ensure that the control group is enforced across channels.

So, before launching our test, we made sure to implement the same process across all our communication channels, using the same targeting rule across channels. 

Measuring What Matters Is Too Important to Avoid, but It Has Its Cost

The cost of leaving customers unattended. But, as we said before, this cost is lower than what you gain from being 100% certain is that what you are doing is working. And if measuring our strategy requires withholding some customers from it until we prove its value, then that’s what we gotta do.

Additionally, using the super control does not necessarily mean some customers will be completely neglected. In my case, I kept two content franchises active even to control customers: Product Updates and Weekly Site Performance. Which are unrelated to the treatment I launched and tested.

Bottom line is, to measure your CRM strategy successfully, you must find the testing methodology that works for you. Once you find the right testing methodology, you must apply it across the board to ensure you’ll always have a way to point at your goal and say with confidence what your contribution is. If your CRM marketing is any good, it will be worth the while for you, your team, and the entire company’s bottom line revenue.

Published on November 25th, 2021

Posted in

• Data Analysis

Dani Amitay

As a Product Marketing Manager at Optimove, Dani's focus is on product adoption and retention among existing customers. She has a rich background at the intersection of product and marketing, leveraging both to create sustainable growth. And if you’re ever traveling to Tel Aviv and need any advice on the trendiest yummy spots, send her a message on Instagram.