Contending Salesforce

Salesforce is… well, you know, Salesforce. 

A reminder: “Salesforce provides customer relationship management (CRM) software and applications focused on sales, customer service, marketing automation, analytics, and application development.” 

Thank you Wikipedia. We’re all familiar with the titan technology company that has grown to dominate the world of core business apps and workflow tools.

Our client, Twiddy & Co, had been using their suite of products for quite some time: CRM, marketing automation, and email services. Looking to improve marketing performance, they layered on another product: Interaction Studio, Salesforce’s content-personalization brain for website and email marketing.

But as we’re constantly reminded, personalization is central to today’s digital buying experience. Not only do customers expect it, but personalized recommendations are responsible for an enormous proportion of online activity:

• 40% of Amazon’s click-through is from product recommendations
• 75% of time spent watching Youtube were based on personalized experiences
• 80% of Netflix hours viewed are from tailored movie & tv suggestions
• 90% of Tripadvisor Bookings were recommended to the travelers *

Given the engagement, conversion, and revenue impact that custom-tailored buyer journeys can have, Twiddy’s marketing team made a bold decision: challenge the tech giant’s personalization capabilities. Put Salesforce “Einstein” artificial intelligence to the test. 

A real head-to-head performance bake off. The new contender: Aidaptive’s machine learning engine.

If you’re keeping score at home, we missed one introduction. In the other corner: Aidaptive is a predictive personalization engine that packages and delivers the power of enterprise-grade machine learning to any business.

Breakdown of the Experiment

Let’s talk about how the test was designed.

The assessment was simple and straightforward. Why? So that the results could be determined easily. In fact, Twiddy hired a 3rd party analyst consulting company to run non-biased data analytics for results comparison.

The goal was to determine the efficacy and accuracy of personalized recommendations to individual website visitors; the method was to use dynamic widgets on the website. 

In order to maintain simplicity, these widgets lived only on the homepage – the primary landing page for new traffic. 

The split in traffic looked almost exactly like a normal A/B test. The marketing team set up their webpage to send 50% of web traffic to Salesforce’s personalization widget, and 50% to that of Aidaptive.

The success criteria was a simple metric of site engagement, to measure the relevance of recommendations. 

We used click-through rate in two flavors: 

• A) clicks on any recommendation provided by one widget or the other, and 
• B) the accuracy of each tool’s recommendation, understood by calculated rank.

To make that clear, each personalization engine calculated top recommendations for each individual on the site. The tool then ordered the recommendations from most-highly-likely to least. For example: clicks on the number one recommendation indicated the prediction was more accurate, versus the 2nd or 3rd ranked recommendation.

The evaluation of this test, as stated, was done by a neutral 3rd party – hired for analytical rigor.

Reviewing the Test

The several-week test generated improved clickthrough for our client – the underlying aim of the whole exercise! And for that, everyone was happy.

What happened more specifically? The results looked like this:

• Aidaptive generated better daily performance over Salesforce for 93% of the duration of the test.
• Aidaptive outperformed Salesforce by 250% in clickthrough rate on recommendations for repeat website visitors and past customers. These are what we call the “known audience” – where data stored in the CRM (like Salesforce), in addition to all of the transaction data and marketing engagement data was available for both tools to use in algorithms and predictions.
• Notably, for the rest of the web audience – those site visitors who are completely anonymous and unknown – Aidaptive drove an additional 12% more traffic down the conversion funnel compared to Salesforce.
• Finally – and greatly appreciated by Twiddy – Aidaptive’s recommendations continued improving over time. This is one of the desirable attributes of machine learning after all. So after 2 additional months of personalization widgets on the clients website, the Aidaptive CTR results were winning against Salesforce’s precedent by 5.3x on click-through for anonymous site visitors.

Why did it work out this way?

In a head-to-head performance test like this (where most variables are kept equal), several factors come into play that can be differentiators:

• The audience target
• Which data each tool uses to inform calculations
• The infrastructure running behind the scenes
• The accuracy and intricacy of the predictive models
• The manner of deploying the predicted output

It’s hard to pin-point exactly “why” the performance had the stated difference, except to say it was a combination of the above.

For example, we know Salesforce had access to all the same data sources – website, CRM, marketing, transactions – but we can’t peek under the hood to see what the Salesforce artificial intelligence is prioritizing. 

That said, we can highlight a major focus point of the Aidaptive engine: machine learning infrastructure and operational plumbing.

But the bottom line… this is the world of marketing, which can be summarized as one huge experiment to find out what works. We do more of what performs and reduce waste from what doesn’t.

Aftermath

As a result of the uncontested experiment testing Aidaptive vs Salesforce on the website experience, our client replaced Salesforce Interaction Studio as the personalization engine behind both the website experience and another facet of their revenue funnel: dynamic email campaigns.

While Salesforce is an incredibly powerful suite of tools – and firmly remains Twiddy’s CRM and marketing automation platform – Aidaptive is the engine layer that sits on top of the marketing, website, CRM, and sales data to generate intelligence, power personalization, and optimize all digital customer experiences.

Their head of marketing recognized the ongoing upward-trend as a differentiator between the two tools:

“I think there was a meaningful difference between Salesforce and Aidaptive results because the Aidaptive machine learning continuously retrained and improved its models.”

While Aidaptive doesn’t currently run advertising optimization in ad-publishers, our team was excited to hear their CEO mention the ROAS (return on ad spend)-impact of using Aidaptive. 

He told us that the efficiency of their website throughput increased to the degree where their cost savings on ad spend were more than the Aidaptive’s annual software fees. (And this was only over the first few months of the calendar year!)

An Underdog Story?

We all love a good underdog story, overcoming the odds. David versus Goliath. The little guy taking on the obvious favorite. 

However, part of the secret behind Aidaptive’s proficiency in delivering world-class machine learning infrastructure is in the pedigree. Much of the team are alumni – even leadership – at Google, possibly the most widely trained and used machine learning on the planet. 

So while a win against Salesforce is extremely exciting, the team at Aidaptive has pretty high standards already.

The next step on the journey, democratize the power of ML. Our mission is to be the most accessible and easy to use end-to-end machine learning platform. As a fully managed MLaaS solution, we enable companies to deploy a revenue generating predictive personalization engine to their website and marketing campaigns in minutes. 

Interested in learning more? You can contact us at [email protected]! 

* Source: webinar