A/B testing: Tips from a growth marketing pro

Jeff Goldenberg launched Abacus, a growth marketing agency, in 2016 with the idea that Abacus would be the best-paid social media and advertising company in the country.

The company is well on its way. It’s become a Facebook partner agency and has produced some of North America’s most innovative social advertising campaigns. As if that were not enough, Abacus has acquired a creative agency and media company to build even more strategic creative campaigns at scale.

Check out their creative case studies here.

We were fortunate to host Jeff as one of our Masters of Growth “Lunch & Grow” webinar speakers recently. He had a lot to share from his experience as a growth leader at Borrowell, a Toronto based fintech leader; as co-author of The Growth Hacker’s Guide to the Galaxy; and as co-founder of the former TRACTION Meetup Toronto (based on the book Traction by Gabriel Weinberg and Justin Mares).

During this session, Jeff explores these three critical themes:

1. A/B testing is not a tactic—it’s a mindset

“Not falling in love with your creative and adapting a ‘testing’ mindset will payoff threefold,” Jeff maintains.

How does this work?

Jeff believes far too many people think they are right. This means that they see their creative as the bomb, that they claim to know what kind of conversion will happen from channel to channel, and that they can predict future success based on the past.

However, these individuals or companies often lack one key understanding: A/B testing should be fundamental. The more you try meaningful and different things, the more you learn and the better your creative campaigns become. A/B testing needs to be part of the DNA of your company and creative campaigns.

Jeff may be on to something here. According to SocialCode data, campaigns that pre-test content are 20–25% more efficient.

Jeff goes on to explain that the secret of marketing is really to have an open mind. Nothing is off limits. Never fall in love with your creative and keep your ego in check.

So, how do we do this as busy growth marketers? This leads us to the next theme.

2. The idea to beat

Everyone on the growth team needs to be brainstorming the “idea to beat.” Sometimes the best idea is simply to get clearer on what exactly the team is testing (e.g., cohorts, time-of-day targeting, messaging). This does not mean testing willy-nilly, but rather, clearly defining the goal of the test, the outcome of the test, and how these tie back to business objectives.

A big mistake many companies make is testing stuff that isn’t meaningful. Jeff cautions that “unless you are Amazon, companies shouldn’t be testing the colour of their buttons.” If you are testing on Facebook or Instagram, it is important to understand how to set up A/B tests and operationalize them in your systems. Connecting CRMs with analytics platforms and testing software is a critical skillset. And being able to pump different landing pages by the market and segment you’re targeting is also key.

But Jeff also warns that if you are a B2B SaaS company, you have to think socially. Meaning, if you are testing content socially, it needs to speak to the audience and be social. No one is reading a white paper or case study socially.

Other tips for B2B companies include:

• Do not blast out content on social that is not social—it doesn’t produce a good ROI.
• Make your content for web as well as mobile, and be vertical-specific.
• Don’t put all your eggs in the Google basket. Algorithms change. It’s crucial to have a diverse approach to channel testing and always have a plan B, C and D for another channel to push to.

3. It’s a data game

When testing, make sure you consider the sample size you are testing against. It needs to be statistically significant in order to move the needle.

Jeff recommends applying growth-hacking principles to A/B testing in order to figure out what will work and what will not. Get your team in the mindset of proving or disproving assumptions. You’re looking for 10x growth, not 2x, so think really, really big in terms of what you test in order to gain a statistically significant result.

What do you test? Jeff recommends not getting too worried about testing buttons and visual layout and to stick to the messaging and creative execution. A lot of companies will test value propositions across multiple landing pages, for example. Tools like Optimizely or Facebook Canvas can help you refine your approach to testing.

CTA for growth marketers: Keep an open mind

Jeff ended his presentation with a call to action for every growth marketer to keep an open mind.

To learn more about the latest growth marketing tactics, tools and resources, I highly recommend you:

• Follow Jeff Goldenberg on Twitter.
• Pick up a copy of his book, co-authored with Mark Hayes: The Growth Hacker’s Guide to the Galaxy.
• Visit Jeff at Abacus.