A multi-touch attribution alternative?
Look, I like multi-touch attribution. It’s a way to nice way to divvy up credit across the multitude of marketing activities needed to get a deal across the finish the line. But good multi-touch attribution is expensive and hard to implement. It can struggle with offline vs online, mobile vs desktop, and impressions vs clicks to name a few.
But as a marketing leader it’s your job to help determine if your marketing activities are actually working. Did that new revised homepage work? What about that big Youtube campaign? What about the substantial ABM investment? How about those billboards?
Marketing is hard. Stakeholders want answers. Your CEO, your board, your CFO, your CRO … better have some solid data because those questions are coming at the next e-staff or board meeting.
So today, I’d like to share a simple yet effective technique I’ve used to help get you those answers.
Control groups.
What’s a Control Group, and Why Does It Matter?
If you’ve ever taken a science class, you’re already familiar with the concept. A control group is the group that doesn’t get the “new thing” you’re testing. It serves as your baseline so you can compare it to the group that does get the new experience.
Why is this so important? Because without a control group, it’s hard to know if the results you’re seeing are due to the change you made or something completely unrelated. Maybe a competitor launched a new product, or a major economic event shifted customer behavior. Maybe you ran an event that same week. Without a baseline for comparison, you’re guessing at best.
Control groups let you measure the real impact of your marketing initiatives. And the best part? It’s free. No fancy tech required.
Real-World Examples of Control Groups in Action
Let’s talk about some practical applications.
Ad Campaigns
At Slack, we ran a lot of ad campaigns, not just to grow but to enter new markets, verticals, or even countries. Here’s how we used control groups to measure success:
• We’d pick three cities, like Denver, Austin, and Seattle, and run the ad campaign there.
• The rest of the U.S. became our control group, where no ads were shown.
• After the campaign, we’d compare KPIs like leads, awareness, pipeline growth, or even brand sentiment between the test cities and the control cities.
What we found was that campaigns didn’t just boost awareness. They lifted metrics across the funnel. More people signed up, used the product, and even spent more. Sometimes, existing customers would see the ads and spend more as well.
Control groups allowed us to measure the difference between the natural growth in awareness and the additional growth driven by the ads.
As we continued to grow we launched a nationwide (USA) campaign. Same playbook here, we used the rest of world as our control group and measured the lift in leads, pipeline and awareness.
Website Changes
When I was at Salesforce, we were rolling out a new homepage design. I loved it and thought it was going to do well. But I was adamant about keeping a control group where some visitors still saw the old homepage.
Why? Because external events can muddy your data. What if a competitor launched a product that same week? What if there was a major outage? Without a control group, we’d have no way of knowing whether the new homepage was actually driving results or if something else was at play.
By isolating the change, we could measure its true impact, whether it was better click-through rates, reduced bounce rates, or higher conversions.
ABM Campaigns
ABM is all the rage now in B2B marketing. I love them as well. They allow you to saturate your key accounts with your content and ads. Blissfully nurturing them to MQL and beyond.
So let’s say for instance you have a 100 target account list. Run the ABM campaign to 50. Have the other 50 NOT get the ABM campaign.
And then measure those two groups … Conversion to MQL, deal size, deal velocity, closed/won. If the ABM campaign is really kicking ass you should be able to see that in the data.
It’s Not Just for Marketing
Control groups aren’t limited to ad campaigns or website changes. They can be applied across your entire go-to-market motion.
Customer Success
Want to prove the ROI of a customer success program? Assign a control group of customers who don’t get a dedicated success manager and compare their performance, such as spend, NPS, or activity, to those who do.
Sales
Ever wonder if your sales team is just harvesting interest that would’ve closed anyway? Use control groups. Let one group of customers go through the normal self-serve flow while assigning a sales rep to another group. See if there’s a measurable difference in outcomes.
Why Control Groups Are So Powerful
Here’s the thing: Marketing teams often try to change too many things at once. They redesign the website, launch a new campaign, and roll out a product update all at the same time. Then, when results improve or don’t, they have no idea which change caused it.
Control groups force you to isolate variables and test one thing at a time. It’s not just a marketing tactic, it’s a decision-making superpower.
You don’t need a big budget or fancy attribution tools to get started. Whether it’s a new design, campaign, or program, just carve out a group that doesn’t get the change. Then, measure the difference. It’s that simple.
Your Turn
Have you used control groups in your marketing or go-to-market strategy? What worked, and what didn’t? Drop your thoughts in the comments. I’d love to hear how you’re applying this approach to measure success.
Thanks for reading!
Bill
Thanks for reading! If you are a podcast lover, please consider subscribing to my SaaS CMO Pro pod. It’s the same content but you get to hear my lovely voice when you are stuck in traffic.👇
🚀 About Bill 🚀
Bill Macaitis is an advisor and board member guiding aspiring AI & SaaS unicorns/decacorns. Bill works with companies to build out capital efficient go-to-market strategies and world class teams. Over his 30 year career Bill has achieved five successful exits and powered growth at three of the fastest-ever growing companies: Slack (CRO/CMO), Zendesk (CMO), and Salesforce (SVP of Marketing).
🦥 Follow SaaS CMO Pro 🦥
Podcast 🎤
Tiktok 🎵
LinkedIn 💼
Youtube 📺
Instagram 📸
Facebook 📘
💼 Work with SaaS CMO Pro 💼
Official website 🌐
Newsletter 📬
Board advisor 👔
My studio gear 🎥
Sponsorships 📢
Speaking opps 🎙️
Contact Bill ✉️
