What happens when a seven-person data team restarts an experimentation program from scratch — and gets the whole company running experiments within months?
Welcome to the first episode of our special miniseries on product experimentation. Now that you can build almost anything faster than ever, what matters most is learning what to build, what to keep, and what to put back on the shelf. Over the next few episodes, we go beyond A/B testing basics to explore how teams build experimentation culture, move faster with data, and make smarter product bets in practice.
Hosting this series are two of Spotify’s finest experimentation nerds: Luke Frake, group product manager for subscriptions, and Mårten Schultzberg, staff data scientist for Spotify’s experimentation platform, Confidence (confidence.spotify.com).
In this first episode, we head outside Spotify to explore how Huel, the UK-founded nutrition brand, approaches experimentation across a rapidly growing omnichannel business. Unlike pure digital products, Huel sells direct-to-consumer on its own site, on Amazon, in supermarkets, and on TikTok Shop — which means you can't always rely on a classical A/B test to measure what's working.
Luke and Mårten are joined by Bhav Patel, Data Director at Huel, to dig into how his lean team rebuilt Huel's experimentation culture from the ground up. They discuss why Huel lets experiment ideas come from anyone and how a founder-led, data-first culture drove strong adoption of experimentation across the entire org. You'll hear how Bhav's team tackles the unique challenges of omnichannel measurement: building synthetic control groups to evaluate TV ads, running geo-lift tests, and stitching together fragmented user data across platforms, plus how AI is fast-tracking experiment design and more.
If you care about building better products with data, check it out — and stay tuned for more episodes in this series dropping soon.
Learn more about how we experiment at Spotify:
Dive deep into experimentation nerdiness on the Confidence blog
👉 Here’s a good post to start with: “Beyond Winning: Spotify's Experiments with Learning Framework”
Get started with Spotify’s product experimentation platform: Confidence
Become an experimentation expert with our free experimentation bootcamp
Follow the Confidence team on LinkedIn for product updates, event announcements, experimentation insights, and more
👋 Follow Luke on Linkedin
👋 Follow Mårten on Linkedin
Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com
You should follow us on Twitter @SpotifyEng, LinkedIn, and YouTube!
Netherlands