Expert answer · 5 min read
How do I set up an A/B test for my discounts in Growth Suite?
The short answer
In depth
How do I set up an A/B test for my discounts in Growth Suite?
Create a campaign with two variants, let Growth Suite split traffic between them randomly, and keep the discipline that makes the result mean something. The setup takes minutes; the patience takes two weeks. Here is the process and the standards behind it.
Step by step
- Pick one question. "Does 10% or 15% convert better?" is a test. "Which campaign is better?" is not, because too many things differ at once.
- Create two variants. Duplicate your offer and change exactly one thing: depth, message, or timing. Everything else stays identical.
- Split traffic randomly. Growth Suite assigns eligible visitors to variants randomly, which is what makes the comparison fair.
- Keep the holdout. A slice of visitors, typically around 10%, should see no offer at all. This is your true baseline for incremental lift.
- Run it long enough. Minimum 1 to 2 weeks and roughly 100 conversions per variant before reading results.
Why the holdout matters more than the variants
Without a no-offer group, you can only learn which discount beat the other, not whether either one added anything. A variant that converts 8% of eligible visitors sounds great until the holdout tells you 6% would have bought anyway. The gap between variant and holdout is the only number that justifies the discount's cost. When any tool or case study claims a big lift, the first question is always: measured against which holdout?
Judge the winner on profit, not applause
Redemption rate measures curiosity. To pick a real winner, compare:
- Incremental conversions: variant conversions minus holdout baseline.
- Revenue per eligible visitor: total revenue divided by everyone who could have seen the offer.
- Margin after discount cost: the deeper code often wins conversions and loses money once you subtract the discount on orders that would have happened anyway.
The variant with the best profit per visitor is the winner, and that is frequently not the biggest discount.
Mistakes that ruin discount tests
- Testing two changes at once. If depth and copy both change, you learn nothing about either.
- Peeking and stopping early. Early leaders flip regularly. Commit to the runtime before you start.
- Tiny samples. Calling a test at 20 orders per side crowns noise as strategy.
- Testing during anomalies. A test running across a big sale or a viral spike measures the event, not the variant.
- Testing things too similar. 10% versus 11% will never separate; pick arms far enough apart to move behavior, like 10% versus 20%.
After the test: adopt, keep measuring, iterate
When a variant wins, roll it out as the new default, but keep a small holdout alive so you always know what the offer still adds. Then queue the next single variable: if depth is settled, test timing; if timing is settled, test the message. Stores that treat testing as a loop rather than a one-off project compound small wins into large ones over a few quarters.
How Growth Suite handles this
Growth Suite's testing is built around holdouts by default: random assignment, a no-offer control slice, and reporting that shows incremental lift rather than raw redemptions. Because offers are targeted at hesitant visitors in the first place, your tests measure the effect on the audience that actually needs convincing.
Frequently asked questions
What should I test first?
Depth. It has the biggest profit impact and the clearest data behind it.
Can I test three variants at once?
You can, but each variant needs its own 100 conversions, so runtime stretches. Two variants plus holdout is the practical default.
My store is small; can I still test?
Yes, with patience. Low-traffic stores simply need longer windows, sometimes a month or more.
What if both variants tie?
Then choose the cheaper one for your margin and test a bigger difference next time.
Should the winner become permanent?
Adopt it, keep the holdout running in the background, and queue the next variable. Optimization is a loop, not a project.
Growth Suite for Shopify
Put this answer to work in your store.
Growth Suite reads what each visitor is about to do and triggers the right offer at the right second — so the strategy above runs on its own.
14-day free trial · No credit card · Cancel anytime
Go deeper
Resources
Shopify Upselling & Cross-Selling
Most Shopify stores leave 10-30% of revenue on the table because they do not upsell or cross-sell effectively. This...
Shopify Conversion Rate: The Complete Resource Hub
Your conversion rate is the most important number in your Shopify store. Learn how to measure it, diagnose problems,...
Shopify Holiday & Seasonal Campaign Strategies
Every holiday has its own shopping psychology. Plan year-round campaigns with the right discount, right timing, and...
Shopify Countdown Timer
Adding a timer takes 5 minutes. Making it convert without destroying trust? That's strategy. The complete guide to...
Shopify Cart Abandonment
70% of Shopify carts are abandoned. Learn why customers leave, how to prevent abandonment in real-time, and recover...
Shopify Discount
Master Shopify discounts without sacrificing your profit margins. Learn strategic discount techniques, timing...