Expert answer · 5 min read

How do I set up an A/B test for my discounts in Growth Suite?

I'm looking to optimize my e-commerce store's discount strategy but I'm unsure how to systematically test different approaches. I want to understand how to set up A/B tests for my promotional campaigns that will help me identify which discount methods drive the most conversions and revenue. My goal is to move beyond guesswork and make data-driven decisions about my pricing and promotional strategies, but I'm not sure where to start or what specific parameters I should be testing.

The short answer

In Growth Suite, you set up a discount A/B test by creating two offer variants inside a campaign, letting the tool split eligible visitors randomly between them, and keeping a holdout group that sees no offer at all so you can measure true incremental lift. The rules that make the result trustworthy matter more than the clicks: change only one variable per test (discount depth OR message OR timing, never two at once), run the test for at least 1 to 2 weeks to cover full weekly cycles, and wait for roughly 100 conversions per variant before reading the outcome. Judge variants on revenue and profit per eligible visitor, not just redemption, because a deeper discount often wins on redemptions while losing on margin. When a winner emerges, roll it out and then test the next single variable. Resist peeking at results mid-test; early leaders flip surprisingly often, and stopping early is how false winners get crowned.

In depth

How do I set up an A/B test for my discounts in Growth Suite?

Create a campaign with two variants, let Growth Suite split traffic between them randomly, and keep the discipline that makes the result mean something. The setup takes minutes; the patience takes two weeks. Here is the process and the standards behind it.

Step by step

  • Pick one question. "Does 10% or 15% convert better?" is a test. "Which campaign is better?" is not, because too many things differ at once.
  • Create two variants. Duplicate your offer and change exactly one thing: depth, message, or timing. Everything else stays identical.
  • Split traffic randomly. Growth Suite assigns eligible visitors to variants randomly, which is what makes the comparison fair.
  • Keep the holdout. A slice of visitors, typically around 10%, should see no offer at all. This is your true baseline for incremental lift.
  • Run it long enough. Minimum 1 to 2 weeks and roughly 100 conversions per variant before reading results.

Why the holdout matters more than the variants

Without a no-offer group, you can only learn which discount beat the other, not whether either one added anything. A variant that converts 8% of eligible visitors sounds great until the holdout tells you 6% would have bought anyway. The gap between variant and holdout is the only number that justifies the discount's cost. When any tool or case study claims a big lift, the first question is always: measured against which holdout?

Judge the winner on profit, not applause

Redemption rate measures curiosity. To pick a real winner, compare:

  • Incremental conversions: variant conversions minus holdout baseline.
  • Revenue per eligible visitor: total revenue divided by everyone who could have seen the offer.
  • Margin after discount cost: the deeper code often wins conversions and loses money once you subtract the discount on orders that would have happened anyway.

The variant with the best profit per visitor is the winner, and that is frequently not the biggest discount.

Mistakes that ruin discount tests

  • Testing two changes at once. If depth and copy both change, you learn nothing about either.
  • Peeking and stopping early. Early leaders flip regularly. Commit to the runtime before you start.
  • Tiny samples. Calling a test at 20 orders per side crowns noise as strategy.
  • Testing during anomalies. A test running across a big sale or a viral spike measures the event, not the variant.
  • Testing things too similar. 10% versus 11% will never separate; pick arms far enough apart to move behavior, like 10% versus 20%.

After the test: adopt, keep measuring, iterate

When a variant wins, roll it out as the new default, but keep a small holdout alive so you always know what the offer still adds. Then queue the next single variable: if depth is settled, test timing; if timing is settled, test the message. Stores that treat testing as a loop rather than a one-off project compound small wins into large ones over a few quarters.

How Growth Suite handles this

Growth Suite's testing is built around holdouts by default: random assignment, a no-offer control slice, and reporting that shows incremental lift rather than raw redemptions. Because offers are targeted at hesitant visitors in the first place, your tests measure the effect on the audience that actually needs convincing.

Frequently asked questions

What should I test first?
Depth. It has the biggest profit impact and the clearest data behind it.

Can I test three variants at once?
You can, but each variant needs its own 100 conversions, so runtime stretches. Two variants plus holdout is the practical default.

My store is small; can I still test?
Yes, with patience. Low-traffic stores simply need longer windows, sometimes a month or more.

What if both variants tie?
Then choose the cheaper one for your margin and test a bigger difference next time.

Should the winner become permanent?
Adopt it, keep the holdout running in the background, and queue the next variable. Optimization is a loop, not a project.

Growth Suite for Shopify

Put this answer to work in your store.

Growth Suite reads what each visitor is about to do and triggers the right offer at the right second — so the strategy above runs on its own.

14-day free trial · No credit card · Cancel anytime

Go deeper

Resources

All resources
Shopify Upselling & Cross-Selling - Growth Suite

Topic ·9 min read

Shopify Upselling & Cross-Selling

Most Shopify stores leave 10-30% of revenue on the table because they do not upsell or cross-sell effectively. This...

Shopify Countdown Timer - Growth Suite

Topic ·4 min read

Shopify Countdown Timer

Adding a timer takes 5 minutes. Making it convert without destroying trust? That's strategy. The complete guide to...

Shopify Cart Abandonment - Growth Suite

Topic ·3 min read

Shopify Cart Abandonment

70% of Shopify carts are abandoned. Learn why customers leave, how to prevent abandonment in real-time, and recover...

Shopify Discount - Growth Suite

Topic ·3 min read

Shopify Discount

Master Shopify discounts without sacrificing your profit margins. Learn strategic discount techniques, timing...