The trap: comparing members to non-members.

Members retain better, spend more, and come more often. But they already had those attributes before joining — that's why they joined. You're not measuring the program; you're measuring that engaged customers self-select into programs.

The rigorous approach:

Regression discontinuity (if there's a signup threshold or free trial).
Matched cohort analysis. For each member, find a non-member with similar pre-join characteristics (AOV, visit frequency, tenure). Compare their post-join behavior to the member's.
Ideal: A/B test. Randomly assign some eligible users to be auto-enrolled (or offered enrollment prominently) vs. others who see no prompt. Measure behavior differences over 6-12 months. Gold standard.
Natural experiment. If the program rolled out in some markets before others, use difference-in-differences on pre/post trajectories.

What to measure:

Incremental spend: member spend minus matched-non-member spend. The question is 'how much of their spend happened because of the program?'
Incremental retention: does being a member slow churn?
Engagement frequency: more visits, higher basket sizes, cross-category purchases?
Margin impact: loyalty points cost the business — you pay out in future value. Net of that cost, what's the margin?

Program-specific metrics:

Redemption rate. What % of earned points get redeemed? Unredeemed points are revenue you've booked as a liability. Low redemption is good for unit economics, bad for program engagement.
Time to first redemption. Slow redemption = weak engagement; fast redemption = genuine value to users.
Point-earning behavior changes. Are members accelerating purchases to hit tier thresholds?

A likely finding:

"6-month loyalty program analysis. Naive comparison: members spend 35% more than non-members. Matched cohort analysis: incremental spend lift = ~8%. Accounting for loyalty-points liability cost (~3% of incremental revenue), net margin impact is roughly 5%. Conclusion: the program is working but the aggregate member-vs-non-member comparison overstates the effect by 4x. Continue, but monitor the redemption-liability ratio."

What the CEO should hear:

Yes/no on whether it's working.
The magnitude, with explicit discount for selection bias.
The cost side (points liability) netted out.
Recommendations: tiered thresholds, targeted offers, removal of features that don't drive behavior.

What senior candidates add:

Reference Sephora's Beauty Insider, Starbucks Rewards, Amazon Prime as industry benchmarks. Programs with published ROI analyses are useful references.
Acknowledge that programs have strategic value beyond ROI — data collection, customer lock-in, competitive moat. These are hard to measure but real.
Propose ongoing monitoring: monthly cohort analysis that detects program fatigue before it shows up in aggregate metrics.

How would you measure the success of a loyalty program?

Draft your answer

Stuck? Peek at a hint

More case studies questions