The two-sided question:

Benefit: users who complete the tutorial may activate faster and retain better.
Cost: users who bail before the tutorial completes have a worse first experience than if they'd been dropped straight into the product. Friction at the front door is expensive.

The experiment:

A/B test on new users only:

Control: no tutorial, dropped directly into product.
Treatment: 4-step tutorial before product access.

Measure at 30 days:

Activation rate (took core action in first week). Primary.
Retention at Day 7 / 14 / 30. The decision metric.
First-session abandonment rate. Did treatment users who abandoned before completing the tutorial come back?
Time-to-first-value. Users who complete tutorial vs. those who skip might differ on when they take core action.
Feature discovery breadth at Day 7. Did tutorial users actually discover features the control users missed?

Possible patterns:

Tutorial helps retention but not activation. Tutorial teaches something users don't figure out on their own. Ship it.
Tutorial hurts activation (frictional) but helps retention of completers. Net effect depends on the magnitudes. Often: trade near-term users for deeper engagement. Hard to call without more signal.
Tutorial helps everyone equally. Rare; usually suggests the tutorial is just a delay, not genuine instruction.
Tutorial hurts both. Clear loss. Kill it.

Refinements to consider:

Skippable tutorials. Let users opt out. Measures tutorial's lift among users who actually wanted it.
Targeted tutorials. Only show to users likely to struggle (e.g., from low-broadband regions, or users who take no action within 60 seconds).
In-context hints vs. upfront tutorial. Trigger help when user shows confusion; don't preempt.

Stakeholder framing:

"Tutorials can help or hurt. Let's A/B test with activation and 30-day retention as the decision metrics. If retention lifts 2+ points, we ship. If activation drops without retention lift, we kill. If it's a tradeoff, we look at LTV math."

What senior candidates add:

Reference the 'progressive disclosure' school of onboarding — teach features as needed, not upfront. Many top apps have moved away from upfront tutorials.
Consider tutorial completion as its own metric. A tutorial that 30% of users complete isn't delivering 70% of its hypothetical value.

Should a product add an onboarding tutorial?

Draft your answer

Stuck? Peek at a hint

More case studies questions