Follow a 5-step framework. Narrate out loud; don't skip steps because you have a hypothesis.

1. Is the drop real? Before you hunt for a cause, rule out the measurement.

Did the pipeline run on time? Any upstream job failures or late arrivals?
Any logging / SDK change shipped yesterday? A renamed event, a reshuffled funnel step, a new version cutover?
Does the raw source (payments, orders) agree with the warehouse number?
Is it the metric definition that shifted (e.g., a filter someone tweaked in the dbt model)?

If you can't answer these, you have a data problem, not a business problem.

2. Segment. Slice before theorizing.

Device (iOS vs Android vs web) — did a release go out?
Geo — concentrated in one country (shipping, currency, competitor)?
Traffic source — is a paid channel drying up or targeting different users?
New vs returning — is it an acquisition-mix problem or a retention problem?
User cohort — any specific signup cohort moving?

3. Decompose the metric. Is it volume or rate? Numerator or denominator?

Did sessions drop (denominator) or did conversions per session drop (rate)?
Which funnel step? Is it pre-checkout attrition or a payment-step failure?

4. Correlate external factors.

Marketing change? (campaign paused, creative swap)
Product launch? (A/B exposure, new feature default)
Competitor move, seasonality, holidays, an outage upstream?

5. Form a hypothesis — then size it. Identify the segment(s) that moved most. Compute how much of the overall drop they account for. If one segment explains most of it, you have your story. If nothing explains it, loop back to step 1 — it might still be a measurement bug.

What to tell leadership at noon: “Here's what dropped, here's which segment drove it, here's my best hypothesis, and here's what I'd do to confirm in the next 24 hours.” Don't guess. Don't blame a bug unless you’ve ruled everything else out.

Metric X dropped 20% overnight. Walk through your investigation.

Draft your answer

Stuck? Peek at a hint

More product metrics questions