A/B TestingMedium
What's the 'peeking problem' in A/B testing?
Seen at: Meta · Netflix · Stripe
An eager PM checks the experiment dashboard every day. On day 4 they see p=0.048 and want to ship. Why is this a problem?
Draft your answer
Saved to this browser only. Try it before you peek at the model answer.
Stuck? Peek at a hint
Reveals the model answer and the self-score rubric.