A/B TestingHard
What is Sample Ratio Mismatch (SRM) and how do you test for it?
Seen at: Meta · Microsoft · Airbnb · Netflix
An experiment assigns 50/50 to control and treatment, but at the end you see 48.2% control / 51.8% treatment across 200,000 users. Is that fine? How would you check, and what could cause it?
Draft your answer
Saved to this browser only. Try it before you peek at the model answer.
Reveals the model answer and the self-score rubric.