The key question: are these outliers noise or signal?

Noise outliers (data errors, sensor malfunction, input mistakes). Remove or fix.
Signal outliers (real rare events, whale customers, legitimate but infrequent). Keep and handle appropriately.

Detection methods:

Z-score / standard deviation method. Values > 3σ from mean. Works for roughly-normal distributions. Fails for skewed data (everything looks like an outlier on one side).
IQR method. Values beyond Q1 − 1.5×IQR or Q3 + 1.5×IQR. Robust to skew; industry default for boxplots.
Modified z-score / MAD. Uses median absolute deviation instead of SD. More robust than classical z-score to outliers themselves (chicken-and-egg problem).
Isolation Forest / other ML approaches. For multivariate outliers. Overkill for single-column.
Domain rules. 'Transactions over $1M are suspicious,' 'Ages over 120 are invalid.' Often the best detector.

Treatments:

Remove. Only if you're confident they're errors. Dropping 1% of data is usually fine; dropping 10% is substantive and biases analysis.
Cap / Winsorize. Replace values above the 99th percentile with the 99th percentile value. Preserves sample size; reduces influence.
Transform. Log-transform brings long tails to bearable. log(1 + x).
Keep, but use robust statistics. Median instead of mean; quantile regression instead of OLS. Changes the analysis, not the data.
Analyze separately. Whales might deserve their own cohort — their behavior is genuinely different.

Context-dependent choices:

For modeling (supervised): Winsorization or log transform often improves performance. Keep outliers as data; reduce their influence.
For descriptive statistics (means, counts): report with AND without outliers; let readers see the impact.
For executive dashboards: usually cap — don't let a single $10M purchase distort the 'typical customer' narrative.
For fraud / anomaly detection: outliers are exactly what you're looking for. Flag them, don't remove.

Example reporting:

"Transaction amounts range from $0.50 to $1.2M. Median is $40; 99.5th percentile is $850. The 0.5% of transactions above $850 (n ~ 500 of 100K) account for 27% of total revenue. These are enterprise / B2B transactions, not errors. For retail-segment analysis, I'll exclude them. For total-revenue analysis, I'll keep them. Reporting both."

Common mistakes:

Auto-removing outliers without investigating. You might delete legitimate high-value customers.
Using 3σ on skewed data. Revenue distributions are right-skewed; many 'outliers' are normal.
Silently treating outliers. A dashboard that drops the top 1% without disclosure misleads stakeholders.
Inconsistent treatment across analyses. If this quarter you capped and last quarter you removed, comparisons are invalid.

What senior candidates add:

'Outliers' is often a symptom of mixed populations. If your transaction data mixes retail and enterprise customers, 'remove the outliers' is just 'remove the enterprise segment.' Segment instead.
Report multiple summary statistics. Mean, median, and trimmed mean together give readers insight into the role of outliers.
Domain knowledge trumps algorithms. A business rule ('no customer should transact more than X per day') is more useful than IQR for most real datasets.

How do you detect and decide what to do with outliers?

Draft your answer

Stuck? Peek at a hint

More data cleaning questions