Spread and Standard Deviation
Understand standard deviation as 'typical distance from the average' — and use it to flag unusual values.
Two call centers both have an average handle time of 8 minutes. But:
- Center A: almost every call is 7–9 minutes (consistent)
- Center B: calls range from 2 minutes to 30 minutes (unpredictable)
Same average, completely different operations. Standard deviation is what captures this difference — it measures how spread out values are around the average.
Std dev = the 'typical distance' of values from the mean.
import pandas as pd
center_a = pd.Series([7.5, 8.0, 8.2, 7.8, 8.1])
center_b = pd.Series([2.0, 15.0, 4.0, 30.0, 5.0])
print(f'A mean={center_a.mean():.1f} std={center_a.std():.1f}') # mean=7.9, std=0.3
print(f'B mean={center_b.mean():.1f} std={center_b.std():.1f}') # mean=11.2, std=11.4Center A: std = 0.3 → calls are typically within 0.3 minutes of the average. Very consistent.
Center B: std = 11.4 → calls vary wildly. High standard deviation = unpredictable process.
Which Series has a higher standard deviation?
import pandas as pd
a = pd.Series([10, 10, 10, 10, 10])
b = pd.Series([1, 5, 10, 15, 19])
print(a.std(), b.std())- A0.0 7.11...
- B7.11... 0.0
- C0.0 0.0
- D5.0 7.0
In most datasets, nearly all 'normal' values fall within 3 standard deviations of the mean:
mean = df['response_time'].mean()
std = df['response_time'].std()
# Flag anything more than 3 std devs away as anomalous
df['is_anomaly'] = (df['response_time'] - mean).abs() > 3 * std
print(f'Anomalies: {df["is_anomaly"].sum()} ({df["is_anomaly"].mean()*100:.1f}%)')This is faster than the IQR method and works well for roughly normal distributions. For heavily skewed data, use IQR instead.
A column has mean=100, std=15. A value of 155 is found. Is it anomalous by the 3-sigma rule?
- ANo — 155 is only slightly above 100
- BYes — 155 is (155-100)/15 = 3.67 standard deviations above the mean, beyond the 3-sigma threshold
- CNo — the 3-sigma rule only flags negative values
- DYes — any value above the mean is anomalous
Std dev is in the same units as the data. You can't directly compare the std dev of salaries ($10,000) to the std dev of test scores (8 points). Coefficient of variation (CV) normalizes spread as a percentage:
# CV = std / mean × 100
cv = df['salary'].std() / df['salary'].mean() * 100
print(f'CV: {cv:.1f}%') # e.g., 22% → moderate spread
# Compare two metrics on different scales
cv_salary = df['salary'].std() / df['salary'].mean() * 100
cv_score = df['score'].std() / df['score'].mean() * 100
print(f'Salary CV: {cv_salary:.1f}%, Score CV: {cv_score:.1f}%')CV < 15% = low variation. CV 15–35% = moderate. CV > 35% = high variation.
Calculate the standard deviation of the 'session_duration' column:
std_dev = df['session_duration'].___()Product A has mean rating 4.2, std=0.3. Product B has mean rating 4.2, std=1.8. What does this tell you?
- ABoth products are equally received — they have the same average
- BProduct A has consistently high ratings; Product B is polarizing — high variance suggests some users love it and others hate it
- CProduct B is better because higher std means more engagement
- DStandard deviation is not useful for rating data