"Average delivery time in the sample was 32 minutes." How sure are we about the true average across all deliveries? A confidence interval answers that by giving a range instead of a single number.
The idea
A sample estimate will differ from the true population value by chance. A 95% confidence interval is a range built by a method that, if repeated on many samples, would capture the true value about 95% of the time.
A practical reading: "Our best estimate is 32 minutes, and the true average is plausibly between 30.5 and 33.5 minutes."
What makes an interval wider or narrower
- More data โ narrower. Width shrinks roughly with the square root of the sample size, so four times the data halves the width.
- More spread โ wider. Highly variable data gives less precise estimates.
- Higher confidence โ wider. A 99% interval is wider than a 95% one.
Interval for a mean
import numpy as np
from scipy import stats
x = np.array(delivery_minutes)
mean = x.mean()
se = x.std(ddof=1) / np.sqrt(len(x)) # standard error
low, high = stats.t.interval(0.95, df=len(x) - 1, loc=mean, scale=se)The standard error measures how much the sample mean would vary from sample to sample.
Interval for a proportion
For a conversion rate or a share of late orders, a simple approximation:
p = conversions / visitors
se = np.sqrt(p * (1 - p) / visitors)
low, high = p - 1.96 * se, p + 1.96 * seFor small samples or rates near 0% or 100%, use better methods such as the Wilson interval (available in statsmodels).
Bootstrap: an interval without formulas
Resample the data with replacement many times, compute the statistic each time, and take the middle 95% of results:
rng = np.random.default_rng(42)
boots = [rng.choice(x, size=len(x), replace=True).mean() for _ in range(5000)]
low, high = np.percentile(boots, [2.5, 97.5])This works for medians, ratios and other statistics where formulas are awkward.
Comparing two groups
When comparing, compute the interval for the difference (B minus A). If that interval excludes zero, the difference is statistically significant at the matching level. Two separate overlapping intervals do not automatically mean "no difference".
Common misreadings
- It is not "a 95% chance the true value is in this particular interval" in the strict sense; the true value is fixed, and the method has the 95% property. In everyday reporting, "plausible range" is a fine phrasing.
- It does not cover individual values: a 95% interval for the average delivery time says nothing about how long a single delivery might take.
- It does not fix bias: a biased sample gives a precise but wrong interval.
Reporting with uncertainty
Instead of "Conversion rose from 4.0% to 4.4%", write "Conversion rose from 4.0% to 4.4% (difference between 0.1 and 0.7 percentage points, 95% CI)". Decision-makers can then judge whether the change is meaningful.
Practise
Run bootstrap intervals yourself with the NumPy random set, and read hypothesis testing explained to see how intervals and p-values fit together.
Practise what you just read
Reading helps; writing the code yourself is what sticks. Start here: