Classification models are judged with confusion matrices. Regression models, which predict numbers like prices or delivery times, need different metrics, all based on the errors: actual minus predicted.
MAE: mean absolute error
The average size of the errors, ignoring direction.
MAE = average of |actual − predicted|If a delivery-time model has an MAE of 4 minutes, predictions are off by about 4 minutes on average. It is in the target's own units and easy to explain to anyone.
MSE: mean squared error
The average of squared errors. Squaring punishes big errors much more than small ones: an error of 10 counts 100 times as much as an error of 1. MSE is in squared units, which makes it hard to interpret directly, but it is what many models minimise during training.
RMSE: root mean squared error
The square root of MSE, back in the target's units. RMSE is always at least as large as MAE. A big gap between them means a few large errors dominate.
R²: coefficient of determination
The share of the variation in the target that the model explains, compared with simply predicting the average every time.
- 1.0: perfect predictions.
- 0: no better than always predicting the mean.
- Below 0: worse than predicting the mean.
R² is useful for comparing models on the same data, but "good" values depend heavily on the problem. Predicting human behaviour rarely reaches the R² of a physics measurement.
In scikit-learn
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score
import numpy as np
pred = model.predict(X_test)
mae = mean_absolute_error(y_test, pred)
rmse = np.sqrt(mean_squared_error(y_test, pred))
r2 = r2_score(y_test, pred)Which one should you use?
| Situation | Prefer |
|---|---|
| Explain accuracy to a business audience | MAE |
| Large errors are especially costly | RMSE |
| Compare models on the same dataset | RMSE or MAE, plus R² |
| Errors matter relative to size (prices from ₹500 to ₹5 lakh) | MAPE or errors on a log scale |
MAPE (mean absolute percentage error) expresses errors as a percentage of actual values, but it breaks down when actual values are near zero.
Always look at the errors, not just the score
- Plot predicted vs actual: points should hug the diagonal.
- Plot residuals (errors) against predictions: a pattern means the model is missing something.
- Check errors by segment: a model may be accurate for small cars and poor for luxury cars.
Compare against a baseline
Before celebrating an MAE, compare it with a naive prediction, such as always predicting the median. If your model barely beats that, it is not adding much.
Practise
The metrics page shows how these numbers react to different errors. Then evaluate a real model in the linear regression lesson and the used car prices practice set.
Practise what you just read
Reading helps; writing the code yourself is what sticks. Start here: