Decision trees and random forests are two of the most popular models for tabular business data. They are closely related: a random forest is literally a collection of decision trees. The difference is in how they trade explainability for accuracy and stability.
How a decision tree works
A tree asks a sequence of yes/no questions about the features, such as "Is tenure under 12 months?" then "Is the last rating below 3?", and ends in a leaf that gives a prediction.
At each step it chooses the question that best separates the outcomes, measured by Gini impurity or entropy for classification and by error reduction for regression.
from sklearn.tree import DecisionTreeClassifier, export_text
tree = DecisionTreeClassifier(max_depth=4, min_samples_leaf=20, random_state=42)
tree.fit(X_train, y_train)
print(export_text(tree, feature_names=list(X_train.columns)))Strengths
- Easy to explain: the rules can be read and shown to a business team.
- Handles numbers and categories, and non-linear patterns.
- Needs little preprocessing (no scaling).
Weaknesses
- Overfits easily if allowed to grow deep.
- Unstable: a small change in data can produce a very different tree.
How a random forest works
A random forest trains many trees, each with two kinds of randomness:
- Each tree sees a random sample of rows (bootstrapping).
- At each split, each tree considers only a random subset of features.
The trees make different mistakes, and averaging their votes cancels much of that error.
from sklearn.ensemble import RandomForestClassifier
forest = RandomForestClassifier(n_estimators=300, min_samples_leaf=5, random_state=42, n_jobs=-1)
forest.fit(X_train, y_train)
forest.feature_importances_Strengths
- Usually more accurate and much more stable than a single tree.
- Less prone to overfitting.
- Gives feature importance scores.
Weaknesses
- Harder to explain: you cannot read 300 trees.
- Larger and slower to predict.
- Feature importances can be misleading with correlated features; permutation importance is a useful check.
Side by side
| Decision tree | Random forest | |
|---|---|---|
| Explainability | High (readable rules) | Lower (importances only) |
| Accuracy | Moderate | Usually higher |
| Overfitting risk | High without limits | Lower |
| Stability | Low | High |
| Speed | Fast | Slower |
Which to choose?
- Need rules people can follow or audit? A shallow decision tree.
- Need the best predictions and explanation is secondary? A random forest (or other ensembles such as gradient boosting).
- Not sure? Train both, compare on the same cross-validation, and consider whether the accuracy gain is worth the loss of explainability.
See them in action
Grow a tree in the decision tree playground and watch it overfit as depth increases. Then see the forest improve on it in the random forest lesson and the online returns practice set.
Practise what you just read
Reading helps; writing the code yourself is what sticks. Start here: