A Support Vector Machine draws a boundary between classes, but not just any boundary: the one with the widest gap to the nearest points on either side. That simple idea makes SVMs robust and effective, especially on medium-sized datasets.
The widest street
Imagine two groups of points, good and defective parts, plotted by two sensor readings. Many lines could separate them. SVM picks the line with the largest margin: the widest "street" between the classes with no points inside it.
A wide margin generalises better to new data, because small measurement changes are less likely to push a point to the wrong side.
Support vectors
The points closest to the boundary, sitting on the edges of the street, are the support vectors. Only they determine where the boundary goes. Points far from the boundary could move without changing anything.
Soft margins and the C parameter
Real data overlaps, so a perfect street is often impossible. SVM allows some points inside the margin or on the wrong side, with a penalty controlled by C:
- Large C: few mistakes allowed; narrow margin; risk of overfitting.
- Small C: more mistakes allowed; wider margin; smoother, more general boundary.
Kernels: curved boundaries
When classes cannot be separated by a straight line, for example when defective parts sit in a ring around good ones, a kernel lets SVM draw curved boundaries. Conceptually, it maps the data into a higher-dimensional space where a straight separation exists, without computing that space directly.
- Linear kernel: straight boundary; good for many features, such as text.
- RBF (Gaussian) kernel: flexible curves; the usual default.
- Polynomial kernel: curved boundaries of a chosen degree.
The RBF kernel has a second parameter, gamma: high gamma makes each point's influence very local (wiggly boundary, overfitting risk); low gamma makes it smoother.
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
svm = make_pipeline(StandardScaler(), SVC(kernel="rbf", C=1.0, gamma="scale", probability=True))
svm.fit(X_train, y_train)
svm.score(X_test, y_test)Tune C and gamma together with cross-validation.
Always scale
Like KNN, SVM depends on distances, so features must be on similar scales.
Strengths
- Effective in high-dimensional spaces.
- Robust thanks to the margin idea.
- Flexible boundaries with kernels.
Weaknesses
- Slow to train on very large datasets.
- Needs careful scaling and tuning.
- Less interpretable than trees or logistic regression.
- Probability outputs need extra calibration.
When to use SVM
Medium-sized datasets with clear but complex boundaries, many features (including text with a linear kernel), and when you have time to tune.
See it in action
The SVM playground lets you move C and switch kernels to watch the margin and boundary change. The SVM lesson and the defective parts practice set show a case where a straight line fails and an RBF kernel works.
Practise what you just read
Reading helps; writing the code yourself is what sticks. Start here: