Naive Bayes is a fast, simple classifier that works surprisingly well on text: spam detection, sentiment, routing support tickets. It is built on a rule from probability called Bayes' theorem.
Bayes' theorem in plain words
It updates a belief using new evidence:
P(class | evidence) ∝ P(evidence | class) × P(class)P(class): how common the class is to begin with (the prior). Maybe a fifth of messages are spam.P(evidence | class): how likely this evidence is in that class. "Free prize" appears far more in spam.- The result: how likely the class is, given what we saw.
Why "naive"?
A message has many words. Naive Bayes assumes each word's presence is independent of the others, given the class. That is clearly not true ("free" and "prize" often appear together), but the simplification makes the model fast and it still ranks classes well in practice.
Classifying a message
For each class, multiply the prior by the probability of each word in that class (in practice, add logarithms to avoid tiny numbers), then pick the class with the highest score.
Text to numbers
from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer
from sklearn.naive_bayes import MultinomialNB
from sklearn.pipeline import make_pipeline
model = make_pipeline(TfidfVectorizer(lowercase=True, ngram_range=(1, 2)), MultinomialNB(alpha=1.0))
model.fit(train_texts, train_labels)
model.predict(["Your broadband is down since morning, please fix"])- CountVectorizer: counts how often each word appears.
- TfidfVectorizer: down-weights words that appear everywhere (like "please") and up-weights distinctive ones.
ngram_range=(1, 2)adds two-word phrases such as "not working".
Smoothing
If a word never appeared in the training data for a class, its probability would be zero, and multiplying by zero wipes out all other evidence. Laplace smoothing (alpha) adds a small count to every word so no probability is exactly zero.
Variants
- MultinomialNB: word counts or TF-IDF; the usual choice for text.
- BernoulliNB: word present or absent.
- GaussianNB: continuous numeric features assumed to be normally distributed.
Strengths
- Very fast to train and predict, even on large text collections.
- Works well with small datasets.
- A strong, quick baseline for any text classification problem.
Weaknesses
- The independence assumption means probability estimates are often poorly calibrated (too confident), even when the ranking is good.
- Struggles with word order and context ("not good" vs "good"), partly helped by n-grams.
- More complex models usually win when there is lots of data.
See it work
The Naive Bayes playground shows how word evidence pushes a message towards one class. The Naive Bayes lesson walks through spam detection, and the support tickets practice set has you route broadband tickets with counts, TF-IDF and smoothing.
Practise what you just read
Reading helps; writing the code yourself is what sticks. Start here: