Building a spam classifier: your first real ML project

Raw emails go in. “Spam” or “Not spam” comes out. We build the whole thing — data, features, model, evaluation — from scratch.

Three posts in, we’ve learned what ML is, built linear regression from scratch, and understood gradient descent. All of it has been setup. Today we actually build something real — a spam classifier that reads emails and decides if they’re spam or not.

This is the ML workflow in full, end to end, for the first time. It will feel like a lot of pieces clicking together at once.

The full pipeline at a glance

Here’s what we’re building today, from raw text to a working prediction:

Raw emails→Clean text→Bag of words→Naive Bayes→Spam / Ham

Each arrow is a transformation. Let’s build each one.

Step 1 — the data

We’ll use the famous SMS Spam Collection dataset — 5,574 messages labelled “spam” or “ham” (ham means legitimate). Here’s what it looks like:

SAMPLE DATA

spam  "WINNER!! You have been selected for a £1000 prize. Call NOW!"
ham   "Are you coming to dinner tonight?"
spam  "FREE entry to our weekly competition. Text WIN to 87121"
ham   "I'll be home by 8, can you pick up some milk?"

The pattern is obvious to us humans. Our job is to teach a machine to see it too — without hardcoding any rules.

Step 2 — bag of words (turning text into numbers)

ML models can’t read text. They need numbers. The simplest way to convert text to numbers is called Bag of Words: build a vocabulary of all words in the dataset, then represent each email as a vector counting how many times each word appears.

EXAMPLE

Vocabulary:  ["winner", "free", "call", "now", "dinner", "home"]
Email A:     "winner free call now"  → [1, 1, 1, 1, 0, 0]
Email B:     "dinner home call"      → [0, 0, 1, 0, 1, 1]

We lose word order — hence “bag” — but the word frequencies alone carry a lot of signal. “Winner”, “free”, “prize”, “urgent” are strong spam signals. “Dinner”, “home”, “tomorrow” are ham signals.

Step 3 — Naive Bayes classifier

Instead of linear regression (which predicts a number), we need a model that predicts a category: spam or ham. We’ll use Naive Bayes — a probabilistic model that’s fast, interpretable, and surprisingly powerful for text.

The core idea: given the words in this email, which class (spam or ham) makes those words most likely? It uses Bayes’ theorem:

BAYES` THEOREM APPLIED TO SPAM

P(\text{spam} \mid \text{words}) \propto P(\text{spam}) \cdot \prod_i P(\text{word}_i \mid \text{spam})

P(\text{spam}) = \text{prior probability}

P(\text{word} \mid \text{spam}) = \text{how often a word appears in spam}

“Naive” means we assume each word is independent — which is obviously not true (“free” and “prize” are correlated), but it works remarkably well in practice anyway.

The full code

Here’s the entire classifier — data loading, feature engineering, training, and evaluation — in one clean script: