Handling Messy Data · Lesson 3 of 7

Categorical variables

What is a categorical variable

A categorical variable takes one of a limited set of values. A survey asking how often you eat breakfast ("Never", "Rarely", "Most days", "Every day") is categorical. So is car brand ("Honda", "Toyota", "Ford"), or our region column ("Urban", "Rural"). Models work with numbers, so feeding them text causes an error. You must convert categories to numbers first.

Ordinal or nominal?

Before encoding, ask whether the categories have a natural order:

Ordinal: there is a clear ranking. "Never" < "Rarely" < "Most days" < "Every day". Education level is another example.
Nominal: there is no ranking. "Honda" is not more or less than "Toyota"; "Urban" is not more or less than "Rural".

This decides which encoding fits best.

Three approaches

1. Drop the categorical columns. Simplest, but you lose whatever information they held. Only sensible if the column was not useful.

2. Ordinal encoding. Give each category an integer: Never=0, Rarely=1, Most days=2, Every day=3. This builds in an order, so it suits ordinal variables, and it works well with tree-based models like random forests.

3. One-hot encoding. Make a new True/False column for each category (a "region_Urban" column and a "region_Rural" column). It assumes no order, so it suits nominal variables. It is the usual safe default, but avoid it when a column has many categories (roughly more than 15), since it creates a column for each one.

Compare them yourself

Python

import pandas as pd
from sklearn.ensemble import RandomForestRegressor
from sklearn.preprocessing import OrdinalEncoder, OneHotEncoder
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_absolute_error

students = pd.DataFrame({
    "study_hours":    [2, 5, 1, 6, 3, 8, 4, 7, 1, 6, 5, 3, 7, 4, 8],
    "attendance_pct": [60, 85, 50, 95, 70, 98, 75, 90, 45, 88, 80, 65, 92, 78, 99],
    "prev_score":     [50, 72, 40, 85, 58, 92, 65, 88, 35, 80, 75, 55, 84, 62, 95],
    "region":         ["Rural","Urban","Rural","Urban","Rural","Urban","Rural","Urban","Rural","Urban","Rural","Rural","Urban","Rural","Urban"],
    "exam_score":     [52, 78, 41, 90, 61, 96, 68, 89, 38, 82, 77, 59, 87, 66, 98],
})
X = students.drop(columns=["exam_score"])
y = students["exam_score"]
train_X, val_X, train_y, val_y = train_test_split(X, y, random_state=1)

num_cols = ["study_hours", "attendance_pct", "prev_score"]
cat_cols = ["region"]

def score(tr, va):
    m = RandomForestRegressor(random_state=1)
    m.fit(tr, train_y)
    return round(mean_absolute_error(val_y, m.predict(va)), 1)

# Approach 1: drop the categorical column
print("Approach 1 (drop):   ", score(train_X[num_cols], val_X[num_cols]))

# Approach 2: ordinal encoding (fit on train, transform both)
oe = OrdinalEncoder()
tr2, va2 = train_X.copy(), val_X.copy()
tr2[cat_cols] = oe.fit_transform(train_X[cat_cols])
va2[cat_cols] = oe.transform(val_X[cat_cols])
print("Approach 2 (ordinal):", score(tr2, va2))

# Approach 3: one-hot encoding
ohe = OneHotEncoder(handle_unknown="ignore", sparse_output=False)
tr_oh = pd.DataFrame(ohe.fit_transform(train_X[cat_cols]), index=train_X.index)
va_oh = pd.DataFrame(ohe.transform(val_X[cat_cols]), index=val_X.index)
tr3 = pd.concat([train_X[num_cols], tr_oh], axis=1); tr3.columns = tr3.columns.astype(str)
va3 = pd.concat([val_X[num_cols], va_oh], axis=1); va3.columns = va3.columns.astype(str)
print("Approach 3 (one-hot):", score(tr3, va3))

Which to use

As a rule: one-hot encoding is the safe default for nominal data with few categories, ordinal encoding when the categories have a real order (and you are using a tree-based model), and dropping only when the column is genuinely useless. On a small dataset like this the scores will be close, but dropping a useful column usually performs worst on real data.

Exercise

Exercise — check your understanding

One-hot encode the "region" column of df with pd.get_dummies. Store the result in encoded.

You’re reading for free. Sign in to keep your progress and earn a certificate when you finish.Sign in to keep my progress →

← PreviousMissing values Next →Pipelines