Introduction

Welcome back

In Intro to Machine Learning you learned the core loop: explore, build, validate, improve. You trained decision trees and a random forest, and measured them honestly with a validation set.

Real data is messier than the tidy tables in that course. It has gaps, it has words instead of numbers, and it hides traps that can quietly ruin a model. This course gives you the tools professionals use to handle all of that:

• Missing values and how to fill them
• Categorical (non-numeric) data and how to encode it
• Pipelines to keep your steps clean and repeatable
• Cross-validation for a more trustworthy score
• Gradient boosting for stronger accuracy
• Data leakage, the subtle bug that fools you into thinking a model is great

Before you start

This is an intermediate course. It assumes you have already built a simple model and are comfortable with a few ideas from Intro to Machine Learning:

• training a model, then validating it on data it has not seen
• mean absolute error (MAE) as a measure of quality
• underfitting and overfitting
• decision trees and random forests

If any of those feel shaky, do the RiseAfrica Intro to Machine Learning course first (and Pandas helps too). They cover everything you need, then come back here.

The data we will use

We will use the same student dataset as before, now with two new wrinkles: a non-numeric region column, and some missing values. Run this to see them:

See the NaN values and the text region column? A model cannot use either as-is. Fixing that is where we start.

Practice

Run the cell and read the missing-values count. Which column has gaps, and how many? Note it down, you will fix it in the next lesson.