Supervised Learning - Learning from Labeled Data

In the previous post, we defined machine learning as teaching computers to learn from data. Now let’s look at the most common and practical approach: supervised learning.

What is Supervised Learning?

Supervised learning is when you train a model using labeled data - meaning each training example comes with both the input and the correct answer.

Think of it like a student learning with an answer key:

flowchart LR
  A["Training Data
(inputs + labels)"] --> B["Learning Algorithm"]
  B --> C["Trained Model"]
  C --> D["New Input"]
  D --> E["Prediction"]
  style B fill:#e8f8f0,stroke:#1a5c38,color:#1a5c38
  style C fill:#e8f8f0,stroke:#1a5c38,color:#1a5c38

The model sees thousands of examples, compares its predictions to the correct answers, and adjusts itself to minimize errors. Once trained, it can make predictions on data it’s never seen before.

A Real-World Dataset

Let’s work with a concrete example: predicting house prices. Here’s a sample of our training data:

In this dataset:

• Features (inputs): Size, Bedrooms, Age, Garage, Neighborhood
• Label (target): Price

The model’s job is to learn the relationship between features and the label so it can predict the price of a house it’s never seen.

The Two Flavors of Supervised Learning

Every supervised learning problem falls into one of two categories:

flowchart TD
  SL["Supervised Learning"] --> REG["Regression"]
  SL --> CLS["Classification"]
  REG --> R1["Predict a continuous number"]
  REG --> R2["Examples: price, temperature, age"]
  CLS --> C1["Predict a category/class"]
  CLS --> C2["Examples: spam/not spam, cat/dog"]
  style SL fill:#f3e8fd,stroke:#5b1a7a,color:#5b1a7a
  style REG fill:#e8f8f0,stroke:#1a5c38,color:#1a5c38
  style CLS fill:#e8f4fd,stroke:#1a5276,color:#1a5276

We’ll cover each in detail in the next two posts. For now, let’s understand the full supervised learning pipeline.

The Training Process

Step 1: Split the Data

You never train and evaluate on the same data. A common split:

pie title Data Split
  "Training Set (70%)" : 70
  "Validation Set (15%)" : 15
  "Test Set (15%)" : 15

• Training set: The model learns from this
• Validation set: Used during training to tune hyperparameters
• Test set: Final evaluation - the model never sees this until the end

Step 2: The Model Learns

During training, the model:

1. Takes a batch of training examples
2. Makes predictions
3. Computes the loss (how far off the predictions are)
4. Adjusts its internal parameters to reduce the loss
5. Repeats for many iterations (epochs)

flowchart TD
  A["Input Batch"] --> B["Model Prediction"]
  B --> C["Compare with True Labels"]
  C --> D["Compute Loss"]
  D --> E["Update Model Parameters"]
  E --> A
  style D fill:#fde8e8,stroke:#7a1a1a,color:#7a1a1a
  style E fill:#e8f8f0,stroke:#1a5c38,color:#1a5c38

Step 3: Evaluate

After training, we check how well the model generalizes to unseen data using the test set.

Feature Types

Real-world data comes in different types, and the model needs to handle each:

One-Hot Encoding Example

The “Neighborhood” column can’t be fed as text to a model. We convert it:

Overfitting vs Underfitting

The most important concept in supervised learning: your model must generalize well.

flowchart LR
  UF["Underfitting
(too simple)"] --- GF["Good Fit
(just right)"] --- OF["Overfitting
(too complex)"]
  style UF fill:#fde8e8,stroke:#7a1a1a,color:#7a1a1a
  style GF fill:#e8f8f0,stroke:#1a5c38,color:#1a5c38
  style OF fill:#fde8e8,stroke:#7a1a1a,color:#7a1a1a

How to Detect Overfitting

If your training error is very low but validation error is high, you’re overfitting.

How to Fix It

• More data - more examples reduce overfitting
• Simpler model - fewer parameters, less complexity
• Regularization - add penalty for overly complex models
• Cross-validation - evaluate on multiple data splits
• Early stopping - stop training when validation error starts increasing

Real-Life Supervised Learning Applications

The Bias-Variance Tradeoff

Every supervised model faces a fundamental tension:

• Bias: Error from overly simple assumptions. High bias → underfitting.
• Variance: Error from sensitivity to training data. High variance → overfitting.

The goal is to find the sweet spot - a model complex enough to capture real patterns but not so complex that it memorizes noise.

Evaluation Metrics

Different tasks need different metrics:

For Regression

• MAE (Mean Absolute Error): Average magnitude of errors
• RMSE (Root Mean Squared Error): Penalizes large errors more
• R² (R-squared): How much variance the model explains (0 to 1)

For Classification

• Accuracy: % of correct predictions (can be misleading with imbalanced data)
• Precision: Of predicted positives, how many are actually positive?
• Recall: Of actual positives, how many did we catch?
• F1 Score: Harmonic mean of precision and recall

What’s Next?

Now that you understand the supervised learning framework, we’ll dive deep into the two subtypes:

flowchart LR
  A["✅ Intro to ML"] --> B["✅ Supervised Learning"]
  B --> C["Regression"]
  C --> D["Classification"]
  D --> E["Unsupervised Learning"]
  style A fill:#e8f8f0,stroke:#1a5c38,color:#1a5c38
  style B fill:#e8f8f0,stroke:#1a5c38,color:#1a5c38
  style C fill:#e8f4fd,stroke:#1a5276,color:#1a5276

In the next post, we’ll explore Regression - predicting continuous values. We’ll build a linear regression model step by step, derive the math, and visualize how the model fits data.

See you in Part 3.