Your First Machine Learning Model: k-Nearest Neighbors

 The Saturday Afternoon Problem

Before we dive into algorithms, let's think about how you already do machine learning without knowing it.

You want to find someone to spend Saturday afternoon with. You're looking for your "nearest neighbor" based on:

• Gender (0 or 1)
• Age (in years)
• Outdoor sports interest (0-10 scale)

Three candidates appear. Who's most similar to you (male, 50 years old, sports score 7)?

Candidate 1: Male, 21 years old, score 5 → Average difference: 10.33
Candidate 2: Female, 51 years old, score 9 → Average difference: 1.33

Wait - the 21-year-old guy seems visually closer, but the math says the 51-year-old woman is more similar? This is the scaling problem we'll solve today.

What Is k-Nearest Neighbors?

It's the most intuitive machine learning algorithm: to classify something new, find what it's closest to and copy that label.

If k=1: Find the single closest penguin and use its species
If k=4: Find the 4 closest penguins and let them vote (majority wins)

That's it. Seriously.

The Real Challenge: Distance Isn't Always What It Looks Like

The Penguin Dataset

We're classifying three penguin species (Adelie, Chinstrap, Gentoo) using:

• Bill length (measured in millimeters: 30-60mm)
• Body mass (measured in grams: 2,500-6,500g)

The problem: Body mass values are 100-1000× larger than bill length values. When we calculate distance, body mass completely dominates.

Visual vs. Mathematical Reality

Look at a penguin at coordinates (45mm, 4500g). Which is closer?

Point N2: (53mm, 4500g) - looks far away horizontally
Point N4: (43mm, 4450g) - looks much closer overall

But the math says:

r

Distance to N2 = sqrt((53-45)² + (4500-4500)²) = 8
Distance to N4 = sqrt((43-45)² + (4450-4500)²) = 50

N2 is actually closer! Why? Because 50 grams difference creates way more mathematical distance than 8mm difference.

The Solution: Scaling

Goal: Transform both variables to comparable ranges so neither dominates.

Common methods:

Rescaling (0 to 1):

y = (x - x_min) / (x_max - x_min)

Z-Score Normalization:

z = (x - mean) / standard_deviation

After scaling, both variables contribute fairly to finding neighbors.

The tidymodels Workflow

R's tidymodels package gives you a standardized process for any machine learning model:

Step 1: Split Your Data

r

Split7030 = initial_split(DataPenguins, prop=0.7, strata=Species)
DataTrain = training(Split7030)
DataTest = testing(Split7030)

Step 2: Create a Recipe (Preprocessing)

r

RecipePenguins = recipe(Species ~ BillLengthMm + BodyMassG, data=DataTrain) |>
  step_naomit() |>
  step_normalize(all_predictors())

Think of this like a cooking recipe - list your ingredients, then describe the preparation steps.

Step 3: Define Your Model Design

r

ModelDesignKNN = nearest_neighbor(neighbors=4) |>
  set_engine("kknn") |>
  set_mode("classification")

Step 4: Build the Workflow and Fit

r

WFModelPenguins = workflow() |>
  add_recipe(RecipePenguins) |>
  add_model(ModelDesignKNN) |>
  fit(DataTrain)

Step 5: Make Predictions

r

DataPredWithTestData = augment(WFModelPenguins, DataTest)

Evaluating Performance: The Confusion Matrix

After predictions, you need to know: how good was the model?

                Truth
Prediction    Adelie  Chinstrap  Gentoo
  Adelie        42        2        1
  Chinstrap      1       19        0
  Gentoo         3        0       36

Accuracy: (42+19+36) / 104 = 93.3% correct
Sensitivity: Of all actual Adelies, what % did we catch? 91.3%
Specificity: Of all non-Adelies, what % did we correctly identify as not Adelie? 96.4%

Choosing k: The Goldilocks Problem

k too small (k=1): Easily fooled by outliers
k too large (k=50): The "neighborhood" is so big it's meaningless
k just right: Requires systematic tuning (covered in future lectures)

Bonus Project: Reading Handwritten Digits

The MNIST dataset contains 60,000 images of handwritten digits (0-9). Each image is 28×28 pixels, meaning 784 pixel values per image.

How it works:

1. Each pixel has a value from 0 (black) to 255 (white)
2. All 784 pixels are stored as one row of data
3. k-NN finds the k closest training images by comparing all 784 pixel values
4. The digits from those k neighbors vote on what digit the unknown image represents

This is Optical Character Recognition (OCR) - the same technology that reads checks, scans documents, and powers postal sorting systems.

Key Takeaways

k-Nearest Neighbors is intuitive: Find what's closest, copy its label
Scaling is critical: Variables with different units need normalization
tidymodels standardizes everything: Same workflow works for any ML model
The confusion matrix tells the truth: Accuracy alone can be misleading
Real-world applications: From penguin species to handwritten digit recognition

What's Next

Lecture 5: Key Machine Learning Concepts - Explained with Linear Regression. We'll explore how regression differs from classification and why understanding one helps you master the other.

Resources:

• Download today's slides: Lecture 4, or download the pdf here
• Course textbook: Free at https://ai.lange-analytics.com/htmlbook/index.html

Challenge: Load the penguin data and try different values of k (1, 4, 10, 20). Watch how the accuracy changes. What patterns do you notice?

You just built your first real machine learning model. Everything from here builds on these same concepts - they just get more sophisticated.