Special Topics in IE 2025

 Last week you learned linear regression for prediction (how much will this house cost?). This week: polynomial regression and the critical problem of overlearning. Same dataset. Different challenges. Same tidymodels framework. The Overlearning Problem: When Models Know Too Much Question: Can a model be too good at learning the training data? Answer: Absolutely, and it's called overlearning. Overlearning happens when your model performs amazingly well on training data but falls flat when facing new data. It's like memorizing test answers without understanding the concepts - you'll ace that specific test but fail when the questions change slightly. This isn't just a minor issue - overlearning is one of the most pressing and still not fully solved problems in machine learning. When Does Overlearning Happen? Three main scenarios make overlearning more likely: Small training dataset : Not enough examples to generalize properly Too many variables/parameters : The ...

 From Classification to Prediction Last week you learned k-Nearest Neighbors for classification (Adelie, Chinstrap, or Gentoo?). This week: linear regression for prediction (how much will this house cost?). Same workflow. Different goal. Same tidymodels framework. The Intuition: Averages Are Predictions Question: What's the price of an average-sized house in King County? Answer: Probably the average price. If the average house is 1,957 square feet and the average price is $521,294, then predicting $521,294 for a 1,957 sqft house makes sense. But what about a 3,000 sqft house? Or a 1,200 sqft house? We need a better model. Enter: The Line of Best Fit Instead of using just the average, linear regression finds the line that best predicts prices based on square footage. Unfitted model: Price = β₁ × Sqft + β₀ Fitted model (after training): Price = 240 × Sqft + 52,509 Translation: Each additional square foot adds $240 to the predicted price, and a 0-sqft house (hypotheti...