Overfitting is not a bug you fix once

Beginners treat overfitting like a compile error — fix it, move on. Practitioners treat it as the permanent weather of the job. Understanding that difference is most of what separates the two.

What overfitting actually is

A model overfits when it learns the noise in your training data as if it were signal. Instead of capturing the general pattern that will hold on new data, it memorises the specific quirks of the exact examples it was shown — the random flukes, the measurement errors, the coincidences that won't repeat. Then you show it something new, the flukes aren't there, and it falls apart.

The tell is unmistakable once you know to look for it: excellent scores on the training data, disappointing scores on data the model hasn't seen. A model that gets 99% on its training set and 71% on fresh data hasn't learned the problem — it has learned the answer key. The picture people use is a scatter of points with a smooth curve drawn through the trend versus a frantic, wiggly curve that contorts itself to pass exactly through every single point. The wiggly curve is “better” on the training points and useless on everything else.

Why it is never permanently solved

Here's the part beginners find frustrating and practitioners find clarifying: there is no setting that makes overfitting go away for good. Every model lives on a spectrum between two failures. Fit the training data too hard and you overfit — you chase noise. Fit it too loosely and you underfit — you're too simple to capture the real pattern and you learn almost nothing. The whole game is finding the sweet spot between those two, and the sweet spot moves with every new dataset, every new feature, every change in how much data you have.

The gap between how a model does on your data and how it does in the world is not an annoyance around the job. It is the job.

This is why treating overfitting as a one-time bug to squash is the wrong mental model. It's a tension you manage continuously, like balance while walking. You never “finish” balancing.

The checklist that catches it early

You can't eliminate the risk, but you can catch it before it embarrasses you in production. This is the routine:

1. Hold out honestly. Split your data into training, validation and test sets before you look at anything or make a single decision. The training set is for learning, the validation set is for tuning, and the test set is opened exactly once, at the very end, to get an honest final number. The moment you start tuning against the test set, it stops being a test.
2. Hunt for leakage. Data leakage is when a feature secretly encodes the answer — a timestamp that happens to line up with the label, an ID number assigned after the outcome was known, a column derived from the very thing you're predicting. Leakage produces gorgeous scores that evaporate in the real world, and it is by far the most common cause of results that look too good to be true. When a result is suspiciously strong, look here first.
3. Watch the two curves. Plot training error and validation error as training progresses. Early on both fall together — the model is genuinely learning. The moment they diverge, with training error still dropping while validation error flattens or climbs, you are watching overfitting happen in real time. That divergence point is your signal to stop.
4. Prefer the simpler model that ties. If a smaller, simpler model matches a big complicated one on validation performance, take the simple one. It has less capacity to memorise noise and it will usually generalise better, quite apart from being cheaper and easier to reason about.
5. Regularise on purpose. Weight penalties, dropout, early stopping, data augmentation and simply getting more data all trade a little training accuracy for a lot of generalisation. That trade is almost always worth making; a model that scores slightly lower on training but far better on reality is the one you want.

The result that should make you suspicious, not excited

Suppose you try something and your model suddenly leaps far ahead of everything you've tried before — a jump so large it feels like a breakthrough. The correct first reaction is not celebration. It is suspicion. In the overwhelming majority of cases, a dramatic unexpected improvement is a bug: a leaky feature, a mislabelled split, an accidental evaluation on the training data, a target that snuck into the inputs.

The professional habit is to actively try to disprove the good result before believing it. Re-check the split. Trace where every feature came from. Confirm you're evaluating on data the model never touched. Nine times out of ten, the boring explanation — a mistake — turns out to be the right one, and the tenth time you've earned real confidence in a genuine win. Verifying the boring explanation first is not pessimism; it's how you avoid shipping a model that looked brilliant in the notebook and fails the instant it meets the world.

What to carry away

Stop imagining overfitting as a switch you flip off once and forget. Picture it as weather: always present, sometimes mild and sometimes severe, and something you dress for every single time you step outside. The forecast changes with every dataset. The skill isn't making it never rain — it's reading the sky, carrying the right gear, and not being surprised when the clouds roll in.

Found an error? Tell us — corrections improve the register.