Overfitting
A model that memorizes noise in the data instead of the real pattern.
What it means
Overfitting occurs when a model is so flexible that it captures idiosyncratic noise in the sample rather than the underlying signal, fitting the training data superbly but predicting new data poorly. It is the bias-variance trade-off run amok: extra parameters reduce error in-sample while increasing error out-of-sample. Symptoms include suspiciously high fit, coefficients that swing wildly with small data changes, and a gap between training and test performance. Remedies include simpler models, regularization that penalizes complexity, and honest evaluation on held-out data through cross-validation.
Examples
A curve threaded through every data point predicts future observations worse than a smoother line that ignores the wiggles.
A backtested trading rule that would have caught every turn of the last decade — buy in odd months, sell after rainy Tuesdays — makes money only in the past it was carved out of.
A hiring model trained on a firm's past stars learns that they went to two universities and play squash. It ranks the old intake beautifully and the next one no better than a coin.
First described in Statistics and machine-learning tradition.