Data leakage is a methodological error in which information reaches training that is not available at the moment of the actual prediction. The model then shows extraordinarily good results in testing but fails in production – making it one of the most treacherous mistakes in machine learning, because it only reveals itself after deployment. The commonest form is target leakage, where the features include a variable derived directly from the predicted phenomenon: a model predicting customer churn has access to the field “date the termination notice was sent”. The second form is pre-processing leakage, where normalisation, missing-value imputation or feature selection are computed over the whole dataset before splitting, so statistics of the test portion seep into training. The third is temporal leakage, where random splitting puts the future into the training set; with time series one may therefore split only chronologically. The fourth is duplicate or group leakage, where the same patient or customer appears in both parts. Prevention lies in strictly separating the test set before any operation, encapsulating pre-processing in a pipeline, and grouping by entity during cross-validation.
It is like a student finding the answer key in the corridor before an exam. He excels in the exam and everyone thinks he has mastered the material. Only in practice does it emerge that he knows nothing. In analytics it looks more innocent: you want to predict which clients will leave, and among the data you inadvertently leave the column “reason for contract termination”. The model spots it immediately, reaches a hundred per cent success rate, and for a while you are proud. Then you let it loose on live clients, where that column is logically empty, and it does not work at all.