A confusion matrix is a table that, in classification, compares actual and predicted classes and thereby breaks a model’s performance down into individual types of error. For a binary task it has four cells: true positives, true negatives, false positives (type I error) and false negatives (type II error). Its value lies precisely in this distinction, which any single summary number hides. Accuracy alone, the proportion of correct predictions, is misleading to the point of danger on imbalanced data: in a task where only one per cent of samples are positive, a model that never finds anything achieves ninety-nine per cent accuracy. All the essential metrics are derived from the confusion matrix – precision and recall, specificity, the F1 score or the Matthews correlation coefficient. Decisive too is that the two error types usually have incommensurable impact: in cancer screening a false negative is unacceptable, in automatic account blocking a false positive is. The choice of decision threshold is therefore always a business decision, not merely a technical one. With multiple classes the matrix shows which classes the model confuses with which.
It is like laying out the outcomes of a pregnancy test in four cells: pregnant and test positive, not pregnant and negative – those are the correct cases. Then two errors that are nowhere near equivalent. The test said “pregnant” and she was not – unpleasant, but resolved by a further examination. The test said “not pregnant” and she was – that is a serious problem. If you only reported that “the test is wrong five per cent of the time”, you would never see this difference. And the whole thing usually hinges on exactly that.