Reading a confusion matrix
A single accuracy number hides which way a classifier is wrong; the matrix shows both directions at once.
The four cells
For a two-class problem the matrix has four counts. Correct positive predictions and correct negative predictions sit on the diagonal. The other two cells are the two distinct ways of being wrong: predicting positive when the truth is negative, and predicting negative when the truth is positive.
Accuracy adds the diagonal and divides by the total. That is a fair summary only when the two classes are roughly balanced and the two errors cost the same, which is rarely true of anything worth building.
Precision and recall
Precision asks: of everything flagged positive, how much really was. Recall asks: of everything that really was positive, how much got flagged. Pushing a decision threshold in either direction improves one and damages the other, so quoting one without the other says very little.
Where the two errors carry different costs, decide which error you are willing to make before choosing the threshold, not after seeing the numbers.
The imbalance trap
If ninety-nine rows in a hundred are negative, a model that predicts negative every single time scores ninety-nine per cent accuracy and has learned nothing at all. The matrix makes this obvious immediately: one whole column is empty.
Reading it in order
- Check the row totals to see how imbalanced the true classes are.
- Check whether either predicted column is nearly empty.
- Read the two off-diagonal cells separately and name the real-world cost of each.
- Compute precision and recall, not accuracy alone.
- Only then decide whether the threshold needs to move.
predicted - predicted +
true - correct wrong, type one
true + wrong, type two correctLayout of the two-class matrix