Object detection is the task of finding all occurrences of the sought categories in an image and returning, for each one, both a label and a position in the form of a bounding rectangle plus a confidence score. Compared with image classification it is substantially harder, because the number of objects is not known in advance and the model must solve localisation and recognition at the same time; the loss function therefore combines a classification term with regression of the box coordinates. Historically, approaches split into two-stage and one-stage. Two-stage ones such as R-CNN and Faster R-CNN first generate candidate regions and then classify them – they are more accurate but slower. One-stage ones such as YOLO and SSD predict boxes and classes in a single pass over a grid of predefined shapes and are suited to real-time operation. Newer approaches like DETR formulate the task as direct set prediction using a transformer and dispense with hand-crafted heuristics. An integral part is non-maximum suppression, which keeps only the best of many overlapping proposals, and the metric is mean average precision computed across IoU values.
The difference from classification is like the difference between the questions “is there a traffic jam in this photo?” and “draw a box around every car in this photo and write down what type it is”. The second task is incomparably more laborious. You do not know in advance whether there will be one car or forty; some are large in the foreground while others are specks in the distance, and half are only half visible behind a pole. And when you finally box one car with three slightly offset rectangles, somebody still has to decide which one counts – because a photo with a thousand boxes around one car helps nobody.