10 KiB
Object Detection
The image classification models we've explored so far take an image as input and produce a categorical output, such as identifying the class 'number' in the MNIST dataset. However, in many scenarios, it's not enough to simply know that an image contains certain objects—we also want to pinpoint their exact locations. This is the purpose of object detection.
Pre-lecture quiz
Image from YOLO v2 website
A Naive Approach to Object Detection
Imagine we want to locate a cat in an image. A very simplistic approach to object detection might look like this:
- Divide the image into a grid of smaller tiles.
- Perform image classification on each tile.
- Identify tiles with sufficiently high activation as containing the object of interest.
Image from Exercise Notebook
However, this method is far from ideal because it only provides a rough estimate of the object's bounding box. For more precise localization, we need to use regression to predict the bounding box coordinates—and for that, we require specialized datasets.
Regression for Object Detection
This blog post offers an excellent introduction to detecting shapes.
Datasets for Object Detection
Here are some commonly used datasets for object detection:
- PASCAL VOC - 20 classes
- COCO - Common Objects in Context. Includes 80 classes, bounding boxes, and segmentation masks.
Object Detection Metrics
Intersection over Union
While evaluating image classification models is straightforward, object detection requires assessing both the accuracy of the predicted class and the precision of the bounding box location. For the latter, we use Intersection over Union (IoU), which measures the overlap between two bounding boxes (or areas).
Figure 2 from this excellent blog post on IoU
The concept is simple: divide the area of intersection between two shapes by the area of their union. For two identical shapes, IoU equals 1. For completely non-overlapping shapes, IoU equals 0. Typically, we only consider bounding boxes with IoU above a certain threshold.
Average Precision
To evaluate how well a specific object class C is detected, we use the Average Precision metric, calculated as follows:
- Plot a Precision-Recall curve, which shows accuracy as a function of the detection threshold (ranging from 0 to 1).
- Depending on the threshold, the number of detected objects and the precision-recall values will vary.
- The resulting curve looks like this:
Image from NeuroWorkshop
The Average Precision for a class C is the area under this curve. More specifically, the Recall axis is divided into 10 segments, and Precision is averaged across these points:
AP = {1\over11}\sum_{i=0}^{10}\mbox{Precision}(\mbox{Recall}={i\over10})
AP and IoU
We only consider detections with IoU above a certain threshold. For example, the PASCAL VOC dataset typically uses \mbox{IoU Threshold} = 0.5, while COCO evaluates AP across multiple \mbox{IoU Threshold} values.
Image from NeuroWorkshop
Mean Average Precision - mAP
The primary metric for object detection is Mean Average Precision (mAP). This is the Average Precision averaged across all object classes, and sometimes across multiple \mbox{IoU Threshold} values. The process of calculating mAP is explained in detail in this blog post and here with code examples.
Different Object Detection Approaches
Object detection algorithms can be broadly categorized into two types:
- Region Proposal Networks (R-CNN, Fast R-CNN, Faster R-CNN): These methods generate Regions of Interest (ROIs) and run a CNN over them to find the highest activations. This approach is somewhat similar to the naive method but uses more sophisticated ROI generation. A major drawback is that these methods are slow because the CNN classifier must process the image multiple times.
- One-pass methods (YOLO, SSD, RetinaNet): These architectures predict both object classes and ROIs in a single pass.
R-CNN: Region-Based CNN
R-CNN uses Selective Search to generate a hierarchical structure of ROIs. These ROIs are passed through CNN feature extractors and SVM classifiers to determine object classes, while linear regression predicts the bounding box coordinates. Official Paper
Image from van de Sande et al. ICCV’11
Images from this blog
F-RCNN - Fast R-CNN
This method is similar to R-CNN, but the regions are defined after applying convolutional layers.
Image from the Official Paper, arXiv, 2015
Faster R-CNN
This approach introduces a neural network to predict ROIs, known as the Region Proposal Network. Paper, 2016
Image from the official paper
R-FCN: Region-Based Fully Convolutional Network
This algorithm is faster than Faster R-CNN. The key idea is:
- Extract features using ResNet-101.
- Process features with a Position-Sensitive Score Map. Each object from
Cclasses is divided intok\times kregions, and the network predicts parts of objects. - For each part of the
k\times kregions, the networks vote for object classes, and the class with the highest vote is selected.
Image from official paper
YOLO - You Only Look Once
YOLO is a real-time, one-pass algorithm. The main idea is:
- Divide the image into
S\times Sregions. - For each region, the CNN predicts
npossible objects, bounding box coordinates, and confidence = probability × IoU.
Image from official paper
Other Algorithms
- RetinaNet: official paper
- SSD (Single Shot Detector): official paper
✍️ Exercises: Object Detection
Continue your learning in the following notebook:
Conclusion
In this lesson, you explored a variety of approaches to object detection!
🚀 Challenge
Explore these articles and notebooks about YOLO and try implementing them:
- Good blog post describing YOLO
- Official site
- YOLO: Keras implementation, step-by-step notebook
- YOLO v2: Keras implementation, step-by-step notebook
Post-lecture quiz
Review & Self Study
- Object Detection by Nikhil Sardana
- A good comparison of object detection algorithms
- Review of Deep Learning Algorithms for Object Detection
- A Step-by-Step Introduction to the Basic Object Detection Algorithms
- Implementation of Faster R-CNN in Python for Object Detection