ce1e136ac3
|
||
|---|---|---|
| .. | ||
| lab | ||
| README.md | ||
| SemanticSegmentationPytorch.ipynb | ||
| SemanticSegmentationTF.ipynb | ||
README.md
Segmentation
Previously, we learned about Object Detection, which helps us locate objects in an image by predicting their bounding boxes. However, for certain tasks, bounding boxes alone are not enough—we need more precise object localization. This task is known as segmentation.
Pre-lecture quiz
Segmentation can be thought of as pixel classification, where we predict the class of each pixel in the image (background being one of the classes). There are two main types of segmentation algorithms:
- Semantic segmentation identifies the class of each pixel but does not differentiate between individual objects of the same class.
- Instance segmentation separates objects of the same class into distinct instances.
For example, in instance segmentation, each sheep is treated as a separate object, whereas in semantic segmentation, all sheep are grouped into a single class.
Image from this blog post
There are various neural network architectures for segmentation, but they all share a similar structure. In some ways, they resemble the autoencoder you learned about earlier, but instead of reconstructing the original image, the goal is to reconstruct a mask. A segmentation network typically consists of the following components:
- Encoder: Extracts features from the input image.
- Decoder: Converts these features into the mask image, which has the same dimensions as the input image, with the number of channels corresponding to the number of classes.
Image from this publication
A key aspect of segmentation is the loss function used during training. In classical autoencoders, we measure the similarity between two images, often using mean square error (MSE). In segmentation, however, each pixel in the target mask image represents a class (one-hot-encoded along the third dimension). Therefore, we use classification-specific loss functions, such as cross-entropy loss, averaged over all pixels. For binary masks, binary cross-entropy loss (BCE) is applied.
✅ One-hot encoding is a method for converting class labels into vectors of length equal to the number of classes. Check out this article for more details on this technique.
Segmentation for Medical Imaging
In this lesson, we will explore segmentation in practice by training a network to identify human nevi (commonly known as moles) in medical images. We will use the PH2 Database of dermoscopy images as our dataset. This dataset contains 200 images categorized into three classes: typical nevus, atypical nevus, and melanoma. Each image is accompanied by a corresponding mask that outlines the nevus.
✅ This technique is particularly well-suited for medical imaging, but what other real-world applications can you think of?
Image from the PH2 Database
Our goal will be to train a model to segment any nevus from its background.
✍️ Exercises: Semantic Segmentation
Explore the notebooks below to learn more about different semantic segmentation architectures, practice using them, and see them in action.
Post-lecture quiz
Conclusion
Segmentation is a powerful image classification technique that goes beyond bounding boxes to achieve pixel-level classification. It is widely used in medical imaging and other applications.
🚀 Challenge
Body segmentation is just one of many tasks that can be performed with images of people. Other important tasks include skeleton detection and pose detection. Experiment with the OpenPose library to explore pose detection.
Review & Self Study
This Wikipedia article provides a comprehensive overview of the various applications of segmentation. Dive deeper into the subdomains of Instance segmentation and Panoptic segmentation to expand your understanding.
Assignment
In this lab, try human body segmentation using the Segmentation Full Body MADS Dataset available on Kaggle.