RomanNum3ral
/
AI-For-Beginners
mirror of https://github.com/microsoft/AI-For-Beginners.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #67 from CinnamonXI/main 

Lesson 9
This commit is contained in:
Jen Looper 2022-05-03 12:35:59 -04:00 committed by GitHub
parent 3cc790122f 98cab4f8a1
commit 55cbfa1997
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
6 changed files with 164 additions and 12 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
etc/quiz-app/src/assets/translations/en/index.js
View File

@ -5,6 +5,7 @@ import x4 from "./lesson-4.json";
import x5 from "./lesson-5.json";
import x7 from "./lesson-7.json";
import x8 from "./lesson-8.json";
import x9 from "./lesson-9.json";
import x23 from "./lesson-23.json";
const quiz = { 0 : x1[0], 1 : x2[0], 2 : x3[0], 3 : x4[0], 4 : x5[0], 5 : x7[0], 6 : x8[0], 7 : x23[0] };
const quiz = { 0 : x1[0], 1 : x2[0], 2 : x3[0], 3 : x4[0], 4 : x5[0], 5 : x7[0], 6 : x8[0], 7 : x9[0], 8 : x23[0] };
export default quiz;

115
etc/quiz-app/src/assets/translations/en/lesson-9.json Normal file
View File

@ -0,0 +1,115 @@
[
{
"title": "AI for Beginners: Quizzes",
"complete": "Congratulations, you completed the quiz!",
"error": "Sorry, try again",
"quizzes": [
{
"id": 109,
"title": "Autoencoders: Pre Quiz",
"quiz": [
{
"questionText": "Self-supervised learning uses ____ data for training",
"answerOptions": [
{
"answerText": "pre-trained",
"isCorrect": false
},
{
"answerText": "raw",
"isCorrect": true
},
{
"answerText": "labeled",
"isCorrect": false
}
]
},
{
"questionText": "Encoder Network coverts input images into latent spaces",
"answerOptions": [
{
"answerText": "true",
"isCorrect": true
},
{
"answerText": "false",
"isCorrect": false
}
]
},
{
"questionText": "VAE is short for?",
"answerOptions": [
{
"answerText": "Variable AutoEncoding",
"isCorrect": false
},
{
"answerText": "Variation auto-encoder",
"isCorrect": true
},
{
"answerText": "Variational automated encoders",
"isCorrect": false
}
]
}
]
},
{
"id": 209,
"title": "Autoencoders: Post Quiz",
"quiz": [
{
"questionText": "Properties of autoencoders include",
"answerOptions": [
{
"answerText": "it is data Specific",
"isCorrect": false
},
{
"answerText": "works on unlabeled data",
"isCorrect": false
},
{
"answerText": "all of the above",
"isCorrect": true
}
]
},
{
"questionText": "Auto encoders can be used to effectively remove noise from images",
"answerOptions": [
{
"answerText": "true",
"isCorrect": true
},
{
"answerText": "false",
"isCorrect": false
}
]
},
{
"questionText": "Variational auto-encoders loss funnction does not consist of which of these?",
"answerOptions": [
{
"answerText": "reconstruction loss",
"isCorrect": false
},
{
"answerText": "KL loss",
"isCorrect": false
},
{
"answerText": "TF loss",
"isCorrect": true
}
]
}
]
}
]
}
]

27
etc/quiz-src/questions-en.txt
View File

@ -185,6 +185,33 @@ Lesson 8E Pre-trained Networks and Transfer Learning: Post Quiz
- batches
+ neural network
Lesson 9B Autoencoders: Pre Quiz
* Self-supervised learning uses ____ data for training
- pre-trained
+ raw
- labeled
* Encoder Network coverts input images into latent spaces
+ true
- false
* VAE is short for?
- Variable AutoEncoding
+ Variation auto-encoder
- Variational automated encoders
Lesson 9E Autoencoders: Post Quiz
* Properties of autoencoders include
- it is data Specific
- works on unlabeled data
+ all of the above
* Auto encoders can be used to effectively remove noise from images
+ true
- false
* Variational auto-encoders loss funnction does not consist of which of these?
- reconstruction loss
- KL loss
+ TF loss
Lesson 23B Multi-Agent Modeling: Pre Quiz
* By modeling the behavior of simple agents, we can understand more complex behaviors of a system.
+ true

2
lessons/4-ComputerVision/09-Autoencoders/AutoEncodersPyTorch.ipynb
View File

File diff suppressed because one or more lines are too long

10
lessons/4-ComputerVision/09-Autoencoders/AutoencodersTF.ipynb
View File

@ -17,7 +17,7 @@
"source": [
"When training CNNs, one of the problems is that we need a lot of labeled data. In the case of image classification, we need to separate images into different classes, which is a manual effort.\n",
"\n",
"However, we might want to use raw (unlabeled) data for training CNN feature extractors, which is called **self-supervised learning**. Instead of labels, we will use training images as both network input and output. The main idea of **autoencoder** is that we will have an **encoder network** that converts input image into some **latent space** (normally it is just a vector of some smaller size), the then **decoder network**, whose goal would be to reconstruct the original image.\n",
"However, we might want to use raw (unlabeled) data for training CNN feature extractors, which is called **self-supervised learning**. Instead of labels, we will use training images as both network input and output. The main idea of **autoencoder** is that we will have an **encoder network** that converts input image into some **latent space** (normally it is just a vector of some smaller size), then the **decoder network**, whose goal would be to reconstruct the original image.\n",
"\n",
"Since we are training autoencoder to capture as much of the information from the original image as possible for accurate reconstruction, the network tries to find the best **embedding** of input images to capture the meaning.\n",
"\n",
@ -629,7 +629,7 @@
"id": "C_u1vG1BDqVx"
},
"source": [
"> **Exercise:** See how denoiser trained on MNIST digits works for different images. As an example, you can take [Fashion MNIST](https://keras.io/api/datasets/fashion_mnist/) dataset, which has the same image size. Note that denoiser works well only on the same image type that is was trained on (i.e. for the same probability distribution of input data)."
"> **Exercise:** See how denoiser trained on MNIST digits works for different images. As an example, you can take [Fashion MNIST](https://keras.io/api/datasets/fashion_mnist/) dataset, which has the same image size. Note that denoiser works well only on the same image type that it was trained on (i.e. for the same probability distribution of input data)."
]
},
{
@ -640,7 +640,7 @@
"source": [
"## Super-resolution\n",
"\n",
"Similarly to denoiser, we can train autoencoders to increase the resolution of the image. To train super-resolution network, we will start with high-res images, and automatically downscale them to produce network inputs. We will then feed autoencoder with small images as inputs and high-res images as outputs.\n",
"Similarly to denoiser, we can train autoencoders to increase the resolution of the image. To train super-resolution network, we will start with high-resolution images, and automatically downscale them to produce network inputs. We will then feed autoencoder with small images as inputs and high-res images as outputs.\n",
"\n",
"Let's downscale MNIST to 14x14:"
]
@ -876,9 +876,9 @@
"source": [
"## Variational Auto-Encoders (VAE)\n",
"\n",
"Traditional autoencoders reduce the dimension of the input data somehow, figuring out the important features of input images. However, latent vectors ofter do not make much sense. In other words, taking MNIST dataset as an example, figuring out which digits correspond to different latent vectors is not an easy task, because close latent vectors would not necessarily correspond to the same digits. \n",
"Traditional autoencoders reduce the dimension of the input data somehow, figuring out the important features of input images. However, latent vectors often do not make much sense. In other words, taking MNIST dataset as an example, figuring out which digits correspond to different latent vectors is not an easy task, because close latent vectors would not necessarily correspond to the same digits. \n",
"\n",
"On the other hand, to train *generative* models it is better to have some understanding of the latent space. This idea leads us to **variational autoencoder** (VAE).\n",
"On the other hand, to train *generative* models it is better to have some understanding of the latent space. This idea leads us to **variational auto-encoder** (VAE).\n",
"\n",
"VAE is the autoencoder that learns to predict *statistical distribution* of the latent parameters, so-called **latent distribution**. For example, we can assume that latent vectors would be distributed as $N(\\mathrm{z\\_mean},e^{\\mathrm{z\\_log\\_sigma}})$, where $\\mathrm{z\\_mean}, \\mathrm{z\\_log\\_sigma} \\in\\mathbb{R}^d$. Encoder in VAE learns to predict those parameters, and then decoder takes a random vector from this distribution to reconstruct the object.\n",
"\n",

19
lessons/4-ComputerVision/09-Autoencoders/README.md
View File

@ -2,7 +2,9 @@
When training CNNs, one of the problems is that we need a lot of labeled data. In the case of image classification, we need to separate images into different classes, which is a manual effort.
However, we might want to use raw (unlabeled) data for training CNN feature extractors, which is called **self-supervised learning**. Instead of labels, we will use training images as both network input and output. The main idea of **autoencoder** is that we will have an **encoder network** that converts input image into some **latent space** (normally it is just a vector of some smaller size), the then **decoder network**, whose goal would be to reconstruct the original image.
## [Pre-lecture quiz](https://black-ground-0cc93280f.1.azurestaticapps.net/quiz/109)
However, we might want to use raw (unlabeled) data for training CNN feature extractors, which is called **self-supervised learning**. Instead of labels, we will use training images as both network input and output. The main idea of **autoencoder** is that we will have an **encoder network** that converts input image into some **latent space** (normally it is just a vector of some smaller size), then the **decoder network**, whose goal would be to reconstruct the original image.
Since we are training autoencoder to capture as much of the information from the original image as possible for accurate reconstruction, the network tries to find the best **embedding** of input images to capture the meaning.л.
@ -14,16 +16,16 @@ Since we are training autoencoder to capture as much of the information from the
While reconstructing original images does not seem useful in its own right, there are a few scenarios where autoencoders are especially useful:
* **Lowering the dimension of images for visualization** or **training image embeddings**. Usually autoencoders give better results than PCA, because it takes into account spatial nature of images and hierarchical features
* **Lowering the dimension of images for visualization** or **training image embeddings**. Usually autoencoders give better results than PCA, because it takes into account spatial nature of images and hierarchical features.
* **Denoising**, i.e. removing noise from the image. Because noise carries out a lot of useless information, autoencoder cannot fit it all into relatively small latent space, and thus it captures only important part of the image. When training denoisers, we start with original images, and use images with artificially added noise as input for autoencoder.
* **Super-resolution**, increasing image resolution. We start with high-res images, and use the image with lower resolution as autoencoder input.
* **Super-resolution**, increasing image resolution. We start with high-resolution images, and use the image with lower resolution as autoencoder input.
* **Generative models**. Once we train autoencoder, the decoder part can be used to create new objects starting from random latent vectors.
## Variational Autoencoders (VAE)
Traditional autoencoders reduce the dimension of the input data somehow, figuring out the important features of input images. However, latent vectors ofter do not make much sense. In other words, taking MNIST dataset as an example, figuring out which digits correspond to different latent vectors is not an easy task, because close latent vectors would not necessarily correspond to the same digits.
Traditional autoencoders reduce the dimension of the input data somehow, figuring out the important features of input images. However, latent vectors ofter do not make much sense. In other words, taking MNIST dataset as an example, figuring out which digits correspond to different latent vectors is not an easy task, because close latent vectors would not necessarily correspond to the same digits.
On the other hand, to train *generative* models it is better to have some understanding of the latent space. This idea leads us to **variational autoencoder** (VAE).
On the other hand, to train *generative* models it is better to have some understanding of the latent space. This idea leads us to **variational auto-encoder** (VAE).
VAE is the autoencoder that learns to predict *statistical distribution* of the latent parameters, so-called **latent distribution**. For example, we may want latent vectors to be distributed normally with some mean z<sub>mean</sub> and standard deviation z<sub>sigma</sub> (both mean and standard deviation are vectors of some dimensionality d). Encoder in VAE learns to predict those parameters, and then decoder takes a random vector from this distribution to reconstruct the object.
@ -38,6 +40,7 @@ To summarize:
> Image from [this blog post](https://ijdykeman.github.io/ml/2016/12/21/cvae.html) by Isaak Dykeman
Variational auto-encoders use complex loss function that consists of two parts:
* **Reconstruction loss** is the loss function that shows how close reconstructed image is to the target (can be MSE). It is the same loss function as in normal autoencoders.
* **KL loss**, which ensures that latent variable distributions stays close to normal distribution. It is based on the notion of [Kullback-Leibler divergence](https://www.countbayesie.com/blog/2017/5/9/kullback-leibler-divergence-explained) - a metric to estimate how similar two statistical distributions are.
@ -52,9 +55,11 @@ Observe how images blend into each other, as we start getting latent vectors fro
<img src="images/vaemnist-diag.png" width="50%"/>
> Image generated by author
## Continue to Notebooks
* [Autoencoders in TensorFlow](AutoencodersTF.ipynb)
* [Autoencoders in PyTorch](AutoEncodersPyTorch.ipynb)
## Properties of Autoencoders
@ -62,6 +67,10 @@ Observe how images blend into each other, as we start getting latent vectors fro
* **Lossy** - reconstructed image is not the same as the original image. The nature of loss is defined by the *loss function* used during training
* Works on **unlabeled data**
## [Post-lecture quiz](https://black-ground-0cc93280f.1.azurestaticapps.net/quiz/209)
> ✅ Todo: Conclusion, Assignment
## Reference
* [Building Autoencoders in Keras](https://blog.keras.io/building-autoencoders-in-keras.html)