merging changes for ch 5
This commit is contained in:
Jen Looper
commit
763f24173e
|
|
@ -289,7 +289,7 @@
|
|||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"We will define our system as a class that subсlasses `KnowledgeEngine`. Each rule is defined by a separate function with `@Rule` annotation, which specifies when the rule should fire. Inside the rule, we can add new facts using `declare` function, and adding those facts will result in some more rules being called by forward inference engine. "
|
||||
"We will define our system as a class that subclasses `KnowledgeEngine`. Each rule is defined by a separate function with `@Rule` annotation, which specifies when the rule should fire. Inside the rule, we can add new facts using `declare` function, and adding those facts will result in some more rules being called by forward inference engine. "
|
||||
]
|
||||
},
|
||||
{
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -49,7 +49,7 @@
|
|||
"source": [
|
||||
"def train(positive_examples, negative_examples, num_iterations = 100):\n",
|
||||
" num_dims = positive_examples.shape[1]\n",
|
||||
" weights = np.zeros((num_dims,1)) # инициализируем веса\n",
|
||||
" weights = np.zeros((num_dims,1)) # initialize weights\n",
|
||||
" \n",
|
||||
" pos_count = positive_examples.shape[0]\n",
|
||||
" neg_count = negative_examples.shape[0]\n",
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ Using the code we have developed in this lesson for binary classification of MNI
|
|||
|
||||
1. For each digit, create a dataset for binary classifier of "this digit vs. all other digits"
|
||||
1. Train 10 different perceptrons for binary classification (one for each digit)
|
||||
1. Define a function that will classify an input digit
|
||||
1. Define a function that will classify an input digit
|
||||
|
||||
> **Hint**: If we combine weights of all 10 perceptrons into one matrix, we should be able to apply all 10 perceptrons to the input digits by one matrix multiplication. Most probable digit can then be found just by applying `argmax` operation on the output.
|
||||
|
||||
|
|
|
|||
|
|
@ -68,7 +68,7 @@
|
|||
"X = X.astype(np.float32)\n",
|
||||
"Y = Y.astype(np.int32)\n",
|
||||
"\n",
|
||||
"# Разбиваем на обучающую и тестовые выборки\n",
|
||||
"# Split into train and test dataset\n",
|
||||
"train_x, test_x = np.split(X, [n*8//10])\n",
|
||||
"train_labels, test_labels = np.split(Y, [n*8//10])"
|
||||
]
|
||||
|
|
|
|||
|
|
@ -19,7 +19,7 @@ Let's start with formalizing the Machine Learning problem. Suppose we have a tra
|
|||
* For regression problem, when we need to predict a number, we can use **absolute error** ∑<sub>i</sub>|f(x<sup>(i)</sup>)-y<sup>(i)</sup>|, or **squared error** ∑<sub>i</sub>(f(x<sup>(i)</sup>)-y<sup>(i)</sup>)<sup>2</sup>
|
||||
* For classification, we use **0-1 loss** (which is essentially the same as **accuracy** of the model), or **logistic loss**.
|
||||
|
||||
For one-level perceptron, function *f* was defined as a linear function *f(x)=wx+b* (here *w* is the weight matrix, *x* is the vector if input features, and *b* is bias vector). For different neural network architectures, this function can take more complex form.
|
||||
For one-level perceptron, function *f* was defined as a linear function *f(x)=wx+b* (here *w* is the weight matrix, *x* is the vector of input features, and *b* is bias vector). For different neural network architectures, this function can take more complex form.
|
||||
|
||||
> In the case of classification, it is often desirable to get probabilities of corresponding classes as network output. To convert arbitrary numbers to probabilities (eg. to normalize the output), we often use **softmax** function σ, and the function *f* becomes *f(x)=σ(wx+b)*
|
||||
|
||||
|
|
@ -68,7 +68,9 @@ In this lesson, we have built our own neural network library, and we have used i
|
|||
|
||||
## 🚀 Challenge
|
||||
|
||||
In the accompanying notebook, you will implement your own framework for building and training multi-layered perceptrons. You will be able to see in detail how modern neural networks operate. Proceed to the [OwnFramework](OwnFramework.ipynb) notebook and work through it
|
||||
In the accompanying notebook, you will implement your own framework for building and training multi-layered perceptrons. You will be able to see in detail how modern neural networks operate.
|
||||
|
||||
Proceed to the [OwnFramework](OwnFramework.ipynb) notebook and work through it.
|
||||
|
||||
## [Post-lecture quiz](https://black-ground-0cc93280f.1.azurestaticapps.net/quiz/8)
|
||||
|
||||
|
|
|
|||
|
|
@ -6,7 +6,6 @@ Lab Assignment from [AI for Beginners Curriculum](https://github.com/microsoft/a
|
|||
|
||||
Solve the MNIST handwritten digit classification problem using 1-, 2- and 3-layered perceptron. Use the neural network framework we have developed in the lesson.
|
||||
|
||||
|
||||
## Stating Notebook
|
||||
|
||||
Start the lab by opening [MyFW_MNIST.ipynb](MyFW_MNIST.ipynb)
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Overfitting
|
||||
|
||||
Overfitting is an extremely important concept in machine learning, and it is very important to get it right!
|
||||
Overfitting is an extremely important concept in machine learning, and it is very important to get it right!
|
||||
|
||||
Consider the following problem of approximating 5 dots (represented by `x` on the graphs below):
|
||||
|
||||
|
|
@ -33,7 +33,7 @@ If you can see that overfitting occurs, you can do one of the following:
|
|||
|
||||
* Increase the amount of training data
|
||||
* Decrease the complexity of the model
|
||||
* Use some [regularization technique](../4-ComputerVision/08-TransferLearning/TrainingTricks.md), such as [Dropout](../4-ComputerVision/08-TransferLearning/TrainingTricks.md#Dropout), which we will consider later.
|
||||
* Use some [regularization technique](../../4-ComputerVision/08-TransferLearning/TrainingTricks.md), such as [Dropout](../../4-ComputerVision/08-TransferLearning/TrainingTricks.md#Dropout), which we will consider later.
|
||||
|
||||
## Overfitting and Bias-Variance Tradeoff
|
||||
|
||||
|
|
@ -69,4 +69,4 @@ Ask yourself the following questions:
|
|||
In this lab, you are asked to solve two classification problems using single- and multi-layered fully-connected networks using PyTorch or TensorFlow.
|
||||
|
||||
* [Instructions](lab/README.md)
|
||||
* [Notebook](lab/LabFrameworks.ipynb)
|
||||
* [Notebook](lab/LabFrameworks.ipynb)
|
||||
|
|
|
|||
|
|
@ -39,4 +39,4 @@ Low-Level API | [TensorFlow+Keras Notebook](IntroKerasTF.ipynb) | [PyTorch](Intr
|
|||
--------------|-------------------------------------|--------------------------------
|
||||
High-level API| [Keras](IntroKeras.ipynb) | *PyTorch Lightning*
|
||||
|
||||
After mastering the frameworks, let's recap the notion of [overfitting](Overfiting.md).
|
||||
After mastering the frameworks, let's recap the notion of [overfitting](Overfitting.md).
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
|
|
@ -19,7 +19,7 @@
|
|||
"\n",
|
||||
"However, we might want to use raw (unlabeled) data for training CNN feature extractors, which is called **self-supervised learning**. Instead of labels, we will use training images as both network input and output. The main idea of **autoencoder** is that we will have an **encoder network** that converts input image into some **latent space** (normally it is just a vector of some smaller size), the then **decoder network**, whose goal would be to reconstruct the original image.\n",
|
||||
"\n",
|
||||
"Since we are training autoencoder to capture as much of the information from the original image as possible for accurate reconstruction, the network tries to find the best **embedding** of input images to capture the meaning.л.\n",
|
||||
"Since we are training autoencoder to capture as much of the information from the original image as possible for accurate reconstruction, the network tries to find the best **embedding** of input images to capture the meaning.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
|
|
@ -41,7 +41,10 @@
|
|||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {
|
||||
"id": "48ua_UfDIqum"
|
||||
"id": "48ua_UfDIqum",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
|
|
@ -62,7 +65,10 @@
|
|||
"height": 109
|
||||
},
|
||||
"id": "-VvA6_e_Ytv9",
|
||||
"outputId": "7f43ddd9-2044-4c88-a57d-7dffbb0741fe"
|
||||
"outputId": "7f43ddd9-2044-4c88-a57d-7dffbb0741fe",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
|
|
@ -93,7 +99,10 @@
|
|||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {
|
||||
"id": "XVXKfTiKZGzs"
|
||||
"id": "XVXKfTiKZGzs",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
|
|
@ -132,7 +141,10 @@
|
|||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {
|
||||
"id": "xRVImJGBZ2kt"
|
||||
"id": "xRVImJGBZ2kt",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
|
|
@ -150,7 +162,10 @@
|
|||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "H6YD611WaAAG",
|
||||
"outputId": "3cf9f53c-8898-4bde-89b7-3670804c2373"
|
||||
"outputId": "3cf9f53c-8898-4bde-89b7-3670804c2373",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
|
|
@ -255,7 +270,10 @@
|
|||
"height": 256
|
||||
},
|
||||
"id": "CTEBvy2FaRBF",
|
||||
"outputId": "b18d0d16-c88f-47bb-fe2e-2ba5871f0563"
|
||||
"outputId": "b18d0d16-c88f-47bb-fe2e-2ba5871f0563",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
|
|
@ -307,7 +325,10 @@
|
|||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "44u_8PR7kaoY",
|
||||
"outputId": "71916b80-0821-4cd2-c481-bbbd5dac2fd4"
|
||||
"outputId": "71916b80-0821-4cd2-c481-bbbd5dac2fd4",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
|
|
@ -333,7 +354,10 @@
|
|||
"height": 164
|
||||
},
|
||||
"id": "DJS4ZdO5oOFn",
|
||||
"outputId": "89df7031-c58b-46ae-9fc7-4ddbf5e46ef9"
|
||||
"outputId": "89df7031-c58b-46ae-9fc7-4ddbf5e46ef9",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
|
|
@ -363,7 +387,10 @@
|
|||
"height": 109
|
||||
},
|
||||
"id": "WXIys9NGz8bF",
|
||||
"outputId": "c0b2ede8-a041-4cac-ccf6-3e40817df8c5"
|
||||
"outputId": "c0b2ede8-a041-4cac-ccf6-3e40817df8c5",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
|
|
@ -414,7 +441,7 @@
|
|||
"\n",
|
||||
"Autoencoders can be effectively used to remove noise from images. In order to train denoiser, we will start with noise-free images, and add artificial noise to them. Then, we will feed autoencoder with noisy images as input, and noise-free images as output.\n",
|
||||
"\n",
|
||||
"Let's see how this works for MNISТ: "
|
||||
"Let's see how this works for MNIST:"
|
||||
]
|
||||
},
|
||||
{
|
||||
|
|
@ -426,7 +453,10 @@
|
|||
"height": 109
|
||||
},
|
||||
"id": "uVprJu59lte0",
|
||||
"outputId": "3e9e924b-ec01-4ff3-c49a-3dd70db84db1"
|
||||
"outputId": "3e9e924b-ec01-4ff3-c49a-3dd70db84db1",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
|
|
@ -461,7 +491,10 @@
|
|||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "mlwxbuaHomTr",
|
||||
"outputId": "24c6c3be-e755-4216-815a-46561448fe7f"
|
||||
"outputId": "24c6c3be-e755-4216-815a-46561448fe7f",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
|
|
@ -551,7 +584,10 @@
|
|||
"height": 201
|
||||
},
|
||||
"id": "7kctWkcwv3kd",
|
||||
"outputId": "00bfe5d9-882b-46d3-b8b2-8310756bc9f9"
|
||||
"outputId": "00bfe5d9-882b-46d3-b8b2-8310756bc9f9",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
|
|
@ -618,7 +654,10 @@
|
|||
"height": 108
|
||||
},
|
||||
"id": "JGV724C9VQ7m",
|
||||
"outputId": "d63d8ad4-55bc-431b-bc95-79f994713ad3"
|
||||
"outputId": "d63d8ad4-55bc-431b-bc95-79f994713ad3",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
|
|
@ -645,7 +684,10 @@
|
|||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {
|
||||
"id": "VXIFG2ulYuGM"
|
||||
"id": "VXIFG2ulYuGM",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
|
|
@ -686,7 +728,10 @@
|
|||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "tie6d1OCY2ql",
|
||||
"outputId": "0a97cea6-3fd3-41e3-c743-e332535fa779"
|
||||
"outputId": "0a97cea6-3fd3-41e3-c743-e332535fa779",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
|
|
@ -775,7 +820,10 @@
|
|||
"height": 200
|
||||
},
|
||||
"id": "C-wLl0BiZGR4",
|
||||
"outputId": "f0104552-e056-4cbf-a238-09820839d70f"
|
||||
"outputId": "f0104552-e056-4cbf-a238-09820839d70f",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
|
|
@ -847,7 +895,10 @@
|
|||
"cell_type": "code",
|
||||
"execution_count": 21,
|
||||
"metadata": {
|
||||
"id": "a8bqQpG6wuJ9"
|
||||
"id": "a8bqQpG6wuJ9",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
|
|
@ -867,7 +918,10 @@
|
|||
"cell_type": "code",
|
||||
"execution_count": 22,
|
||||
"metadata": {
|
||||
"id": "y6cJ93kwQkM3"
|
||||
"id": "y6cJ93kwQkM3",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
|
|
@ -885,7 +939,10 @@
|
|||
"cell_type": "code",
|
||||
"execution_count": 23,
|
||||
"metadata": {
|
||||
"id": "Bf3UqBMXQwpQ"
|
||||
"id": "Bf3UqBMXQwpQ",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
|
|
@ -917,7 +974,10 @@
|
|||
"cell_type": "code",
|
||||
"execution_count": 24,
|
||||
"metadata": {
|
||||
"id": "czXAjHUJR-BE"
|
||||
"id": "czXAjHUJR-BE",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
|
|
@ -939,7 +999,10 @@
|
|||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "zky-60lERlG-",
|
||||
"outputId": "baed11e4-b165-42d5-d86b-1037741554c8"
|
||||
"outputId": "baed11e4-b165-42d5-d86b-1037741554c8",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
|
|
@ -1047,7 +1110,10 @@
|
|||
"height": 256
|
||||
},
|
||||
"id": "YF8hHAZVR5x_",
|
||||
"outputId": "146edef1-676d-4592-8b9f-250ff4b98fa6"
|
||||
"outputId": "146edef1-676d-4592-8b9f-250ff4b98fa6",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
|
|
@ -1100,7 +1166,10 @@
|
|||
"height": 433
|
||||
},
|
||||
"id": "jZ0PzYmInDi5",
|
||||
"outputId": "b114fa04-ba23-47e3-b63e-1e3eac248463"
|
||||
"outputId": "b114fa04-ba23-47e3-b63e-1e3eac248463",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
|
|
@ -1142,7 +1211,10 @@
|
|||
"height": 303
|
||||
},
|
||||
"id": "AZQfTelyOigw",
|
||||
"outputId": "3c0fc864-e3e5-4b57-d84f-572358f2e61b"
|
||||
"outputId": "3c0fc864-e3e5-4b57-d84f-572358f2e61b",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
|
|
|
|||
|
|
@ -23,7 +23,10 @@
|
|||
"cell_type": "code",
|
||||
"execution_count": 13,
|
||||
"metadata": {
|
||||
"id": "vh9JiNcw80sd"
|
||||
"id": "vh9JiNcw80sd",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
|
|
@ -52,7 +55,10 @@
|
|||
"cell_type": "code",
|
||||
"execution_count": 14,
|
||||
"metadata": {
|
||||
"id": "iOWm55Bd5DNl"
|
||||
"id": "iOWm55Bd5DNl",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
|
|
@ -81,9 +87,9 @@
|
|||
},
|
||||
"source": [
|
||||
"A few tricks used in generator:\n",
|
||||
"* Instead of ReLU, we use **Leaky ReLU**, i.e. a ReLU which is not exactly 0 for negative $x$, but rather another linear function with very small slope.\n",
|
||||
"* Instead of ReLU, we use **Leaky ReLU**, i.e. a ReLU which is not exactly 0 for negative $x$, but rather another linear function with very small slope. This is important, because it helps gradient descent to propagate values even if we are on the negative side of ReLU (where values are 0)\n",
|
||||
"* We use Batch Normalization in order to stabilize training\n",
|
||||
"* The activation function on last layer is `tanh`, so the output is in the range [-1,1].\n",
|
||||
"* The activation function on last layer is `tanh`, so the output is in the range [-1,1]\n",
|
||||
"\n",
|
||||
"## Discriminator\n",
|
||||
"\n",
|
||||
|
|
@ -94,7 +100,10 @@
|
|||
"cell_type": "code",
|
||||
"execution_count": 15,
|
||||
"metadata": {
|
||||
"id": "krxyG2oh87pU"
|
||||
"id": "krxyG2oh87pU",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
|
|
@ -124,7 +133,10 @@
|
|||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {
|
||||
"id": "TsJH1VWQ9yeB"
|
||||
"id": "TsJH1VWQ9yeB",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
|
|
@ -148,7 +160,10 @@
|
|||
"cell_type": "code",
|
||||
"execution_count": 17,
|
||||
"metadata": {
|
||||
"id": "ZzX0kMrW-dQA"
|
||||
"id": "ZzX0kMrW-dQA",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
|
|
@ -166,15 +181,21 @@
|
|||
"\n",
|
||||
"On each step of the training, we have two phases:\n",
|
||||
"\n",
|
||||
"* Training discriminator. We generate some random vectors `noise` (training happens in minibatches, so we use 100 vectors at a time), produce для этого передаём ему на вход последовательность из 50% реальных и 50% искусственных изображений (и соответствующие labels)\n",
|
||||
" * Обучаем генератор (неявно через обучение генеративно-состязательной модели с замороженными весами дискриминатора) на искусственных данных"
|
||||
"* Training discriminator:\n",
|
||||
" - We generate some random vectors `noise`. Training happens in minibatches, so we use `batch//2` vectors to produce `batch//2` generated images\n",
|
||||
" - Sample `batch//2` random images from the dataset\n",
|
||||
" - Train discriminator on 50% real and 50% generated images, providing corresponding labels (0 or 1)\n",
|
||||
" * Train the generator by using combined adversarial model, passing random vectors as input, and expecting 1's as output (which corresponds to real images)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 18,
|
||||
"metadata": {
|
||||
"id": "aISx_wo4W1qJ"
|
||||
"id": "aISx_wo4W1qJ",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
|
|
@ -196,7 +217,10 @@
|
|||
"height": 673
|
||||
},
|
||||
"id": "mNdcpEa3_Pqs",
|
||||
"outputId": "75ebec09-e608-4747-8298-0cb2281d95af"
|
||||
"outputId": "75ebec09-e608-4747-8298-0cb2281d95af",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
|
|
@ -357,7 +381,9 @@
|
|||
"source": [
|
||||
"## DCGAN\n",
|
||||
"\n",
|
||||
"**Deep Convolutional GAN** is pretty obvious idea of using convolutional layers for generator and discriminator. The main difference here is using `Conv2DTranspose` layer in the generator."
|
||||
"In the previous example, we have used dense networks for both generator and discriminator, but we know than CNNs provide better performance when dealing with images. **Deep Convolutional GAN** is similar to the architecture above, but it uses convolutional layers for generator and discriminator. \n",
|
||||
"\n",
|
||||
"The main difficulty here is to build an architecture for denerator, because it has to do an inverse task comaring to traditional CNN - it has to generate image from feature vector. In a way, this is similar to decoder part of autoencoders.That's why we will be using `Conv2DTranspose` layers in the generator."
|
||||
]
|
||||
},
|
||||
{
|
||||
|
|
@ -368,7 +394,10 @@
|
|||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "fkD2WufskBdm",
|
||||
"outputId": "b4a22c66-a493-4abd-dc79-55c9fd7be137"
|
||||
"outputId": "b4a22c66-a493-4abd-dc79-55c9fd7be137",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
|
|
@ -393,7 +422,10 @@
|
|||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "tAk-hsF0TzEv",
|
||||
"outputId": "68bba515-be2d-44f5-f079-b461456ed082"
|
||||
"outputId": "68bba515-be2d-44f5-f079-b461456ed082",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
|
|
@ -460,7 +492,10 @@
|
|||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {
|
||||
"id": "t9FZHScTUsCd"
|
||||
"id": "t9FZHScTUsCd",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
|
|
@ -492,7 +527,10 @@
|
|||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"metadata": {
|
||||
"id": "2Jz166pQWRba"
|
||||
"id": "2Jz166pQWRba",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
|
|
@ -512,7 +550,10 @@
|
|||
"height": 1000
|
||||
},
|
||||
"id": "kohaucz-WTue",
|
||||
"outputId": "edf64874-5ec9-4c10-cb36-95e2bb5111cf"
|
||||
"outputId": "edf64874-5ec9-4c10-cb36-95e2bb5111cf",
|
||||
"vscode": {
|
||||
"languageId": "python"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
|
|
@ -23,7 +23,7 @@ CNN discriminator consists of the following layers: several convolutions+pooling
|
|||
|
||||
### Generator
|
||||
|
||||
Generator is slightly more tricky. You can consider it to be a reversed discriminator - starting from latent vector (in place of a feature vector), it has fully-connected layer to convert it into required size/shape, followed by deconvolutions+upscaling.
|
||||
Generator is slightly more tricky. You can consider it to be a reversed discriminator - starting from latent vector (in place of a feature vector), it has fully-connected layer to convert it into required size/shape, followed by deconvolutions+upscaling. This is similar to *decoder* part of [autoencoder](../09-Autoencoders/README.md).
|
||||
|
||||
> Because convolution layer is implemented as a linear filter traversing the image, deconvolution is essentially similar to convolution, and can be implemented using the same layer logic.
|
||||
|
||||
|
|
@ -33,7 +33,7 @@ Generator is slightly more tricky. You can consider it to be a reversed discrimi
|
|||
|
||||
### Training the GAN
|
||||
|
||||
GANs are called **adversarial** because there is a constant competition between generator and discriminator. During this cometition, both generator and discriminator improve, thus the network learns to produce better and better pictures.
|
||||
GANs are called **adversarial** because there is a constant competition between generator and discriminator. During this competition, both generator and discriminator improve, thus the network learns to produce better and better pictures.
|
||||
|
||||
The training happens in two stages:
|
||||
|
||||
|
|
@ -43,8 +43,8 @@ The training happens in two stages:
|
|||
During this process, both generator and discriminator losses are not going down significantly. In the ideal situation, they should oscillate, corresponding to both networks improving their performance.
|
||||
|
||||
## Go to Notebook
|
||||
* [GAN Notebook in TensorFlow/Keras](GANs.ipynb)
|
||||
|
||||
* [GAN Notebook in TensorFlow/Keras](GANTF.ipynb)
|
||||
* [GAN Notebook in PyTorch](GANPyTorch.ipynb)
|
||||
### Problems with GAN training
|
||||
|
||||
GANs are known to be especially difficult to train. Here are a few problems:
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
Binary file not shown.
|
After Width: | Height: | Size: 131 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 211 KiB |
|
|
@ -63,7 +63,7 @@ For a gentle introduction to *AI in the Cloud* topic you may consider taking the
|
|||
<tr><td>7</td><td>Convolutional Neural Networks<br/>CNN Architectures</td><td><a href="4-ComputerVision/07-ConvNets/README.md">Text</a><br/><a href="4-ComputerVision/07-ConvNets/CNN_Architectures.md">Text</a></td><td><a href="4-ComputerVision/07-ConvNets/ConvNetsPyTorch.ipynb">PyTorch</a></td><td><a href="4-ComputerVision/07-ConvNets/ConvNetsTF.ipynb">TensorFlow</a></td><td><a href="4-ComputerVision/07-ConvNets/lab/README.md">Lab</a></td></tr>
|
||||
<tr><td>8</td><td>Pre-trained Networks and Transfer Learning<br/>Training Tricks</td><td><a href="4-ComputerVision/08-TransferLearning/README.md">Text</a><br/><a href="4-ComputerVision/08-TransferLearning/TrainingTricks.md">Text</a></td><td><a href="4-ComputerVision/08-TransferLearning/TransferLearningPyTorch.ipynb">PyTorch</a></td><td><a href="4-ComputerVision/08-TransferLearning/TransferLearningTF.ipynb">TensorFlow</a><br/><a href="4-ComputerVision/08-TransferLearning/Dropout.ipynb">Dropout sample</a></td><td><a href="4-ComputerVision/08-TransferLearning/lab/README.md">Lab</a></td></tr>
|
||||
<tr><td>9</td><td>Autoencoders and VAEs</td><td><a href="4-ComputerVision/09-Autoencoders/README.md">Text</a></td><td><a href="4-ComputerVision/09-Autoencoders/AutoEncodersPytorch.ipynb">PyTorch</td><td><a href="4-ComputerVision/09-Autoencoders/AutoencodersTF.ipynb">TensorFlow</a></td><td></td></tr>
|
||||
<tr><td>10</td><td>Generative Adversarial Networks</td><td><a href="4-ComputerVision/10-GANs/README.md">Text</a></td><td><a href="4-ComputerVision/10-GANs/GANPytorch.ipynb">PyTorch</td><td><a href="4-ComputerVision/10-GANs/GANs.ipynb">TensorFlow</a></td><td></td></tr>
|
||||
<tr><td>10</td><td>Generative Adversarial Networks</td><td><a href="4-ComputerVision/10-GANs/README.md">Text</a></td><td><a href="4-ComputerVision/10-GANs/GANPyTorch.ipynb">PyTorch</td><td><a href="4-ComputerVision/10-GANs/GANTF.ipynb">TensorFlow</a></td><td></td></tr>
|
||||
<tr><td>11</td><td>Object Detection</td><td>Text</td><td>PyTorch</td><td>TensorFlow</td><td></td></tr>
|
||||
<tr><td>12</td><td>Semantic Segmentation. U-Net</td><td><a href="4-ComputerVision/12-Segmentation/README.md">Text</a></td><td><a href="4-ComputerVision/12-Segmentation/SemanticSegmentationPytorch.ipynb">PyTorch</td><td><a href="4-ComputerVision/12-Segmentation/SemanticSegmentationTF.ipynb">TensorFlow</td><td></td></tr>
|
||||
<tr><td>V</td><td colspan="2"><b><a href="5-NLP/README.md">Natural Language Processing</a></b></td>
|
||||
|
|
|
|||
Loading…
Reference in New Issue