diff --git a/3-NeuralNetworks/05-Frameworks/Overfitting.md b/3-NeuralNetworks/05-Frameworks/Overfitting.md index 515036f9..adbf14ce 100644 --- a/3-NeuralNetworks/05-Frameworks/Overfitting.md +++ b/3-NeuralNetworks/05-Frameworks/Overfitting.md @@ -1,6 +1,6 @@ # Overfitting -Overfitting is an extremely important concept in machine learning, and it is very important to get it right! +Overfitting is an extremely important concept in machine learning, and it is very important to get it right! Consider the following simple problem of approximating 5 dots (represented by `x` on the graphs below): @@ -33,12 +33,13 @@ If you can see that overfitting occurs, you can do one of the following: * Increase the amount of training data * Decrease the complexity of the model - * Use some [regularization technique](../4-ComputerVision/08-TransferLearning/TrainingTricks.md), such as [Dropout](../4-ComputerVision/08-TransferLearning/TrainingTricks.md#Dropout), which we will consider later. + * Use some [regularization technique](../../4-ComputerVision/08-TransferLearning/TrainingTricks.md), such as [Dropout](../../4-ComputerVision/08-TransferLearning/TrainingTricks.md#Dropout), which we will consider later. ## Overfitting and Bias-Variance Tradeoff Overfitting is actually a case of more generic problem in statistics, called [Bias-Variance Tradeoff](https://en.wikipedia.org/wiki/Bias%E2%80%93variance_tradeoff). If we consider possible sources of error in our model, we can see two types of errors: + * **Bias errors** are caused by our algorithm not being able to capture the relationship between training data correctly. It can result from the fact that our model is not powerful enough (**underfitting**). * **Variance errors**, which are caused by the model approximating noise in the input data instead of meaningful relationship (**overfitting**). -During training, bias error decreases (as our model learns to approximate the data), and variance error increases. It is important to stop training - either manually (when we detect overfitting) or automatically (by introducing regularization) to prevent overfitting. +During training, bias error decreases (as our model learns to approximate the data), and variance error increases. It is important to stop training - either manually (when we detect overfitting) or automatically (by introducing regularization) to prevent overfitting. diff --git a/3-NeuralNetworks/05-Frameworks/README.md b/3-NeuralNetworks/05-Frameworks/README.md index 2b721e8e..3e9f7ce2 100644 --- a/3-NeuralNetworks/05-Frameworks/README.md +++ b/3-NeuralNetworks/05-Frameworks/README.md @@ -19,7 +19,7 @@ High-level API| [Keras](https://keras.io/) | [PyTorch Lightning](https://pytorch **High-level APIs** pretty much consider neural network as a **sequence of layers**, and make constructing most of the neural networks much easier. Training the model usually requires preparing the data and then calling `fit` function to do the job. -High-level API allows you to construct typical neural networks very fast, without worrying about lots of details. At the same time, low-level API offer much more control over training process, and thus they are used a lot in research, when you are dealing with new neural network architectures. +High-level API allows you to construct typical neural networks very fast, without worrying about lots of details. At the same time, low-level API offer much more control over training process, and thus they are used a lot in research, when you are dealing with new neural network architectures. It is also important to understand that you can use both APIs together, eg. you can develop your own network layer architecture using low-level API, and then use it inside the larger network constructed and trained with high-level API. Or you can define a network using high-level API as a sequence of layers, and then use your own low-level training loop to perform optimization. Both APIs use the same basic underlying concepts, and they are designed to work well together. @@ -35,4 +35,4 @@ Low-Level API | [TensorFlow+Keras Notebook](IntroKerasTF.ipynb) | [PyTorch](Intr --------------|-------------------------------------|-------------------------------- High-level API| [Keras](IntroKeras.ipynb) | *PyTorch Lightning* -After mastering the frameworks, let's recap the notion of [overfitting](Overfiting.md). \ No newline at end of file +After mastering the frameworks, let's recap the notion of [overfitting](Overfitting.md). diff --git a/3-NeuralNetworks/images/Overtitting.png b/3-NeuralNetworks/images/Overfitting.png similarity index 100% rename from 3-NeuralNetworks/images/Overtitting.png rename to 3-NeuralNetworks/images/Overfitting.png