Merge fe61851beb into 33e781bf7b

lessons/3-NeuralNetworks/04-OwnFramework/OwnFramework.ipynb

@@ -706,20 +706,22 @@
     }
    },
    "source": [
-    "To compute $\\partial\\mathcal{L}/\\partial W$ we can use the **chaining rule** for computing derivatives of a composite function, as you can see in the formulae above. It corresponds to the following idea:\n",
+    "To compute $\\partial\\mathcal{L}/\\partial W$, we use the **chain rule** for derivatives of a composite function, as shown above. Backpropagation applies this rule from the output toward the parameters:\n",
     "\n",
-    "* Suppose under given input we have obtanes loss $\\Delta\\mathcal{L}$\n",
-    "* To minimize it, we would have to adjust softmax output $p$ by value $\\Delta p = (\\partial\\mathcal{L}/\\partial p)\\Delta\\mathcal{L}$  \n",
-    "* This corresponds to the changes to node $z$ by $\\Delta z = (\\partial\\mathcal{p}/\\partial z)\\Delta p$\n",
-    "* To minimize this error, we need to adjust parameters accordingly: $\\Delta W = (\\partial\\mathcal{z}/\\partial W)\\Delta z$ (and the same for $b$)\n",
+    "* For a given input, the forward pass produces a loss $\\mathcal{L}$.\n",
+    "* At the softmax output $p$, the upstream gradient is $\\partial\\mathcal{L}/\\partial p$.\n",
+    "* Propagating through the operation that computes $p$ from $z$ gives $\\partial\\mathcal{L}/\\partial z = (\\partial\\mathcal{L}/\\partial p)(\\partial p/\\partial z)$.\n",
+    "* Propagating one step further gives parameter gradients such as $\\partial\\mathcal{L}/\\partial W = (\\partial\\mathcal{L}/\\partial z)(\\partial z/\\partial W)$ (and similarly for $b$).\n",
+    "\n",
+    "An optimizer can then use these gradients to update the parameters in a direction that reduces the loss.\n",
     "\n",
     "<img src=\"images/ComputeGraphGrad.PNG\" width=\"400px\" align=\"right\"/>\n",
     "\n",
-    "This process starts distributing the loss error from the output of the network back to its parameters. Thus the process is called **back propagation**.\n",
+    "This process propagates gradients from the network's output back to its parameters, which is why it is called **backpropagation**.\n",
     "\n",
     "One pass of the network training consists of two parts:\n",
     "* **Forward pass**, when we calculate the value of loss function for a given input minibatch\n",
-    "* **Backward pass**, when we try to minimize this error by distributing it back to the model parameters through the computational graph."
+    "* **Backward pass**, when we compute the gradients of the loss with respect to the model parameters by propagating derivatives backward through the computational graph."
    ]
   },
   {