AI-For-Beginners/translations/pcm/lessons/3-NeuralNetworks/04-OwnFramework

Introduction to Neural Networks. Multi-Layered Perceptron

For di last section, you don learn about di simplest neural network model - one-layered perceptron, wey be linear two-class classification model.

For dis section, we go expand di model make e dey more flexible, so e go fit:

• do multi-class classification join di two-class one
• solve regression problems join di classification
• separate classes wey no fit separate with straight line

We go also build our own modular framework for Python wey go help us create different neural network architectures.

Pre-lecture quiz

Formalization of Machine Learning

Make we start with how Machine Learning problem dey formalize. Imagine say we get training dataset X with labels Y, and we need build model f wey go give di most correct predictions. Di quality of di predictions dey measure by Loss function ℒ. Di following loss functions dey common:

• For regression problem, when we need predict number, we fit use absolute error ∑_i|f(x⁽ⁱ⁾)-y⁽ⁱ⁾|, or squared error ∑_i(f(x⁽ⁱ⁾)-y⁽ⁱ⁾)²
• For classification, we dey use 0-1 loss (wey be di same as accuracy of di model), or logistic loss.

For one-level perceptron, function f na linear function f(x)=wx+b (here w na di weight matrix, x na di vector of input features, and b na bias vector). For different neural network architectures, dis function fit get more complex form.

For classification, e dey good to get probabilities of di classes as network output. To change random numbers to probabilities (like to normalize di output), we dey use softmax function σ, and di function f go be f(x)=σ(wx+b)

For di definition of f above, w and b na parameters θ=⟨w,b⟩. If we get dataset ⟨X,Y⟩, we fit calculate di overall error for di whole dataset as function of parameters θ.

✅ Di goal of neural network training na to reduce di error by changing parameters θ

Gradient Descent Optimization

One popular method for function optimization na gradient descent. Di idea be say we fit calculate di derivative (for multi-dimensional case na gradient) of loss function with respect to parameters, and change di parameters so di error go reduce. E fit formalize like dis:

• Start parameters with random values w⁽⁰⁾, b⁽⁰⁾
• Repeat di following step many times:
  • w⁽ⁱ⁺¹⁾ = w⁽ⁱ⁾-η∂ℒ/∂w
  • b⁽ⁱ⁺¹⁾ = b⁽ⁱ⁾-η∂ℒ/∂b

For training, di optimization steps suppose calculate based on di whole dataset (remember say loss dey calculate as sum of all training samples). But for real life, we dey use small portions of di dataset wey dem dey call minibatches, and calculate gradients based on di subset of data. Because di subset dey pick randomly each time, dis method na stochastic gradient descent (SGD).

Multi-Layered Perceptrons and Backpropagation

One-layer network, as we don see before, fit classify linearly separable classes. To build richer model, we fit join plenty layers for di network. Mathematically, e mean say di function f go get more complex form, and e go calculate in steps:

• z₁=w₁x+b₁
• z₂=w₂α(z₁)+b₂
• f = σ(z₂)

Here, α na non-linear activation function, σ na softmax function, and parameters θ=<w₁,b₁,w₂,b₂>.

Di gradient descent algorithm go still dey di same, but e go hard to calculate gradients. With di chain differentiation rule, we fit calculate derivatives like dis:

• ∂ℒ/∂w₂ = (∂ℒ/∂σ)(∂σ/∂z₂)(∂z₂/∂w₂)
• ∂ℒ/∂w₁ = (∂ℒ/∂σ)(∂σ/∂z₂)(∂z₂/∂α)(∂α/∂z₁)(∂z₁/∂w₁)

✅ Di chain differentiation rule dey use to calculate derivatives of di loss function with respect to parameters.

Notice say di left-most part of all di expressions dey di same, so we fit calculate derivatives well starting from di loss function and go "backwards" through di computational graph. So di method of training multi-layered perceptron na backpropagation, or 'backprop'.

TODO: image citation

✅ We go talk about backprop well for di notebook example.

Conclusion

For dis lesson, we don build our own neural network library, and we don use am for simple two-dimensional classification task.

🚀 Challenge

For di notebook wey follow, you go build your own framework for creating and training multi-layered perceptrons. You go see how modern neural networks dey work.

Go di OwnFramework notebook and work through am.

Post-lecture quiz

Review & Self Study

Backpropagation na common algorithm for AI and ML, e good make you study am well

Assignment

For dis lab, dem ask you to use di framework wey you build for dis lesson to solve MNIST handwritten digit classification.

• Instructions
• Notebook

---

Disclaimer:
Dis docu wey you dey see don use AI translation service Co-op Translator take translate am. Even though we dey try make sure say e correct, make you sabi say automatic translation fit get mistake or no too accurate. Di original docu for di language wey dem first write am na im you go take as di correct one. If na important information, e go better make professional human translator check am. We no go fit take blame for any misunderstanding or wrong interpretation wey fit happen because you use dis translation.