RomanNum3ral
/
AI-For-Beginners
mirror of https://github.com/microsoft/AI-For-Beginners.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #44 from jlooper/main 

Perceptron Edits plus homepage tweaks for NN segment
This commit is contained in:
Dmitri Soshnikov 2022-04-21 16:03:10 +03:00 committed by GitHub
parent 5926cba8e1 1b0228e265
commit 6c2953e954
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
6 changed files with 265 additions and 32 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
1-Intro/README.md
View File

@ -4,7 +4,7 @@
> Sketchnote by [Tomomi Imura](https://twitter.com/girlie_mac)
## [Pre-lecture quiz](link-to-quiz-app) TODO
## [Pre-lecture quiz](https://black-ground-0cc93280f.1.azurestaticapps.net/quiz/0)
**Artificial Intelligence** is an exciting scientific discipline that studies how we can make computers exhibit intelligent behavior, e.g. do those things that human beings are good at doing.
@ -140,7 +140,7 @@ Over the past few years we have witnessed huge successes with large language mod
Do a tour of the internet to determine where, in your opinion, AI is most effectively used. Is it in a Mapping app, or some speech-to-text service or a video game? Research how the system was built.
## [Post-lecture quiz](link-to-quiz-app) TODO
## [Post-lecture quiz](https://black-ground-0cc93280f.1.azurestaticapps.net/quiz/1)
## Review & Self Study

6
2-Symbolic/README.md
View File

@ -6,7 +6,7 @@
The quest for artificial intelligence is based on a search for knowledge, to make sense of the world similar to how humans do. But how can you go about doing this?
## [Pre-lecture quiz](link-to-quiz-app) TODO
## [Pre-lecture quiz](https://black-ground-0cc93280f.1.azurestaticapps.net/quiz/2)
In the early days of AI, the top-down approach to creating intelligent systems (discussed in the previous lesson) was popular. The idea was to extract the knowledge from people into some machine-readable form, and then use it to automatically solve problems. This approach was based on two big ideas:
@ -90,7 +90,7 @@ Simplified structure of a human neural system | Architecture of a knowledge-base
Expert systems are built like the human reasoning system, which contains **short-term memory** and **long-term memory**. Similarly, in knowledge-based systems we distinguish the following components:
* **Problem memory**: contains the knowledge about the problem being currently solved, i.e. the temperature or blood pressure of a patient, whether he has inflammation or not, etc. This knowledge is also called **static knowledge**, because it contains a snapshot of what we currently know about the problem - the so-called *problem state*.
* **Knowledge base**: represents long-term knowledge about a problem domain. It is extracted manually from human experts, and does not change from consulation to consultation. Because it allows us to navigate from one problem state to another, it is also called **dynamic knowledge**.
* **Knowledge base**: represents long-term knowledge about a problem domain. It is extracted manually from human experts, and does not change from consultation to consultation. Because it allows us to navigate from one problem state to another, it is also called **dynamic knowledge**.
* **Inference engine**: orchestrates the whole process of searching in the problem state space, asking questions of the user when necessary. It is also responsible for finding the right rules to be applied to each state.
As an example, let's consider the following expert system of determining an animal based on its physical characteristics:
@ -220,7 +220,7 @@ Nowadays, AI is often considered to be a synonym for *Machine Learning* or *Neur
In the three notebooks associated to this lesson, there are challenges at the end - pick one, and try to solve it!
## [Post-lecture quiz](link-to-quiz-app) TODO
## [Post-lecture quiz](https://black-ground-0cc93280f.1.azurestaticapps.net/quiz/3)
## Review & Self Study

49
3-NeuralNetworks/03-Perceptron/README.md
View File

@ -1,20 +1,25 @@
# Introduction to Neural Networks. Perceptron.
# Introduction to Neural Networks: Perceptron
One of the first attempts to implement something similar to a modern neural network was done by Frank Rosenblatt from Cornell Aeronautical Laboratory in 1957. It was hardware implementation called "Mark-1", designed to recognize primitive geometric figures, such as triangles, squares and circles.
## [Pre-lecture quiz](https://black-ground-0cc93280f.1.azurestaticapps.net/quiz/4)
<img src='images/Rosenblatt-wikipedia.jpg' alt='Frank Rosenblatt'/> | <img src='images/Mark_I_perceptron_wikipedia.jpg' alt='The Mark 1 Perceptron' />
-----|-----
One of the first attempts to implement something similar to a modern neural network was done by Frank Rosenblatt from Cornell Aeronautical Laboratory in 1957. It was a hardware implementation called "Mark-1", designed to recognize primitive geometric figures, such as triangles, squares and circles.
| | |
|--------------|-----------|
|<img src='images/Rosenblatt-wikipedia.jpg' alt='Frank Rosenblatt'/> | <img src='images/Mark_I_perceptron_wikipedia.jpg' alt='The Mark 1 Perceptron' />|
> Images [from Wikipedia](https://en.wikipedia.org/wiki/Perceptron)
An input image was represented by 20x20 photocell array, so the neural network had 400 inputs and one binary output. Simple network contained one neuron, also called **threshold logic unit**. Neural network weights were potentiometers that required manual adjustment during the training phase.
An input image was represented by 20x20 photocell array, so the neural network had 400 inputs and one binary output. A simple network contained one neuron, also called a **threshold logic unit**. Neural network weights acted like potentiometers that required manual adjustment during the training phase.
> New York Times wrote about perceptron at that time:
> ✅ A potentiometer is a device that allows the user to adjust the resistance of a circuit.
> The New York Times wrote about perceptron at that time:
> *the embryo of an electronic computer that [the Navy] expects will be able to walk, talk, see, write, reproduce itself and be conscious of its existence.*
## Perceptron Model
Suppose we have N features in our model, in which case the input vector would be a vector of size N. Perceptron is a **binary classification** model, i.e. it can distinguish between two classes of input data. We will assume that for each input vector x the output of our perceptron would be either +1 or -1, depending on the class. The output will be computed using the formula
Suppose we have N features in our model, in which case the input vector would be a vector of size N. A perceptron is a **binary classification** model, i.e. it can distinguish between two classes of input data. We will assume that for each input vector x the output of our perceptron would be either +1 or -1, depending on the class. The output will be computed using the formula:
y(x) = f(w<sup>T</sup>x)
@ -25,19 +30,20 @@ where f is a step activation function
## Training the Perceptron
To train a perceptron we need to find weights vector w that classifies most of the values correctly, i.e. results in the smallest **error**. This error is defined by **perceptron criterion** on the following manner:
To train a perceptron we need to find a weights vector w that classifies most of the values correctly, i.e. results in the smallest **error**. This error is defined by **perceptron criterion** in the following manner:
E(w) = -&sum;w<sup>T</sup>x<sub>i</sub>t<sub>i</sub>
where
where:
* the sum is taken on those training data points i that result in the wrong classification
* x<sub>i</sub> is the input data, and t<sub>i</sub> is either -1 or +1 for negative and positive examples accordingly.
This criteria is considered as a function of weights w, and we need to minimize it. Often, a method called **gradient descent** is used, in which we start with some initial weights w<sup>(0)</sup>, and then at each step update the weights according to the formula
This criteria is considered as a function of weights w, and we need to minimize it. Often, a method called **gradient descent** is used, in which we start with some initial weights w<sup>(0)</sup>, and then at each step update the weights according to the formula:
w<sup>(t+1)</sup> = w<sup>(t)</sup> - &eta;&nabla;E(w)
Here &eta; is so-called **learning rate**, and &nabla;E(w) denotes the **gradient** of E. After we calculate the gradient, we end up with
Here &eta; is the so-called **learning rate**, and &nabla;E(w) denotes the **gradient** of E. After we calculate the gradient, we end up with
w<sup>(t+1)</sup> = w<sup>(t)</sup> + &sum;&eta;x<sub>i</sub>t<sub>i</sub>
@ -63,13 +69,26 @@ def train(positive_examples, negative_examples, num_iterations = 100, eta = 1):
return weights
```
## [Proceed to Notebook](Perceptron.ipynb)
## Conclusion
To see how we can use perceptron to solve some toy as well as real-life problems, and to continue learning - go to [Perceptron](Perceptron.ipynb) notebook.
In this lesson, you learned about a perceptron, which is a binary classification model, and how to training it by using a weights vector.
## [Lab](lab/README.md)
## 🚀 Challenge
In this lesson, we have implemented perceptron for binary classification task, and we have used it to classify between two handwritten digits. In this lab, you are asked to solve the problem of digit classification entirely, i.e. determine which digit is most likely to correspond to a given image.
If you'd like to try to build your own perceptron, try [this lab on Microsoft Learn](https://docs.microsoft.com/en-us/azure/machine-learning/component-reference/two-class-averaged-perceptron?WT.mc_id=academic-57639-dmitryso) which uses the [Azure ML designer](https://docs.microsoft.com/en-us/azure/machine-learning/concept-designer?WT.mc_id=academic-57639-dmitryso).
## [Post-lecture quiz](https://black-ground-0cc93280f.1.azurestaticapps.net/quiz/5)
## Review & Self Study
To see how we can use perceptron to solve a toy problem as well as real-life problems, and to continue learning - go to [Perceptron](Perceptron.ipynb) notebook.
Here's an interesting [article about perceptrons](https://towardsdatascience.com/what-is-a-perceptron-basics-of-neural-networks-c4cfea20c590
) as well.
## [Assignment](lab/README.md)
In this lesson, we have implemented a perceptron for binary classification task, and we have used it to classify between two handwritten digits. In this lab, you are asked to solve the problem of digit classification entirely, i.e. determine which digit is most likely to correspond to a given image.
* [Instructions](lab/README.md)
* [Notebook](lab/PerceptronMultiClass.ipynb)

4
3-NeuralNetworks/03-Perceptron/lab/README.md
View File

@ -4,13 +4,13 @@ Lab Assignment from [AI for Beginners Curriculum](https://github.com/microsoft/a
## Task
Using the code we have developed in this lesson for binary classification of MNIST handwritten digits, create a multi-class classified that would be able to recognize any digit. Compute classification accuracy on train and test dataset, and print out the confusion matrix.
Using the code we have developed in this lesson for binary classification of MNIST handwritten digits, create a multi-class classified that would be able to recognize any digit. Compute the classification accuracy on the train and test dataset, and print out the confusion matrix.
## Hints
1. For each digit, create a dataset for binary classifier of "this digit vs. all other digits"
1. Train 10 different perceptrons for binary classification (one for each digit)
1. Define a function that will classify input digit
1. Define a function that will classify an input digit
> **Hint**: If we combine weights of all 10 perceptrons into one matrix, we should be able to apply all 10 perceptrons to the input digits by one matrix multiplication. Most probable digit can then be found just by applying `argmax` operation on the output.

22
3-NeuralNetworks/README.md
View File

@ -2,23 +2,25 @@
![Summary of Intro Neural Networks content in a doodle](../sketchnotes/ai-neuralnetworks.png)
As we have discussed in the introduction, one of the ways to achieve intelligence is to train a **computer model** or an **artificial brain**. Since the middle of 20th century, researchers tried different mathematical models, until in recent years this direction proved to by hugely successful. Such mathematical models of the brain are called **Neural Networks** (sometimes *Artificial Neural Networks*, ANNs, in order to indicate that we are talking about models, not real networks of neurons).
As we discussed in the introduction, one of the ways to achieve intelligence is to train a **computer model** or an **artificial brain**. Since the middle of 20th century, researchers tried different mathematical models, until in recent years this direction proved to by hugely successful. Such mathematical models of the brain are called **neural networks**.
> Sometimes neural networks are called *Artificial Neural Networks*, ANNs, in order to indicate that we are talking about models, not real networks of neurons.
## Machine Learning
Neural Networks are a part of larger discipline called **Machine Learning**, whose goal is to use data to train computer models that are able to solve problems. Machine Learning constitutes a large part of Artificial Intelligence, however, we do not cover classical ML in this curricula.
Neural Networks are a part of a larger discipline called **Machine Learning**, whose goal is to use data to train computer models that are able to solve problems. Machine Learning constitutes a large part of Artificial Intelligence, however, we do not cover classical ML in this curricula.
> Visit our separate **[Machine Learning for Beginners](http://github.com/microsoft/ml-for-beginners)** curriculum to learn more about Machine Learning.
> Visit our separate **[Machine Learning for Beginners](http://github.com/microsoft/ml-for-beginners)** curriculum to learn more about classic Machine Learning.
In Machine Learning, we assume that we have some dataset of examples **X**, and corresponding output values **Y**. Examples are often N-dimensional vectors that consist of **features**, and outputs are called **labels**.
We will consider two most common machine learning problems:
We will consider the two most common machine learning problems:
* **Classification**, where we need to classify an input object into two or more classes.
* **Regression**, where we need to predict a numerical number for each of the input samples.
> When representing inputs and outputs as tensors, input dataset is a matrix of size M&times;N, where M is number of samples, N is the number of features. Output labels Y is the vector of size M.
> When representing inputs and outputs as tensors, the input dataset is a matrix of size M&times;N, where M is number of samples and N is the number of features. Output labels Y is the vector of size M.
In this curricula, we will only focus on neural network models.
In this curriculum, we will only focus on neural network models.
## A Model of a Neuron
@ -28,19 +30,19 @@ From biology we know that our brain consists of neural cells, each of them havin
----|----
Real Neuron *([Image](https://en.wikipedia.org/wiki/Synapse#/media/File:SynapseSchematic_lines.svg) from Wikipedia)* | Artificial Neuron *(Image by Author)*
Thus, simplest mathematical model of a neuron contains several inputs X<sub>1</sub>, ..., X<sub>N</sub> and an output Y, and a series of weights W<sub>1</sub>, ..., W<sub>N</sub>. An output is calculated as
Thus, the simplest mathematical model of a neuron contains several inputs X<sub>1</sub>, ..., X<sub>N</sub> and an output Y, and a series of weights W<sub>1</sub>, ..., W<sub>N</sub>. An output is calculated as:
<img src="images/netout.png" alt="Y = f\left(\sum_{i=1}^N X_iW_i\right)" width="131" height="53" align="center"/>
where f is some non-linear **activation function**.
> Early models of neuron were described in classical paper [A logical calculus of the ideas immanent in nervous activity](http://www.springerlink.com/content/61446605110620kg/fulltext.pdf) by Warren McCullock and Walter Pitts in 1943. Donald Hebb in his book "[The Organization of Behavior: A Neuropsychological Theory](https://books.google.com/books?id=VNetYrB8EBoC)" proposed the way those networks can be trained.
> Early models of neuron were described in the classical paper [A logical calculus of the ideas immanent in nervous activity](http://www.springerlink.com/content/61446605110620kg/fulltext.pdf) by Warren McCullock and Walter Pitts in 1943. Donald Hebb in his book "[The Organization of Behavior: A Neuropsychological Theory](https://books.google.com/books?id=VNetYrB8EBoC)" proposed the way those networks can be trained.
## In this Section
In this section we will learn about:
* [Perceptron](03-Perceptron/README.md), one of the earliest neural network models for two-class classification
* [Multi-layered networks](04-OwnFramework/README.md) and [how to build our own framework](04-OwnFramework/OwnFramework.ipynb)
* [Neural Network Frameworks](05-Frameworks/README.md), such as [PyTorch](05-Frameworks/IntroPyTorch.ipynb) and [Keras/TensorFlow](05-Frameworks/IntroKerasTF.ipynb)
* [Multi-layered networks](04-OwnFramework/README.md) with a paired notebook [how to build our own framework](04-OwnFramework/OwnFramework.ipynb)
* [Neural Network Frameworks](05-Frameworks/README.md), with these notebooks: [PyTorch](05-Frameworks/IntroPyTorch.ipynb) and [Keras/Tensorflow](05-Frameworks/IntroKerasTF.ipynb)
* [Overfitting](05-Frameworks/Overfitting.md)

212
quiz-app/src/assets/translations/en/group-1.json
View File

@ -109,6 +109,218 @@
]
}
]
},
{
"id": 2,
"title": "Knowledge Representation and Expert Systems: Pre-Quiz",
"quiz": [
{
"questionText": "The top-down approach to creating intelligent systems was based on:",
"answerOptions": [
{
"answerText": "knowledge seeking and reading",
"isCorrect": "false"
},
{
"answerText": "knowledge representation and reasoning",
"isCorrect": "true"
},
{
"answerText": "knowledge reasoning and seeking",
"isCorrect": "false"
}
]
},
{
"questionText": "Knowledge is the same as information",
"answerOptions": [
{
"answerText": "true",
"isCorrect": "false"
},
{
"answerText": "false",
"isCorrect": "true"
}
]
},
{
"questionText": "Knowledge is obtained by an:",
"answerOptions": [
{
"answerText": "active learning process",
"isCorrect": "true"
},
{
"answerText": "passive learning process",
"isCorrect": "false"
},
{
"answerText": "both of these",
"isCorrect": "false"
}
]
}
]
},
{
"id": 3,
"title": "Knowledge Representation and Expert Systems: Post-Quiz",
"quiz": [
{
"questionText": "The simplest method of knowledge representation is:",
"answerOptions": [
{
"answerText": "algorithmic",
"isCorrect": "true"
},
{
"answerText": "symbolic",
"isCorrect": "false"
},
{
"answerText": "synergistic",
"isCorrect": "false"
}
]
},
{
"questionText": "Scenarios can represent complex situations that can unfold in time",
"answerOptions": [
{
"answerText": "true",
"isCorrect": "true"
},
{
"answerText": "false",
"isCorrect": "false"
}
]
},
{
"questionText": "Forward inference starts with initial data and then:",
"answerOptions": [
{
"answerText": "executes a reasoning loop",
"isCorrect": "true"
},
{
"answerText": "looks for a goal",
"isCorrect": "false"
},
{
"answerText": "starts over",
"isCorrect": "false"
}
]
}
]
},
{
"id": 4,
"title": "Introduction to Neural Networks - Perceptron: Post-Quiz",
"quiz": [
{
"questionText": "Early neural networks required",
"answerOptions": [
{
"answerText": "manual weight adjusting",
"isCorrect": "true"
},
{
"answerText": "terabytes of data",
"isCorrect": "false"
},
{
"answerText": "special reasoning",
"isCorrect": "false"
}
]
},
{
"questionText": "A simple neuron is also called a 'threshold logic unit'",
"answerOptions": [
{
"answerText": "true",
"isCorrect": "true"
},
{
"answerText": "false",
"isCorrect": "false"
}
]
},
{
"questionText": "A perceptron is a ___ type of model",
"answerOptions": [
{
"answerText": "linear regression",
"isCorrect": "false"
},
{
"answerText": "clustering",
"isCorrect": "false"
},
{
"answerText": "binary classification",
"isCorrect": "true"
}
]
}
]
},
{
"id": 5,
"title": "Introduction to Neural Networks - Perceptron: Post-Quiz",
"quiz": [
{
"questionText": "To train a perceptron, find a weights vector that results in the smallest ___.",
"answerOptions": [
{
"answerText": "size",
"isCorrect": "false"
},
{
"answerText": "error",
"isCorrect": "true"
},
{
"answerText": "nodes",
"isCorrect": "false"
}
]
},
{
"questionText": "To minimize the function of weights, you can use gradient descent",
"answerOptions": [
{
"answerText": "true",
"isCorrect": "true"
},
{
"answerText": "false",
"isCorrect": "false"
}
]
},
{
"questionText": "During gradient descent, each step updates the ___",
"answerOptions": [
{
"answerText": "learning rate",
"isCorrect": "false"
},
{
"answerText": "weights",
"isCorrect": "true"
},
{
"answerText": "gradient",
"isCorrect": "false"
}
]
}
]
}
]
}