diff --git a/3-NeuralNetworks/05-Frameworks/README.md b/3-NeuralNetworks/05-Frameworks/README.md index df3e6a94..2b721e8e 100644 --- a/3-NeuralNetworks/05-Frameworks/README.md +++ b/3-NeuralNetworks/05-Frameworks/README.md @@ -9,9 +9,9 @@ While `numpy` library can do the first part, we need some mechanism to compute g Another important thing is to be able to perform computations on GPU, or any other specialized compute units, such as [TPU](https://en.wikipedia.org/wiki/Tensor_Processing_Unit). Deep neural network training requires *a lot* of computations, and to be able to parallelize those computations on GPUs is very important. -Currently, there are two most popular neural frameworks: [Tensorflow](http://tensorflow.org), and [PyTorch](https://pytorch.org/). Both provide low-level API to operate with tensors on both CPU and GPU. On top of the low-level API, there is also higher-level API, called [Keras](https://keras.io/) and [PyTorch Lightning](https://pytorchlightning.ai/) correspondingly. +Currently, there are two most popular neural frameworks: [TensorFlow](http://TensorFlow.org), and [PyTorch](https://pytorch.org/). Both provide low-level API to operate with tensors on both CPU and GPU. On top of the low-level API, there is also higher-level API, called [Keras](https://keras.io/) and [PyTorch Lightning](https://pytorchlightning.ai/) correspondingly. -Low-Level API | [TensorFlow](http://tensorflow.org) | [PyTorch](https://pytorch.org/) +Low-Level API | [TensorFlow](http://TensorFlow.org) | [PyTorch](https://pytorch.org/) --------------|-------------------------------------|-------------------------------- High-level API| [Keras](https://keras.io/) | [PyTorch Lightning](https://pytorchlightning.ai/) @@ -25,7 +25,7 @@ It is also important to understand that you can use both APIs together, eg. you ## Learning -In this course, we offer most of the content both for PyTorch and Tensorflow. You can chose your preferred framework and only go through the corresponding notebooks. If you are not sure which framework to chose - read some discussions on the internet regarding **PyTorch vs. Tensorflow**. You can also have a look at both frameworks to get better understanding. +In this course, we offer most of the content both for PyTorch and TensorFlow. You can chose your preferred framework and only go through the corresponding notebooks. If you are not sure which framework to chose - read some discussions on the internet regarding **PyTorch vs. TensorFlow**. You can also have a look at both frameworks to get better understanding. Where possible, we will use High-Level APIs for simplicity. However, we believe it is important to understand how neural networks work from the ground up, thus in the beginning we start by working with low-level API and tensors. However, if you want to get going fast and do not want to spend a lot of time on details, you can skip those, and go straight into high-level API notebooks. diff --git a/3-NeuralNetworks/README.md b/3-NeuralNetworks/README.md index 16b73517..a5598b61 100644 --- a/3-NeuralNetworks/README.md +++ b/3-NeuralNetworks/README.md @@ -40,5 +40,5 @@ where f is some non-linear **activation function**. In this section we will learn about: * [Perceptron](03-Perceptron/README.md), one of the earliest neural network models for two-class classification * [Multi-layered networks](04-OwnFramework/README.md) and [how to build our own framework](04-OwnFramework/OwnFramework.ipynb) -* [Neural Network Frameworks](05-Frameworks/README.md), such as [PyTorch](05-Frameworks/IntroPyTorch.ipynb) and [Keras/Tensorflow](05-Frameworks/IntroKerasTF.ipynb) +* [Neural Network Frameworks](05-Frameworks/README.md), such as [PyTorch](05-Frameworks/IntroPyTorch.ipynb) and [Keras/TensorFlow](05-Frameworks/IntroKerasTF.ipynb) * [Overfitting](05-Frameworks/Overfitting.md) diff --git a/4-ComputerVision/07-ConvNets/README.md b/4-ComputerVision/07-ConvNets/README.md index 669f662a..9c004a8c 100644 --- a/4-ComputerVision/07-ConvNets/README.md +++ b/4-ComputerVision/07-ConvNets/README.md @@ -30,7 +30,7 @@ The way CNNs work is based on the following important ideas: Let's continue exploring how convolutional neural networks work, and how we can achieve trainable filters, in corresponding notebooks: * [Convolutional Neural Networks - PyTorch](ConvNetsPyTorch.ipynb) -* [Convolutional Neural Networks - Tensorflow](ConvNetsTF.ipynb) +* [Convolutional Neural Networks - TensorFlow](ConvNetsTF.ipynb) ## Pyramid Architecture diff --git a/4-ComputerVision/08-TransferLearning/README.md b/4-ComputerVision/08-TransferLearning/README.md index 331e5b20..2a7d995d 100644 --- a/4-ComputerVision/08-TransferLearning/README.md +++ b/4-ComputerVision/08-TransferLearning/README.md @@ -23,4 +23,4 @@ In this example, we will use a dataset of [Cats and Dogs](https://www.microsoft. Let's see transfer learning in action in corresponding notebooks: * [Transfer Learning - PyTorch](TransferLearningPyTorch.ipynb) -* [Transfer Learning - Tensorflow](TransferLearningTF.ipynb) +* [Transfer Learning - TensorFlow](TransferLearningTF.ipynb) diff --git a/4-ComputerVision/09-Autoencoders/README.md b/4-ComputerVision/09-Autoencoders/README.md index 54849686..b3c3107e 100644 --- a/4-ComputerVision/09-Autoencoders/README.md +++ b/4-ComputerVision/09-Autoencoders/README.md @@ -49,7 +49,7 @@ Observe how images blend into each other, as we start getting latent vectors fro ## Continue to Notebooks -* [Autoencoders in Tensorflow](AutoencodersTF.ipynb) +* [Autoencoders in TensorFlow](AutoencodersTF.ipynb) ## Properties of Autoencoders diff --git a/4-ComputerVision/10-GANs/README.md b/4-ComputerVision/10-GANs/README.md index a711f99e..53a62e50 100644 --- a/4-ComputerVision/10-GANs/README.md +++ b/4-ComputerVision/10-GANs/README.md @@ -39,7 +39,7 @@ The training happens in two stages: During this process, both generator and discriminator losses are not going down significantly. In the ideal situation, they should oscillate, corresponding to both networks improving their performance. ## Go to Notebook -* [GAN Notebook in Tensorflow/Keras](GANs.ipynb) +* [GAN Notebook in TensorFlow/Keras](GANs.ipynb) ### Problems with GAN training diff --git a/5-NLP/13-TextRep/README.md b/5-NLP/13-TextRep/README.md index 3853e33a..c76f384f 100644 --- a/5-NLP/13-TextRep/README.md +++ b/5-NLP/13-TextRep/README.md @@ -45,4 +45,4 @@ However, none of those approaches can fully take into account the semantics of t ## Continue to Notebooks * [Text Representation with PyTorch](TextRepresentationPyTorch.ipynb) -* [Text Representation with Tensorflow](TextRepresentationTF.ipynb) +* [Text Representation with TensorFlow](TextRepresentationTF.ipynb) diff --git a/5-NLP/14-Embeddings/README.md b/5-NLP/14-Embeddings/README.md index b94abb0f..ef024409 100644 --- a/5-NLP/14-Embeddings/README.md +++ b/5-NLP/14-Embeddings/README.md @@ -13,7 +13,7 @@ By using embedding layer as a first layer in our classifier network, we can swit ## Continue in Notebooks * [Embeddings with PyTorch](EmbeddingsPyTorch.ipynb) -* [Embeddings Tensorflow](EmbeddingsTF.ipynb) +* [Embeddings TensorFlow](EmbeddingsTF.ipynb) ## Semantic Embeddings: Word2Vec diff --git a/5-NLP/15-LanguageModeling/README.md b/5-NLP/15-LanguageModeling/README.md index 7541e1f5..95f55196 100644 --- a/5-NLP/15-LanguageModeling/README.md +++ b/5-NLP/15-LanguageModeling/README.md @@ -16,5 +16,5 @@ The idea of CBoW is exactly predicting a missing word, however, to do this we ta ## More Info * [Official PyTorch tutorial on Language Modeling](https://pytorch.org/tutorials/beginner/nlp/word_embeddings_tutorial.html). -* [Official TensorFlow tutorial on training Word2Vec model](https://www.tensorflow.org/tutorials/text/word2vec). +* [Official TensorFlow tutorial on training Word2Vec model](https://www.TensorFlow.org/tutorials/text/word2vec). * Using **gensim** framework to train most commonly used embeddings in a few lines of code is as described [in this documentation](https://pytorch.org/tutorials/beginner/nlp/word_embeddings_tutorial.html). diff --git a/5-NLP/16-RNN/README.md b/5-NLP/16-RNN/README.md index df1db0ed..240afefe 100644 --- a/5-NLP/16-RNN/README.md +++ b/5-NLP/16-RNN/README.md @@ -50,7 +50,7 @@ Recurrent network, one-directional or bidirectional, captures certain patterns w ## Continue to Notebooks * [RNNs with PyTorch](RNNPyTorch.ipynb) -* [RNNs with Tensorflow](RNNTF.ipynb) +* [RNNs with TensorFlow](RNNTF.ipynb) ## RNNs for other tasks diff --git a/5-NLP/17-GenerativeNetworks/README.md b/5-NLP/17-GenerativeNetworks/README.md index 5011a485..903b98c3 100644 --- a/5-NLP/17-GenerativeNetworks/README.md +++ b/5-NLP/17-GenerativeNetworks/README.md @@ -27,7 +27,7 @@ When generating text (during inference), we start with some **prompt**, which is ## Continue to Notebooks * [Generative Networks with PyTorch](GenerativePyTorch.ipynb) -* [Generative Networks with Tensorflow](GenerativeTF.ipynb) +* [Generative Networks with TensorFlow](GenerativeTF.ipynb) ## Soft text generation and temperature diff --git a/5-NLP/18-Transformers/README.md b/5-NLP/18-Transformers/README.md index 2497dbd4..0e102279 100644 --- a/5-NLP/18-Transformers/README.md +++ b/5-NLP/18-Transformers/README.md @@ -55,7 +55,7 @@ Next, we need to capture some patterns within our sequence. To do this, transfor In transformers, we use **Multi-Head Attention**, in order to give network the power to capture several different types of dependencies, eg. long-term vs. short-term word relations, co-reference vs. something else, etc. -[Tensorflow Notebook](TransformersTF.ipynb) contains more detains on the implementation of transformer layers. +[TensorFlow Notebook](TransformersTF.ipynb) contains more detains on the implementation of transformer layers. ### Encoder-Decoder Attention @@ -75,12 +75,12 @@ Since each input position is mapped independently to each output position, trans ![picture from http://jalammar.github.io/illustrated-bert/](images/jalammarBERT-language-modeling-masked-lm.png) -There are many variations of Transformer architectures including BERT, DistilBERT. BigBird, OpenGPT3 and more that can be fine tuned. The [HuggingFace package](https://github.com/huggingface/) provides repository for training many of these architectures with both PyTorch and Tensorflow. +There are many variations of Transformer architectures including BERT, DistilBERT. BigBird, OpenGPT3 and more that can be fine tuned. The [HuggingFace package](https://github.com/huggingface/) provides repository for training many of these architectures with both PyTorch and TensorFlow. ## Continue to Notebooks * [Transformers in PyTorch](TransformersPyTorch.ipynb) -* [Transformers in Tensorflow](TransformersTF.ipynb) +* [Transformers in TensorFlow](TransformersTF.ipynb) ## Related materials diff --git a/5-NLP/README.md b/5-NLP/README.md index 89b9f55a..742a7bac 100644 --- a/5-NLP/README.md +++ b/5-NLP/README.md @@ -28,7 +28,7 @@ If you are using local Python installation to run this course, you may need to i ```bash pip install -r requirements-torch.txt ``` -**For Tensorflow** +**For TensorFlow** ```bash pip install -r requirements-tf.txt ``` @@ -39,7 +39,7 @@ In this section, in some of the examples we will be training quite large models. When running on GPU, you may experience situations when you run out of GPU memory. During training, the amount of GPU memory consumed depends on many factors, including minibatch size. If you experience any memory problems - you may try to minimize the minibatch size in the code. -Also, some older versions of Tensorflow do not release GPU memory correctly if we are training multiple models in one Python kernel. In order to use GPU memory cautiously, you may set tensorflow option to grow GPU memory allocation only when required. You would need to include the following code in your notebooks: +Also, some older versions of TensorFlow do not release GPU memory correctly if we are training multiple models in one Python kernel. In order to use GPU memory cautiously, you may set TensorFlow option to grow GPU memory allocation only when required. You would need to include the following code in your notebooks: ```python physical_devices = tf.config.list_physical_devices('GPU') diff --git a/5-NLP/requirements-tf.txt b/5-NLP/requirements-tf.txt index 1ecdc307..18b4002d 100644 --- a/5-NLP/requirements-tf.txt +++ b/5-NLP/requirements-tf.txt @@ -7,7 +7,7 @@ opencv-python==4.5.1.48 Pillow==7.1.2 scikit-learn scipy -tensorflow -tensorflow_datasets -tensorflow_text +TensorFlow +TensorFlow_datasets +TensorFlow_text transformers==4.3.3 \ No newline at end of file diff --git a/Mindmap.md b/Mindmap.md index 677325f9..7624cc72 100644 --- a/Mindmap.md +++ b/Mindmap.md @@ -14,7 +14,7 @@ - [Multi-Layered Networks](https://github.com/microsoft/AI-For-Beginners/blob/main/3-NeuralNetworks/04-OwnFramework/README.md) - [Intro to Frameworks](https://github.com/microsoft/AI-For-Beginners/blob/main/3-NeuralNetworks/05-Frameworks/README.md) - [PyTorch](https://github.com/microsoft/AI-For-Beginners/blob/main/3-NeuralNetworks/05-Frameworks/IntroPyTorch.ipynb) - - [Tensorflow](https://github.com/microsoft/AI-For-Beginners/blob/main/3-NeuralNetworks/05-Frameworks/IntroKerasTF.md) + - [TensorFlow](https://github.com/microsoft/AI-For-Beginners/blob/main/3-NeuralNetworks/05-Frameworks/IntroKerasTF.md) - [Overfitting](https://github.com/microsoft/AI-For-Beginners/blob/main/3-NeuralNetworks/05-Frameworks/Overfitting.md) ## Computer Vision diff --git a/Mindmap.svg b/Mindmap.svg index 26887ea8..fabdd4fd 100644 --- a/Mindmap.svg +++ b/Mindmap.svg @@ -11,4 +11,4 @@ .mm-rl4bst-3-fo strong { font-weight: bolder; } .mm-rl4bst-3-fo pre { margin: 0; padding: .2em .4em; } -
GloVE
Word2Vec
TF/IDF
Bag of Words
Architectures
Training Tricks
Overfitting
Tensorflow
PyTorch
Multi-Agent Systems
Deep Reinforcement Learning
Genetic Algorithms
Text Generation and GPT
Named Entity Recognition
Transformers and BERT
Generative Recurrent Networks
Recurrent Neural Networks
Language Modeling
Semantic Embeddings
Text Representation
Segmentation
Object Detection
Generative Adversarial Networks
Autoencoders and VAEs
Trasnsfer Learning
Convolutional Networks
Intro to CV. OpenCV
Intro to Frameworks
Multi-Layered Networks
Perceptron
Expert Systems
Knowledge Representation
Approaches to AI
History of AI
AI Definition
AI Ethics
Other Techniques
Natural Language Processing
Computer Vision
Neural Networks
Sybmbolic AI
Introduction to AI
AI
 \ No newline at end of file +
GloVE
Word2Vec
TF/IDF
Bag of Words
Architectures
Training Tricks
Overfitting
TensorFlow
PyTorch
Multi-Agent Systems
Deep Reinforcement Learning
Genetic Algorithms
Text Generation and GPT
Named Entity Recognition
Transformers and BERT
Generative Recurrent Networks
Recurrent Neural Networks
Language Modeling
Semantic Embeddings
Text Representation
Segmentation
Object Detection
Generative Adversarial Networks
Autoencoders and VAEs
Trasnsfer Learning
Convolutional Networks
Intro to CV. OpenCV
Intro to Frameworks
Multi-Layered Networks
Perceptron
Expert Systems
Knowledge Representation
Approaches to AI
History of AI
AI Definition
AI Ethics
Other Techniques
Natural Language Processing
Computer Vision
Neural Networks
Sybmbolic AI
Introduction to AI
AI
 \ No newline at end of file diff --git a/README.md b/README.md index d12d8c40..a610df6e 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@ Azure Cloud Advocates at Microsoft are pleased to offer a 12-week, 24-lesson curriculum all about **Artificial Intelligence**. In this curriculum, you will learn: * Different approaches to Artificial Intelligence, including "good old" symbolic approach with **Knowledge Representation** and reasoning. -* **Neural Networks** and **Deep Learning**, which are at the core of modern AI. We will try to illustrate all ideas using code in two most popular frameworks - Tensorflow(http://tensorflow.org) and PyTorch(http://pytorch.org). +* **Neural Networks** and **Deep Learning**, which are at the core of modern AI. We will try to illustrate all ideas using code in two most popular frameworks - TensorFlow(http://TensorFlow.org) and PyTorch(http://pytorch.org). * **Neural Architectures** for working with Images and Text. We will try to cover very recent models, but may lack a little bit on state-of-the-art. * Less popular AI approaches, such as **Genetic Algorithms** @@ -29,7 +29,7 @@ For a gentle introduction to *AI in the Cloud* topic you may consider taking [Ge # Content - + @@ -42,42 +42,42 @@ For a gentle introduction to *AI in the Cloud* topic you may consider taking [Ge - + - + - + - - - - - - + + + + + + - + - - - - - - - - + + + + + + + + - +
NoLessonIntroPyTorchKeras/TensorflowLab
NoLessonIntroPyTorchKeras/TensorFlowLab
IIntroduction to AIPAT
1Introduction and History of AIText
Notebook
4 Multi-Layered Perceptron and Creating our own FrameworkTextNotebook
5Intro to Frameworks (PyTorch/Tensorflow)
Overfitting		Intro to Frameworks (PyTorch/TensorFlow)
Overfitting		 Text
Text		 PyTorchKeras/Tensorflow
Keras/TensorFlow
IVComputer Vision MS LearnMS LearnMS Learn PAT
6Intro to Computer Vision. OpenCVTextNotebook
7Convolutional Neural Networks
CNN Architectures
Training Tricks		Text
Text		PyTorchTensorflow
8Pre-trained Networks and Transfer LearningText
Text		PyTorchTensorflow
Dropout sample
9Autoencoders and VAEsTextPyTorchTensorflow
10Generative Adversarial NetworksTextPyTorchTensorflow
11Object DetectionTextPyTorchTensorflow
12Instance Segmentation. U-NetTextPyTorchTensorflow
7Convolutional Neural Networks
CNN Architectures
Training Tricks		Text
Text		PyTorchTensorFlow
8Pre-trained Networks and Transfer LearningText
Text		PyTorchTensorFlow
Dropout sample
9Autoencoders and VAEsTextPyTorchTensorFlow
10Generative Adversarial NetworksTextPyTorchTensorFlow
11Object DetectionTextPyTorchTensorFlow
12Instance Segmentation. U-NetTextPyTorchTensorFlow
VNatural Language Processing MS LearnMS LearnMS Learn PAT
13Text Representation. Bow/TF-IDFTextPyTorchTensorflow
14Semantic word embeddings. Word2Vec and GloVeTextPyTorchTensorflow
15Language Modeling. Training your own embeddingsTextPyTorchTensorflow
16Recurrent Neural NetworksTextPyTorchTensorflow
17Generative Recurrent NetworksTextPyTorchTensorflow
18Transformers. BERT.TextPyTorchTensorflow
19Named Entity RecognitionTextPyTorchTensorflow
20Text Generation using GPTTextPyTorchTensorflow
13Text Representation. Bow/TF-IDFTextPyTorchTensorFlow
14Semantic word embeddings. Word2Vec and GloVeTextPyTorchTensorFlow
15Language Modeling. Training your own embeddingsTextPyTorchTensorFlow
16Recurrent Neural NetworksTextPyTorchTensorFlow
17Generative Recurrent NetworksTextPyTorchTensorFlow
18Transformers. BERT.TextPyTorchTensorFlow
19Named Entity RecognitionTextPyTorchTensorFlow
20Text Generation using GPTTextPyTorchTensorFlow
VIOther AI TechniquesPAT
21Genetic AlgorithmsTextNotebook
22Deep Reinforcement LearningTextPyTorchTensorflow
22Deep Reinforcement LearningTextPyTorchTensorFlow
23Multi-Agent SystemsText
VIIAI EthicsPAT
24AI Ethics and Responsible AIText
-Each lesson contains some pre-reading material (linked as **Text** above), and some executable Jupyter Notebooks, which are often specific to the framework (**PyTorch** or **Tensorflow**). The executable notebook also contains a lot of theoretical material, so to understand the topic you need to go through at least one version of the notebooks (either PyTorch or Tensorflow). There are also **Labs** available for some topics, which give you an opportunity to try applying the material you have learnt to some specific problem. +Each lesson contains some pre-reading material (linked as **Text** above), and some executable Jupyter Notebooks, which are often specific to the framework (**PyTorch** or **TensorFlow**). The executable notebook also contains a lot of theoretical material, so to understand the topic you need to go through at least one version of the notebooks (either PyTorch or TensorFlow). There are also **Labs** available for some topics, which give you an opportunity to try applying the material you have learnt to some specific problem. Some sections also contain links to **MS Learn** modules that cover related topics. Microsoft Learn provides convenient GPU-enabled learning environment, although in terms of content you can expect this curriculum to go a bit deeper. @@ -92,7 +92,7 @@ However, if you are serious about the course, we suggest to fork the entire repo - Start with a pre-lecture quiz. - Read the intro text for the lecture -- If the lecture has additional notebooks, go through them, reading and executing the code. If both Tensorflow and PyTorch notebooks are provided, you can focus on one of them - chose your favourite framework. +- If the lecture has additional notebooks, go through them, reading and executing the code. If both TensorFlow and PyTorch notebooks are provided, you can focus on one of them - chose your favourite framework. - Notebooks often contain some of the challenges that require you to tweak the code a little bit to experiment. Do not by lazy and do it! - Take the post-lecture quiz. - If there is a lab attached to the module - complete the assignment. diff --git a/ai-for-beginners.png b/ai-for-beginners.png new file mode 100644 index 00000000..77e62c33 Binary files /dev/null and b/ai-for-beginners.png differ