From 0931c79d5a3e84a96bd1eda53507b179bbc96bb4 Mon Sep 17 00:00:00 2001 From: Dmitri Soshnikov Date: Wed, 30 Mar 2022 14:41:02 +0300 Subject: [PATCH] Add Genetic and Multiagent --- .../21-GeneticAlgorithms/Diophantine.ipynb | 36 + 6-Other/21-GeneticAlgorithms/Genetic.ipynb | 696 ++++++++++++++++++ 6-Other/21-GeneticAlgorithms/README.md | 57 ++ 6-Other/23-MultiagentSystems/README.md | 26 + 7-Ethics/README.md | 37 + README.md | 36 +- 6 files changed, 873 insertions(+), 15 deletions(-) create mode 100644 6-Other/21-GeneticAlgorithms/Diophantine.ipynb create mode 100644 6-Other/21-GeneticAlgorithms/Genetic.ipynb create mode 100644 6-Other/21-GeneticAlgorithms/README.md create mode 100644 6-Other/23-MultiagentSystems/README.md create mode 100644 7-Ethics/README.md diff --git a/6-Other/21-GeneticAlgorithms/Diophantine.ipynb b/6-Other/21-GeneticAlgorithms/Diophantine.ipynb new file mode 100644 index 00000000..9f71ed9a --- /dev/null +++ b/6-Other/21-GeneticAlgorithms/Diophantine.ipynb @@ -0,0 +1,36 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Assignment: Diophantine Equations\n", + "\n", + "> This assignment is part of [AI for Beginners Curriculum](http://github.com/microsoft/ai-for-beginners) and is inspired by [this post](https://habr.com/post/128704/).\n", + "\n", + "Your goal is to solve so-called **Diophantine equation** - an equation with integer roots and integer coefficients. For example, consider the following equation:\n", + "\n", + "$$a+2b+3c+4d=30$$\n", + "\n", + "You need to find integer roots $a$,$b$,$c$,$d\\in\\mathbb{N}$ that satisfy this equation.\n", + "\n", + "Hints:\n", + "1. You can consider roots to be in the interval [0;30]\n", + "1. As a gene, consider using the list of root values" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [] + } + ], + "metadata": { + "language_info": { + "name": "python" + }, + "orig_nbformat": 4 + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/6-Other/21-GeneticAlgorithms/Genetic.ipynb b/6-Other/21-GeneticAlgorithms/Genetic.ipynb new file mode 100644 index 00000000..d9cb2c62 --- /dev/null +++ b/6-Other/21-GeneticAlgorithms/Genetic.ipynb @@ -0,0 +1,696 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "collapsed": true + }, + "source": [ + "# Genetic Algorithms\n", + "\n", + "This notebook is part of [AI for Beginners Curriculum](http://github.com/microsoft/ai-for-beginners)." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "trusted": true + }, + "outputs": [], + "source": [ + "import random\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "import math\n", + "import time" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Some Theory\n", + "\n", + "**Genetic Algorithms** (GA) are based on **evolutionary approach** to AI, in which methods of evolution of population is used to obtain an optimal solution for a given problem. They were proposed in 1975 by [John Henry Holland](https://en.wikipedia.org/wiki/John_Henry_Holland).\n", + "\n", + "Genetic Algorithms are based on the following ideas:\n", + "* Valid solutions to the problem can be represented as **genes**\n", + "* **Crossover** allows us to combine two solutions together to obtain new valid solution\n", + "* **Selection** is used to select more optimal solutions using some **fitness function**\n", + "* **Mutations** are introduced to destabilize optimization and get us out of the local minimum \n", + "\n", + "If you want to implement a Genetic Algorithm, you need the following:\n", + "\n", + " * To find a method of coding our problem solutions using **genes** $g\\in\\Gamma$\n", + " * On the set of genes $\\Gamma$ we need to define **fitness function** $\\mathrm{fit}: \\Gamma\\to\\mathbb{R}$. Smaller function values would correspond to better solutions.\n", + " * To define **crossover** mechanism to combine two genes together to get a new valid solution $\\mathrm{crossover}: \\Gamma^2\\to\\Gamma$.\n", + " * To define **mutation** mechanism $\\mathrm{mutate}: \\Gamma\\to\\Gamma$.\n", + "In many cases, crossover and mutation are quite simple algorithms to manipulate genes as numeric sequences or bit vectors.\n", + "\n", + "Specific implementation of a genetic algorithm can vary from case to case, but overall structure is the following:\n", + "\n", + "1. Select initial population $G\\subset\\Gamma$\n", + "2. Randomly select one of the operations that will be performed at this step: crossover or mutation \n", + "3. **Crossover**:\n", + " * Randomly select two genes $g_1, g_2 \\in G$\n", + " * Compute crossover $g=\\mathrm{crossover}(g_1,g_2)$\n", + " * If $\\mathrm{fit}(g)<\\mathrm{fit}(g_1)$ or $\\mathrm{fit}(g)<\\mathrm{fit}(g_2)$ - replace corresponding gene in the population by $g$.\n", + "4. **Mutation** - select random gene $g\\in G$ and replace it by $\\mathrm{mutate}(g)$\n", + "5. Repeat from step 2, until we get sufficiently small value of $\\mathrm{fit}$, or until the limit on the number of steps is reached.\n", + "\n", + "Tasks typically solved by GA:\n", + "1. Schedule optimization\n", + "1. Optimal packing\n", + "1. Optimal cutting\n", + "1. Speeding up exhaustive search\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Problem 1: Fair Treasure Split\n", + "\n", + "**Task**: \n", + "Two people found a treasure that contains diamonds of different sizes (and, correspondingly, different price). They need to split the treasure in two parts in such a way that the difference in the price is 0 (or minimal).\n", + "\n", + "**Formal definition**: \n", + "We have a set of numbers $S$. We need to split it into two subsets $S_1$ and $S_2$, such that $$\\left|\\sum_{i\\in S_1}i - \\sum_{j\\in S_2}j\\right|\\to\\min$$ and $S_1\\cup S_2=S$, $S_1\\cap S_2=\\emptyset$.\n", + "\n", + "First of all, let's define the set $S$:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "trusted": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[8344 2197 9335 3131 5863 9429 3818 9791 15 5455 1396 9538 4872 6549\n", + " 8587 5986 6021 9764 8102 5083 5739 7684 8498 3007 6599 820 7490 2372\n", + " 9370 5235 3525 3154 859 1906 8159 3950 2173 2988 2050 349 8713 2284\n", + " 4177 6033 1651 9176 5049 8201 171 5081 1216 3756 4711 2757 7738 1272\n", + " 5650 6584 5395 9004 7797 969 8104 1283 1392 4001 5768 445 274 256\n", + " 8239 8015 4381 9021 1189 8879 1411 3539 6526 8011 136 7230 2332 451\n", + " 5702 2989 4320 2446 9578 8486 4027 2410 9588 8981 2177 1493 3232 9151\n", + " 4835 5594 6859 8394 369 3200 126 4259 2283 7755 2014 2458 8327 8082\n", + " 7413 7622 1206 5533 8751 3495 5868 8472 6850 3958 3149 4672 4810 6274\n", + " 4700 6134 4627 4616 6656 9949 884 2256 7419 1926 7973 5319 5967 9158\n", + " 3823 7697 9466 5675 5412 9784 5426 8209 3421 1136 6047 4429 8001 4417\n", + " 1381 722 7350 6018 6235 7860 5853 7660 5937 6242 1 9552 3971 8302\n", + " 2633 9227 7283 154 8599 4269 9392 8539 1630 368 2409 9351 3838 9814\n", + " 6186 5743 5083 1325 1610 779 3643 3262 5768 8725 961 4611 6310 4788\n", + " 1648 5951 8118 7779]\n" + ] + } + ], + "source": [ + "N = 200\n", + "S = np.array([random.randint(1,10000) for _ in range(N)])\n", + "print(S)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's encode each possible solution of the problem by a binary vector $B\\in\\{0,1\\}^N$, where the number on $i$-th position shows to which of the sets ($S_1$ or $S_2$) the $i$-th number in the original set $S$ belongs. `generate` function will generate those random binary vectors." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "trusted": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[1 0 0 1 1 1 1 1 0 1 1 1 0 0 1 0 1 1 1 0 0 1 1 0 1 1 0 0 1 0 1 0 1 0 1 1 1\n", + " 0 1 1 1 0 1 0 0 1 0 0 1 1 0 1 0 1 1 0 0 1 0 0 0 1 1 0 1 1 0 0 0 0 1 0 1 0\n", + " 1 0 0 0 0 0 1 1 0 1 0 0 1 0 1 0 0 1 1 0 0 1 1 1 0 0 1 1 0 1 1 0 0 0 0 1 1\n", + " 1 0 1 0 0 1 1 1 1 1 1 1 1 0 1 0 1 1 1 1 1 1 0 1 0 1 0 1 0 0 1 1 1 0 0 1 1\n", + " 0 1 1 0 1 1 0 0 0 1 1 0 0 0 0 0 0 0 0 1 1 0 1 1 1 0 0 1 1 0 1 1 0 0 1 1 0\n", + " 0 0 0 1 0 1 1 0 1 1 0 1 0 0 0]\n" + ] + } + ], + "source": [ + "def generate(S):\n", + " return np.array([random.randint(0,1) for _ in S])\n", + "\n", + "b = generate(S)\n", + "print(b)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's now define `fit` function that calculates the \"cost\" of the solution. It will be the difference between sum or two sets, $S_1$ and $S_2$:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "trusted": true + }, + "outputs": [ + { + "data": { + "text/plain": [ + "133784" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "def fit(B,S=S):\n", + " c1 = (B*S).sum()\n", + " c2 = ((1-B)*S).sum()\n", + " return abs(c1-c2)\n", + "\n", + "fit(b)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we need to define functions for mutation and crossover:\n", + "* For mutation, we will select one random bit and negate it (change from 0 to 1 and vice versa)\n", + "* For crossover, we will take some bits from one vector, and some bits from another one. We will use the same `generate` function to randomly select, which bits to take from which of the input masks." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "trusted": true + }, + "outputs": [], + "source": [ + "def mutate(b):\n", + " x = b.copy()\n", + " i = random.randint(0,len(b)-1)\n", + " x[i] = 1-x[i]\n", + " return x\n", + "\n", + "def xover(b1,b2):\n", + " x = generate(b1)\n", + " return b1*x+b2*(1-x)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's create initial population of the solutions $P$ of the size `pop_size`:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "trusted": true + }, + "outputs": [], + "source": [ + "pop_size = 30\n", + "P = [generate(S) for _ in range(pop_size)]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now, the main function to perform the evolution. `n` is the number of steps of evolution to undergo. At each step:\n", + "* With the probability of 30% we perform a mutation, and replace the element with the worst `fit` function by the mutated element\n", + "* With the probability of 70% we perform crossover\n", + "\n", + "The function returns the best solution (gene corresponding to the best solution), and the history of minimal fit function in the population on each iteration." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "trusted": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[0 0 0 1 1 0 0 0 0 1 1 1 0 1 0 0 0 1 0 1 0 1 0 1 0 1 1 0 0 0 0 0 1 0 1 1 0\n", + " 0 0 0 1 1 0 0 1 0 0 0 0 0 1 0 0 1 1 1 1 1 1 1 0 1 1 0 1 1 1 1 1 0 1 0 0 0\n", + " 0 1 1 1 0 1 0 1 1 1 1 1 0 0 0 1 1 0 1 0 0 1 0 0 1 1 1 1 1 1 1 1 0 1 0 1 1\n", + " 0 1 1 0 0 0 0 1 1 1 1 0 1 0 0 1 0 1 1 1 0 1 0 0 0 0 0 0 1 1 0 0 0 1 1 0 0\n", + " 1 0 1 1 1 1 1 0 1 0 1 0 1 1 1 0 0 0 1 1 0 0 0 0 0 0 1 1 1 0 1 0 0 0 1 0 1\n", + " 0 1 0 1 0 0 1 1 1 0 1 1 0 0 1] 4\n" + ] + } + ], + "source": [ + "def evolve(P,S=S,n=2000):\n", + " res = []\n", + " for _ in range(n):\n", + " f = min([fit(b) for b in P])\n", + " res.append(f)\n", + " if f==0:\n", + " break\n", + " if random.randint(1,10)<3:\n", + " i = random.randint(0,len(P)-1)\n", + " b = mutate(P[i])\n", + " i = np.argmax([fit(z) for z in P])\n", + " P[i] = b\n", + " else:\n", + " i = random.randint(0,len(P)-1)\n", + " j = random.randint(0,len(P)-1)\n", + " b = xover(P[i],P[j])\n", + " if fit(b)" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "plt.plot(hist)\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Problem 2: N Queens Problem\n", + "\n", + "**Task**:\n", + "You need to place $N$ queens on a chess board of the size $N\\times N$ in such a way that they do not attack each other.\n", + "\n", + "First of all, let's solve the problem without genetic algorithms, using full search. We can represent the state of the board by the list $L$, where $i$-th number in the list is the horizontal position of the queen in $i$-th row. It is quite obvious that each solution will have only one queen per row, and each row would have a queen.\n", + "\n", + "Our goal would be to find the first solution to the problem, after which we will stop searching. You can easily extend this function to generate all possible positions for queens." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "trusted": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[1, 5, 8, 6, 3, 7, 2, 4]\n" + ] + }, + { + "data": { + "text/plain": [ + "True" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "N = 8\n", + "\n", + "def checkbeats(i_new,j_new,l):\n", + " for i,j in enumerate(l,start=1):\n", + " if j==j_new:\n", + " return False\n", + " else:\n", + " if abs(j-j_new) == i_new-i:\n", + " return False\n", + " return True\n", + "\n", + "def nqueens(l,N=8,disp=True):\n", + " if len(l)==N:\n", + " if disp: print(l)\n", + " return True\n", + " else:\n", + " for j in range(1,N+1):\n", + " if checkbeats(len(l)+1,j,l):\n", + " l.append(j)\n", + " if nqueens(l,N,disp): return True\n", + " else: l.pop()\n", + " return False\n", + " \n", + "nqueens([],8)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now let's measure how long does it take to get a solution for 20-queens problem:" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "trusted": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "10.6 s ± 2.17 s per loop (mean ± std. dev. of 7 runs, 1 loop each)\n" + ] + } + ], + "source": [ + "%timeit nqueens([],20,False)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now let's solve the same problem using genetic algorithm. This solution is inspired by [this blog post](https://kushalvyas.github.io/gen_8Q.html).\n", + "\n", + "We will represent each solution by the same list of length $N$, and as a `fit` function we will take the number of queens that attack each other:" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "trusted": true + }, + "outputs": [], + "source": [ + "def fit(L):\n", + " x=0\n", + " for i1,j1 in enumerate(L,1):\n", + " for i2,j2 in enumerate(L,1):\n", + " if i2>i1:\n", + " if j2==j1 or (abs(j2-j1)==i2-i1): x+=1\n", + " return x" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Since calculating fitness function is time consuming, let's store each solution in the population together with the value of fitness function. Let's generate the initial population:" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "trusted": true + }, + "outputs": [ + { + "data": { + "text/plain": [ + "[(array([2, 3, 8, 7, 5, 4, 1, 6]), 4),\n", + " (array([3, 4, 5, 1, 2, 8, 6, 7]), 8),\n", + " (array([1, 3, 7, 4, 5, 8, 6, 2]), 6),\n", + " (array([1, 5, 4, 6, 8, 3, 7, 2]), 4),\n", + " (array([3, 5, 7, 1, 8, 6, 4, 2]), 3)]" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "def generate_one(N):\n", + " x = np.arange(1,N+1)\n", + " np.random.shuffle(x)\n", + " return (x,fit(x))\n", + "\n", + "def generate(N,NP):\n", + " return [generate_one(N) for _ in range(NP)]\n", + "\n", + "generate(8,5)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we need to define mutation and crossover functions. Crossover would combine two genes together by breaking them at some random point and concatenating two parts from different genes together." + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "trusted": true + }, + "outputs": [ + { + "data": { + "text/plain": [ + "array([1, 2, 7, 8])" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "def mutate(G):\n", + " x=random.randint(0,len(G)-1)\n", + " G[x]=random.randint(1,len(G))\n", + " return G\n", + " \n", + "def xover(G1,G2):\n", + " x=random.randint(0,len(G1))\n", + " return np.concatenate((G1[:x],G2[x:]))\n", + "\n", + "xover([1,2,3,4],[5,6,7,8])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We will enhance gene selection process by selecting more genes with better fitness function. The probability of selection of a gene would depend on the fitness function: " + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": { + "trusted": true + }, + "outputs": [], + "source": [ + "def choose_rand(P):\n", + " N=len(P[0][0])\n", + " mf = N*(N-1)//2 # max fitness fn\n", + " z = [mf-x[1] for x in P]\n", + " tf = sum(z) # total fitness\n", + " w = [x/tf for x in z]\n", + " p = np.random.choice(len(P),2,False,p=w)\n", + " return p[0],p[1]\n", + "\n", + "def choose(P):\n", + " def ch(w):\n", + " p=[]\n", + " while p==[]:\n", + " r = random.random()\n", + " p = [i for i,x in enumerate(P) if x[1]>=r]\n", + " return random.choice(p)\n", + " N=len(P[0][0])\n", + " mf = N*(N-1)//2 # max fitness fn\n", + " z = [mf-x[1] for x in P]\n", + " tf = sum(z) # total fitness\n", + " w = [x/tf for x in z]\n", + " p1=p2=0\n", + " while p1==p2:\n", + " p1 = ch(w)\n", + " p2 = ch(w)\n", + " return p1,p2" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now let's define the main evolutionary loop. We will make the logic slightly different from previous example, to show that one can get creative. We will loop until we get the perfect solution (fitness function=0), and at each step we will take the current generation, and produce the new generation of the same size. This is done using `nxgeneration` function, using the following steps:\n", + "\n", + "1. Discard the most unfit solutions - there is `discard_unfit` function that does that\n", + "1. Add some more random solutions to the generation\n", + "1. Populate new generation of size `gen_size` using the following steps for each new gene:\n", + " - select two random genes, with probability proportional to fitness function\n", + " - calculate a crossover\n", + " - apply a mutation with the probability `mutation_prob`" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": { + "trusted": true + }, + "outputs": [ + { + "data": { + "text/plain": [ + "(array([4, 7, 5, 3, 1, 6, 8, 2]), 0)" + ] + }, + "execution_count": 20, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "mutation_prob = 0.1\n", + "\n", + "def discard_unfit(P):\n", + " P.sort(key=lambda x:x[1])\n", + " return P[:len(P)//3]\n", + "\n", + "def nxgeneration(P):\n", + " gen_size=len(P)\n", + " P = discard_unfit(P)\n", + " P.extend(generate(len(P[0][0]),3))\n", + " new_gen = []\n", + " for _ in range(gen_size):\n", + " p1,p2 = choose_rand(P)\n", + " n = xover(P[p1][0],P[p2][0])\n", + " if random.random()0:\n", + " #print(\"Generation {0}, fit={1}\".format(n,mf))\n", + " n+=1\n", + " mf = min([x[1] for x in P])\n", + " P = nxgeneration(P)\n", + " mi = np.argmin([x[1] for x in P])\n", + " return P[mi]\n", + "\n", + "genetic(8)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "It is interesting that in most of the times we are able to get a solution pretty quickly, but in some rare cases optimization reaches local minimum, and the process is stuck for a long time. It is important to take that into account when you are measuring average time: while in most of the cases genetic algorithm will be faster than full search, in some cases it can take longer. To overcome this problem, it often makes sense to limit the number of generations to consider, and if we are not able to find the solution - we can start from scratch. " + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": { + "trusted": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "The slowest run took 18.71 times longer than the fastest. This could mean that an intermediate result is being cached.\n", + "26.4 s ± 28.7 s per loop (mean ± std. dev. of 7 runs, 1 loop each)\n" + ] + } + ], + "source": [ + "%timeit genetic(10)" + ] + } + ], + "metadata": { + "interpreter": { + "hash": "86193a1ab0ba47eac1c69c1756090baa3b420b3eea7d4aafab8b85f8b312f0c5" + }, + "kernelspec": { + "display_name": "Python 3.6", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.5" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/6-Other/21-GeneticAlgorithms/README.md b/6-Other/21-GeneticAlgorithms/README.md new file mode 100644 index 00000000..3118cad2 --- /dev/null +++ b/6-Other/21-GeneticAlgorithms/README.md @@ -0,0 +1,57 @@ +# Genetic Algorithms + +**Genetic Algorithms** (GA) are based on **evolutionary approach** to AI, in which methods of evolution of population is used to obtain an optimal solution for a given problem. They were proposed in 1975 by [John Henry Holland](https://en.wikipedia.org/wiki/John_Henry_Holland). + +Genetic Algorithms are based on the following ideas: + +* Valid solutions to the problem can be represented as **genes** +* **Crossover** allows us to combine two solutions together to obtain new valid solution +* **Selection** is used to select more optimal solutions using some **fitness function** +* **Mutations** are introduced to destabilize optimization and get us out of the local minimum + +If you want to implement a Genetic Algorithm, you need the following: + + * To find a method of coding our problem solutions using **genes** g∈Γ + * On the set of genes Γ we need to define **fitness function** fit: Γ→**R**. Smaller function values correspond to better solutions. + * To define **crossover** mechanism to combine two genes together to get a new valid solution crossover: Γ2→Γ. + * To define **mutation** mechanism mutate: Γ→Γ. +In many cases, crossover and mutation are quite simple algorithms to manipulate genes as numeric sequences or bit vectors. + +Specific implementation of a genetic algorithm can vary from case to case, but overall structure is the following: + +1. Select initial population G⊂Γ +2. Randomly select one of the operations that will be performed at this step: crossover or mutation +3. **Crossover**: + * Randomly select two genes g1, g2 ∈ G + * Compute crossover g=crossover(g1,g2) + * If fit(g)1) or fit(g)2) - replace corresponding gene in the population by g. +4. **Mutation** - select random gene g∈G and replace it by mutate(g) +5. Repeat from step 2, until we get sufficiently small value of fit, or until the limit on the number of steps is reached. + +## Typical Tasks + +Tasks typically solved by GA: +1. Schedule optimization +1. Optimal packing +1. Optimal cutting +1. Speeding up exhaustive search + + +## Notebooks + +Go to [Genetic.ipynb](Genetic.ipynb) notebooks to see two examples of using Genetic Algorithms: + +1. Fair division of treasure +1. 8 Queen Problem + +## Assignment + +Your goal is to solve so-called **Diophantine equation** - an equation with integer roots. For example, consider the equation a+2b+3c+4d=30. You need to find integer roots that satisfy this equation. + +Hints: +1. You can consider roots to be in the interval [0;30] +1. As a gene, consider using the list of root values + +Use [Diophantine.ipynb](Diophantine.ipynb) as a starting point. + +*This assignment is inspired by [this post](https://habr.com/post/128704/).* diff --git a/6-Other/23-MultiagentSystems/README.md b/6-Other/23-MultiagentSystems/README.md new file mode 100644 index 00000000..8ee8cfb4 --- /dev/null +++ b/6-Other/23-MultiagentSystems/README.md @@ -0,0 +1,26 @@ +# Multiagent Systems + +One of the possible ways of achieving intelligence is so-called **emergent** (or **synergetic**) approach, which is based on the fact that combined behavior of many relatively simple agents can result in the overall more complex (or intelligent) behavior of the system as a whole. Theoretically, this is based on the principles of [Collective Intelligence](https://en.wikipedia.org/wiki/Collective_intelligence), [Emergentism](https://en.wikipedia.org/wiki/Global_brain) and [Evolutionary Cybernetics](https://en.wikipedia.org/wiki/Global_brain), which state that higher-level systems gain some sort of added value when being properly combined from lower-level systems (so-called *principle of metasystem transition*). + +The direction of **Multi-Agent Systems** has emerged in AI in 1990s as a response to growth of Internet and distributed systems. On of the classical AI textbooks, [Artificial Intelligence: A Modern Approach](https://en.wikipedia.org/wiki/Artificial_Intelligence:_A_Modern_Approach), focuses on the view of classical AI from the point of view of Multi-agent systems. + +Central to Multi-agent approach is the notion of **Agent** - an entity that lives in some **environment**, which it can perceive, and act upon. This is a very broad definition, and there could be many different types and classifications of agents: + +* By their ability to reason: + - **Reactive** agents usually have simple request-response type of behavior + - **Deliberative** agents employ some sort of logical reasoning and/or planning capabilities +* By the place where agent execute its code: + - **Static** agents work on a dedicated network node + - **Mobile** agents can move their code between network nodes +* By their behavior: + - **Passive agents** do not have specific goals. Such agents can react to external stimuli, but will not initiate any actions themselves. + - **Active agents** have some goals which they pursue + - **Cognitive agents** involve complex planning and reasoning + +Multi-agent systems are nowadays used in a number of applications: +* In games, many non-player characters employ some sort of AI, and can be considered to be intelligent agents +* In video production, rendering complex 3D scenes that involve crowds is typically done using multi-agent simulation +* In systems modeling, multi-agent approach is used to simulate the behavior of a complex model. For example, multi-agent approach has been successfully used to predict the spread of COVID-19 disease worldwide. Similar approach can be used to model traffic in the city, and see how it reacts to changes in traffic rules. +* In complex automation systems, each device can act as an independent agent, which makes the whole system less monolith and more robust. + +## NetLogo \ No newline at end of file diff --git a/7-Ethics/README.md b/7-Ethics/README.md new file mode 100644 index 00000000..37cab163 --- /dev/null +++ b/7-Ethics/README.md @@ -0,0 +1,37 @@ +# Ethical and Responsible AI + +You have almost finished this course, and I hope that by now you clearly see that AI is based on a number of formal mathematical methods that allow us to find relationships in data and train models to replicate the human behavior in some areas. At this point in history, we consider AI to be a very powerful tool to extract patterns from data, and to apply those patterns to solve new problems. + +However, in science fiction we often see stories where AI presents a danger to the humankind. Usually those stories are centered around some sort of AI rebellion, when AI decides to confront human beings. This implies that AI has some sort of emotions, or can take decisions unforeseen by its developers. + +The kind of AI that we have learnt about in this course is nothing more than large matrix arithmetics. It is a very powerful tool to help us solve our problems, and as any other powerful tool - it can be used for good and for bad purposes. What's also important, it can be *misused*. + +## Principles of Responsible AI + +To avoid this accidental misuse of AI, Microsoft states important [Principles of Responsible AI](https://www.microsoft.com/ai/responsible-ai). + +* **Fairness** is related to the important problem of *model biases*, which can be caused by using biased data for training. For example, when we try to predict the probability of getting a software developer job for a person, the model is likely to give higher preference to males - just because the training dataset was likely biased towards male audience. We need to carefully balance training data and investigate the model to avoid biases, and make sure that the model takes into account more relevant features. +* **Reliability and Safety**. By their nature, AI models can make mistakes. A neural network returns probabilities, and we need to take it into account when making decisions. Every model has some precision and recall, and we need to understand that to prevent harm that a wrong advice can cause. +* **Privacy and Security** have some AI-specific flavour. For example, when we use some data for training a model, this data becomes somehow "integrated" into the model. On one hand, that increases security and privacy, on the other - we need to remember which data the model was trained on. +* **Inclusiveness** means that we are not building AI to replace people, but rather to augment people and make our work more creative. It is also related to fairness, because when dealing with underrepresented communities, most of the datasets we collect are likely to be biased, and we need to make sure that those communities are included and correctly handled by AI. +* **Transparency**. This includes making sure that we are always clear about AI being used. Also, wherever possible, we want to use AI systems that are *interpretable*. +* **Accountability**. When AI models come up with some decisions, it is not always clear who is responsible for those decisions. We need to make sure that we understand the responsibility of AI decisions. In most of the cases we would want to include human being into the loop of taking important decisions, and people are made accountable. + +## Tools for Responsible AI + +At Microsoft, we have developed [Responsible AI Toolbox](https://github.com/microsoft/responsible-ai-toolbox), which contains a set of tools: + +* Interpretability Dashboard (InterpretML) +* Fairness Dashboard (FairLearn) +* Error Analysis Dashboard +* Responsible AI Dashboard that includes + - EconML - tool for Causal Analysis, which focuses on what-if questions + - DiCE - tool for Counterfactual Analysis allows you to see which features need to be changed to affect the decision of the model + +## Model Interpretability + +* Glass box models +* Black box models + +## Model Fairness + diff --git a/README.md b/README.md index e9ed1632..29a7f28e 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -[![GitHub license](https://img.shields.io/github/license/microsoft/AI-For-Beginners.svg)](https://github.com/microsoft/AI-For-Beginners/blob/master/LICENSE) +[![GitHub license](https://img.shields.io/github/license/microsoft/AI-For-Beginners.svg)](https://github.com/microsoft/AI-For-Beginners/blob/main/LICENSE) [![GitHub contributors](https://img.shields.io/github/contributors/microsoft/AI-For-Beginners.svg)](https://GitHub.com/microsoft/AI-For-Beginners/graphs/contributors/) [![GitHub issues](https://img.shields.io/github/issues/microsoft/AI-For-Beginners.svg)](https://GitHub.com/microsoft/AI-For-Beginners/issues/) [![GitHub pull-requests](https://img.shields.io/github/issues-pr/microsoft/AI-For-Beginners.svg)](https://GitHub.com/microsoft/AI-For-Beginners/pulls/) @@ -16,18 +16,22 @@ Azure Cloud Advocates at Microsoft are pleased to offer a 12-week, 24-lesson curriculum all about **Artificial Intelligence**. In this curriculum, you will learn: -* Different approaches to Artificial Intelligence, including the "good old" symbolic approach with **Knowledge Representation** and reasoning. -* **Neural Networks** and **Deep Learning**, which are at the core of modern AI. We will illustrate the concepts behind these important topics using code in two of the most popular frameworks - TensorFlow(http://Tensorflow.org) and PyTorch(http://pytorch.org). + +* Different approaches to Artificial Intelligence, including the "good old" symbolic approach with **Knowledge Representation** and reasoning ([GOFAI](https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence)). +* **Neural Networks** and **Deep Learning**, which are at the core of modern AI. We will illustrate the concepts behind these important topics using code in two of the most popular frameworks - [TensorFlow](http://Tensorflow.org) and [PyTorch](http://pytorch.org). * **Neural Architectures** for working with images and text. We will cover recent models but may lack a little bit on the state-of-the-art. -* Less popular AI approaches, such as **Genetic Algorithms**. +* Less popular AI approaches, such as **Genetic Algorithms** and **Multi-Agent Systems**. What we will not cover in this curriculum: -* **Classic Machine Learning**, which is well described in our [Machine Learning for Beginners Curriculum](http://github.com/Microsoft/ML-for-Beginners) -* Practical AI applications built using **[Cognitive Services](https://azure.microsoft.com/services/cognitive-services/?WT.mcid=academic-33554-dmitryso)**. For this, we recommend that you start with modules Microsoft Learn for [vision](https://docs.microsoft.com/learn/paths/create-computer-vision-solutions-azure-cognitive-services/?WT.mcid=academic-33554-dmitryso), [natural language processing](https://docs.microsoft.com/learn/paths/explore-natural-language-processing/?WT.mcid=academic-33554-dmitryso) and others. -* Specific ML **Cloud Frameworks**, such as [Azure Machine Learning](https://azure.microsoft.com/services/machine-learning/?WT.mcid=academic-33554-dmitryso). There is a great learning path called [Build and operate machine learning solutions with Azure Machine Learning](https://docs.microsoft.com/learn/paths/build-ai-solutions-with-azure-ml-service/?WT.mcid=academic-33554-dmitryso) for this topic. -* **Conversational AI** and **Chat Bots**. There is a separate [Create conversational AI solutions](https://docs.microsoft.com/learn/paths/create-conversational-ai-solutions/?WT.mcid=academic-33554-dmitryso) learning path, and you can also refer to [this blog post](https://soshnikov.com/azure/hello-bot-conversational-ai-on-microsoft-platform/) for more detail. -For a gentle introduction to *AI in the Cloud* topic you may consider taking the [Get started with artificial intelligence on Azure](https://docs.microsoft.com/learn/paths/get-started-with-artificial-intelligence-on-azure/?WT.mcid=academic-33554-dmitryso) Learning Path. +* Business cases of using **AI in Business**. Consider taking [Introduction to AI for business users](https://docs.microsoft.com/learn/paths/introduction-ai-for-business-users/?WT.mc_id=academic-33554-dmitryso) learning path on Microsoft Learn, or [AI Business School](https://www.microsoft.com/ai/ai-business-school/?WT.mc_id=academic-33554-dmitryso), developed in cooperation with [INSEAD](https://www.insead.edu/). +* **Classic Machine Learning**, which is well described in our [Machine Learning for Beginners Curriculum](http://github.com/Microsoft/ML-for-Beginners) +* Practical AI applications built using **[Cognitive Services](https://azure.microsoft.com/services/cognitive-services/?WT.mc_id=academic-33554-dmitryso)**. For this, we recommend that you start with modules Microsoft Learn for [vision](https://docs.microsoft.com/learn/paths/create-computer-vision-solutions-azure-cognitive-services/?WT.mc_id=academic-33554-dmitryso), [natural language processing](https://docs.microsoft.com/learn/paths/explore-natural-language-processing/?WT.mc_id=academic-33554-dmitryso) and others. +* Specific ML **Cloud Frameworks**, such as [Azure Machine Learning](https://azure.microsoft.com/services/machine-learning/?WT.mc_id=academic-33554-dmitryso) or [Azure Databricks](). Consider using [Build and operate machine learning solutions with Azure Machine Learning](https://docs.microsoft.com/learn/paths/build-ai-solutions-with-azure-ml-service/?WT.mc_id=academic-33554-dmitryso) and [Build and O perate Machine Learning Solutions with Azure Databricks](https://docs.microsoft.com/learn/paths/build-operate-machine-learning-solutions-azure-databricks/?WT.mc_id=academic-33554-dmitryso) learning paths. +* **Conversational AI** and **Chat Bots**. There is a separate [Create conversational AI solutions](https://docs.microsoft.com/learn/paths/create-conversational-ai-solutions/?WT.mc_id=academic-33554-dmitryso) learning path, and you can also refer to [this blog post](https://soshnikov.com/azure/hello-bot-conversational-ai-on-microsoft-platform/) for more detail. +* **Deep Mathematics** behind deep learning. For this, we would recommend [Deep Learning](https://www.amazon.com/Deep-Learning-Adaptive-Computation-Machine/dp/0262035618) by Ian Goodfellow, Yoshua Bengio and Aaron Courville, which is also available online at [https://www.deeplearningbook.org/](https://www.deeplearningbook.org/). + +For a gentle introduction to *AI in the Cloud* topic you may consider taking the [Get started with artificial intelligence on Azure](https://docs.microsoft.com/learn/paths/get-started-with-artificial-intelligence-on-azure/?WT.mc_id=academic-33554-dmitryso) Learning Path. --- # Content @@ -55,8 +59,8 @@ For a gentle introduction to *AI in the Cloud* topic you may consider taking the MS Learn PAT 6Intro to Computer Vision. OpenCVTextNotebook -7Convolutional Neural Networks
CNN Architectures
Training TricksText
TextPyTorchTensorFlow -8Pre-trained Networks and Transfer LearningText
TextPyTorchTensorFlow
Dropout sample +7Convolutional Neural Networks
CNN ArchitecturesText
TextPyTorchTensorFlow +8Pre-trained Networks and Transfer Learning
Training TricksText
TextPyTorchTensorFlow
Dropout sample 9Autoencoders and VAEsTextPyTorchTensorFlow 10Generative Adversarial NetworksTextPyTorchTensorFlow 11Object DetectionTextPyTorchTensorFlow @@ -74,11 +78,13 @@ For a gentle introduction to *AI in the Cloud* topic you may consider taking the 19Named Entity RecognitionTextPyTorchTensorFlow 20Text Generation using GPTTextPyTorchTensorFlow VIOther AI TechniquesPAT -21Genetic AlgorithmsTextNotebook -22Deep Reinforcement LearningTextPyTorchTensorFlow -23Multi-Agent SystemsText +21Genetic AlgorithmsTextNotebook +22Deep Reinforcement LearningTextPyTorchTensorFlow +23Multi-Agent SystemsText VIIAI EthicsPAT -24AI Ethics and Responsible AIText +24AI Ethics and Responsible AIText +Extras +1Multi-Modal Networks, CLIP and VQGANText Each lesson contains some pre-reading material (linked as **Text** above), and some executable Jupyter Notebooks, which are often specific to the framework (**PyTorch** or **TensorFlow**). The executable notebook also contains a lot of theoretical material, so to understand the topic you need to go through at least one version of the notebooks (either PyTorch or TensorFlow). There are also **Labs** available for some topics, which give you an opportunity to try applying the material you have learnt to a specific problem.