diff --git a/1-Intro/README.md b/1-Intro/README.md index 1cb99639..671e6db6 100644 --- a/1-Intro/README.md +++ b/1-Intro/README.md @@ -17,7 +17,7 @@ The task of solving a specific human-like problem, such as determining a person' One of the problems when dealing with the term **[Intelligence](https://en.wikipedia.org/wiki/Intelligence)** is that there is no clear definition of this term. One can argue that intelligence is connected to **abstract thinking**, or to **self-awareness**, but we cannot properly define it. -[Photo of a Cat](images/photo-cat.jpg) | To see the ambiguity of a term *intelligence*, try answering a question: "Is a cat intelligent?". Different people tend to give different answer to this question, as there is not universally accepted test to prove this true or not. And if you think there is - try running your cat through an IQ test... +![Photo of a Cat](images/photo-cat.jpg) | To see the ambiguity of a term *intelligence*, try answering a question: "Is a cat intelligent?". Different people tend to give different answer to this question, as there is not universally accepted test to prove this true or not. And if you think there is - try running your cat through an IQ test... ----|---- However, when speaking about AGI we need to have some way to tell if we have created a truly intelligent system. [Alan Turing](https://en.wikipedia.org/wiki/Alan_Turing) proposed such a way called **[Turing Test](https://en.wikipedia.org/wiki/Turing_test)**, which also acts like a definition of intelligence. The idea of such a test is that we compare our system to something inherently intelligent - a real human being. And because any automatic comparison can be easily bypassed by a computer program - we use human interrogator. So, if a human being is unable to distinguish between a real person and a computer system in text-based dialogue - the system is considered intelligent. @@ -26,4 +26,19 @@ However, when speaking about AGI we need to have some way to tell if we have cre ## Different Approaches to AI -If we want computer to behave like a human, we probably \ No newline at end of file +If we want computer to behave like a human, we need somehow to model inside a computer our way of thinking. Consequently, we need to try to understand what makes a human being intelligent. + +There are two possible approaches to this problem: + +Top-down Approach (Symbolic Reasoning) | Bottom-up Approach (Neural Networks) +---------------------------------------|------------------------------------- +Modelling the way a person reasons to solve a problem. It involves extracting **knowledge** from a human being, and representing it in a computer-readable form. We also need to develop a way to model **reasoning** inside a computer. | Modelling a structure of a human brain, consisting of huge number of simplest units called **neurons**. Each neuron acts like a simple weighted average of its inputs, and we can train a network of neurons to solve useful problems by providing **training data**. + +There are also some other possible approaches to intelligence that we will mention here, but the two above are the main ones. +### Top-Down Approach + +In **top-down approach**, we try to model our reasoning. People tend to have some rules in their head, for example, when a doctor is diagnosing a patient, he may realize that a person has a fever, and thus there might be some inflammation going on in his body. By applying a large set of rules to a specific problem a doctor may be able to come up with the final diagnosis. + +This approach relies heavily on **knowledge representation** and **reasoning**. Extracting knowledge from a human expert might be the most difficult part, because a doctor in many cases would not know exactly why he or she is coming up with a particular diagnosis. Sometimes the solution just comes up in his/her head without explicit thinking. Some tasks, such as determining the age of a person from photograph, cannot be at all reduced to manipulating knowledge. + +Another \ No newline at end of file diff --git a/3-NeuralNetworks/05-Frameworks/IntroKerasTF.ipynb b/3-NeuralNetworks/05-Frameworks/IntroKerasTF.ipynb index 94dd075c..f172f8e1 100644 --- a/3-NeuralNetworks/05-Frameworks/IntroKerasTF.ipynb +++ b/3-NeuralNetworks/05-Frameworks/IntroKerasTF.ipynb @@ -1,34 +1,4 @@ { - "nbformat": 4, - "nbformat_minor": 0, - "metadata": { - "celltoolbar": "Slideshow", - "kernelspec": { - "name": "python3", - "display_name": "Python 3", - "language": "python" - }, - "language_info": { - "mimetype": "text/x-python", - "nbconvert_exporter": "python", - "name": "python", - "file_extension": ".py", - "version": "3.7.4-final", - "pygments_lexer": "ipython3", - "codemirror_mode": { - "version": 3, - "name": "ipython" - } - }, - "livereveal": { - "start_slideshow_at": "selected" - }, - "colab": { - "name": "IntroKerasTF.ipynb", - "provenance": [], - "collapsed_sections": [] - } - }, "cells": [ { "cell_type": "markdown", @@ -36,37 +6,25 @@ "id": "En2vX4FuwHlu" }, "source": [ - "# Введение в нейронные сети\n", + "## Introduction to Tensorflow and Keras\n", "\n", - "## Эпизод 2а: Многослойный персептрон на TensorFlow и Keras\n", + "> This notebook is a part of [AI for Beginners Curricula](http://github.com/microsoft/ai-for-beginners). Visit the repository for complete set of learning materials.\n", "\n", - "Дмитрий Сошников | dmitri@soshnikov.com\n", + "### Neural Frameworks\n", "\n", - "http://github.com/shwars/NeuroWorkshop\n", - "-> Notebooks -> IntroKerasTF.ipynb" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "fVjlmWBwwHl2" - }, - "source": [ - "## Нейросетевые фреймворки\n", + "We have learnt that to train neural networks you need:\n", + "* Quickly multiply matrices (tensors)\n", + "* Compute gradients to perform gradient descent optimization\n", "\n", - "Мы видели, что для обучения нейросетей нужно:\n", - "* Быстро умножать матрицы (тензоры)\n", - "* Считать производные для вычисления градиента для метода обратного распространения ошибки\n", + "What neural network frameworks allow you to do:\n", + "* Operate with tensors on whatever compute is available, CPU or GPU, or even TPU\n", + "* Automatically compute gradients (they are explicitly programmed for all built-in tensor functions)\n", "\n", - "Что позволяют делать нейросетевые фреймворки:\n", - "* Оперировать с тензорами, как на CPU, так и на GPU\n", - "* Автоматически вычислять производные (они вручную прописаны для всех элементарных функций)\n", - "\n", - "Опционально:\n", - "* Конструктор для нейросетей (описание сети как набора слоёв)\n", - "* Простые функции для обучения (`fit`, как в Scikit Learn)\n", - "* Набор алгоритмов оптимизации\n", - "* Набор абстракций для работы с данными" + "Optionally:\n", + "* Neural Network constructor / higher level API (describe network as a sequence of layers)\n", + "* Simple training functions (`fit`, as in Scikit Learn)\n", + "* A number of optimization algorithms in addition to gradient descent\n", + "* Data handling abstractions (that will ideally work on GPU, too)" ] }, { @@ -75,48 +33,47 @@ "id": "8cACQoFMwHl3" }, "source": [ - "## Основные фреймворки\n", + "### Most Popular Frameworks\n", "\n", - "* Tensorflow 1.0 - первый, получивший широкое распространение (Google). Позволял определять статический computation graph, и затем в явном виде выполнять вычисления\n", - "* PyTorch - Facebook\n", - "* Keras - надстройка над Tensorflow/PyTorch для унификации (Francois Chollet)\n", - "* Tensorflow 2.0 + Keras - динамический вычислительный граф, код получается похожим на обычные вычисления в numpy\n", + "* Tensorflow 1.x - first widely available framework (Google). Allowed to define static computation graph, push it to GPU, and explicitly evaluate it\n", + "* PyTorch - a framework from Facebook that is growing in popularity\n", + "* Keras - higher level API on top of Tensorflow/PyTorch to unify and simplify using neural networks (Francois Chollet)\n", + "* Tensorflow 2.x + Keras - new version of Tensorflow with integrated Keras functionality, which supports **dynamic computation graph**, allowing to perform tensor operations very similar to numpy (and PyTorch)\n", "\n", - "Мы рассмотрим Tensorflow 2.0 и Keras. Вам необходимо убедиться, что у вас установлена версия 2.x.x Tensorflow:\n", + "We will consider Tensorflow 2.x and Keras. Make sure you have version 2.x.x of Tensorflow installed:\n", "```\n", "pip install tensorflow\n", "```\n", - "или\n", + "or\n", "```\n", "conda install tensorflow\n", - "```\n", - "или выполняйте код в [Google Colab](https://colab.research.google.com/)" + "```" ] }, { "cell_type": "code", + "execution_count": 1, "metadata": { - "tags": [], "colab": { "base_uri": "https://localhost:8080/" }, "id": "xwqVx9-bwHl3", - "outputId": "2aa591b4-b647-441f-9c8e-4e0da2d517a0" + "outputId": "2aa591b4-b647-441f-9c8e-4e0da2d517a0", + "tags": [] }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "2.3.0\n" + ] + } + ], "source": [ "import tensorflow as tf\n", "import numpy as np\n", "print(tf.__version__)" - ], - "execution_count": 1, - "outputs": [ - { - "output_type": "stream", - "text": [ - "2.4.1\n" - ], - "name": "stdout" - } ] }, { @@ -125,14 +82,14 @@ "id": "6tp2xGV7wHl4" }, "source": [ - "## Основные понятия в TensorFlow\n", + "## Basic Concepts: Tensor\n", "\n", - "**Тензор** - это многомерный массив произвольной размерности. Удобно использовать при обучении нейросетей, например:\n", - "* 400x400 - чёрно-белая картинка\n", - "* 400x400x3 - цветная картинка\n", - "* 16x400x400x3 - minibatch из 16 картинок, используемый для одного шага обучения\n", - "* 25x400x400x3 - секунда видео\n", - "* 8x25x400x400x3 - minibatch из 8 1-секундных видео" + "**Tensor** is a multi-dimensional array. It is very convenient to use tensors to represent different types of data:\n", + "* 400x400 - black-and-white picture\n", + "* 400x400x3 - color picture \n", + "* 16x400x400x3 - minibatch of 16 color pictures\n", + "* 25x400x400x3 - one second of 25-fps video\n", + "* 8x25x400x400x3 - minibatch of 8 1-second videos" ] }, { @@ -141,47 +98,49 @@ "id": "qG2bsaR7wHl4" }, "source": [ - "### Простые тензоры" + "### Simple Tensors\n", + "\n", + "You can easily create simple tensors from lists of np-arrays, or generate random ones:" ] }, { "cell_type": "code", + "execution_count": 2, "metadata": { - "trusted": true, "colab": { "base_uri": "https://localhost:8080/" }, "id": "ybpnk08HwHl4", - "outputId": "fad9ed4a-df82-44a0-84ea-324bc71ea46f" + "outputId": "fad9ed4a-df82-44a0-84ea-324bc71ea46f", + "trusted": true }, - "source": [ - "a = tf.constant([[1,2],[3,4]])\n", - "print(a)\n", - "a = tf.random.normal(shape=(10,3))\n", - "print(a)" - ], - "execution_count": 2, "outputs": [ { + "name": "stdout", "output_type": "stream", "text": [ "tf.Tensor(\n", "[[1 2]\n", " [3 4]], shape=(2, 2), dtype=int32)\n", "tf.Tensor(\n", - "[[ 1.1846514 0.7508411 -1.2235912 ]\n", - " [ 0.9710432 -0.15184928 -0.10070106]\n", - " [ 0.01476252 -0.19518584 0.43259642]\n", - " [ 1.8912938 0.6803918 -0.41853315]\n", - " [-1.0234195 -0.46338254 -1.6521171 ]\n", - " [-2.6472843 0.72314113 -0.8514363 ]\n", - " [-0.2650735 -0.22901714 -0.4702833 ]\n", - " [ 0.4691955 -0.6513927 -0.22865944]\n", - " [-1.0068002 2.3035717 -0.8701199 ]\n", - " [ 0.35074237 0.192767 -1.013126 ]], shape=(10, 3), dtype=float32)\n" - ], - "name": "stdout" + "[[-0.50882405 1.1335199 -0.78524745]\n", + " [ 0.5572137 0.8736796 -0.05827063]\n", + " [-0.4501901 0.2969733 -1.660203 ]\n", + " [-0.13222459 0.23592396 -0.32563084]\n", + " [-0.9020366 -0.24209827 0.72447866]\n", + " [ 0.22193347 1.3671101 -0.13040254]\n", + " [-0.35891932 -2.0387182 0.7902264 ]\n", + " [ 2.590151 -0.40071017 -0.7124725 ]\n", + " [ 1.1319299 0.34047043 1.0180026 ]\n", + " [ 0.57838875 -1.5080605 0.8308813 ]], shape=(10, 3), dtype=float32)\n" + ] } + ], + "source": [ + "a = tf.constant([[1,2],[3,4]])\n", + "print(a)\n", + "a = tf.random.normal(shape=(10,3))\n", + "print(a)" ] }, { @@ -190,11 +149,12 @@ "id": "AXFMsV3r09Ux" }, "source": [ - "С тензорами можно производить обычные вычисления, которые производятся поэлементно (как в numpy). При этом тензоры автоматически дополняются до нужной размерности. Можно извлечь numpy-массив из тензора при помощи `.numpy()`:" + "You can use arithmetic operations on tensors, which are performed element-wise, as in numpy. Tensors are automatically expanded to required dimension, if needed. To extract numpy-array from tensor, use `.numpy()`:" ] }, { "cell_type": "code", + "execution_count": 3, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -202,30 +162,29 @@ "id": "e5Nu5Xgj1DnQ", "outputId": "0dfc8758-4ffd-4968-c7bf-6ba8d435df2e" }, - "source": [ - "print(a-a[0])\r\n", - "print(tf.exp(a)[0].numpy())" - ], - "execution_count": 3, "outputs": [ { + "name": "stdout", "output_type": "stream", "text": [ "tf.Tensor(\n", "[[ 0. 0. 0. ]\n", - " [-0.21360815 -0.90269035 1.1228901 ]\n", - " [-1.1698889 -0.9460269 1.6561877 ]\n", - " [ 0.7066424 -0.07044929 0.80505806]\n", - " [-2.2080708 -1.2142236 -0.42852592]\n", - " [-3.8319356 -0.02769995 0.3721549 ]\n", - " [-1.4497249 -0.9798582 0.75330794]\n", - " [-0.7154559 -1.4022338 0.99493176]\n", - " [-2.1914515 1.5527306 0.35347128]\n", - " [-0.83390903 -0.5580741 0.2104652 ]], shape=(10, 3), dtype=float32)\n", - "[3.2695467 2.1187813 0.29417184]\n" - ], - "name": "stdout" + " [ 1.0660378 -0.2598403 0.7269768 ]\n", + " [ 0.05863395 -0.8365466 -0.87495553]\n", + " [ 0.37659946 -0.89759594 0.4596166 ]\n", + " [-0.39321256 -1.3756182 1.509726 ]\n", + " [ 0.73075753 0.23359025 0.6548449 ]\n", + " [ 0.14990473 -3.172238 1.5754738 ]\n", + " [ 3.0989752 -1.53423 0.07277495]\n", + " [ 1.640754 -0.79304945 1.8032501 ]\n", + " [ 1.0872128 -2.6415803 1.6161287 ]], shape=(10, 3), dtype=float32)\n", + "[0.60120213 3.1065722 0.45600685]\n" + ] } + ], + "source": [ + "print(a-a[0])\n", + "print(tf.exp(a)[0].numpy())" ] }, { @@ -234,15 +193,16 @@ "id": "uQ5zN6cVyrG7" }, "source": [ - "## Переменные\r\n", - "\r\n", - "Переменные могут содержать какие-то значения, которые мы затем можем модифицировать с помощью методов `assign` и `assign_add`. \r\n", - "\r\n", - "Например, вот глупый способ посчитать сумму всех строк тензора `a`" + "## Variables\n", + "\n", + "Variables are useful to represent tensor values that can be modified using `assign` and `assign_add`. They are often used to represent neural network weights.\n", + "\n", + "As an example, here is a silly way to get a sum of all rows of tensor `a`:" ] }, { "cell_type": "code", + "execution_count": 4, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -250,22 +210,21 @@ "id": "7pu0UZ-_yqfB", "outputId": "6708c83e-02e6-4442-8757-45918eb1fbc2" }, - "source": [ - "s = tf.Variable(tf.zeros_like(a[0]))\r\n", - "for i in a:\r\n", - " s.assign_add(i)\r\n", - "\r\n", - "print(s)" - ], - "execution_count": 4, "outputs": [ { + "name": "stdout", "output_type": "stream", "text": [ - "\n" - ], - "name": "stdout" + "\n" + ] } + ], + "source": [ + "s = tf.Variable(tf.zeros_like(a[0]))\n", + "for i in a:\n", + " s.assign_add(i)\n", + "\n", + "print(s)" ] }, { @@ -274,11 +233,12 @@ "id": "rIh1EHcezlNo" }, "source": [ - "Умный способ:" + "Much better way to do it:" ] }, { "cell_type": "code", + "execution_count": 5, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -286,23 +246,20 @@ "id": "aQIdWZ1kzn6P", "outputId": "1c123d9a-ecd2-4f2e-828e-5ade85ac8f63" }, - "source": [ - "tf.reduce_sum(a,axis=0)" - ], - "execution_count": 8, "outputs": [ { - "output_type": "execute_result", "data": { "text/plain": [ - "" + "" ] }, - "metadata": { - "tags": [] - }, - "execution_count": 8 + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" } + ], + "source": [ + "tf.reduce_sum(a,axis=0)" ] }, { @@ -311,25 +268,37 @@ "id": "U-auwezDwHl6" }, "source": [ - "## Вычисляем производные\r\n", - "\r\n", - "Для обратного распространения ошибки, нам нужно уметь вычислять градиенты. Это делается с помощью `tf.GradientTape()`:\r\n", - " * Оборачиваем интересующие нас вычисления в `with tf.GradientTape`\r\n", - " * Помечаем интересующие нас тензоры вызовом `tape.watch` (переменные отслеживаются автоматически)\r\n", - " * Проводим вычисления\r\n", - " * Получаем градиенты через `tape.gradient` " + "## Computing Gradients\n", + "\n", + "For back propagation, you need to compute gradients. This is done using `tf.GradientTape()` idiom:\n", + " * Add `with tf.GradientTape` block around our computations\n", + " * Mark those tensors with respect to which we need to compute gradients by calling `tape.watch` (all variables are watched automatically)\n", + " * Compute whatever we need (build computational graph)\n", + " * Obtain gradients using `tape.gradient` " ] }, { "cell_type": "code", + "execution_count": 5, "metadata": { - "trusted": true, "colab": { "base_uri": "https://localhost:8080/" }, "id": "m8vFOXr7wHl6", - "outputId": "860ac72e-50c7-4ff2-f258-747f27194f90" + "outputId": "860ac72e-50c7-4ff2-f258-747f27194f90", + "trusted": true }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "tf.Tensor(\n", + "[[-0.9959172 -0.79481506]\n", + " [ 0.74432266 0.7713953 ]], shape=(2, 2), dtype=float32)\n" + ] + } + ], "source": [ "a = tf.random.normal(shape=(2, 2))\n", "b = tf.random.normal(shape=(2, 2))\n", @@ -340,18 +309,6 @@ " # What's the gradient of `c` with respect to `a`?\n", " dc_da = tape.gradient(c, a)\n", " print(dc_da)" - ], - "execution_count": 9, - "outputs": [ - { - "output_type": "stream", - "text": [ - "tf.Tensor(\n", - "[[0.9086961 0.954213 ]\n", - " [0.85484374 0.9623889 ]], shape=(2, 2), dtype=float32)\n" - ], - "name": "stdout" - } ] }, { @@ -360,28 +317,29 @@ "id": "8sfjBMBu59B5" }, "source": [ - "## Пример 1: Линейная регрессия\r\n", - "\r\n", - "Попробуем с помощью полученных знаний решить классическую задачу линейной регрессии. Для этого сгенерируем небольшой синтетический датасет:" + "## Example 1: Linear Regression\n", + "\n", + "Now we know enough to solve the classical problem of **Linear regression**. Let's generate small synthetic dataset:" ] }, { "cell_type": "code", + "execution_count": 6, "metadata": { - "trusted": true, - "id": "j723455WwHl7" + "id": "j723455WwHl7", + "trusted": true }, + "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "from sklearn.datasets import make_classification, make_regression\n", "from sklearn.model_selection import train_test_split\n", "import random" - ], - "execution_count": 10, - "outputs": [] + ] }, { "cell_type": "code", + "execution_count": 7, "metadata": { "colab": { "base_uri": "https://localhost:8080/", @@ -390,41 +348,37 @@ "id": "WJNK_J6v6I-Z", "outputId": "eb4a66a6-6b9a-4c8a-bc24-d81eeb2d3f27" }, - "source": [ - "np.random.seed(13) # pick the seed for reproducability - change it to explore the effects of random variations\r\n", - "\r\n", - "train_x = np.linspace(0, 3, 120)\r\n", - "train_labels = 2 * train_x + 0.9 + np.random.randn(*train_x.shape) * 0.5\r\n", - "\r\n", - "plt.scatter(train_x,train_labels)" - ], - "execution_count": 11, "outputs": [ { - "output_type": "execute_result", "data": { "text/plain": [ - "" + "" ] }, - "metadata": { - "tags": [] - }, - "execution_count": 11 + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" }, { - "output_type": "display_data", "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAb2ElEQVR4nO3df4xlZXkH8O+zwwUuP8pVmVoYXZemzZIKZZedEJo1RtZYbFHcgFEb0xbbZpM2bYVQ2tE/ivYfJjFWbW1qJpQWW6pLXdhQUJFkN6FuFDvDLgKyNChuZbBllB0QZsrODE//uPcsd86c97zvuec957zvvd9PsnF25+yZ9+yV5773eZ/3eUVVQURE4drU9ACIiCgfAzURUeAYqImIAsdATUQUOAZqIqLAnVLFTc8991zdsmVLFbcmIhpKc3NzP1HV8azvVRKot2zZgtnZ2SpuTUQ0lETkmOl7TH0QEQWOgZqIKHAM1EREgWOgJiIKHAM1EVHgKqn6ICIaJfsPz+NT9z+JZxeXcX6njZuu3Ird2ye83Z+BmoiohP2H5/Gxux7F8soaAGB+cRkfu+tRAPAWrJn6ICIq4VP3P3kySCeWV9bwqfuf9PYzGKiJiEp4dnG50J8PgoGaiKiE8zvtQn8+CAZqIiJ0c807pw/ggqn7sHP6APYfnnf6ezdduRXt1ti6P2u3xnDTlVu9jY2LiUQ08sosCCbfZ9UHEVGF8hYEXQLu7u0TXgNzGlMfRDTy6lgQLMMaqEVkq4gc6fv1oohcX8fgiIjqUMeCYBnWQK2qT6rqNlXdBmAHgCUAd1c+MiKimtSxIFhG0Rz1OwF8X1WNDa6JiGJTZEGwf7v4Oe0WRIDFpZVKFhEToqruF4vcBuBhVf18xvf2ANgDAJs3b95x7BhjORENl3R1SFq7NYZbrrl4oGAtInOqOpn1PefFRBE5FcDVAP4t6/uqOqOqk6o6OT6eeewXEVHw8uqps6pD+vneOp4okvr4DXRn0//rfRRERAGw1VO7VIFUUSlSpDzvtwB8yfsIiIgCYWuw5FIFUkWliFOgFpEzAbwLwF3eR0BEFAhbPXVWdUi/qipFnFIfqvoygDd4/+lERAE5v9PGfEawTmbJ6eqQuqo+uIWciKjnpiu3bqjqSM+Sq94unoWBmoiop44GS4NgoCYi6tPEjNmGTZmIiALHQE1EFDgGaiKiwDFHTUQjob+ZUiiLhK4YqIloKOQF4kGP2kruOb+4jDERrKliooEgz0BNRIWEODPNC8QAcOOdj2At1SnUdtRW+p7J3y9ynqIvhdqcupqcnNTZ2Vnv9yWiZmW1+SzT2tOXndMHMncUdtotvLL6am7Hu4lOO/NNx3TP/r93aGpX+cH3eGlzSkRka1rUFFOPjsXlldwgLejOkBWvzZSTtqa2Lnh1nqfIQE1EzkI9BHaQjnUCIJ1PKNIpr87zFBmoiciZKTid024Zm+3XwXTm4evOaGVePyayIUgnXDrl1X2eIgM1ETnLCl6tTYKXT6waUwhVSJ/CAgC3XHMxJjptCLr541uuuRg3v/etmQH80x+4BBOWk8d3b584eU+gG9zRd29WfRBRkLKaFi2dWMXxpZV119kqKsowVXjccs3FxsU9U5VKiJ3ysrDqg4hKuWDqvsw0ggB4evoq7z/PVI0xSBVGSKWGeVUfnFETUSm2Zvu++VzQDGXGbMMcNRGVYlrIq2qxzfQGUGcVRt0YqImolP5Ft/6FvKpmqnW/MYSAqQ8iKq3OFEKop7BUySlQi0gHwK0ALkK3Rvz3VPVbVQ6MiMgkltyyL64z6s8B+Lqqvl9ETgVwRoVjIiLKFFKVRp2sgVpEzgHwdgDXAYCqngBwotphERGtN2ir0mHgsph4AYAFAP8oIodF5FYROTN9kYjsEZFZEZldWFjwPlAialZ6N2Dd28RDbQhVB5dAfQqASwH8vapuB/AygKn0Rao6o6qTqjo5Pj7ueZhE1KRkNlvnNvH+n53XcrTphlB1cMlRPwPgGVV9qPf7ryAjUBPR8DLNZj9xz+O5p6qUzSdn9b9OG+b66YQ1UKvq/4jIj0Rkq6o+CeCdAL5X/dCIKBR5/Z4Xl7t9PtKnqvjIJ2e9QfQb9vrphGvVx58AuKNX8fEDAB+pbkhEFBrTNvG0/pyxKZ9cJFDnpTWaOLuwKU6BWlWPAMhsFkJEw++mK7daUxCJvOBaNJ9seoPwfQxW6LiFnIissraJm5ryn99pO/XjcKkiGcXt4lm4hZyInKR3A5oOuk2CaN73XGqik8XI5ZU1jIlgTXWk0h39GKiJRliZygyXnhum7+XVRO/ePrEhkK+pngz0oxakAR4cQDSyTDPiOo6ZMh02AHTTKqaFy6zc9LBsK887OIA5aqIR1eROP1MOW4Dc6pL0YmSTG3HqxEBNNKJ8npRSVNYioQDGWXYiHeBHZVs5AzXRkDNVV5hmtQpU3ssjXUXSabesQTqr2qPJN5s6cTGRaIjlVVfk1Ub77kxnyiP3LxzmMVV71H1eY1MYqImGUBIYs4JYkhpIFuVs15UN1LZSvLxt4rbFzaw3m3ZrDFdcOI6d0weiX2BMMPVB5EHTLUDTY0kW2EyS1MDu7RM4NLULYrmuDFseOe9n2CpQsjbiXLtjAvvm5odqgZEzaqKSQmtob2tkBGxMDfhMIaTTHLb2pHnbxF3+/dIbcXZOH/DSZyQknFETldRE5UHeDN42C85alPO1VTurXM40W0/eBHxvE3dZYAzpE5ALzqiJSqq78sA2g8+bxY6JrHsTSWaYvk72znrTUmSX3i2dWMX+w/PeTxW3fToI7ROQCwZqopLqrjywbb/OWmBrbRJAgJW1brjMCk4+TvY2vTkpuiV4Se9qADi+tLJuDL6CpGmBMZmh2/79QsTUB1FJdXd4s83gsxbYzjr9lJNBOmFKz5RJC5jenCY6bZx52sZ5YRUpoqzn71+UjLH2mjNqopJ8f3S3cZnBp2eoF0zdl3kv05bsQdMCebPZG/YecRqDD3kz9BhrrzmjJvIgKXN7evoqHJraVelH6EFm8C79oYHyC6N5s1nXMVQtxh7XnFETRWaQGbwtb5vwkRYwzWZdx1C1uj8B+cBATRShootvrsGpyrRASAHS5+JlHdiPmihQTfRZbrJH9ajL60fNGTVRgJqq9Q1p1kuvcQrUIvJDAD8DsAZg1RT1iciPJmt9Y0sLjIIiM+orVPUnlY2EiE6KsdaXqsPUB1EA0vnozhktHF9a2XBdyLW+VB3XOmoF8A0RmRORPVkXiMgeEZkVkdmFhQV/IyQaclmNjF76v1W0xta3Mwq91peq4zqjfpuqzovIzwN4QESOquqD/Reo6gyAGaBb9eF5nERDIz17XjqxuiEfvfKqotNu4czTTuGiHrkFalWd7/3vcyJyN4DLADyY/7eIKC2rmsPkheUVHLn51+saGgXMmvoQkTNF5OzkawC/DuCxqgdGNIxcmvonmI+mhMuM+o0A7haR5Pp/VdWvVzoqoiHlWrXBfDT1swZqVf0BgEtqGAvR0DNt0WY+mvKwPI+i1MT2ah9MjYk+cfVbjePvf9Zz2i2IAItLK1E9N5XDQE3RqXN7te0NoegbRtEt2uln7T8hJeu5Y30Do3xsykTR2Tl9wHhq9aGpXd5+jq1BUR0NjEzP2i95bjZUilteUyYeHEDRqWt7ta2Jfh2nj7s8U3JNE6ehUz0YqCk6dZ0UYntDqOMNw+WZkmvYH2R4MVBTdOo6Ssn2hlDHG0bWs/brf+5Qjroi/xioKTq2U6Z9sb0h1PGGkX7WTruF153RynzuKsZT5kRy8oeLiUQ5fFd9ND3eovfi4mR98hYTGaiJIlfVm0Vd1TXUxaO4iCJRNOhWWVPOxclwMEdNlNJUXjarL/XH7no09+dXWZLHxclwMFBTVKoOooMES9f72sY9SNCtctZbV3UN2TFQUzSqCqL9qpihuo57kKBb5ay3ruoasmOOmqJRx8ncVcxQXcdt6qyXF3RNTZ58zXp5InkYGKgpGnXtBCwaLG1M45tfXMbO6QMnFw6vuHAc++bmCwXdok2eKE4M1BSNKoJoWhUzVNO4Ba8dxTW/uIx9c/O4dscEDh5dKBR0OesdfgzUFI2qP+YD+TPUQeuVs8YtANI7GJZX1nDw6EItNcqhbdShfNzwQlFpKsBk7dIDgNed0cLN781u+p/X8N/UulQAPD19VRWPsG5c3HEYHm54oaFh+5hfVSA3HUp7fGklc4NJVsP/dmsMn/ngNuzePmHc9VdHjXIdi7LkF8vzyIsQmvdUWb6Xt2CZVb5nK/NrskaZOw7j4xyoRWRMRA6LyL1VDojiU0d9swtTcLzxzkdKj8U2000HOVswbLJGmTsO41Mk9fFRAE8A+LmKxkKRCuWjtCk4rqmW7n+RtSDYrz/I7T88j00iWMtY/+m/rqlqjToWZckvpxm1iLwJwFUAbq12OBSjUD5K580Iy+4uTGbAnXZrw/f6g1zy6SIrSIcSDLnjMD6uM+rPAvhzAGebLhCRPQD2AMDmzZvLj4yiUUd9swvbrLfsG0cyA85bsDQtOo6JBBUMWXsdF2ugFpH3AHhOVedE5B2m61R1BsAM0C3P8zZCCl4oH6WTwHPjnY9Y0w5lf44pyJneDF5VZWCkgbmkPnYCuFpEfgjgywB2ici/VDoqikpIH6V3b5/Apz9wSWMVFVyooyoU2vDSm1H/maq+J+86bnihpoW0MYabScgFN7zQyGkqB8smSVQFbiGnxjTZb4K9Lig0nFFTcHye9efrnMHZY88X7lxHVAfOqKkRpl4XQHcx0jVIFskJJwE9ryFS/38NzC1TnfJm1Oz1QY3Iq2kusgXd5eis/Yfnse2T38D1e48YgzSQ3XbUxyGxRGUxUFMjbOVqrkHStisymXEvLq8UH2TO/YnqxEBNznx2yMvqHpfmEiRtdcumnYJpUvD+RHVioCYnvjvk9W+SMXEJkrZ2oS7BfqLTxocv39zYJhkiG1Z9kJMqOuT1984YdAu6rW457zSV9GLh5Ftez5I9ChIDNTmpskNe2U0ieZtbTI2aso7QYqMiChUD9RCoY/NG1R3yqgqS3ClIw4CBOnI+N47kCaVD3iA4U6bYcTExci51xD6E1CGPaNRwRh25Ok9XqWNm2p/GOafdgkj3pO+x3tFWRXYtEg0LBurIhXK6io1LHj2dxunfpJIcBFBVaocoZEx9RM5WRxwC1xps180p3NpNo4aBOnIx5I5NefTr9x5Zt8OxSLqGW7tplDD1MQRCr2pwacAE5G9OSQsttUNUJQZq8i6dj+6c0cLxJXNTpCSVYTtFPBFaaoeoagzU5FVWXXdrk6A1JlhZM/c+f3ZxecPmFFZ9EHUxUJNXWfnolVcVnXYLZ552ijG1kaQy6k7j8EguigEXE8krUz76heUVHJrahc9+cFswVSq+OwISVcUaqEXkdBH5jog8IiKPi8gn6xgYvcZnH+iq2fpDh1SlUteuTqKyXFIfrwDYpaoviUgLwDdF5Guq+u2Kx0aor5eHLy49QUKpUqlzVydRGdZArd3Tb1/q/bbV++X/RFzK5LMPdB352Ji61cWyq5PI6RRyERkDMAfglwD8nar+RcY1ewDsAYDNmzfvOHbsmOehjqYLpu7LfFcUAE9PX+V8n6zm/Mmp21mVFKOwyFbkBHOiquWdQu5U9aGqawC2iUgHwN0icpGqPpa6ZgbADABMTk5yxu2Jr1lf1sw8eZHS6RRbuiWrcdLi0kp0AT2m2T+NtkLleaq6KCIHAbwbwGO264dVnbNNX32gbXnX/nSKbZHN1Dgp9Px5llDy5UR5XKo+xnszaYhIG8C7ABytemChqruky1eVhMsMPAnmeYtstsZJrJog8s9lRn0egNt7eepNAO5U1XurHVa4qjjk1cbHrM9le3YSzE3plk0iTr04WDVB5JdL1cd3AWyvYSxRiLGkK0nVLK+sndyKnSwkJvrTKaagnvX3siQBfxQWJInqwJ2JBdk2dISmP1UDdINtuzWGD1++2ZhOSdItYyIb7qfoVouYJAGfu/6I/GGvj4JCOuTVZcZqStUcPLqAQ1O7jPfevX0CN+w9kvm9pKQvr+pj5/SB2lNERMOKgbqgUEq6XHcslknVmHLVE512bpAv+3OJaD0G6gGEUNLluqhZpg67zKcH7voj8oc56opV1VDJdcZa5kzFMqWBMZzlSBQLzqgrVGVDJdcZa9lUzaCfHkJJERENA6deH0VNTk7q7Oys9/vGZuf0gcxgOiaCT3/gklJBi30qiIZL6V4fNBhTemJNtfTMmjNWotHBQF2hvFO1fZSq+V7U5AYVojBxMbFCWQtq/UIqVeMGFaJwMVBXKG+HHxBWqRqPpSIKF1MfFUtSB3XvZiyaxuAGFaJwMVDXwGXhz2d+eJCyQG5QIQoXA3VN8hb+fNdbD9KKNaQeJkS0HnPUAfCZH95/eN5YaZKXxvB1QAER+ccZdQB85YeTmbmJLY0RQg8TItqIM+oA+OpxnXdMFtMYRPHijLokH4uApvzwFReOY+f0gdKVGwCYxiCKGAN1Cb4WAbOqQq64cBz75ua9VG5MdNoM0kQRG9pAXXamW+b0FNet4Xk/Y5ATUli5QTScrIFaRN4M4IsA3ojuKUwzqvq5qgdWRtmZbh2np9h+xiD3HqRRE/t7EIXPZUa9CuBGVX1YRM4GMCciD6jq9yoe28DKznTrOD3F9jMGvXeRyo0q+2UTkT/Wqg9V/bGqPtz7+mcAngAQ9H/FZcvd6jg9xfYz6jghhf09iOJQqDxPRLYA2A7goYzv7RGRWRGZXVhY8DO6AZUtd3P9+2U2idh+Rh0bUNjfgygOzouJInIWgH0ArlfVF9PfV9UZADNA94QXbyN0kM6zpismgGKz0SKLcoNuEnH5GVVvQGF/D6I4OM2oRaSFbpC+Q1XvqnZIxWT1Ud43N49rd0wMPBsdZDZb9BDbELZs8wBaojhYz0wUEQFwO4DnVfV6l5vWeWai6VzCiU4bh6Z2lb6/S1VEzOcXsuqDKAx5Zya6BOq3AfgPAI8CeLX3xx9X1a+a/k6VgTodWEwNiATA09NXFb5ff6DKCsCCbo3iRN+1Vb9ZENHwK3W4rap+E9341LiscrIkcKa55Flt5WlZVRHJz+q/lotyRFSlqJoymQJn+l3ENc9qK0+zBdrkWl9NlYiIskQVqE2BM0lFCIBOu4XTW5tww94j1kU920zYJdA+u7jMRTkiqlRUgdoUOJNc8Gc+uA2vrL6K40srTidp22bCtlPEk2tDqOAgouEVVVMmW+1x0a3jtvv1987Iyoenr2VgJqIqRBWobU2Hii7quTQx6g/AIZSyhTAGIqqXtTxvEHXWUfcb9jK5mOu1iShfXnleVDlqm1gW9YruYkywiRLRaIoq9WEzSD/mupVpLcp6baLRFHWgNuVrBw3MvvK/efcp0yubTZSIRlO0gdp303tf9yt7cktekOdRW0SjKdocte98ra/72e6TV7ud1Qmwvw6c9dpEoym4GbVr+sF3vtbX/VxObjHNil3SIqzXJho9QQVqU9pg9tjzOHh0YV3w9p2v9XU/233yFjxv2Hsk855cLCQabUEFatOM8o5v//eGrnXX7pgodYpLmq/8b5mTW7hYSERZgspR5zVd6re8soaDRxe85mt95X/L3CeWOnAiqldQOxNNOwuzuB4MEBtuEScaTaUODqhTVtqgzMEAMeJiIRGlBZX6yEobfPjyzUwHENFIC2pGDWTPKCff8nqmA4hoZAUXqLMwHUBEo8waqEXkNgDvAfCcql5U/ZCqw4U6IoqRy4z6nwB8HsAXqx3KRj4Dq+/eIEREdbEuJqrqgwCer2Es69j6XhTFXs5EFKugqj76+Q6s7OVMRLHyFqhFZI+IzIrI7MLCQun7+Q6sthPHiYhC5S1Qq+qMqk6q6uT4+Hjp+/kOrNyeTUSxCjb14TOwJouSyytrGBMBAHTaLZze2oQb9h4pdG4hEVHdrIFaRL4E4FsAtorIMyLy+9UPy1+TpP5FSQBYU0Vrk+DlE6s4vrTiZaGSiKhKQTVlqkKRRk8TnTYOTe2qeERERBvlNWUKNvXhS5HFR1aAEFGIhj5QF1l8ZAUIEYVo6AN11qJka5OgNSbr/owVIEQUqiiaMpVhOqMw68+4lZyIQjT0i4lERDGI4oQXdrYjIsoWRKBmZzsiIrMgFhPZ2Y6IyCyIQM3OdkREZkEEana2IyIyCyJQs7MdEZFZEIuJplpnLiQSEQUSqAGeNE5EZBJE6oOIiMwYqImIAsdATUQUOAZqIqLAMVATEQWuku55IrIA4NiAf/1cAD/xOJwmDcuzDMtzAHyWEA3LcwDlnuUtqjqe9Y1KAnUZIjJravUXm2F5lmF5DoDPEqJheQ6gumdh6oOIKHAM1EREgQsxUM80PQCPhuVZhuU5AD5LiIblOYCKniW4HDUREa0X4oyaiIj6MFATEQWusUAtIu8WkSdF5CkRmcr4/mkisrf3/YdEZEv9o7RzeI7rRGRBRI70fv1BE+O0EZHbROQ5EXnM8H0Rkb/pPed3ReTSusfoyuFZ3iEiL/S9Jn9Z9xhdicibReSgiHxPRB4XkY9mXBP8a+P4HFG8LiJyuoh8R0Qe6T3LJzOu8Ru/VLX2XwDGAHwfwC8COBXAIwB+JXXNHwH4Qu/rDwHY28RYPTzHdQA+3/RYHZ7l7QAuBfCY4fu/CeBrAATA5QAeanrMJZ7lHQDubXqcjs9yHoBLe1+fDeC/Mv4/Fvxr4/gcUbwuvX/ns3pftwA8BODy1DVe41dTM+rLADylqj9Q1RMAvgzgfalr3gfg9t7XXwHwThGRGsfowuU5oqCqDwJ4PueS9wH4onZ9G0BHRM6rZ3TFODxLNFT1x6r6cO/rnwF4AkC6cXvwr43jc0Sh9+/8Uu+3rd6vdFWG1/jVVKCeAPCjvt8/g40v2slrVHUVwAsA3lDL6Ny5PAcAXNv7SPoVEXlzPUPzzvVZY/FrvY+uXxORtzY9GBe9j8/b0Z3B9Yvqtcl5DiCS10VExkTkCIDnADygqsbXxEf84mJi9f4dwBZV/VUAD+C1d1lqzsPo9lW4BMDfAtjf8HisROQsAPsAXK+qLzY9nkFZniOa10VV11R1G4A3AbhMRC6q8uc1FajnAfTPLN/U+7PMa0TkFADnAPhpLaNzZ30OVf2pqr7S++2tAHbUNDbfXF6zKKjqi8lHV1X9KoCWiJzb8LCMRKSFbnC7Q1XvyrgkitfG9hyxvS4AoKqLAA4CeHfqW17jV1OB+j8B/LKIXCAip6KbbL8ndc09AH639/X7ARzQXmY+INbnSOUKr0Y3NxejewD8Tq/C4HIAL6jqj5se1CBE5BeSfKGIXIbufwehTQIAdCs6APwDgCdU9a8NlwX/2rg8Ryyvi4iMi0in93UbwLsAHE1d5jV+NXK4raquisgfA7gf3cqJ21T1cRH5KwCzqnoPui/qP4vIU+guDH2oibHmcXyOPxWRqwGsovsc1zU24Bwi8iV0V93PFZFnANyM7iIJVPULAL6KbnXBUwCWAHykmZHaOTzL+wH8oYisAlgG8KEAJwGJnQB+G8CjvZwoAHwcwGYgqtfG5TlieV3OA3C7iIyh+2Zyp6reW2X84hZyIqLAcTGRiChwDNRERIFjoCYiChwDNRFR4BioiYgCx0BNRBQ4BmoiosD9P3fFq1WlCzEwAAAAAElFTkSuQmCC\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAbVklEQVR4nO3df4ylVXkH8O+zw0XuAjJaphVGx8XULKkQdtmJoZnGyGrFij82YNTGtKltsmmTNkIo7egfRf9iEmPVxqbNxNJqSnVpFzYGVDRZDHWj2Bl2EZClUXErs7aMZQfEmbqzw9M/7n2XO++85z3nve857z3n3u8n2bC795075903PPfMc57zHFFVEBFRvLYNegBERFSOgZqIKHIM1EREkWOgJiKKHAM1EVHkzgnxphdffLHu2LEjxFsTEQ2lxcXFn6nqRNFrQQL1jh07sLCwEOKtiYiGkoicML3G1AcRUeQYqImIIsdATUQUOQZqIqLIMVATEUUuSNUHEdEoOXR0CZ+4/0mcXFnDpeNt3HrdTuzbPent/RmoiYhqOHR0CR+5+1GsrW8AAJZW1vCRux8FAG/BmqkPIqIaPnH/k2eDdGZtfQOfuP9Jb9+DgZqIqIaTK2uV/r4fDNRERDVcOt6u9Pf9YKAmIkIn1zwzdxiXzd6HmbnDOHR0yenrbr1uJ9qtsU1/126N4dbrdnobGxcTiWjk1VkQzF5n1QcRUUBlC4IuAXff7kmvgTmPqQ8iGnlNLAjWYQ3UIrJTRI71/HpeRG5qYGxERI1oYkGwDmugVtUnVXWXqu4CsAfAKoB7Qg+MiKgpTSwI1lE1R/0WAD9UVWODayKi1FRZEOzdLn5RuwURYGV1PcgiYkZU1f1ikTsAPKyqny14bT+A/QAwNTW158QJxnIiGi756pC8dmsMt99wZV/BWkQWVXW66DXnxUQRORfAuwH8a9HrqjqvqtOqOj0xUXjsFxFR9MrqqYuqQ3r53jqeqZL6+B10ZtP/430UREQRsNVTu1SBhKgUqVKe97sAvuh9BEREkbA1WHKpAglRKeIUqEVkO4DfBnC39xEQEUXCVk9dVB3SK1SliFPqQ1VXAfyK9+9ORBSRS8fbWCoI1tksOV8d0lTVB7eQExF13Xrdzi1VHflZcujt4kUYqImIupposNQPBmoioh6DmDHbsCkTEVHkGKiJiCLHQE1EFDnmqIloJPQ2U4plkdAVAzURDYWyQNzvUVvZey6trGFMBBuqmBxAkGegJqJKYpyZlgViALjlrkewkesUajtqK/+e2ddXOU/Rl0ptTl1NT0/rwsKC9/closEqavNZp7WnLzNzhwt3FI63W/jlmRdLO95NjrcLP3RM79n7dUdm99YffJeXNqdERLamRYNi6tGxsrZeGqQFnRmy4qWZctbW1NYFr8nzFBmoichZrIfA9tOxTgDk8wlVOuU1eZ4iAzUROTMFp4vaLWOz/SaYzjx8xfZW4fVjIluCdMalU17T5ykyUBORs6Lg1dom+MXpM8YUQgj5U1gA4PYbrsTkeBuCTv749huuxG3vekNhAP/k+67CpOXk8X27J8++J9AJ7uh5b1Z9EFGUipoWrZ4+g1Or65uus1VU1GGq8Lj9hiuNi3umKpUYO+UVYdUHEdVy2ex9hWkEAfDU3PXev5+pGqOfKoyYSg3Lqj44oyaiWmzN9n3zuaAZy4zZhjlqIqrFtJAXarHN9AHQZBVG0xioiaiW3kW33oW8UDPVpj8YYsDUBxHV1mQKIdZTWEJyCtQiMg7gcwCuQKdG/A9V9dsBx0VEZJRKbtkX1xn1ZwB8TVXfKyLnAtgecExERIViqtJokjVQi8jLAbwJwB8AgKqeBnA67LCIiDbrt1XpMHBZTHwdgGUA/ygiR0XkcyJyfv4iEdkvIgsisrC8vOx9oEQ0WPndgE1vE4+1IVQTXAL1OQCuBvB3qrobwC8AzOYvUtV5VZ1W1emJiQnPwySiQcpms01uE+/93mUtRwfdEKoJLjnqpwE8raoPdf/8bygI1EQ0vEyz2Y99+fHSU1Xq5pOL+l/nDXP9dMYaqFX1v0XkJyKyU1WfBPAWAN8PPzQiikVZv+eVtU6fj/ypKj7yyUUfEL2GvX4641r18WcA7uxWfPwIwIfCDYmIYmPaJp7XmzM25ZOrBOqytMYgzi4cFKdArarHABQ2CyGi4XfrdTutKYhMWXCtmk82fUD4PgYrdtxCTkRWRdvETU35Lx1vO/XjcKkiGcXt4kW4hZyInOR3A5oOus2CaNlrLjXR2WLk2voGxkSwoTpS6Y5eDNREI6xOZYZLzw3Ta2U10ft2T24J5BuqZwP9qAVpgAcHEI0s04y4iWOmTIcNAJ20imnhsig3PSzbyssODmCOmmhEDXKnnymHLUBpdUl+MXKQG3GaxEBNNKJ8npRSVdEioQDGWXYmH+BHZVs5AzXRkDNVV5hmtQoE7+WRryIZb7esQbqo2mOQHzZN4mIi0RArq64oq4323ZnOlEfuXTgsY6r2aPq8xkFhoCYaQllgLApiWWogW5SzXVc3UNtK8cq2idsWN4s+bNqtMVx7+QRm5g4nv8CYYeqDyINBtwDNjyVbYDPJUgP7dk/iyOxeiOW6Omx55LLvYatAKdqIc+OeSRxcXBqqBUbOqIlqiq2hva2REbA1NeAzhZBPc9jak5ZtE3f598tvxJmZO+ylz0hMOKMmqmkQlQdlM3jbLLhoUc7XVu2icjnTbD37EPC9TdxlgTGmn4BccEZNVFPTlQe2GXzZLHZMZNOHSDbD9HWyd9GHlqK49G719BkcOrrk/VRx208Hsf0E5IKBmqimpisPbNuvixbYWtsEEGB9oxMui4KTj5O9TR9Oik4JXta7GgBOra5vGoOvIGlaYMxm6LZ/vxgx9UFUU9Md3mwz+KIFtgvOO+dskM6Y0jN10gKmD6fJ8TbOf9nWeWGIFFHR/fcuSqZYe80ZNVFNvn90t3GZwednqJfN3lf4XqYt2f2mBcpmszcfOOY0Bh/KZugp1l5zRk3kQVbm9tTc9Tgyuzfoj9D9zOBd+kMD9RdGy2azrmMILcUe15xREyWmnxm8LW+b8ZEWMM1mXccQWtM/AfnAQE2UoKqLb67BKWRaIKYA6XPxsgnsR00UqUH0WR5kj+pRV9aPmjNqoggNqtY3plkvvcQpUIvIjwH8HMAGgDOmqE9Efgyy1je1tMAoqDKjvlZVfxZsJER0Voq1vhQOUx9EEcjno8e3t3BqdX3LdTHX+lI4rnXUCuDrIrIoIvuLLhCR/SKyICILy8vL/kZINOSKGhm98H9n0Brb3M4o9lpfCsd1Rj2jqidF5FcBfENEjqvqg70XqOo8gHmgU/XheZxEQyM/e149fWZLPnr9RcV4u4XzX3YOF/XILVCr6snuf58RkXsAvBHAg+VfRUR5RdUcJs+trePYbW9ramgUMWvqQ0TOF5ELs98DeBuAx0IPjGgYuTT1zzAfTRmXGfWvAbhHRLLr/0VVvxZ0VERDyrVqg/lo6mUN1Kr6IwBXNTAWoqFn2qLNfDSVYXkeJWkQ26t9MDUm+ti732Acf++9XtRuQQRYWV1P6r6pHgZqSk6T26ttHwhVPzCqbtHO32vvCSlF953qBxiVY1MmSs7M3GHjqdVHZvd6+z62BkVNNDAy3Wuv7L7ZUCltZU2ZeHAAJaep7dW2JvpNnD7uck/ZNYM4DZ2awUBNyWnqpBDbB0ITHxgu95Rdw/4gw4uBmpLT1FFKtg+EJj4wiu61V+99x3LUFfnHQE3JsZ0y7YvtA6GJD4z8vY63W3jF9lbhfYcYT50TyckfLiYSlfBd9THo8VZ9Ly5ONqdsMZGBmihxoT4smqquoQ4exUWUiKpBN2RNORcn48EcNVHOoPKyRX2pP3L3o6XfP2RJHhcn48FATUkJHUT7CZau72sbdz9BN+Sst6nqGrJjoKZkhAqivULMUF3H3U/QDTnrbaq6huyYo6ZkNHEyd4gZquu4TZ31yoKuqcmTr1kvTySPAwM1JaOpnYBVg6WNaXxLK2uYmTt8duHw2ssncHBxqVLQrdrkidLEQE3JCBFE80LMUE3jFrx0FNfSyhoOLi7hxj2TeOD4cqWgy1nv8GOgpmSE/jEfKJ+h9luvXDRuAZDfwbC2voEHji83UqMc20YdKscNL5SUQQWYol16APCK7S3c9q7ipv9lDf9NrUsFwFNz14e4hU3j4o7D+HDDCw0N24/5oQK56VDaU6vrhRtMihr+t1tj+NT7d2Hf7knjrr8mapSbWJQlv1ieR17E0LwnZPle2YJlUfmercxvkDXK3HGYHudALSJjInJURO4NOSBKTxP1zS5MwfGWux6pPRbbTDcf5GzBcJA1ytxxmJ4qqY8PA3gCwMsDjYUSFcuP0qbguKFau/9F0YJgr94gd+joEraJYKNg/af3ukFVazSxKEt+Oc2oReTVAK4H8Lmww6EUxfKjdNmMsO7uwmwGPN5ubXmtN8hlP10UBelYgiF3HKbHdUb9aQB/AeBC0wUish/AfgCYmpqqPTBKRxP1zS5ss966HxzZDLhswdK06DgmElUwZO11WqyBWkTeCeAZVV0UkTebrlPVeQDzQKc8z9cAKX6x/CidBZ5b7nrEmnao+31MQc70YfCiKgMj9c0l9TED4N0i8mMAXwKwV0T+OeioKCkx/Si9b/ckPvm+qwZWUcGFOgqh0oaX7oz6z1X1nWXXccMLDVpMG2O4mYRccMMLjZxB5WDZJIlC4BZyGphB9ptgrwuKDWfUFB2fZ/35Omdw4cSzlTvXETWBM2oaCFOvC6CzGOkaJKvkhLOAXtYQqff/BuaWqUllM2r2+qCBKKtprrIF3eXorENHl7Dr41/HTQeOGYM0UNx21MchsUR1MVDTQNjK1VyDpG1XZDbjXllbrz7IkvcnahIDNTnz2SGvqHtcnkuQtNUtm3YK5knF9ydqEgM1OfHdIa93k4yJS5C0tQt1CfaT42188JqpgW2SIbJh1Qc5CdEhr7d3Rr9b0G11y2WnqeQXC6df+0qW7FGUGKjJScgOeXU3iZRtbjE1aio6QouNiihWDNRDoInNG6E75IUKktwpSMOAgTpxPjeOlImlQ14/OFOm1HExMXEudcQ+xNQhj2jUcEaduCZPV2liZtqbxrmo3YJI56Tvse7RVlV2LRINCwbqxMVyuoqNSx49n8bp3aSSHQQQKrVDFDOmPhJnqyOOgWsNtuvmFG7tplHDQJ24FHLHpjz6TQeObdrhWCVdw63dNEqY+hgCsVc1uDRgAso3p+TFltohComBmrzL56PHt7dwatXcFClLZdhOEc/EltohCo2BmrwqqutubRO0xgTrG+be5ydX1rZsTmHVB1EHAzV5VZSPXn9RMd5u4fyXnWNMbWSpjKbTODySi1LAxUTyypSPfm5tHUdm9+LT798VTZWK746ARKFYA7WInCci3xWRR0TkcRH5eBMDo5f47AMdmq0/dExVKk3t6iSqyyX18UsAe1X1BRFpAfiWiHxVVb8TeGyE5np5+OLSEySWKpUmd3US1WEN1No5/faF7h9b3V/+T8SlQj77QDeRj02pW10quzqJnE4hF5ExAIsAfh3A36rqXxZcsx/AfgCYmprac+LECc9DHU2Xzd5X+KkoAJ6au975fYqa82enbhdVUozCIluVE8yJQis7hdyp6kNVNwDsEpFxAPeIyBWq+ljumnkA8wAwPT3NGbcnvmZ9RTPz7CHl0ym2dEtR46SV1fXkAnpKs38abZXK81R1RUS+CeDtAB6zXD60mpxt+uoDbcu79qZTbItspsZJsefPi8SSLycq41L1MdGdSUNE2gDeCuB44HFFq+mSLl9VEi4z8CyYly2y2RonsWqCyD+XGfUlAD7fzVNvA3CXqt4bdljxCnHIq42PWZ/L9uwsmJvSLdtEnHpxsGqCyC+Xqo/vAdjdwFiSkGJJV5aqWVvfOLsVO1tIzPSmU0xBvejrimQBfxQWJImawJ2JFdk2dMSmN1UDdIJtuzWGD14zZUynZOmWMZEt76foVIuYZAGfu/6I/GGvj4piOuTVZcZqStU8cHwZR2b3Gt973+5J3HzgWOFrWUlfWdXHzNzhxlNERMOKgbqiWEq6XHcs1knVmHLVk+Pt0iBf9/sS0WYM1H2IoaTLdVGzTh12nZ8euOuPyB/mqAML1VDJdcZa50zFOqWBKZzlSJQKzqgDCtlQyXXGWjdV0+9PD7GkiIiGgVOvj6qmp6d1YWHB+/umZmbucGEwHRPBJ993Va2gxT4VRMOldq8P6o8pPbGhWntmzRkr0ehgoA6o7FRtH6Vqvhc1uUGFKE5cTAyoaEGtV0ylatygQhQvBuqAynb4AXGVqvFYKqJ4MfURWJY6aHo3Y9U0BjeoEMWLgboBLgt/PvPD/ZQFcoMKUbwYqBtStvDnu966n1asMfUwIaLNmKOOgM/88KGjS8ZKk7I0hq8DCojIP86oI+ArP5zNzE1saYwYepgQ0VacUUfAV4/rsmOymMYgShdn1DX5WAQ05YevvXwCM3OHa1duAGAagyhhDNQ1+FoELKoKufbyCRxcXPJSuTE53maQJkrY0AbqujPdOqenuG4NL/se/ZyQwsoNouFkDdQi8hoAXwDwKgAvAphX1c+EHlgddWe6TZyeYvse/bx3P42a2N+DKH4uM+ozAG5R1YdF5EIAiyLyDVX9fuCx9a3uTLeJ01Ns36Pf965SuRGyXzYR+WOt+lDVn6rqw93f/xzAEwCi/r+4brlbE6en2L5HEyeksL8HURoqleeJyA4AuwE8VPDafhFZEJGF5eVlT8PrT91yN9evr7NJxPY9mtiAwv4eRGlwXkwUkQsAHARwk6o+n39dVecBzAOdE168jdBBPs+ar5gAqs1GqyzK9btJxOV7hN6Awv4eRGlwmlGLSAudIH2nqt4ddkjVFPVRPri4hBv3TPY9G+1nNlv1ENsYtmzzAFqiNFjPTBQRAfB5AM+q6k0ub9rkmYmmcwknx9s4Mru39vu7VEWkfH4hqz6I4lB2ZqJLoP4tAP8O4FF0yvMA4KOq+hXT14QM1PnAYmpAJACemru+8vv1BqqiACwAFJ0Pguza0B8WRDT8ah1uq6rfQic+DVxROVkWOPNc8qy28rSiqojse/Vey0U5IgopqaZMpsCZ/xRxzbPaytNsgTa71ldTJSKiIkkFalPgzFIRAmC83cJ5rW24+cAx66KebSbsEmhPrqxxUY6IgkoqUJsCZ5YL/tT7d+GXZ17EqdV1p5O0bTNh2yni2bUxVHAQ0fBKqimTrfa46tZx2/v19s4oyofnr2VgJqIQkgrUtqZDVRf1XJoY9QbgGErZYhgDETXLWp7XjybrqHsNe5lcyvXaRFSurDwvqRy1TSqLelV3MWbYRIloNCWV+rDppx9z0+q0FmW9NtFoSjpQm/K1/QZmX/nfsvep0yubTZSIRlOygdp303tf71f35JayIM+jtohGU7I5at/5Wl/vZ3ufstrtok6AvXXgrNcmGk3Rzahd0w++87W+3s/l5BbTrNglLcJ6baLRE1WgNqUNFk48iweOL28K3r7ztb7ez/Y+ZQueNx84VvieXCwkGm1RBWrTjPLO7/zXlq51N+6ZrHWKS56v/G+dk1u4WEhERaLKUZc1Xeq1tr6BB44ve83X+sr/1nmfVOrAiahZUe1MNO0sLOJ6MEBquEWcaDTVOjigSUVpgzoHA6SIi4VElBdV6qMobfDBa6aYDiCikRbVjBoonlFOv/aVTAcQ0ciKLlAXYTqAiEaZNVCLyB0A3gngGVW9IvyQwuFCHRGlyGVG/U8APgvgC2GHspXPwOq7NwgRUVOsi4mq+iCAZxsYyya2vhdVsZczEaUqqqqPXr4DK3s5E1GqvAVqEdkvIgsisrC8vFz7/XwHVtuJ40REsfIWqFV1XlWnVXV6YmKi9vv5Dqzcnk1EqYo29eEzsGaLkmvrGxgTAQCMt1s4r7UNNx84VuncQiKiplkDtYh8EcC3AewUkadF5I/CD8tfk6TeRUkA2FBFa5vgF6fP4NTqupeFSiKikKJqyhRClUZPk+NtHJndG3hERERblTVlijb14UuVxUdWgBBRjIY+UFdZfGQFCBHFaOgDddGiZGuboDUmm/6OFSBEFKskmjLVYTqjsOjvuJWciGI09IuJREQpSOKEF3a2IyIqFkWgZmc7IiKzKBYT2dmOiMgsikDNznZERGZRBGp2tiMiMosiULOzHRGRWRSLiaZaZy4kEhFFEqgBnjRORGQSReqDiIjMGKiJiCLHQE1EFDkGaiKiyDFQExFFLkj3PBFZBnCizy+/GMDPPA5nkIblXoblPgDeS4yG5T6AevfyWlWdKHohSKCuQ0QWTK3+UjMs9zIs9wHwXmI0LPcBhLsXpj6IiCLHQE1EFLkYA/X8oAfg0bDcy7DcB8B7idGw3AcQ6F6iy1ETEdFmMc6oiYioBwM1EVHkBhKoReTtIvKkiPxARGYLXhcR+Zvu698TkasHMU4XDvfyZhF5TkSOdX/91SDGaSMid4jIMyLymOH1lJ6J7V5SeSavEZEHROQJEXlcRD5ccE0Sz8XxXlJ5LueJyHdF5JHuvXy84Bq/z0VVG/0FYAzADwG8DsC5AB4B8Bu5a94B4KsABMA1AB5qepwe7+XNAO4d9Fgd7uVNAK4G8Jjh9SSeieO9pPJMLgFwdff3FwL4z4T/X3G5l1SeiwC4oPv7FoCHAFwT8rkMYkb9RgA/UNUfqeppAF8C8J7cNe8B8AXt+A6AcRG5pOmBOnC5lySo6oMAni25JJVn4nIvSVDVn6rqw93f/xzAEwDyTduTeC6O95KE7r/1C90/trq/8lUZXp/LIAL1JICf9Pz5aWx9YC7XxMB1nL/Z/THpqyLyhmaG5l0qz8RVUs9ERHYA2I3O7K1Xcs+l5F6ARJ6LiIyJyDEAzwD4hqoGfS6DOOFFCv4u/2nkck0MXMb5MDp7+F8QkXcAOATg9aEHFkAqz8RFUs9ERC4AcBDATar6fP7lgi+J9rlY7iWZ56KqGwB2icg4gHtE5ApV7V0T8fpcBjGjfhrAa3r+/GoAJ/u4JgbWcarq89mPSar6FQAtEbm4uSF6k8ozsUrpmYhIC53Adqeq3l1wSTLPxXYvKT2XjKquAPgmgLfnXvL6XAYRqP8DwOtF5DIRORfABwB8OXfNlwH8fnfl9BoAz6nqT5seqAPrvYjIq0REur9/Izr/5v/b+EjrS+WZWKXyTLpj/AcAT6jqXxsuS+K5uNxLQs9lojuThoi0AbwVwPHcZV6fS+OpD1U9IyJ/CuB+dKom7lDVx0Xkj7uv/z2Ar6CzavoDAKsAPtT0OF043st7AfyJiJwBsAbgA9pdFo6JiHwRnVX3i0XkaQC3obNIktQzAZzuJYlnAmAGwO8BeLSbDwWAjwKYApJ7Li73kspzuQTA50VkDJ0Pk7tU9d6QMYxbyImIIsediUREkWOgJiKKHAM1EVHkGKiJiCLHQE1EFDkGaiKiyDFQExFF7v8B4SC4LI9GLoEAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": { - "tags": [], "needs_background": "light" - } + }, + "output_type": "display_data" } + ], + "source": [ + "np.random.seed(13) # pick the seed for reproducability - change it to explore the effects of random variations\n", + "\n", + "train_x = np.linspace(0, 3, 120)\n", + "train_labels = 2 * train_x + 0.9 + np.random.randn(*train_x.shape) * 0.5\n", + "\n", + "plt.scatter(train_x,train_labels)" ] }, { @@ -433,37 +387,37 @@ "id": "Ng4rZmGc6oxk" }, "source": [ - "Линейная регрессия вычисляется как $f_{W,b}(x) = Wx+b$, где $W, b$ - параметры модели, которые необходимо найти. Функция ошибки на наборе данных $\\{x_i,y_u\\}_{i=1}^N$ может быть определена как среднеевадратичное отклонение\r\n", - "$$\r\n", - "\\mathcal{L}(W,b) = {1\\over N}\\sum_{i=1}^N (f_{W,b}(x_i)-y_i)^2\r\n", - "$$\r\n", - "\r\n", - "Опишем модель и функцию ошибки:" + "Linear regression is defined by a straight line $f_{W,b}(x) = Wx+b$, where $W, b$ are model parameters that we need to find. An error on our dataset $\\{x_i,y_u\\}_{i=1}^N$ (also called **loss function**) can be defined as mean square error:\n", + "$$\n", + "\\mathcal{L}(W,b) = {1\\over N}\\sum_{i=1}^N (f_{W,b}(x_i)-y_i)^2\n", + "$$\n", + "\n", + "Let's define our model and loss function:" ] }, { "cell_type": "code", + "execution_count": 8, "metadata": { "id": "QxhI4GlB6aiH" }, + "outputs": [], "source": [ - "input_dim = 1\r\n", - "output_dim = 1\r\n", - "learning_rate = 0.1\r\n", - "\r\n", - "# This is our weight matrix\r\n", - "w = tf.Variable([[100.0]])\r\n", - "# This is our bias vector\r\n", - "b = tf.Variable(tf.zeros(shape=(output_dim,)))\r\n", - "\r\n", - "def f(x):\r\n", - " return tf.matmul(x,w) + b\r\n", - "\r\n", - "def compute_loss(labels, predictions):\r\n", + "input_dim = 1\n", + "output_dim = 1\n", + "learning_rate = 0.1\n", + "\n", + "# This is our weight matrix\n", + "w = tf.Variable([[100.0]])\n", + "# This is our bias vector\n", + "b = tf.Variable(tf.zeros(shape=(output_dim,)))\n", + "\n", + "def f(x):\n", + " return tf.matmul(x,w) + b\n", + "\n", + "def compute_loss(labels, predictions):\n", " return tf.reduce_mean(tf.square(labels - predictions))" - ], - "execution_count": 14, - "outputs": [] + ] }, { "cell_type": "markdown", @@ -471,33 +425,33 @@ "id": "JUxwj3367gD2" }, "source": [ - "Обучать модель будем на сериях примеров - minibatches. Для обучения используем градиентный спуск, подстраивая парметры в соответствии с формулой:\r\n", - "$$\r\n", - "\\begin{array}{l}\r\n", - "W^{(n+1)}=W^{(n)}-\\eta\\frac{\\partial\\mathcal{L}}{\\partial W} \\\\\r\n", - "b^{(n+1)}=b^{(n)}-\\eta\\frac{\\partial\\mathcal{L}}{\\partial b} \\\\\r\n", - "\\end{array}\r\n", + "We will train the model on a series of minibatches. We will use gradient descent, adjusting model parameters using the following formulae:\n", + "$$\n", + "\\begin{array}{l}\n", + "W^{(n+1)}=W^{(n)}-\\eta\\frac{\\partial\\mathcal{L}}{\\partial W} \\\\\n", + "b^{(n+1)}=b^{(n)}-\\eta\\frac{\\partial\\mathcal{L}}{\\partial b} \\\\\n", + "\\end{array}\n", "$$" ] }, { "cell_type": "code", + "execution_count": 11, "metadata": { "id": "-991PErM7fJU" }, + "outputs": [], "source": [ - "def train_on_batch(x, y):\r\n", - " with tf.GradientTape() as tape:\r\n", - " predictions = f(x)\r\n", - " loss = compute_loss(y, predictions)\r\n", - " # Note that `tape.gradient` works with a list as well (w, b).\r\n", - " dloss_dw, dloss_db = tape.gradient(loss, [w, b])\r\n", - " w.assign_sub(learning_rate * dloss_dw)\r\n", - " b.assign_sub(learning_rate * dloss_db)\r\n", + "def train_on_batch(x, y):\n", + " with tf.GradientTape() as tape:\n", + " predictions = f(x)\n", + " loss = compute_loss(y, predictions)\n", + " # Note that `tape.gradient` works with a list as well (w, b).\n", + " dloss_dw, dloss_db = tape.gradient(loss, [w, b])\n", + " w.assign_sub(learning_rate * dloss_dw)\n", + " b.assign_sub(learning_rate * dloss_db)\n", " return loss" - ], - "execution_count": 13, - "outputs": [] + ] }, { "cell_type": "markdown", @@ -505,25 +459,26 @@ "id": "idr2VEWb9rr0" }, "source": [ - "Теперь приступаем к обучению: делаем несколько проходов по всему датасету (эпох), разбиваем его на minibatches, и вызываем функцию обучения:" + "Let's do the training. We will do several passes through the dataset (so-called **epochs**), divide it into minibatches and call the function defined above:" ] }, { "cell_type": "code", + "execution_count": 12, "metadata": { "id": "nOuu0qpx-wAp" }, + "outputs": [], "source": [ - "# Shuffle the data.\r\n", - "indices = np.random.permutation(len(train_x))\r\n", - "features = tf.constant(train_x[indices],dtype=tf.float32)\r\n", + "# Shuffle the data.\n", + "indices = np.random.permutation(len(train_x))\n", + "features = tf.constant(train_x[indices],dtype=tf.float32)\n", "labels = tf.constant(train_labels[indices],dtype=tf.float32)" - ], - "execution_count": 12, - "outputs": [] + ] }, { "cell_type": "code", + "execution_count": 13, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -531,35 +486,42 @@ "id": "3zdIf6c_85Ht", "outputId": "43b04684-8b90-4c65-d5ff-20ebac61c73c" }, - "source": [ - "batch_size = 4\r\n", - "for epoch in range(10):\r\n", - " for i in range(0,len(features),batch_size):\r\n", - " loss = train_on_batch(tf.reshape(features[i:i+batch_size],(-1,1)),tf.reshape(labels[i:i+batch_size],(-1,1)))\r\n", - " print('Epoch %d: last batch loss = %.4f' % (epoch, float(loss)))" - ], - "execution_count": 15, "outputs": [ { + "name": "stdout", "output_type": "stream", "text": [ - "Epoch 0: last batch loss = 94.5247\n", - "Epoch 1: last batch loss = 9.3428\n", - "Epoch 2: last batch loss = 1.4166\n", - "Epoch 3: last batch loss = 0.5224\n", - "Epoch 4: last batch loss = 0.3807\n", - "Epoch 5: last batch loss = 0.3495\n", - "Epoch 6: last batch loss = 0.3413\n", - "Epoch 7: last batch loss = 0.3390\n", - "Epoch 8: last batch loss = 0.3384\n", - "Epoch 9: last batch loss = 0.3382\n" - ], - "name": "stdout" + "Epoch 0: last batch loss = 63.5879\n", + "Epoch 1: last batch loss = 6.0149\n", + "Epoch 2: last batch loss = 0.9082\n", + "Epoch 3: last batch loss = 0.3712\n", + "Epoch 4: last batch loss = 0.2928\n", + "Epoch 5: last batch loss = 0.2765\n", + "Epoch 6: last batch loss = 0.2724\n", + "Epoch 7: last batch loss = 0.2712\n", + "Epoch 8: last batch loss = 0.2709\n", + "Epoch 9: last batch loss = 0.2708\n" + ] } + ], + "source": [ + "batch_size = 4\n", + "for epoch in range(10):\n", + " for i in range(0,len(features),batch_size):\n", + " loss = train_on_batch(tf.reshape(features[i:i+batch_size],(-1,1)),tf.reshape(labels[i:i+batch_size],(-1,1)))\n", + " print('Epoch %d: last batch loss = %.4f' % (epoch, float(loss)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We now have obtained optimized parameters $W$ and $b$. Note that their values are similar to the original values used when generating the dataset ($W=2, b=1$)" ] }, { "cell_type": "code", + "execution_count": 14, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -567,28 +529,26 @@ "id": "US6q0nCBD-LL", "outputId": "65a79620-a3eb-445b-aafb-60a60575ab0e" }, - "source": [ - "w,b" - ], - "execution_count": 16, "outputs": [ { - "output_type": "execute_result", "data": { "text/plain": [ - "(,\n", - " )" + "(,\n", + " )" ] }, - "metadata": { - "tags": [] - }, - "execution_count": 16 + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" } + ], + "source": [ + "w,b" ] }, { "cell_type": "code", + "execution_count": 15, "metadata": { "colab": { "base_uri": "https://localhost:8080/", @@ -597,39 +557,35 @@ "id": "_e6xRMZFDnyI", "outputId": "d202b7fe-4383-4d82-b98e-a20f3180093e" }, - "source": [ - "plt.scatter(train_x,train_labels)\r\n", - "x = np.array([min(train_x),max(train_x)])\r\n", - "y = w.numpy()[0,0]*x+b.numpy()[0]\r\n", - "plt.plot(x,y)" - ], - "execution_count": 17, "outputs": [ { - "output_type": "execute_result", "data": { "text/plain": [ - "[]" + "[]" ] }, - "metadata": { - "tags": [] - }, - "execution_count": 17 + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" }, { - "output_type": "display_data", "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZhU1Z3/8ffp7uqFrRulQWhoQFZFZOsxGowKiBhN3KNJRo3JTJiJGbcYlGQWZ/L8ZgTcMyYTiZplYhKNInFiZFFQXFFWUeluEGRp9qWbpffq8/uju9qmum7Vra5b1beaz+t5eNJ03br3XMp876lzvud7jLUWERHxr4zOboCIiESnQC0i4nMK1CIiPqdALSLicwrUIiI+l5WMk/bp08cOGTIkGacWEemSVq9efcBaWxjptaQE6iFDhrBq1apknFpEpEsyxmxzek1DHyIiPqdALSLicwrUIiI+p0AtIuJzCtQiIj6XlKwPEZGTycK1FTywuIxdlTUMKMhj1oxRXDWhyLPzK1CLiCRg4doKfrRgAzUNQQAqKmv40YINAJ4Faw19iIgk4IHFZa1BOqSmIcgDi8s8u4YCtYhIAnZV1sT1+45QoBYRScCAgry4ft8RCtQiIjSPNU+es4yhs19m8pxlLFxb4ep9s2aMIi+QecLv8gKZzJoxyrO2aTJRRE56iUwIhl5X1oeISBJFmxB0E3CvmlDkaWAOp6EPETnppWJCMBExA7UxZpQxZl2bP0eMMXemonEiIqmQignBRMQM1NbaMmvteGvteGASUA28mPSWiYikSComBBMR7xj1NOBTa61jgWsRkXQTz4Rg2+Xi+XkBjIHK6oakTCKGGGut+4ONeRpYY619PMJrM4GZAMXFxZO2bVMsF5GuJTw7JFxeIJP7rxnboWBtjFltrS2J9JrryURjTDZwBfCnSK9ba+dba0ustSWFhRG3/RIR8b1o+dSRskPa8nrpeEg8Qx9fprk3vdfzVoiI+ECsfGo3WSDJyBSJJz3vG8AfPG+BiIhPxCqw5CYLJBmZIq4CtTGmOzAdWOB5C0REfCJWPnWk7JC2kpUp4mrow1p7HDjV86uLiPjIgII8KiIE61AvOTw7JFVZH1pCLiLSYtaMUe2yOsJ7ycleLh6JArWISItUFFjqCAVqEZE2OqPHHIuKMomIeOCjiiqeWZmchX7qUYuIJOCzA8d5cEkZf/lwN3175nDNhIHkZTtnhnSEArWISAfsO1rLf7+2mT+8v51AZga3TR3Ody843fMgDQrUInKSaFtMKZFJwqO1DcxfsYWn3tpKfWMTXz9nELdPG0HfnrlJaHUzBWoR6RKiBeKObrUVOmdFZQ0ZQBOQYaDJwlfO7s8PLxnFkD7dk31rCtQiEh+veqZet8kpEAPc/dx6gmGVQmNttRV+zqaW3zdZyMnK4OIz+qUkSEOcZU7dKikpsatWrfL8vCLSuSKV+UyktKdXJs9ZFnFFYUFegLrGpqgV74oK8iI+dL54/2vsqqqN+r63Z09NvPEtopU5VY9aRFxLdBPYZHGq0VFZ0xD1fQZaA3zbXnjxqd2iBulo10wGBWoRcc2vm8A61eiIxgDh4wk1DUFmL/iQ2oam1rHoaNdMFS14ERHXnIJTfl7Asdh+Kjjtedi7WyDi8ZnGtAvSIbUNTdw9fST3Xz3WsVJeqvdTVKAWEdciBcRAhuF4fSMVlTVYPh9CSGawDt+FBeD+a8ZSVJCHoXn8+P5rxnLfV8dEDOAPXT+OIoeHzmm9crlt2ghuOKe49ZzQHNxpc+5UDvVo6ENEXItUtKi6vpHD1SeOBSdz3Nopw+P+a8Y6Tu6FZ6nMGHMar27c2264JC+Qyewvj279u1/qfijrQ0QSMnT2yxGHEQywdc7lnl/PKcPDTRZGY7CJP63eyaOvlrP3SB1jBvRi/9E69h+t6/RUQ2V9iEjSxCq277WOTGhaa1n88R7mLS5jy/7jTCwu4L+/MZFzhp6SlDZ6TYFaRBLipti+l+J9MLz76UHmLipl3Y5KRvTtwfybJjH9zH6YljHndKBALSIJSXWxfbcPhk92HWHuolLeKN9P//xc5l13NtdOHEhmRvoE6BAFahFJWCon3WI9GHYcquahJWX8ef0ueuUG+PFlo7n5vCHkRtmU1u9cBWpjTAHwJHAWzTni37HWvpvMhomIOIn0YDhwrI7Hl23mmZXbyMww/OOFw/jHC4eRnxc5lzqduO1RPwYsstZeZ4zJBrolsU0iIhFFKgh18Zn9ePLNLfxyxRZqG5u4vmQQd148gn69kld2NNViBmpjTD5wAXALgLW2HqhPbrNERE4UKX961p/WkxPI5FhdI18+6zR+OGMUwwp7dHJLveemRz0U2A/8yhgzDlgN3GGtPd72IGPMTGAmQHFxsdftFJFO1tnlTSMVhGposphgEwu/P5nxgwpS1pZUc7OEPAuYCPyPtXYCcByYHX6QtXa+tbbEWltSWFjocTNFpDOFerOpXCbe9tpOi1wA6hubunSQBnc96p3ATmvtypa/P0+EQC0iXZdTedN/f+njqLuqJNoDj1T/OpxTzY6uJGagttbuMcbsMMaMstaWAdOAT5LfNBHxi2j1nkM1n8N3VenI1lfh/uuvG6MG6VRXsessbrM+bgOeacn42AJ8O3lNEhG/cVvvOVSMKfRzpNfcBOq9R2p59NVN7Dta53hMkU+2AUsFV4HaWrsOiFgsRES6vkirAZ1Eq7kRa4OBqpoGnnjjU55+eyvBJkv37EyO17e/ptfbYPmdViaKSExuy5vC5zU3YtXjaDuG3T8/l5Ihp/BG+X6O1DZw5bgB/GD6KNZsP5zSOiJ+pUAtIq6ErwZ02ug2FESjvRb+3l1Vtby0fhejT+vJ77/7BcYMyG8N5DUNQTKNIWjtSTXc0ZYCtchJLJHMDDfFmJxem7eoNOIwytHaxtYg3TaQB61tDfQnW5AGbRwgctJy6hEne5up97ce4vonnEsFFUWZuIw0Nt3ZC3G8oo0DRKQdp9zoZG2hVbrnCA8sKuO10n2OO3wbIo9th4RPRjptywXxpQH6nQK1yEmqIzuldMTOw9U8vLScF9dW0CMni3svHc2p3bO576WPT3hQGHDcGTwkfHOAVD9sOosCtUgX5zQ04JQbbWnelzDRIYRDx+v52fLN/O+728DAzC+dzvcuGkZBt2wAsrMyWtuVnxdoXTjjJFK2R6oeNp1NgVqkC4s2NBAtNzqRIYTq+kaeenMr81ds4Xh9I9dNGsgZ/XvxZMvv2j4srppQ1NrGaJyyPVK9X2NnUaAW6YJCvehIQSw0NBCalIt1nNtA3RBs4o8f7OCnr21i/9E6LjmzH7NmjOLjXUeijiNHGr4IiTW56bQt15TRhUyesyztJxhDFKhFPOCnzAM3hYxCQwOhXu3Q2S9HHB92M4TQ1GR5ecNuHlpSxmcHqzlnyCn84sZJTBrcG4BbfvVB1HHkaNeIlYESKUVwyuhCXlhd0aUmGBWoRRLkt8yDaD3UkPChgY4OIby16QBzF5WyoaKK0af15OlbSqiqbuD2P6xtDZxOWRyhAO10TFFBnqt/v/CFOJPnLOtyE4xu6lGLSBTRMg+SJVSjeejsl5k8Z9kJdaFj9YIjTcrNmjGKvLDNX6Mt1d6ws4obn1zJjU+t5NDxeh6+fhwv3/4ljtQ08uMXPzqhbrXTnt+hh0C8147FzQRjtH8/P1KPWiRBqc48iNWDj9aLzTTmhIdIqIfpZpUhwGcHjvPgkjL+8uFuencL8K9fOZMbzy0mJyuz9f3hDy1L5NS76vpGFq6tcH1tt2J9O/DbNyA3FKhFEpTqzINYucORJtgCGQYMNASbw2Wk4BRpZ++QfUdr+elrm/jj+zsIZGZw29ThfPeC0+mVe+IO304PJwsUhKXgHa5uOKENXgVJpwnGUA89HXOvFahFEhQrMHgtVg/ebaU7p+DUdmL0tPxcxhbl8+amAzQEm/jGOcXcNm04fXtG3uE72ngz0C5XOhkBMlYPPR1zrxWoRRLk9Vf3WNz04MN7qENnvxzxXLGWZO+uqmV3VS0TBhXwyA3jGdKne9S2RXto3fXsOldt8EK0Hno65l5rMlHEA1dNKOLt2VPZOudy3p49NalfoTsy+eYUhMJ/71TVbt/RuphBGpr/He6/ZixFBXkYmnvSoRQ7t21INq8nL1NBPWqRNNORHnys4RlrLcvL9rGrqjbi++Pp9Tr1ZlM9ROQk1d+AvKBALZKG4p18ixacVm87zNxXSnn/s0NkZhiCEcraedHr9VOA9HLyMhVUj1rEp5K92nHT3qPMW1zG0k/2UtgzhzumjSAvkMm/LPwo5TWqRfWoRdJOMnN9d1XW8Oir5Ty/eifds7P44SUj+c75Q+mW3RwOMjOML3q98jlXgdoY8xlwFAgCjU5RX0S8kYxc38rqen7++qf8+p3PwMJ3Jg/l1inDOaV79gnHpduwwMkgnh71FGvtgaS1RERaeZnrW1Mf5FfvbOV/Xv+UY3WNXDNhIHdNH8HA3t0SbaakiIY+RHwgfDy6oFug3QIViG9SrzHYxHOrdvLYa+XsPVLHtNF9mXXpKEaf1svLpksKuA3UFlhijLHAE9ba+eEHGGNmAjMBiouLvWuhSBcXaTw6kGEIZJrWJd/gPpXNWsuij/bwwJIytuw/zqTBvXn8mxP5myGnJO0eJLncBurzrbUVxpi+wFJjTKm1dkXbA1qC93xozvrwuJ0iXUZ477m6vrHdeHRDk6UgL0D3nKy4JvXe+fQAcxeVsX5HJSP69uCXN5dw8Rl9Mcaphp2kA1eB2lpb0fK/+4wxLwLnACuiv0tEwkXqPTupqmlg3X2XuDrvx7uqmLeojDfK9zMgP5d5153NtRMHkpmhAN0VxAzUxpjuQIa19mjLz5cAP0l6y0S6IDdF/UPcjEdvP1jNQ0vL+PO6XeTnBfjny87gpvMGkxu2RFrSm5sedT/gxZavTlnA7621i5LaKpEuym3WRqzx6APH6nh82WaeWbmNzAzDrRcN4x8uHEZ+XsDxPZK+YgZqa+0WYFwK2iLS5TlVbnM7Hn2srpFfrtjCk29uobaxietLBnHnxSPo1yty2VHpGpSeJ2nJT5vJxsOpMNG/XzHGsf0L11Ywb1Epu6pqT9gppW/PHL4w9BQF6ZOAArWknVRupRTrgRDvAyPewkQL1uzk3hc+bE3Ta5tOte9oXbv7TtcHmESnokySdibPWea4i8jbs6d6dp3wBwKcWKAo1uuJsNbyRvl+vvvbVSfkUkcSuu9ktkeSL1pRJm0cIGknVVspxdpdPFm7j6/bUck3fvket/zqg5hBGj6/787YDV1SQ4Fa0k6qdgqJ9UDw+oHx6f5jfO93q7nqZ2+zed8x/uOKMQzIjz3+HLrvdNwLUNxRoJa0k6qtlGI9ELx6YOypquVHCz7kkkdWsKJ8P3ddPJLXZ03hW18cwj2Xjm53r221vW+/bHUl3lOglrQTbV8+L8V6ICT6wKiqaWDuolIuenA5z6/eyU3nDuaNe6Zwx8Uj6JHTPM8ffq8FeQF6dwtEvO9kPMAWrq1g8pxlDJ39MpPnLGPh2ooOn0s6TpOJIlF4nfUBUNsQ5DfvfMbPX/+UI7UNXDW+iB9MH8mgUxIvO+pl1ocmJ1Mr2mSiArVIijQGm1iwpoJHXi1nd1UtF40q5J4ZozlzQGJlR5OVkpeq7Bpppq24RDqRtZYln+zlgcVlbN53jHGDCnj4+vGcN+zUdsfGG3STvWVXPL+X5FGgFgnjZQ/1/a2HmPPKRtZsr+T0wu784saJzBhzWsSyox0JusnYsivEabm7JidTT5OJklaSPbkVCpYVlTVYPg+W8V6ndM8RvvPrD7j+iXepqKzhhpJB1NYH+d7v1nD+3OURz9eRPOhk9npTlV0jsalHLWkjFUvHE+2h7jxczcNLy3lxbQU9c7K499LRnNo9m/te+jhmuzsSdJPZ6413ubskjwK1pI1kfs0P6WgP9dDxeh5ftpnfvbcNY2DmBadz64XDye8WYPKcZa7a3ZGg61Tkyater3Yk9wcFakkbqZjcijdYVtc38tSbW3lixRaq6xv52qRB3Dl9BP3zPz/eqX0VlTVMnrOstbc6ZXQhL6yuiCvoqtd7clCglrSRiskttz3UhmATf3x/O4+9tpkDx+q45Mx+3HPpKIb37em63YbPt+KqqKzhhdUVXDupiOWl++MKuur1dn0K1JI2kv01H6L3UNvWhc7MMASbLOcMPYUnbprEpMG942p327rSITUNQZaX7k9JjrLKoaYXBWpJG6n6mh+ph7pwbQX3PP8h9cEmAIJNzWG2fM8Rdhyqjhio2wbD/LwAuYEMKqsbHHvYkJoc5VTW8xZvKFBLWon1NT8ZPcUPd1Zy7wufB+m2KmsaIwa58GBYWdNAXiCTR24Yz1UTihxX/aUiRzkVk7LiLeVRiyf8ULzHqxzokK0HjvP936/hisffpq6xfZAOiZTrHCsnujNzlLXiMP247lEbYzKBVUCFtfYryWuSpBu/fJV2Co53P7c+rrbsO1rLT1/bxB/f30F2Vga3Tx3On1bvZHdVreN7woNcrGDYmdkaWnGYfuIZ+rgD2AgkVkFGuhy/fJV2Co5Ba109OI7UNjD/jS089dZWGoJNfPMLxdw2dQSFPXM4vbBHuwnBttoGuYVrK8gwhmCEgmdtj+usbI1UTMqKt1wFamPMQOBy4D+BHyS1RZJ2/PJVOtokXbQHR21DkN+9t42fLd/M4eoGvjpuAHdPH8mQPt1bjwm9799f+pjKmoYT3t82yIW+XUQK0n4Jhsq9Tj9ue9SPAvcA7ZNEWxhjZgIzAYqLixNvmaQNv3yVjtRTbCv8wRFssry4toJHlpZTUVnDl0b04d5LR3NWUX7E94d6wNEmLCN9uwDINMZXdZyVe51eYgZqY8xXgH3W2tXGmIucjrPWzgfmQ3M9as9aKL7nl6/SocBz93Prow47WGtZVrqPeYvKKNt7lLMH5jPvurOZPLyP6+s4BTmnbxFN1iowSoe56VFPBq4wxlwG5AK9jDG/s9bemNymSbrw01fp0DWdHhyrtx1iziulfPDZYYac2o2ffXMil42NXHa0I/zy7UK6lrh2eGnpUf8wVtaHdniRzhY+PHHjucWs3lbJqxv3UtgzhzumjeCGvxlEINPbDFVtXyUdpR1e5KQTGp7YVVnDI0vLeWBxGd2zs5g1YxTfnjyEbtnJ+U/fT98upOvQnonSaZJZb6Kyup6fv/4pv37nM7Bw83mD+f6U4fTunp30a4t0hHrU4jteLpJpG3RPy89lYnFvVmzaz7G6Rq6dOJC7po+kKCzPOdK1V207FHflOpFUUI9aOoVTrQto3uXabZCMNCYMMGZALx6+fjyjTut5wrEPLC5zvG54RTuNLUsqqUctvhNtMUw8vet5i0oj5i1XVje0BumFaysiLlQJF6nsqAoViR+oKJN0iljparE2dQV459MD7HKovxF6EIR63LGCtBMVKhI/UI9aXPNyAi7WKkJwDpIfVVQxb3EZK8r3kxmjpobTSsFwkQr5tz2PSGdSoBZXvK6Q1zaNzWnMODxIbj9YzYNLynhp/S4KugX458vOoKBbgH/788eOqyLd9IiLOrhfoUiqKFCLK8mokNe2dka0Jej7j9bx+LJN/P797WRmGL4/ZRgzLxhGfl4AgEBmhmNPP1qhpvDJwpLBpyhlT3xJgVpcSWaFPKdFIhef2Y+Hl5bz5JtbqGts4oa/GcQd00bQr1duu/c7BVSnIZbe3QLc99UxJ7xPhYrErxSou4BULN5Idg2LtkGyrjHI71du58J5yzl4vJ7Lx/bn7ktGcnphjw6dF7RSUNKbAnWaS9XuKqmokNfUZHlp/S4eXFLGzsM1fHHYqdx76WjGDSpI6LzqKUu6U6BOc6naXSWZPVNrLa+X72feojI27j7Cmf178dvvjOVLI/p4VtVOJJ0pUKe5VO6ukoye6drth5nzSikrtx6i+JRu3HTuYF7buJdvPf0++XkBjIHD1Q2taXjxrFoU6SoUqNNcutQ/Dh9Hv/m8wazdXsmij/fQp0c2P7lyDN0Cmfxrm1S7totUQrnSnbVxrkhn0srENDdrxijyApkn/M5v+b+hcfSKyhoszcH2/ldKWV62j7suHskbs6Zw83lDeOTVTa4Wp7hZtSjSlShQp7mrJhRx/zVjKSrIw9C8eMNvhYScVgfWNTbx3KodLP1kLxDfcI2WdsvJREMfXYCfsxpqG4KOC07gxKGMaItTwvltaEckmRSoxXML11Ywb1Epu6pqyXCRtBEaynBT/wP8N7QjkmwK1OKpF9fs5N4XNlAfbAKgyUKmgYwMQ0PQufb5rsqadimAyvoQaaZALZ5ZueXgCUE6JGihZ3YW3XOyYhZgSvUwjrbkknSgQC0J27j7CPMWlbK8bL/jMVU1Day775KYBZhSKVWrOkUSFTNQG2NygRVATsvxz1tr70t2w+Rzfu317ThUzSNLy3lxXQU9c7KY/eXR/PadzyIW82/bYwZ/1N5I1apOkUS56VHXAVOttceMMQHgLWPMK9ba95LcNsGfvb6Dx+p4fPlmnnlvO8bAzAtO59YLh5PfLcBpvXJj9pj9kqWSylWdIomIGaht8+63x1r+Gmj54/2OuBKRl72+RHvmx+saeeqtrcxfsYXq+kauLxnEHRePoH/+56lyfuoxx5IuqzpFXO1CbozJBFYDw4GfWWvvjXDMTGAmQHFx8aRt27Z53NST09DZL0d8Khpg65zLXZ8n0thwaPupSJkUbYN6//xczht2Km+UH+DAsTpmjOnHrBmjGN63Z/sLpRGn8XK/LRiSk0PCu5Bba4PAeGNMAfCiMeYsa+1HYcfMB+YDlJSUqMftEa96fZF65qEPKXw4JTyA7aqq5YU1FQwr7M78myex/WA133r6gxNS6CqrG3zde44knXr/cnKLK+vDWltpjFkOXAp8FOv4riqVk3te1YGONe7adjjFacl3TX2Q7QerT2hP28JJfhg/j5dfxstFoolZ68MYU9jSk8YYkwdMB0qT3TC/ilRg6EcLNrBwbUVSrudVLQ83PfBdlTV8uLPSMdd5d1VtzF29VTBJxHsxx6iNMWcDvwEyaQ7sz1lrfxLtPSUlJXbVqlWeNdJPJs9ZFjGQFRXk8fbsqZ3QIncijceGywtkUtMQJMM0rygMF1odGEu84+cikuAYtbX2Q2CC561KU+mY0hUaqqlpCLYG29BEYltBa7l92ggG5OfyH//3Sbug7vS+cKHeu1/zv0XSjVYmxindUrrCe9JBa8kLZPLVcf3564Y9HKtrBOD84X145IbxFPbMASA3kMndz61v14O2EDVYh8bP/Zj/LZKuVI86Tn4q1L9wbQWT5yxj6OyXmTxnWcRxcqc87D+t3smxukauGDeAN2ZdxO/+/gutQRqag2mTwzBHKKXPAAV5AXp3C7QbP4+W/y0i8VGPOk5+Sely22N1GpKxFv5y2/mcVZTveA2nbw9uxuPTcYhIxK8UqDvADyldblcs9s/PjVh7o6ggL2qQhsRSA9NtiEjEzzT0kWRuhic6wk2PddVnh8gJG6YB98E2kdRAPw0RiaQ79aiTKJkTatF6rOV7jzJvURmvbtxL3545fG3SQN7efIDdVbVxD9V09NuDX4aIRLoCV7U+4tWV86jj4ZRznWkMD10/LqGgFSkvOicrg3EDC1i17RDds7P4x4uG8e3JQ+iWreexiN8lXOtDOsZpeCJobcI967Y91orKGnrkZFHbEGTdzkr+7vyh3HrRcHp3z+5Yw0XEVxSokyjartpeFKi/ZEw/Kipr+MXrn3K8vpFrJg7krukjKerghJ0WqIj4kwJ1EsXaVbujqWoNwSaeW7WDx17dxL6jdVx8Rj/uuXQUI/t1vOyoFqiI+JcCdRKFAlykFX4Qf6qatZa/btjDg0vK2HrgOCWDe/Pzv51IyZBTEm6rtqUS8S8F6iQLBblES5W+s/kAcxeVsn5nFSP79eDJm0uYdkZfjDERj493GEMLVET8S4E6BdykqjkF1o8qqpi7qJQ3Nx1gQH4uD35tHFdPKCIzI3KADp0r3mEMLVAR8S+l5/mAU6rdmAG9WLO9koJuAf5pynBuPHcwuREWsITrSClWbUsl0rmUnudzkcaH6xqbWLu9ku9PGcY/XDiMXrkBV+dauLbCMdMk2jCGFqiI+JcCtQ84Fk4CZs0Y7fo8oV6xk1jDGH6oYSIi7anWRyerawzSKy9ybznefOho22SpzoZI+lKPOkEdXSTS1GT58/oKHlpSTlVNQ7vtr/ICmUwZXcjkOcsSztwANNYsksYUqBPQkewKay2vl+1n7qJSSvccZcyAXvzX1WM5eKyOB5eUtwblKaMLeWF1hSeZG0UFeQrSImmsywbqRJdDu3l/vItE1mw/zNxXSlm59RCDT+3GzecN5tVP9vKtp99vd43Jc5bFvQAlkfrRIuJfMQO1MWYQ8FugH83zW/OttY8lu2GJSHQ5dKK7p4T/fvO+YzywuJTFH++lT49sfnLlGLoFMvnXP3/seI2OLEDpSOaG6nuI+J+bHnUjcLe1do0xpiew2hiz1Fr7SZLb1mGJLod2+/5Yi0T2VNXy6KvlPLdqB92ys/jB9JH83flD6Z6TFbPH3NEFKPFkbqi+h0h6iJn1Ya3dba1d0/LzUWAj4Ov/Fye6HNrt+512Mfn+lGHc/8pGLnxgOS+s2cm3vjiEN2ZdxO3TRtA9J8vVNVKxQ4o2oBVJD3GNURtjhgATgJURXpsJzAQoLi72oGkdl+hyaLfvDx9q6J+fy6TBvZnzSilH6xq5enwRd00fyaBTusV9jVQsQFF9D5H04DpQG2N6AC8Ad1prj4S/bq2dD8yH5iXknrXQhfBx1vCMCYivNxrPpNxVE4r4ytn9eX71Th59dRP/9+Fupowq5J5LR3NG/14JXSPZC1BU30MkPbha8GKMCdAcpJ+x1i5IbpPiExpnraiswdI8zvrC6gqunVTUoU1Zwf2mrtZaFn20hxmPrmD2gg0cPF4HQPneY5TtOerJNZJJG9CKpIeYRZlMcx3N3wCHrLV3ujlpKosydaQAUTycsiLe23KQuYtKWbu9kr49c6isbjgHR4kAAAn7SURBVKA+2NT6vnQpaKSsDxF/iFaUyU2gPh94E9gAhCLRj621f3V6TzIDdXhgcSpAZICtcy6P+3xtA1WkinJt5ecF+PFlo3ns1U3sqqpt97pXDwsR6foSqp5nrX2L5rjX6SKlkxmak7vDuRlnjZWeFq12BkBdQ5CcrEx2RwjSoEk5EfFGWhVlihQ4Le2fIm7HWWOlp8UKtLWNTTywuMzxoaBJORHxQloF6mjlQEOTcgV5AXIDGdz17Domz1nGwrUVcZ9vV2UNx+sa6ZEbOylmV2WNJuVEJKnSKlA79VBDY8GP3DCeusYmDlc3tGaA/GjBBsdg7XS+XnkBLnxgOUdrG4my41XrOfyQwSEiXVdaBepYPdd4V9pFOp8BqmoaGFbYgwW3fpGHrx/fWhc62hDLVROKeHv2VLbOuZy3Z09VkBYRz6RV9bxYq/XiXWl31YQirLX8v5c3cvB4PQD983P5z2vGctHIQowxTCzufUIWSGensvmhDSKSWmkVqCH6ar14V9qt31HJn1bv5ODxegb2zuPuS0Zy5bgiMhzGOzp7qyoVURI5OaVdoI7G7dLvLfuP8dCScl7esJtTumdz31fP5JtfKCYnK/YO317oaK840aqAIpKeulSgjjU0su9ILY++tolnP9hBTlYGt08bwXe/NJSeLnf49kIivWIVURI5OaV1oHbqmYYHvCO1DTzxxqc89dZWgk2WG79QzD9NHUFhzxxX5/OqXZBYr1hFlEROTmkbqN30TGsbgvzvu9v42eubqaxu4IpxA7j7kpEMPrV7h87nRbti9YqjBXlttSVyckrbQB2tZ/rVcQNYsGYnjywtZ1dVLReMLOSeGaM4qyi/Q+eLJ1DHOk+0XnGsIJ+KGtUi4j++C9Ruhx+ceqYVlTV8+bEVlO89xriB+Tz4tXF8cXifmNf1avzXzc4tTr1iNw+Lzs48EZHU81WgdupRrtp2iOWl+08I3tEq5zUGLT//24l8+azTaK7SGptX47+J7Nxy17PrIp5Tk4UiJzdfBWqnHuUz721vrZAXCt7XTipqt4sLwPUlA/nPq8cSyIxv0aVX47+J7NyiyUIRicRXS8ijFV1qq6YhyNJP9jK2zZhzr9ws5l17NvOuGxd3kAbvdlxJ5Dwq7iQikcTcOKAjOrpxgNNuLU6yszK45YtD+N6Fw+jdPTvu6/mRloiLnJwS2jgglSINGzhtDNAtO5OlP7iwtWBSV6HJQhEJ56uhj0jDBt84ZxCBsNobOVkZ/NfVY7tckBYRicRXPWr4vEdpreXlDbt5aEk5DU2W7MwM6oNNFGk4QEROMr4L1ABvbz7A3EWlfLizipH9evDkzSVMO6Ov61Q7EZGuJGagNsY8DXwF2GetPSuZjfmoooq5i0p5c9MBigryePBr47h6QhGZsbZZcUkTdSKSjtz0qH8NPA78NpkNqapp4Gu/eJecQAb/cvkZ3HjuYBZ9tIcL5i33JLCqlrOIpKuYgdpau8IYMyTZDcnPC/DETZMYX1xAr9yA54FVtZxFJF35KuvjgpGF9GqpDR3v/oexqJaziKQrzwK1MWamMWaVMWbV/v37Ez6f14HVaRm2lmeLiN95FqittfOttSXW2pLCwsKEz+d1YNXybBFJV74a+mjLy8AayvaoaQiS2ZLiV5AXIDeQwV3PrmPynGUsXFvhSbtFRLwWM1AbY/4AvAuMMsbsNMb8XfKb5V2RpNCkZKiGSNBaAhmG4/WNHK5uwPL5RKWCtYj4ka+KMiVDPIWeigryeHv21CS3SESkvWhFmXw79OGVeCYflQEiIn7U5QN1PJOPygARET/q8oE60qRkIMMQyDxxWboyQETEr3xZlMlLTnsURvqdViiKiB91+clEEZF0kBY7vKiynYhIZL4I1KpsJyLizBeTiV4XYBIR6Up8EahV2U5ExJkvArUq24mIOPNFoFZlOxERZ76YTHTKddZEooiITwI1NAdrBWYRkfZ8MfQhIiLOFKhFRHxOgVpExOcUqEVEfE6BWkTE55JSPc8Ysx/Y1sG39wEOeNicztRV7qWr3AfoXvyoq9wHJHYvg621hZFeSEqgToQxZpVTqb9001XupavcB+he/Kir3Ack71409CEi4nMK1CIiPufHQD2/sxvgoa5yL13lPkD34kdd5T4gSffiuzFqERE5kR971CIi0oYCtYiIz3VaoDbGXGqMKTPGbDbGzI7weo4x5tmW11caY4akvpWxubiPW4wx+40x61r+/H1ntDMWY8zTxph9xpiPHF43xpifttznh8aYialuo1su7uUiY0xVm8/k31LdRreMMYOMMcuNMZ8YYz42xtwR4RjffzYu7yMtPhdjTK4x5n1jzPqWe/mPCMd4G7+stSn/A2QCnwKnA9nAeuDMsGNuBX7R8vPXgWc7o60e3MctwOOd3VYX93IBMBH4yOH1y4BXAAOcC6zs7DYncC8XAX/p7Ha6vJf+wMSWn3sC5RH+G/P9Z+PyPtLic2n5d+7R8nMAWAmcG3aMp/Grs3rU5wCbrbVbrLX1wB+BK8OOuRL4TcvPzwPTjDEmhW10w819pAVr7QrgUJRDrgR+a5u9BxQYY/qnpnXxcXEvacNau9tau6bl56PARiC8cLvvPxuX95EWWv6dj7X8NdDyJzwrw9P41VmBugjY0ebvO2n/obUeY61tBKqAU1PSOvfc3AfAtS1fSZ83xgxKTdM85/Ze08V5LV9dXzHGjOnsxrjR8vV5As09uLbS6rOJch+QJp+LMSbTGLMO2AcstdY6fiZexC9NJibf/wFDrLVnA0v5/CkrnWcNzXUVxgH/DSzs5PbEZIzpAbwA3GmtPdLZ7emoGPeRNp+LtTZorR0PDATOMcaclczrdVagrgDa9iwHtvwu4jHGmCwgHziYkta5F/M+rLUHrbV1LX99EpiUorZ5zc1nlhastUdCX12ttX8FAsaYPp3cLEfGmADNwe0Za+2CCIekxWcT6z7S7XMBsNZWAsuBS8Ne8jR+dVag/gAYYYwZaozJpnmw/aWwY14CvtXy83XAMtsyMu8jMe8jbKzwCprH5tLRS8DNLRkG5wJV1trdnd2ojjDGnBYaLzTGnEPz/w/81gkAmjM6gKeAjdbahx0O8/1n4+Y+0uVzMcYUGmMKWn7OA6YDpWGHeRq/OmVzW2ttozHmn4DFNGdOPG2t/dgY8xNglbX2JZo/1P81xmymeWLo653R1mhc3sftxpgrgEaa7+OWTmtwFMaYP9A8697HGLMTuI/mSRKstb8A/kpzdsFmoBr4due0NDYX93Id8D1jTCNQA3zdh52AkMnATcCGljFRgB8DxZBWn42b+0iXz6U/8BtjTCbND5PnrLV/SWb80hJyERGf02SiiIjPKVCLiPicArWIiM8pUIuI+JwCtYiIzylQi4j4nAK1iIjP/X8W5zmilIONHQAAAABJRU5ErkJggg==\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAArjUlEQVR4nO3deXxU1fnH8c8hBAiCRCQiRCloMViQRSg/K3UBq6gsplpbcalaFZe6gCgGtCxuoLhXqwXFFQUtiIoiRRZxqwoCigLWBZQABYSwJUASzu+Pm8QwzHInc2dyb/J9v168BObOzLnOi2dOnvOc5xhrLSIi4l91qnsAIiISnQK1iIjPKVCLiPicArWIiM8pUIuI+FzdZLxos2bNbOvWrZPx0iIiNdKiRYs2WWuzwj2WlEDdunVrFi5cmIyXFhGpkYwxqyM9ptSHiIjPKVCLiPicArWIiM8pUIuI+JwCtYiIzyWl6kNEpDaZvjifcbNWsragiJaZGdzcO4fcLtmevb4CtYhIAqYvzmfYtC8oKi4FIL+giGHTvgDwLFgr9SEikoBxs1ZWBOlyRcWljJu10rP3UKAWEUnA2oKiuP6+KhSoRUQS0DIzI66/rwoFahERnFxzj7FzaZP3Jj3GzmX64nxXz7u5dw4Z6Wn7/F1Geho3987xbGxaTBSRWi+RBcHyx1X1ISKSRNEWBN0E3Nwu2c51hYXQsKHn41PqQ0RqvYQXBNesgQsugB49oLQ09vVxihmojTE5xpgllX5tM8YM8nwkIiLVpMoLgkVFcMcdkJMDU6dC375QUuL5+GIGamvtSmttZ2ttZ6ArUAi86vlIRESqSdwLgtbCK69Au3YwYgSceSasWOEE7fr1PR9fvKmPU4BvrbURG1yLiARNbpdsxpx9DNmZGRggOzODMWcfEzY/Pe+lWSw5ohP88Y+s3F2Xyy8dR5sjL6HH5O9cV4rEy1hr3V9szETgM2vto2EeGwgMBGjVqlXX1asVy0WkBtmwgVVXDabVqy+xJaMx9534Z6Z0PJW9dX6eiWekp0UM8LEYYxZZa7uFfcxtoDbG1APWAu2ttf+Ldm23bt2sjuISkSAKbbA0tFcbznr/VRg9mpIdO3nm2L480mMA2xo0Cvv87MwMPsjrFff7RgvU8ZTnnYEzm44apEVEgiq0nrrtogV0HPdn2JwPp59O78PO4tuDD4/6Gl5uHS8XT456APCS5yMQEfGJ8nrqI3/6kadfGckz/xrNXuCmi++GmTPZdeRRMV/Dy63j5VwFamNMQ+BUYJrnIxAR8Ynt6zdy25wJvD3xWrquWc4dPS/j9L88ytRDOwLhq0Mq83rreDlXqQ9rbSFwsOfvLiLiB6Wl8NRTvPvkUJrs3MbkTqdx/wkX8dMBmYCTd4b9t4s3yUjHGCgoLE7K1vFy2kIuIrXbu+/CDTfA0qWUdvk//tD1Qj47uE3Fw6Gz5Irt4imkLeQiUjutWgXnngsnnwxbtsCUKTRb9BF/vub3ruqpU0kzahGpXXbuhLFjYdw4qFMHRo+Gm26qaKZUHTPmWBSoRaR2sBZefBFuuQXy8+H8852AfXj0cjs/UOpDRGq+Tz91OttdeCEceii8/z5MmhSIIA0K1CJSk61bB5deCt27w3ffwcSJ8MknTtAOEAVqEal5du1y0hpHHeXMnIcOZcYr8+nxv1/QZvjMuI7a8gPlqEWkRpi+OJ9xb6+g/afzGPnuRLI3r4P+/eG++5i+o2GVjtoq7/uRX1BEmjGUWkt2EuulI1GgFpG4hDYtSnXQijSmiU+8wT2znuC3q5fy9cGtuGzAnfS7+RLYAUNeXkppSAO6WEdthfb9KH9+POcpeiWuNqduqXueSM0UGrwgsdaenvjpJ6b2vYzcj99gR70MHjjhQiZ1PoOStLpkZqSzu2TvfuchVpadmRH2S6fH2LnkR2mwVNUueZFE656nHLWIuBbtENiUKymBRx+Ftm3J/fgNXuhyBiddOYFnu/ajJM1JFhQUFUcN0gZnhmz5eaZcnruO1QUvGV3yIlHqQ0RcS/gQWK+88w4MGgRffgm9enHJMQN4r0GLuF7CAKH5hMrpkJaZGVFn1MnokheJZtQi4lqk4NQkI50eY+fSJu/N5FZUfPMN5ObCqac6B8u++iq88w7nXHxG2DMPD2qYHvZl0ozZL0iXK//SidYpL1ld8iJRoBYR18IFr/Q6hp17SiKmEDyxfTvk5UH79vDOO3x5XR49L3mMNv9Jp8c98wDCnnk4sl/7sAH8/j92quiIF6plpU555a8JTnCH6un/odSHiLgW2uazZWYGhXtK2FJYvM91sSoqXNu7F557DoYNg/Xr4eKLefv86xj83kaKdu5bajfm7GMiLu5FqlIJtzBa3Z3ywlHVh4gkpE3em2HTCAb4fmyfqr/whx867UcXLoTjjoOHH4bu3SNWY1SlCsNPpYZenZkoIrKfSItuVV5sW7PGaZz04ovQsiW88AIMGOB0usPbBU2/zJhjUY5aRBISLm9dpcW2oiK44w7IyYGpU+G222DlSrjggoogDZG/AFJZhZFqCtQikpDKi25VarZvLbzyChx9NIwYAWeeCcuXO0G7UaP9LvfsiyFAlPoQkYRVOYWwZImTh16wADp2hHnznBNXYrwXRF4grIlcBWpjTCbwJNABp0b8L9baj5I4LhGpyTZudFIbEyZA06bwxBNw+eWQFvmE78qCklv2itsZ9cPA29baPxhj6gENkzgmEamp9uyBxx5zjr/audOZTY8YAQcd5OrpfqrSSKWYgdoYcyBwInAJgLV2D7AnucMSkRpn5kwYPNhZIOzdGx580MlLuxTaEKo6uthVFzeLiUcAG4GnjTGLjTFPGmMOCL3IGDPQGLPQGLNw48aNng9URKrX9MX5VdsmvnIl9OnjLBLu3QszZjhBO44gDT5rCJVibgJ1XeBY4HFrbRdgJ5AXepG1dry1tpu1tltWVpbHwxSR6lQ+m41rm3hBAdx4I3To4JxReN99sGyZE7TLtmO7fe9oLUdT3hCqGrjJUa8B1lhrPy77878IE6hFpOaKNJsd9fqX++eMOx4KTz3F7luGkb51C1M6nsYLfa/gil7HkVuvXlzvG67/daiaXD9dLmagttauN8b8aIzJsdauBE4Bvkr+0ETELyLNWguKiikocvp85BcUMe3BSfT8+FmafP0VXxzegZG5I/my+ZFQQpXyyeG+ICqr6fXT5dxWfVwHTCqr+PgOuDR5QxIRv4nVmzl76waGzZtI35Xvs77JIdw34G88f3j3fVIcVWnUFC2tUR1nF1YXV4HaWrsECNssRERqvpt754RNQWTs2cVVH/+LKz+ZhsXwwG8vYEL337MrvUHY14k3nxzpC8LrY7D8TjsTRSSm/XYDNmnAiYtmc/2sJ2mx4ydeO/okxp58CesOzKro3xyrUZObmuhwXxC1Jd1RmQK1iLhSsRtw4UJno8qHH/Llob/kurOGsvCw9sC+QTRagHVTE10eyIuKS0kzhlJra1W6ozIFapFaLK6dfuvXw/Dh8PTTcMgh8NRT/LfTqayb/V9MhOdHeu1oNdG5XbL3C+Sl1lYE+toWpEEHB4jUWuFK3zLS0/bvfLd7Nzz0ENx5p/P7QYOcPh0HHljl94502AA4+edIC5fhctM1ZVt5tIMD1OZUpJaKudPPWnjtNeecwrw86NnTOfX73nsTCtIQufbZED63XS50MbJKG3ECSIFapJaKelLKsmVw2mnOid/16sGsWfD669C2rSfvHa6ntIGIs+xyoQG+tmwrV45apIaLlBoIV/rWpGg7g9+fRMm4t9h7QGPqPfIIXHUVpKd7OqbQKpImGekVG2ciCVft4eWxXH6mQC1Sg0Wrrqhc+pa2t5Tzl8zkxvcmceDunUzqfAaP97yIW377W3I9CNKRviwqLxxGE6naw/PzGn1KgVqkBioPjOGCWHlqoHxRbt5jL3LNa4+Ss+kHPmzVkdG/G8jKrNYAce8kjDSWaKV40baJh13crCRSnXXPdln0GDs38AuM5RSoRTzgp8oDN42M1hYUwbffkjt6CLmvvcYPTZpz5e+HM6vtb/bZ9u1FCiFWKV6094h19mK4Y7l6tsti6qL8GtW3WoFaJEF+a2gfq5HRAbsLyVs8DR6c5uSe776bi0u68P3O/Z9TlRRC6JdWrPak0baJu/n/F3osV4+xc6N+MQSRqj5EElQdlQfRmvhHmqEau5c/fPEO85+8kovenQznnQdffw3DhnFD32M8Odk7XLlcpM7T5V8CXp8q7maBscqHIFQTzahFEpTqyoNYM/hwM9Rj85cz8p3xdFr/Xxa3yGHYn++k7+VnkduyZcXzIPGTvcN9aVnCl94V7ilh+uJ8z08Vj7XA6LefgNxQoBZJUKorD2LlfCsvsB26bRO3vPsMv/9qPusbHcygvkN47VcnYU0dPggJTl6c7B3py8kCmSEleFsKi/cJkF4FyViNnGL9//MjBWqRBKW6w1usGXxul2zq7Cpi/ci7uXD+i6TZvUw4cQAPdj2bwno/f3lECk6JLIxGyzcD+9VKJyNAxpqhB7H2WoFaJEFe/+geS9QZvLUwdSr9b7oJVq+Gc86BceO4+59fhd31F2lLdlXTAtG+tAZPWRL2OckIkNFm6EGsvdZioogHcrtk80FeL74f24cP8nol9UfoSItvd7YucfpxnHsuNGkCc+fCv/4FbdpEDEJeb8nO7ZLNmLOPITszA4Mzky4vsXM7hmTzevEyFTSjFgmY0Bn8r+ru5tGvptHm7pfgoIPg8cfh8suh7s//vN2mZ7xIC0SazfrlEIBU/wTkBQVqkQDK7ZJNbvsseOwxGD0adu6E66+HESOcYB3meogdnJKZFvBTgPRy8TIV1I9axKeiLurNnAmDB8PKldC7Nzz4IBx9tCfv6apHtXguWj9qzahFfCjSol6jVd/yuyfvgbfeclqOzpgBZ565z7bvRPhp1is/cxWojTGrgO1AKVASKeqLiDdCF/UO3LWD6+ZO5uS734BGB8B998F11zm9oj0WtLRAbRDPjLqntXZT0kYiIhXKF+/q7C3lj5/P5qb3nqdp4TZe7ngq5/37OWjevJpHKKmk1IeID4TmozMbptN25WJGvjOe9hu+45PDfsXF546moN0xnKcgXeu4DdQW+LcxxgL/tNaOD73AGDMQGAjQqlUr70YoUsPtt4C3ejV3vvs0fZa/R37jLK7tP5QZ7U4go15dxvi41leSx22g7mGtXWuMOQSYbYxZYa1dUPmCsuA9HpyqD4/HKVJjhM6eC/eUUFRcSsaeXVz18VSu/GQqFsPjJ1/EKz3P4/tCG/GEE6kdXAVqa+3asv9uMMa8CnQHFkR/loiEClfNgbX0X/4uefOfoeX2Tbx+9ImMPfkS1h14CN+POLOaRyx+EDNQG2MOAOpYa7eX/f404Pakj0ykBgqt5jhm3X8ZOWc83fKX80XzI7m+/80sPKw98HMjIxE3M+rmwKvGqdOsC7xorX07qaMSqaHKqzmydmzh5gXP8ocv5vBTwybcfMb1TO1wCnvrOD0o/N57QlIrZqC21n4HdErBWERqvF80SqP3nClc++EU6pcUM77773n0+POom9mEFvXrapOJhKXyPAkkPx0m64q18MYbzPjn9TRas5rZv+zOXT0vY1XTbKfzXf/2Ecdf+V6bZKRjDBQUFgfjvsUTCtQSOKk8SinWF4KrL4wvv3T6csyeTaOjj+bDR19g1PYWrC0oilnNEXqvlRvvh7vvwH2BiSsK1BI4qTpKKdYXQswvjM2bYdQo+Mc/oHFjePhhuPpqjk9P5wOXY4h1onjl+w7iWYDijg4OkMBJ1VFKsZroR3r8gbe+ctqPtm3r/HfgQPjvf502pOnpcY3BzT2VX1Mdp6FLamhGLYGTqqOUYn0hhHv8+FVLGDlnPGz6wTlt5aGHoGPHKo8h0r2GXuNmvBJcmlFL4KTqKKVYR0dVfrzVlnX8c9qdvDjlNhqX7oGpU2HOnISCNIS/18oq37dfjroS7ylQS+BEO5fPS7G+EG7uncPBe3cz9N1nmP3U1fx21RIe6HkJC996H84+25Me0aH3mpmRzkEN08PedzK+wKYvzqfH2Lm0yXuTHmPnMn1xfgJ3I1WlE15EoohYRbF3Lzz/PLtuGkqDTRuY2qEXz/W7kkvP7VGtC3deVn3otJfUinbCiwK1SLw++ghuuAE+/RS6d4dHHoH/+79qG06ySvJ6jJ0bNj+enZnBB3m9En592ZeO4hLxQn4+5OXBCy9Aixbw3HNwwQVQx7sMYrxBN5kleVqc9A8FapEQocHylpNa0X/OZBgzBkpLYfhwGDYMGjXy/H3jDbrJrClPVXWNxKZALYGS7J13+wRLa+n4n9l0HTsRtm5wFgjHjYMjjkjKuKsSdJM56725d07YHLWaRaWeArUERip23pUHy6M3fMfId8Zz3I/LWJ7VmrGX38ffJwxJ6rirEnSTOevVieT+oUAtgZGKreO71q7nrvee57yl/2Zrg0bceto1TO7Um7110vh7FV/T7birEnSTPevVieT+oEAtgZHUxa3iYnjsMeZPuI2M3UU8e2xfHvrt+Wxr4OShE2niH2l8+QVF9Bg7t2K22rNdFlMX5ccVdDXrrR0UqCUwkvZj/ttvO93tVqyg8LiTGNBpAMsyD6t4ONEZaqRxG6j4+/yCIqYuyuecrtnMW7ExrqCrWW/Np0AtgeH5j/lffw033ghvvgm//CW88QbN+/Th8iVrw85Qq7qQGW7cBgjdwVBUXMq8FRtTUqOsdqjBog0vEiieBJitW+GOO5y2oxkZMGKE09muXr2o7xsabAEOapjOyH7hm/5Ha/gfqdGSAb4f2ye++4mTdhz6kza8SI0R68f8qIG8tBQmToRbb4VNm+Avf4G77oLmzWO+b6S+0FsKi8NWcIRr+J+RnsaDf+pMbpfsiLv+UlGjnKp+3uIdNWUST/iheU95cMwvKMLycxnc9MX5sGABdOvm9IbOyYGFC+HJJ10FaYi+YBmu53Os3tCp6gAYjnYcBo/rQG2MSTPGLDbGzEjmgCR4ogbIFAoXHJtuWkf6+QPgpJPgp59g8mQnaB97bFyvHWumGxrkYgXDVHUADEftUIMnntTHDcBy4MAkjUUCyi8/SlcOjhl7dnHVx1O58pOpWAx/P/FCWo8ZSb/jf1ml1w63IFhZ5SA3fXE+dYyhNMz6T+XrqqtaQzsOg8dVoDbGHAb0Ae4CbkzqiCRw/PKjdMvMDPK3FNJ/+QLy5j9Ny+2beP3oExl78iWsPfAQshf8UOVAXR5QR73+5T4HzMK+Qa78p4twQdovwVC118Hjdkb9EDAUaBzpAmPMQGAgQKtWrRIemASHX5r33Hn4bg587Ba6rvmKZc2P5IZ+N/Hp4R0qHk/0i6N8BhxtwTLSomOaMb6qqlDtdbDEDNTGmL7ABmvtImPMyZGus9aOB8aDU57n1QDF/6r9R+n16+HWW+n59NPsOuhg8s64npc7nMLeOvsu1nn1xREtyEX6MthrrQKjVJmbxcQeQH9jzCpgMtDLGPNCUkclgVJtC2O7dzvd7I46Cp5/HoYMocF333DcXUOpX3/fmuhUfXFooU6SIa4NL2Uz6pustX2jXacNL5JU1sKMGc6uwm++gb594f77nYBdprp23mkziVSVNrxIzfHVV05fjn//G9q1g5kz4fTT97usunKwWqiTZIgrUFtr5wPzkzISqXXimvVu3gyjRsE//gGNGzvbv6++GtLTk//ecdJCnXhNM2qpFq4PASgpgfHj4W9/g4ICuPJKuP12aNZsn9fy4pzBhas3x925TiQV1JRJqkWkXhfgLEbe3DuH3C0rndO+ly2Dk092ZtEdO+5zbTw54fKAHq0hUuV/DcotSypFy1Gr14dUi2g1zXVWfc8BA/4Ip5wCO3bA1Kkwd+5+QRpi99QAJ0B3Hv1vBk1ZEjFIQ/i2o6E9PESqg1IfUi3CbZI5YHch1/znFS7/9FVK6tTln6f9hStfewwaNIj4OrF2RUZqT+qWGhWJHyhQi2teLsBV3iRj7F7OXjaPoQuepfmOzUxt35N7T7qYDY2bcWWUIA2xd0VG2ikYKlwj/8qvI1KdFKjFFa9PAC9/zlsTXuWa6Y/Qed1/WdLiKK7KHc7i7HaAu3MKY+2KdDMjzq7ieYUiqaJALa543iEvP5/cB/LIfeEFipo1J6//EKa0OwlrnGUTt0EyVt1ytNNUQhcLu/2iqeqfxZcUqMUVzzrkFRXBAw/A3Xc7pXfDhpExbBjHfbON96oYJKPVLUdqTxruCC3VP4tfKVDXAKnYLp1whzxrYdo0uOkmWLUKzj7b6dNxxBEA5HZpnJQgqZ2CUhMoUAec17njSBLqkLd0KQwaBPPnQ4cOMGcO9Er+SdvlNFOWoFMddcC5qSP2QpU65G3c6GzzPvZY+PxzZ/v34sUpDdIiNYFm1AGXytNVXM9Mi4udoDxqFGzfDtdeCyNHQtOmMZ9aOY3TJCMdY5yTvtPKjrbKVupCaiEF6oDzy+kqFd5+2+lut2IFnHoqPPQQ/OpXTgAePzdqnjg0jVP5yKvyo62SldoR8TOlPgLu5t45ZKTve5JJtdT/fv210xf6jDOcao7XX4dZsyqCtJtTyt1uTtHWbqltFKgDrtpOVym3datTydGhAyxY4FRyLFsG/fqBMUDkPPqgKUvoMXZuRcCOJ12jrd1Smyj1UQNUS1VDaSk8/TQMHw6bNsFf/gJ33QXNm+93abSgWjmVEW1zSiht7ZbaRIFa4vfee0770cWLoUcP55SVrl0rHg6t685smM6WwuKIL1eeyoi0OSWUtnZLbaNALe798AMMHQpTpsBhh8FLL8Gf/lSR4oDwdd3pdQzpaYbi0si9z9cWFO23OUVVHyIOBWqJrbAQ7r0X7rnH+fPIkU7Abthwv0vD5aOL91oyM9I5oH7diKmN8lRGqtM41XUIrkg8tJgokVkLkydDTg6MHg1nnQUrVzr10WGCNETOR28tKuaDvF489KfO/qhSAdfVKCLVLWagNsY0MMZ8YoxZaoz50hgzOhUDk59NX5xPj7FzaZP35j5VEkm1aBGccAIMGABZWU5Fx+TJ0KpV1KdFWuSrPGOu1iqVSlK1q1MkUW5SH7uBXtbaHcaYdOB9Y8xMa+1/kjw2IXW9PCqsXw+33upUdGRlwYQJcOmlkJYW+7m46wnil94bqdzVKZKImIHaOqff7ij7Y3rZL+9PxJWwvOwDHTUfu3s3PPII3HEH7NoFQ4bAbbdBkyZxvUeQutX5blenSASuTiE3xqQBi4BfAo9Za28Jc81AYCBAq1atuq5evdrjodZObfLeDPutaIDvx/Zx/Trhzg40gLWWP65fwoh5T9Hox1XO7sL772f6zgMCEWwTEc8J5iLJFu0UcldVH9baUqCzMSYTeNUY08FauyzkmvHAeIBu3bppxu0Rr2Z94WbmR276gRFzJnDiqsV8e/DhfP735zn+2gtjplvCNU4qKCwOXEAP0uxfare4yvOstQXGmPnA6cCyGJfXWKks6UqoD3QllfOuB+7aweD3J3HRZ29SWC+D0adcwfNd+tB8R2M+IPYiW6TGSUFsmOSXfLlINDEDtTEmCyguC9IZwO+Ae5I+Mp9K9eKeV7O+lpkZrN+8gwFLZ3Hjey/QZNcOXurUmwdOuJDNDZ08dHkwj7bIFqtxUkLnKIpIWG5m1C2AZ8vy1HWAl621M5I7LP/y/JBXF7yY9d1z0CayHryFnA2r+KjVMdx+yhUsP+SIfa4pT6dESrfUMcZVLw5VTYh4y03Vx+dAlxSMJRACV9L13XesveJafjt3Jj82ac41ucN566jfYCpt+4Z90ymRem6UWussQMZ4y/KAr11/It7QFvI4Baaka8cOGDOG0vvuJ9PCuBMu4slf57I7vT4Z6Wmc0zWbeSs2hg2i5f8d8vLSiob95SxEDdblAT/l9d8iNZgCdZy8WtzzQtgZa6cW8MILkJcH69bx7y6nMuo3F/C/xs0qnldUXMq8FRv5IC/y2YW5XbIZPGVJ2Mcszo7CaFUfPcbOTXmKSKSmUqCOk19KusLNWF/6+yuc+OmzNF22BLp3h2nTuGb6T2Fnv25SNZF+esjOzIga5KO9vm9TRCI+pkBdBX4o6aq8qHnI9p+45d1nOOfLeWxq3BSefRYuvBDq1KHl/LlVTtUk8tNDYFJEIgGg7nlJlqyGSmsLiqhfsodrPnqZeROupO+K93jsuHM5+bIn4M9/hjrOR5vImYqJNFDyzVmOIjWAZtRJlLQFNWs5f82nXDXjCQ7f+j/ePuo33NXzMn7MPJTskBlroqmaqv704JcUkUhN4KrXR7y6detmFy5c6PnrBk2PseHTDmnGcP8fO1UtaH3+uXMM1vz5fJ3VmlG9LufD1p0B9akQCbJovT6U+kiiSAtnpdbG36B+0ya4+mro0sUJ1o89xvI357O682+qva+ziCSXUh9JFO1UbdelasXF8I9/OKeqbN8Of/2r8/umTTkLOOvXv/BsvNqgIuJPmlEnUbgFtcpilqrNmgUdO8KgQfDrX8PSpU7P6KZNvR0oOpZKxM8UqJOovGoiLWS7drmIpWpffw39+sHppzsz6tdec4J2+/ZJG6uOpRLxL6U+kqw8deCqHnnrVrjzTnj4YWjQwDn5+/rroX79uN833jSGNqiI+JcCdQrELFUrLWXx7Q/yiwfuInPHVt7s1pu6d9/NGadWrRdWVcoCtUFFxL8UqFMkYj3y++9TcMU1dFnxBQuzj2b02SP5okVbMhb8j93N8qu0mFeVVqx+6mEiIvtSoK4uP/wAQ4fClCnsbpLF9f1u5vWjT4SyfHYiB9hGqjSJlsbQBhUR/1KgTrXCQif3fO+9YC2MHEnPbcdQWK/BfpfGmx8uT3lEEiuN4YceJiKyP1V9pIq1MHkytGsHo0dD//6wYgWMGsVBhxwU9ileHGBbTmkMkeDSjDpBrqorFi1ytn1/8IGzs3DSJDjhhIqHI+WHe7bLosfYuQlXbgDatSgSYArUCYhZXfG//8Gtt8LEidCsGUyYAJdeCmn7boIJlx/u2S6LqYvyPancyM7MUJAWCbAaG6gT3Q7t5vmRqisefHMZuXNegttvh6IiuPFG+NvfoEkT1+9RlRNSVLkhUjPFDNTGmMOB54BDgb3AeGvtw8keWCISbS/q9vn7pRqspde3n/K3uU/ClrXQpw/cfz/k7B8oY71HVTagVKVyQ/09RPzPzYy6BBhirf3MGNMYWGSMmW2t/SrJY6uyqtQRV+X5lVMNR276kRFzJ3DS95+xOutweOstOOOMKr9HVTegxFO5oQNoRYIhZtWHtXadtfazst9vB5YDvv5XnOh2aLfPv7l3Ds1LChnxznhmTfwrXdauZMypA1kyY0HUIO3mPVJxQor6e4gEQ1w5amNMa6AL8HGYxwYCAwFatWrlxdiqLNHt0K6eX1JC7n9e54ynbqXu1q1M7nQaL/a5givO+T/OcjEbjfUeqdiAov4eIsHgOlAbYxoBU4FB1tptoY9ba8cD48E54cWzEboQmmcNrZiA+GajMRfl5s1zyu2++IL6J50EDz/MBZ06cUEcY3az8JfsDSjq7yESDK42vBhj0nGC9CRr7bTkDik+4fooT12Uzzlds6t0KCtEOdQ1cw+ccw706gXbtsG//uUE7U6d4j7ENpGDY72iA2hFgiHmmYnGGAM8C2y21g5y86KpPDMx0rmE2ZkZfJDXK+HXn744n0dfX8LvZz3H5Z++SlrdutS97Van5C4jo+KacLPjIGwyUdWHiD9EOzPRTeqjB3AR8IUxZknZ3w231r7l0fjiEhpYqtKAKNrrVQ5U0xf9yEe3P8KkuRNpvmMz09r35N6TLmb99mZkP/xRxbWJVplUJ/X3EPG/mIHaWvs+EP6IkhQLV05mgHA/E7jJs0YtT9uzhrbnXkruj8tZ0qItV+cO47PsoyueW/laLcqJSDIFamdiuJmrhf2Ctds8a7jXa7x5A/UvuxQWzybrgIMYcuZgpnXoiTX7p/PLZ81alBORZApU97xIM1QLFYtymRnpNEivw+ApS2Iu6lV+vfole7jmo5eZN+FKen0+D/LyuODm55h6zClhg3Tl19CinIgkU6ACdaQZavnC4YN/6szukr1sKSx2dZJ2y8wMsJbeX3/I7CevZuiC53i/dWcuGvw0jBnDX/t3iXqKePlr+KGCQ0RqrkClPmLVHse7qHfHEXvJuOU2frNqKSubteKCP93JZ227MubsY4B9N52Ey4dXfm8tyolIsgQqUMfared6UW/TJhgxgl7//Cd7Gh/Iff2v54mcU2jetBFjQsrTKgdgP5Sy+WEMIpJaMeuoqyKVddSVxaypLi6Gxx+HkSNh+3a45hoYNQqaNk35WKsiyPXaIhJdtDrqQOWoY4m6qDdrFnTq5Gz97tYNli6FRx6pliAd7y7GcmqiJFI7BSr1EUu41Mioo9M5dcRVMGMGHHkkvPYa9OtXcdp3qiXSWlT12iK1U6ADdaR8bW6XbNi6Fe68E859GOrXh3vucWbT9evH/XpejQsS65Wtem2R2imwgTrizLS0lNyls2H4cNiwwTmj8O674dBDq/Z6xNdEP9GTW6IFeR21JVI7BTZHHW5m2v77z2l31u/g8sudNMcnnzgHy8YI0pFeryr531ivE2n22zIzI2wnwMp14KrXFqmdfDejdpt+qDwzbbltA3nzn6H/8gWsa3QwTJoEAwbElYf2Kv/r5uSWSLNiN2kR1WuL1D6+CtSR0gYLV29m3oqN+wTvlpkZ/LRxC1d+PI2rPp6KwfLw8efxeu8LmXN+n7jf26v8byIntwyesiTsa2qxUKR281WgjjSjnPSfHyp2BOYXFDFs6ueM3LWME58cR8ttG5nR7gTGnHwpm5u1YEy/Y6r03l7lfxM5uUWLhSISjq8CdbSmS+Xar/+GkXPG033NVxTktOea84Yz86C2tMzM2G9XYTy8OqMwkdfRYqGIhOOrnYmRdhYCNNu5hSELnudPn89mc8MDuf+Eixjz5kOQFr1pUtBoi7hI7ZToCS8pE25GWa+0mIsXvsF1H04mo2Q3T/36LP5+/Hk0PjSrxgVp0GKhiOzPV4F6n7TBlkL+sH4pt8yeQLN1q5lz5K+5q+dlfHfwYUoHiEit4qtADWUzygbbYPBgpz9HTg4fDn+OETuczSLZSgeISC3jr0C9ZQuMHg2PPgqNGsGDD8Jf/8rx6el8UN1jExGpJjEDtTFmItAX2GCt7ZC0kRQUQE6O0yt64EC44w7IyvL0LbRQJyJB5GZG/QzwKPBcUkeSmQm33AKnnAKdOwPeBlavenmIiKRazF4f1toFwOYUjAWGDNknSEfrexEv9XIWkaDybVMmrwOrejmLSFB5FqiNMQONMQuNMQs3btyY8Ot5HVijda0TEfEzzwK1tXa8tbabtbZblgeLgF4H1qjHdImI+JhvUx9eBtbyRcmi4lLSylqfZmak0yC9DoOnLInr3EIRkVSLGaiNMS8BHwE5xpg1xpjLkj8s75rkV16UBCi1lvQ6hp17SthSWOzJQqWISDL5qilTMkRr9BQqOzODD/J6JXlEIiL7i9aUybepD6/Es/ioChAR8aMaH6jjWXxUBYiI+FGND9ThFiXT6xjS0/Y9T1EVICLiV/5qypQEkU5cCfd32kouIn5U4xcTRUSCIBAnvKiznYhIeL4I1OpsJyISmS8WE9XZTkQkMl8EanW2ExGJzBeBWp3tREQi80WgVmc7EZHIfLGYGKnWWQuJIiI+CdTgBGsFZhGR/fki9SEiIpEpUIuI+JwCtYiIzylQi4j4nAK1iIjPJaV7njFmI7C6ik9vBmzycDjVqabcS025D9C9+FFNuQ9I7F5+Ya3NCvdAUgJ1IowxCyO1+guamnIvNeU+QPfiRzXlPiB596LUh4iIzylQi4j4nB8D9fjqHoCHasq91JT7AN2LH9WU+4Ak3YvvctQiIrIvP86oRUSkEgVqERGfq5ZAbYw53Riz0hjzjTEmL8zjxhjzSNnjnxtjjq2Ocbrh4l5ONsZsNcYsKfs1ojrGGYsxZqIxZoMxZlmEx4P0mcS6l6B8JocbY+YZY5YbY740xtwQ5ppAfC4u7yUon0sDY8wnxpilZfcyOsw13n4u1tqU/gLSgG+BI4B6wFLgVyHXnAnMBAxwHPBxqsfp4b2cDMyo7rG6uJcTgWOBZREeD8Rn4vJegvKZtACOLft9Y+DrAP9bcXMvQflcDNCo7PfpwMfAccn8XKpjRt0d+MZa+521dg8wGTgr5JqzgOes4z9ApjGmRaoH6oKbewkEa+0CYHOUS4Lymbi5l0Cw1q6z1n5W9vvtwHIgtGl7ID4Xl/cSCGX/r3eU/TG97FdoVYann0t1BOps4MdKf17D/h+Ym2v8wO04f1P2Y9JMY0z71AzNc0H5TNwK1GdijGkNdMGZvVUWuM8lyr1AQD4XY0yaMWYJsAGYba1N6udSHSe8mDB/F/pt5OYaP3Azzs9w9vDvMMacCUwH2iZ7YEkQlM/EjUB9JsaYRsBUYJC1dlvow2Ge4tvPJca9BOZzsdaWAp2NMZnAq8aYDtbaymsinn4u1TGjXgMcXunPhwFrq3CNH8Qcp7V2W/mPSdbat4B0Y0yz1A3RM0H5TGIK0mdijEnHCWyTrLXTwlwSmM8l1r0E6XMpZ60tAOYDp4c85OnnUh2B+lOgrTGmjTGmHnAe8HrINa8Dfy5bOT0O2GqtXZfqgboQ816MMYcaY0zZ77vj/D//KeUjTVxQPpOYgvKZlI3xKWC5tfaBCJcF4nNxcy8B+lyyymbSGGMygN8BK0Iu8/RzSXnqw1pbYoy5FpiFUzUx0Vr7pTHmqrLHnwDewlk1/QYoBC5N9TjdcHkvfwCuNsaUAEXAebZsWdhPjDEv4ay6NzPGrAFG4iySBOozAVf3EojPBOgBXAR8UZYPBRgOtILAfS5u7iUon0sL4FljTBrOl8nL1toZyYxh2kIuIuJz2pkoIuJzCtQiIj6nQC0i4nMK1CIiPqdALSLicwrUIiI+p0AtIuJz/w+0sBpb5YETRAAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": { - "tags": [], "needs_background": "light" - } + }, + "output_type": "display_data" } + ], + "source": [ + "plt.scatter(train_x,train_labels)\n", + "x = np.array([min(train_x),max(train_x)])\n", + "y = w.numpy()[0,0]*x+b.numpy()[0]\n", + "plt.plot(x,y,color='red')" ] }, { @@ -638,32 +594,32 @@ "id": "0giuwC9GHzi8" }, "source": [ - "## Вычислительный граф\r\n", - "\r\n", - "Для проведения вычислений Tensorflow строит внутри себя вычислительный граф, который, в т.ч., может вычисляться на GPU. Однако в нашем случае, поскольку мы использовали пользовательские Python-функции, они не включались в вычислительный граф, и при вычислениях на GPU производилась бы передача данных между GPU и CPU и обратно.\r\n", - "\r\n", - "Для ускорения высчислений и построения единого статического графа, необходимо отметить все функции соответствующим декоратором:" + "## Computational Graph and GPU Computations\n", + "\n", + "Whenever we compute tensor expression, Tensorflow builds a computational graph that can be computed on the available computing device, e.g. CPU or GPU. Since we were using arbitrary Python function in our code, they cannot be included as part of computational graph, and thus when running our code on GPU we would need to pass the data between CPU and GPU back and forth, and compute custom function on CPU.\n", + "\n", + "Tensorflow allows us to mark our Python function using `@tf.function` decorator, which will make this function a part of the same computational graph. This decorator can be applied to functions that use standard Tensorflow tensor operations. " ] }, { "cell_type": "code", + "execution_count": 16, "metadata": { "id": "HK7HPLz3Hyrl" }, + "outputs": [], "source": [ - "@tf.function\r\n", - "def train_on_batch(x, y):\r\n", - " with tf.GradientTape() as tape:\r\n", - " predictions = f(x)\r\n", - " loss = compute_loss(y, predictions)\r\n", - " # Note that `tape.gradient` works with a list as well (w, b).\r\n", - " dloss_dw, dloss_db = tape.gradient(loss, [w, b])\r\n", - " w.assign_sub(learning_rate * dloss_dw)\r\n", - " b.assign_sub(learning_rate * dloss_db)\r\n", + "@tf.function\n", + "def train_on_batch(x, y):\n", + " with tf.GradientTape() as tape:\n", + " predictions = f(x)\n", + " loss = compute_loss(y, predictions)\n", + " # Note that `tape.gradient` works with a list as well (w, b).\n", + " dloss_dw, dloss_db = tape.gradient(loss, [w, b])\n", + " w.assign_sub(learning_rate * dloss_dw)\n", + " b.assign_sub(learning_rate * dloss_db)\n", " return loss" - ], - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "markdown", @@ -671,13 +627,16 @@ "id": "J7HusxWkGjLX" }, "source": [ - "## Dataset API\r\n", - "\r\n", - "Для работы с данными в Tensorflow присутствует удобное API, которым мы в данном случае можем воспользоваться:" + "The code has not changed, but if you were running this code on GPU and on larger dataset - you would have noticed the difference in speed. \n", + "\n", + "## Dataset API\n", + "\n", + "Tensorflow contains a convenient API to work with data. Let's try to use it. We will also train our model from scratch." ] }, { "cell_type": "code", + "execution_count": 17, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -685,22 +644,9 @@ "id": "oYro9Lbr8q0M", "outputId": "78c0a6de-71bd-4eef-8819-439495b28672" }, - "source": [ - "w.assign([[10.0]])\r\n", - "b.assign([0.0])\r\n", - "\r\n", - "# Create a tf.data.Dataset object for easy batched iteration\r\n", - "dataset = tf.data.Dataset.from_tensor_slices((train_x.astype(np.float32), train_labels.astype(np.float32)))\r\n", - "dataset = dataset.shuffle(buffer_size=1024).batch(256)\r\n", - "\r\n", - "for epoch in range(10):\r\n", - " for step, (x, y) in enumerate(dataset):\r\n", - " loss = train_on_batch(tf.reshape(x,(-1,1)), tf.reshape(y,(-1,1)))\r\n", - " print('Epoch %d: last batch loss = %.4f' % (epoch, float(loss)))" - ], - "execution_count": 18, "outputs": [ { + "name": "stdout", "output_type": "stream", "text": [ "Epoch 0: last batch loss = 173.4585\n", @@ -712,10 +658,22 @@ "Epoch 6: last batch loss = 2.9458\n", "Epoch 7: last batch loss = 2.7311\n", "Epoch 8: last batch loss = 2.5332\n", - "Epoch 9: last batch loss = 2.3508\n" - ], - "name": "stdout" + "Epoch 9: last batch loss = 2.3507\n" + ] } + ], + "source": [ + "w.assign([[10.0]])\n", + "b.assign([0.0])\n", + "\n", + "# Create a tf.data.Dataset object for easy batched iteration\n", + "dataset = tf.data.Dataset.from_tensor_slices((train_x.astype(np.float32), train_labels.astype(np.float32)))\n", + "dataset = dataset.shuffle(buffer_size=1024).batch(256)\n", + "\n", + "for epoch in range(10):\n", + " for step, (x, y) in enumerate(dataset):\n", + " loss = train_on_batch(tf.reshape(x,(-1,1)), tf.reshape(y,(-1,1)))\n", + " print('Epoch %d: last batch loss = %.4f' % (epoch, float(loss)))" ] }, { @@ -724,21 +682,24 @@ "id": "A10prCPowHl7" }, "source": [ - "## Пример\n", - "Рассмотрим пример двухмерной задачи классификации на 2 класса. Примером такой задачи может быть классификация опухоли на 2 типа - доброкачественная и злокачественная, в зависимости от её размера и возраста.\n", + "## Example 2: Classification\n", "\n", - "Сгенерируем тестовые данные случайным образом:\n" + "Now we will consider binary classification problem. A good example of such a problem would be a tumour classification between malignant and benign based on it's size and age.\n", + "\n", + "The core model is similar to regression, but we need to use different loss function. Let's start by generating sample data:\n" ] }, { "cell_type": "code", + "execution_count": 18, "metadata": { + "id": "j0OTPkGpwHl7", "scrolled": false, - "trusted": true, - "id": "j0OTPkGpwHl7" + "trusted": true }, + "outputs": [], "source": [ - "np.random.seed(0) # pick the seed for reproducability - change it to explore the effects of random variations\n", + "np.random.seed(0) # pick the seed for reproducibility - change it to explore the effects of random variations\n", "\n", "n = 100\n", "X, Y = make_classification(n_samples = n, n_features=2,\n", @@ -749,17 +710,17 @@ "split = [ 70*n//100, (15+70)*n//100 ]\n", "train_x, valid_x, test_x = np.split(X, split)\n", "train_labels, valid_labels, test_labels = np.split(Y, split)" - ], - "execution_count": 19, - "outputs": [] + ] }, { "cell_type": "code", + "execution_count": 19, "metadata": { + "id": "c-_BjSHPwHl8", "scrolled": false, - "trusted": true, - "id": "c-_BjSHPwHl8" + "trusted": true }, + "outputs": [], "source": [ "def plot_dataset(features, labels, W=None, b=None):\n", " # prepare the plot\n", @@ -778,40 +739,45 @@ " ax.plot(cx,cy,'g')\n", " ax.set_ylim(min_y,max_y)\n", " fig.show()" - ], - "execution_count": 20, - "outputs": [] + ] }, { "cell_type": "code", + "execution_count": 20, "metadata": { - "scrolled": false, - "trusted": true, "colab": { "base_uri": "https://localhost:8080/", "height": 283 }, "id": "tq0vFchQwHl8", - "outputId": "9a5aa6a0-c92f-4d72-9e78-c0f615804bff" + "outputId": "9a5aa6a0-c92f-4d72-9e78-c0f615804bff", + "scrolled": false, + "trusted": true }, - "source": [ - "plot_dataset(train_x, train_labels)" - ], - "execution_count": 21, "outputs": [ { - "output_type": "display_data", + "name": "stderr", + "output_type": "stream", + "text": [ + "C:\\Users\\dmitryso\\AppData\\Local\\Temp/ipykernel_66332/2721537645.py:17: UserWarning: Matplotlib is currently using module://matplotlib_inline.backend_inline, which is a non-GUI backend, so cannot show the figure.\n", + " fig.show()\n" + ] + }, + { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEKCAYAAAASByJ7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO2dd5iU5dX/v2dmtrINdpfOsoBKV9AFsQRFNIgx1thiNJZojDH6mkSNKSbRn0l8TaKmGDUxRo2vFbEiYgsIEaWIIFUEgaXsLmzvOzP374/vPM6WqbvT93yua67dnXn2mTPzzJxz36eKMQaKoiiKYou3AIqiKEpioAZBURRFAaAGQVEURfGgBkFRFEUBoAZBURRF8aAGQVEURQEAOOItQF8oKioypaWl8RZDURQlqVizZs1BY0xx9/uT2iCUlpZi9erV8RZDURQlqRCRXb7uV5eRoiiKAkANgqIoqYgxQHs70NERb0mSiqR2GSmK0gecTmDjRuDNN4GdOwG7HZg+HZgzBxgzBhCJt4Th09YGrFoFLFoE7NvH+w47DDjjDOCoo/gaFb+oQVCiijFAYyN/5uQANt2TJgatrcCDDwKffALk5gKFhYDbDXz4IfD++8BZZwHnnptcRqGpCbj/fmDrVqCoCBg9mvdXVgL33QccfzxwzTWAQ9WeP/SdUaJCRwfwwQdcqFVUUK/k5wPz5wNf+QqQlRVvCfs5TzwBbNgAlJZ2VfrDh3PnsHAhlers2XETMWwefxz4/POeu5tBg4CBA4GVK4Fhw4BzzomfjAlOwqzXRGSUiLwnIptEZKOI3BRvmZTe0d4O/PnPwN//zt9LSnhzOIB//xv43e+AhoZ4S9mPqayktR41yvcOwOEAhg4FXnoJcLliL19vqKqiq2jkSN+vSYTGbvFi7o4UnySMQQDgBPAjY8wkALMAfF9EJsVZJqUXLFwIrF8PjB0L5OV5v58DBvC+8nLgscfiK2O/5uOP+TOQ/27AAKCmhrGFZGDDBvolA72mjAyuUD77LHZyJRkJ4zIyxuwHsN/ze4OIbAYwAsCmuAqmhEVTE/D228CIEf7dzyNHUidVVABDhsRWPgVAbS2Qlhb8OJuNFzRW1NRwlb9vH5CZCUydCkyYEFoguKkp9ICx7hD8kjAGoTMiUgpgOoAP4yuJEi7bt9MFHUjfiHAxt3EjMHgwvRJ2e3LFL5OavLzQ0jGNiU2wx+Wie+r11/mcmZn8EC1ezA/ID35A91YgCgpCc29Z2Q2KTxLOIIhIDoAFAP7HGFPv4/FrAVwLACUlJTGWTglGW1toit3lApYtA155hQtWhwM49ljg1FMZE1SiyLRpwLPPUjn6u1gtLcw+Gjs2+vIsWAC8+iqzgrpnAB06xKDTHXcE3k5OncodjbW68IX1mg4/PHKypxiJFEOAiKSBxuApY8yLvo4xxjxijCkzxpQVF/doxaHEmbw86plANDQAa9YAH30EpKcz0WXoUHoLfv1r4N13YyJq/2XYMODooxnM8XWx3G66bc48M/opmlVVwBtv8EPg67kKC7nKeP11733G0O3TeUdQUADMnQvs2uX7NTmdfE3nnqtppwFImHdGRATAowA2G2P+GG95lN5x2GFML21uBrKzez7ucgHLl/P3adNoEAB+R4cPZ8zvX//i7xMmxEzs/sfVVzNnf9s2pmValvzgQRaOfPWrVLDR5oMPuLIP5P8fOhRYsQI47TQGn5YsYcxABJg5k/ePGwdceCFXGx98wA9fQQFf06FDdJGdcw5w8snRf01JjJhgy7kYISInAngfwAYAbs/dPzXGLPL3P2VlZUab2yUe778PPPIIU027xxLKy4H33gPKyoCJE33/f1UVPRU/+lH0Ze3XtLcDq1dzhV5eTgU7dSqNwaRJsQnqPPAAA0+FhYGP27yZsQW7nXGFzEyuLioruYO47DIaMLebRu6tt1iTIAIceSSrr0eP1kCVBxFZY4wp635/wuwQjDHLAejVSgFOPBGorwdeeIGLv4ED+T2srfWmo44f7///CwsZcG5ooMtXiRLp6azePf54KlKR2CtMh4PPHQinE/j0U+4GOscN7Xa6v9rbWZQ2ciQ/WBMm6PaylySMQVBSBxHga1+jm3r5choBY2goLAUfKF3cZuM5rBigEgPi1VNk+nQGjwLFA3ftotHwl2mUns7MoUWLAq80lKCoQVCixrBhwAUX8Gbxl78AmzbRZe0Pt5sGxFcMQkkxpk/nhW5s9J0Oagw/MIcfHthoFRVx5eHvPEpIJFSWkZL6nHQSA86BqKwEjjlGv9f9gqws4PrrGfg9eLBrhlBbG/DFF/Q5jhsX+DzWtrKtLaripjpqEJSYMnEid/779/t+vKWFGYVnnBFbuZQ4MmUK8POf84OxaxewZw+wezeDTmefzQ9De3vgc7hcNAi6rewT6jJSYorDAdx8M/CHP3DxV1DAtjkdHVwg2mzADTdocVrCcegQW0s4HOxLEkrri3AYNw647Tb2M6mu5vOUlLD/0Cef0B0UKM5QWQnMmqVtdPuIGgQl5gwaBPziF8C6dUwpr6hgFuGZZzLwPHhwvCVUvmTXLuDFF6mQbTZvcOf004F58yJvGIYM6VmRPHmyd1s5bFjP/2lq4opi3rzIytIPSZg6hN6gdQiKEkW2bgXuvZdKv7jYG9RtaaFynjaNfYYibRR8UV3NbWV5eddt5aFD3E18//uciKaERMLXISiKkkC0tXGoRV5ez5SwrCz69NatA955h7uFaDNoEPsZffIJi84qK7mtPO881lEMGhT6uZqaaNDcbm5HCwqiJ3eSoQZBiQvxqoNSQmTdOirOoiLfj4uwpcQbb7AjYSz6A2VksDht5sze/X9DA/Dyy8DSpV37IM2cyR5H2otdDYISO+rr2ZLmrbfoAcjMZMxgzhzGKZUE4uOPgwdos7OZCXDgAKuEE5n6enZN3b+fjbIsN5fLxU6Ln34K/PSnfKwfo2mnSkzYu5eB5Oee42Jy9Gi2qFi6lPevXBlvCZUutLeHNnBGJDnGbD7zDLMXRo/uGvOw27kaMQZ46KHgrXpTHDUIStRpbWU80OXi9zE7m3okLY3fxcGDgYcfZhqqkiCUlASfluZyUYEOHBgbmXpLbS1XHIFW/0VFrH9IlpGhUUINghJ11q6li8ifOzozk+1oliyJrVxh4XJxbu8//gHcdx/w5JPsppmqK8rjjuNrC7T6r6hg7n+gPiSJgLXSCLTjsYJZ27ZFXZxERmMIStRZtowzEgIxeDDw4YfAt7/N2GFCUVHBNs3WrN/0dPbXeecd5sh/73up12djyBDOGVi8mNu67sq0poZK9Otfj4984RCqS8tmC220aAqjBkGJOrW1wZW83c4FaUtLghmE+nrgnnvo9yot7fqYMezT/6c/AbfemnqTuC66iEryzTep/LOyqFxbWxkAuv1234ViiUZhobdjYqC0Nrc7OV5PFEmxT7CSiAwcyKByoDYzTqdX5yQUy5ZxNTx6dM/HRFhBu3UrdwxHHhl7+aKJ3Q5cfDHTSleu5EXMyGCH0smTk8cAjh7N61Rb6z/e0drK1zZ1amxlSzCS5IoqycycOcCDD/qvHWpsZNq7zcaF9ogRHNo1ZUqcdY7bzcBGoF4aInQXvfVW6hkEi6Ii9hVJVkSAb30L+O1vmcnQ3b3X1kZj953vJNj2NPZoUFmJOkcdRZ1aWdnzsV276JHYvZuT1LKzmexx333A//4vjUXcaG1lMVOwbUtuLuMLSuIyfjzwwx/SJ7lrF2snKioYcD54ELjySmD27HhLGXd0h6BEnYwMfhfvvZffv8JCKv59+zh/ecAA7iKGDvUeP2gQR+0+9BBnK8elotnanrjdgYezuFwMNCuJzZFHAn/8I9tfbNrE6zp2LAd8p1pSQC9Rg6DEhKFDgTvv5LTEt97i4mzzZg7CmjKFRqEzlnv+009pROLSDjs9ncLt2BG49fKhQ+zb3wusrFVt4REj+tr+IsVRg6DEjAEDgJNP5q2qivGCkhL/ytAqXvvgg14ahMZGYONGZgoNGABMmhR+I7PTT2eW0cCBvgMaLS3cPZxwQsindDpZ0vDGG9wFiXBw0Lx5jNXGa7yxoqhBUOJCYyOTWIKtjDMz6eINC6eTPfyXLOHvFjYbZ3hefHHowcNJk7j6f+kl+rry8ym0MRSssZF1CP6q7rrR1gb87W8s1svP97YA2rmTLrXZs4ErrkieBB4ltdCPnRIXMjO9nQ8CGYX29jALYd1u4LHHGJwoKemqWV0u4L336OK58cbQtK4IWyyXlACvvMLot83G57GMxRFHhCzes88yo2rMmK6vu7iY9mbZMv4899wwXrOiRAg1CEpcGDqUrWXq6wNXMbe3A8ceG8aJt24Fli9nEVl334vdzpz0devYzXPGjNDOKcJjy8qYKtXWxiBkOD34wTT4//yHsRFfRtBm445h8WJg/nwaTUWJJeqtVOKCCHDWWYwldPbqdMbqqjx+fBgnfvttpjD5c8SLMB6waFHYMkOELR1KSsI2BoA3sSVQS530dBrBLVvCF09R+ooaBCVuzJxJb8zu3Vx4u928v6mJmUW5ucBNN4UZZN26NXj3zYICOu1j3La5oSH0bKKWlujKoii+SCiXkYj8E8CZACqNMVPiLY8SXUToK584kcVpn3zC+wsKgG9+kxk3tbVAXR3dSyG1tQhV48YhzzMvL/TmqIHafChKtEgogwDgXwD+AuCJOMuhxAgr5XLiRC7YnU4agZdf5kwTK6HH4QBOOYUdFALWEE2ezBiBVeXmi+pqFkCEMgAmgkyezKd0Ov3Hs9vamAAVlptMUSJEQrmMjDHLAFTHWw4lPtjtTAC66y4WsI0YwQBsSQmzOhcvZklAQ0OAk8ydS3+LP3eQMdxyzJ8fldcQiLw8GrU9e3zvFNxuoLwc+NrXNKCsxIeEMghK/8YY5ugbQ2PQeQGfns7Eob17geefD3CSww5jH/8vvuByuzMdHYwdzJoVt0Z0F17I2MnOnd6AutPJyu1du2jPzjgjLqIpSsK5jIIiItcCuBYASkpK4iyNEkk+/5wr5ECXdfhwYMUK4Bvf8FOfIMIAxMCBwKuvdjUKaWlMbTr77Ji7izqLcN11LEB7803gs88o8qRJtGMTJmgbCyV+JJ1BMMY8AuARACgrK0vR+YX9EyvVMpBCdDi4g/jiiwCLfJuNfpdTT+VJGxsZpR0/PiGitXY72+7389b7SgKSdAZBSV06OkJfuPurXehCRgZ7byuKEhIJFUMQkacBfABgvIiUi8jV8ZZJiR3DhwdX9MYw+FpYGBuZFKU/kVA7BGPMJfGWQYkM9fXeRqO5uUy5DNSiAqALKCPDm3oJAHZXO4bsX4fDPn8TefXlaOlw4OCYGShxnwygNMqvQlH6FwllEJTkp6ODWUDvvMPMT6uOwGYDvlpWjfOG/Rfp2zfz4IkTgeOP/7INRFYW57o/9hjTTXPQiGM/uB+F1Z+hNbMANWmD0druwpyMFZBfvceD58/XKKyiRAg1CErEcLmAv/8d+PBDKvQvi6+MQen2tzD8vmewbSAw6dhctqPYvJltqi+6iEOURTBnDt1Gzz1rcOT6h5HZuBPl2aVwtgnS0oCZJ9pRMGQELc/TT3M2Z1lZPF+2oqQMahCUiLFpE7ByZc/WziW738f0DU+idsgobKtPR2E7MGwYmBra3g78+9/cHsyeDRHahuNH7EbDDzdg/9DRyLYJiotZfPylkUlLY7XaSy8BxxyjuwRFiQAJFVRWkpslSxgv6Kybba4OTN74PBpyh8M40pGZySlhX5Kezmjyc89x1e8hZ/MqDBtpx9HHCKZPZ9fTHu0e8vI4mFkH3CtKRFCDoESMbdt6NhotOrgF6e0NcKaxM11WFlsJdWndkJXFWoHNm733VVcH798gwuBEU1PfhW9sZN+M1ta+nyuWVFez7HnfPm+7WEXpJeoyUiKGzdazR09WSzXS25swuGID0tsb4LKlQWwjANcQwNGt6KC21vt7fn7P1hPdMYaBi74Um23eDLz+Ov1dIrwdfzwHHI8Y0fvzRpvPPwcWLmQql/XGFxUBX/865zvrYGalF6hBUCLGpEnUq0OG8G+HsxVHbH0FQw98jLaMPLhtaXB2uDHJsRfydiZ7CnUeep+W5v195kxOoQ80Y7OhgU/WW8W9eDHwf/9HP5c1xszpZCDkv/8FfvhDvqhEY+1a4M9/piEcNcqr/BsagEceAXbsAC67TI2CEjb6iVHC4uBB4LXX2ITu0Uepm9rb+dhpp9F743YDMAbT1/wduQ170Z6eg/b0XHQ4stAiA5A9vICKfsWKTv8AtqS2KC1lYx9/8QGnk93hzjmndwHlrVuZpTRqFAcaW+dwOBjTyM8H/vQndkZNJOrqgIceoszFxV2Vfm4uI/rvvAOsWRM/GZWkRXcISki4XMALL3BRLcLFqcvFWfb5+cAPfsBWQaecArz7LjA1eweG71+D6sLxyGyrx4D6/ahHPvLzgQEDAEgWq9a2b2fZ8YwZdHlYiADf+x7wxz/SR15YSIXndtMQtLRw3FpYA5c78eabfBGddyUADdWhQ3yOigrm0V53XZAhDDHkww9pDP25yWw274jQsjLNvlLCQg1CClBVxWZvbjfT8ktLI68Hnn+eOmb06J79hurqgP/9X+AXv6CnorAQqP7jMtS1pKMWghr7UTjc2YiitFoMyM6B9bFz2jNg1m1Ey/zzkP3Ny3t+GPPzgdtvB1avpvto924+eVkZ+0QfdljvXmhbG/Dxx9wddH8hq1YxwGyzMevpySfZIO+882JfBFdfT3k++ojbsJISYMMGP21eO1FQwF7aDQ3Bj1WUTqhBSGIOHmQKvzV6UoRGYeRI4PLLgSOOiNzzLF7s2xgA1NstLSwJ+MEPGNfsWF2OA5tysP4LoL4lA58POxGNTTswuPxzHNrTBIcDaHc50Jxegservov2X+Vi/nzq+S6L9sxM4MQTebMi1n1Vyu3t3gwli8ZGYPly3m/FNTo6+JxDhtC95HZzZFssWLeOfrn2dr7BdjvwwQeMbwwezPiLw0GlX1/P/8nJoQGwguMhdQBUFC9qEJKUQ4eA3/yGeqxzXNEYJuv89rfALbdEJia6ciX1S6BOpIMHc9FdU0OPRYdkYOd2F9w2FpSJpKO1aAK2NR2OAztb4GwDikZmYnTaXhSNyYVUb8MX/+9DvP9wDWafPRCOE2b13AFEanWelcX6h/Z2/gS4C3C5uq6oOzqojNPSuDp/8UUaps6B8GiwfTvwwAN0oQ0Y4L0/N5dWeetW73tRVcWfVo+QgQN50dPSeLwSHk4nd2GLF3t3pEcfDcyZE52td4KhQeUk5bnnuDgcPrzrQleEOmHgQODhhyOzSNyzJ3hmp83GW7VnAOp/mmfC1ljfpVDN7QbK99vhzMyBLS8HTZWNaEgvQtlHf8Npq3+Dsub34dz8GfY9swy4+25avGgEdR0ObkUOHODfra0cxdY9TtDeDowdy9/T0qhwP/oo8vJ058UX+YZ3NgYWpaWUf+1aoLKSBquggD/z8xmkf/ttDlvoHh9RAtPcDPzhD8D99zOZobCQC4SVK4Ff/5oDl3zNPk0h1CAkITU1dC0HmiOfl0ddunFj358vPT20mie32+vFePXADKTnZSK9vfHLx5uavAPm7eJGfkcV9h1MQ379btQVlKIxbziksBAba0bAPWo0AyN/+EN0isXmzKE7qqaGiqC7C6m+ngp28GDvfdnZDHBHk4MHWRvROcDemUGDuJOxZm/62kFlZHAUW4orr4jzz39yp1haSmPgcHgr6UeOZCBt5cp4SxlVwjYIIjJAROIzf1ABwMUsEHyYjMPRrU1EL5k+nTGCQLS0cEE7fDiTc9ocA7Dq2BuQ2VKDnMYDEOP+Mlab7azHsNad2JAxA1vaxqAxz1tHkJbGhXlDo7C+YNcuroYjTVERcOut/H3vXhqd9nZardpaWtTjjuv6JlsWL5rU1fFN8ueaaGykIcvP57H19TRojY2UOy2Nk+JqayNz8fsL+/YxVbekxPd773BwcbBwYUpXhAc1CCJiE5FvisjrIlIJYAuA/SKySUTuFZHDoi+m0plAtVqdsYLMfWXqVK/+8SfP/v0s7rW8FCJA1eDJWHrSHagYPAV59eUY2FSOYR274LSl45Vh38VHOBbt6b7TOb+Ue+BA+nOjQWkp8LvfATfc4A3GFhXREMye3bN1RktL9OdepqcHXtnX11PO4mLGCsaO5Wp2+HDKPXeuNw6ya1d0ZU0l1q71BuP9kZvLmM3u3bGTK8aEstx5D8DbAG4H8Kkxxg0AIjIIwBwA94jIQmPMv6MnptKZoUOpMN3uwMWoHR3UeX0lLQ248UbgnnuoEwcP9j5vSwuNwZFHskupJZ+V5FJXMBqrZt6AjNY6lG+qx9YdaXAVDoaBIG/3Cgwb2rUPkaULv9TFVgO7aJGVRffRLbfQR+zvDWts5BYo2iM5hw9nTKCpyXcMwVoNuFwMund2afk6VgmN6mpvgkEgItU7K0EJxWV0qjHmLmPMessYAIAxptoYs8AYcz6AZ6MnotKd4mIq4IoK/8c0N9Pl7XcQfZiMGwfccQcnn5WX87Z7N5/nwgtpMKzvU04OF9id9XhbZj6yx4/CQcdQuIwNHU6Bw+7C+OzyLs/T1ESDkpXlucPp9BkcdbkivHM/4wyutnft6tJ1FcbQr3/oEHD99d5RbtHCbmfe7oEDvl9gTg6tcHa2/ziDRSL3Yko08vO7Xnd/GNPpw5l6BN0hGGM6RGQCgBEAPjTGfBklFJHTjTGLjTEhvJNKJLnoIuCuu7iDLSrqutNtaqKx+MEPIqu/Ro7kOWtquKCyujz4SmY56yxm7+3dy9kHNhsXvOPGMWbqcABnT6jFoOZ9aEwfDoD1Ym43O1Z8SVUVm7V5Hl+1ijVqe/fynFOncmcyaVIfMwKzsoAf/5g+4v/8hxYHoECHHQZcfDF/xoLZs9m8bulSXlzLndXeTpdRXh6LTPxtD+vr6UYaPz428qYCRx/Nax/IH9vUxN3b6NGxlS2GiAmyrRSRGwF8H8BmANMA3GSMednz2FpjzNFRl9IPZWVlZvXq1fF6+rhTXs7OCpar2EpFz89nxXC8B4nV1ABPPMEaKwur84TdDkzK2olzN/waNVkj0OZOQ0YGe9oVFnoO7uig5v/1r9EwaDTuu496ctAg6kRr8d7YyPjFJZdEqJ9bUxO3P04nhRk2LPb55243M1oWLfJaP4eD7q0jjmDRWnZ2z37j9fV843/8Y27nlNAwhm1SNm/myqc7Lhe/aFddBZx8cszFizQissYY00NDhGIQNgA4zhjTKCKlAF4A8KQx5gER+dgYMz0aAodCfzcIAD/HO3eywaXLRd01aVL0k2HCobLSK19hIXvYHTzIwltZ/AYmfPw08scMRNG4AjjSxFtdV10NXHopzLzTcd99TKHt3m0C4Hm/+AK48kr2UkoprPfC5WJQ09ry7djB1cCBA11XtYMGUWmpMQifhgb4XHVUVdE3evrp3CmmQHFaXwzCRmPM5E5/54BGYROAU4wx0yItbKioQUgBjOEW4qWXuCq32bg6Li0Fzj4bmDYN5eXAz3/Onbq/72JzMz0qv/99YhnDqOJ2M7V0927+PmIE/W3B8pEV/7S1eXtn7d3LD9y0aWzlO2FCShgDwL9BCOWrUyEi04wx6wDAs1M4E8A/AUQ5B09JeURY6DBtGrcSVjR88OAvv3zr1gVOzQf4LwcPcqcQK1d/3LHZ6D6KVNMqhTuwE07gze0OnoqaYoRiEC4H0KUBgjHGCeByEXk4KlIp/Q8R72SdbtTVhdaFQSR4AZ2ihEw/HDAUSpZReYDHVkRWHEXpycCBoWUEut2+U/eTjpoaNrBra6Mfe+LE4POlFSUC9Bdvq5LEHH00m/kFyghsbGTAOhKFeHGjuZkjPf/73641CJmZnAw3b16/cl8osUcNgpKQGMPEGrudhWqzZnFYmK9WM04n6y6uuy6Jd/ltbcxw2b6daY+dA8NtbcBTT9HqnX++GgUlaoRsEEREAFwKYKwx5k4RKQEw1BgTsX7AInI6gAcA2AH8wxjzu0idW0kO6uuZjvrGG8y2dDhYmzB7NvXhhg3MvszP5yL64EFmF11wAXD88fGWvg+sWEE30ZgxPRV+RgZTrF57jZbRV568okSAcHYIDwJwAzgFwJ0AGgAsADAjEoJ4Oqj+FcBpAMoBrBKRV4wxmyJxfiXxOXCAozhratieo7SUu4Q1a6gvL7mE2X9vvslsIrud/dxOOSVwSmrC43azAK1TZlUPHA7eli4FLr00tvIp/YZwDMKxxpijReRjADDG1IhICN2gQmYmgO3GmB0AICLPADgbrHdQUpyODhaKtrV17Qxgt7PYrr2d7vWf/MTbtTplaGxkEV5JSeDjCgpYSasoUSIcj2uHZxVvAEBEisEdQ6QYAWBPp7/LPfd1QUSuFZHVIrK6yhofqCQcLhdX+jU13rZAgfj0U5YhFBf7fjw9nQk3r70WWTkVRfESzg7hTwAWAhgsIncD+AaAn0dFqgAYYx4B8AjASuVYP78SmNZW4P336QGx5ifk5wPz5/seMWCxfHnwlNHCQg60qqvjOVOGnBy2Smhs7DnGszO1tZx3kAg0NbGid8MGWvxx4+i/+7IRVQRoaGDDLrebNSrBursqfSYkg+AJKC8DsAbAXAAC4BxjTCT3r3sBdO5UM9Jzn5IkNDXR7fPZZ/z+Wh6Qpibg3/9mltAPf+hb8dfXB+/MahWNtrSkmEGw2YCvfQ147DG+Ob7iCE4nFe9XvhJ7+bqzZg3wyCP07+XkUP5PPgEWLGB67Fln9S2gU1fHc63oVObkdnMWxYUXssWuEhVCchkZNjxaZIzZYoz5qzHmLxE2BgCwCsDhIjLGE5u4GMArEX4OJYo8/TQb7Y0d21XpDxjA+3buZPakLwYNCl5l7HYzHbUvxWetrRxt0NDQ+3NEheOPZ6+c3bt7+tja2thp88wz459htHEj8Oc/0yKXlnLVPmgQrf+IEcALLzBFrLfU1gK/+Q23jMOGsZuhdduyBbjzTmDPnuDnUXpFOC6jtSIywxizKhqCGGOcInIDgDfBtNN/GmMiMAuADNAAACAASURBVCJeiQU1NaynCjSTZcQIppRecEHPrs2zZtHN1NLCZJqiop7upaoqtjzKzQ1fvv37mZ20YgX1rTEcF/C1rwFTpiRAhlJGBnDzzbSqy5d7C9NEOKvhW9/yjqSLF8ZQvvx8No/qjsNBw7BwIXDSSb2z3M8+y1zi7gF2m40FKYcOAQ89xGEgSVt0kriElWUE4FIR2QWgCXQbGWNMhGZyAcaYRQAWRep8SuzYupU6LFCjTbudOmXrVhoAgP+zZAl1yJ49fCw7m3qwpITKOi2NhqKlhQo8XLZtYxdUY+jKcjj4+969wL33starr16OsLDmjhrDKLo1Azkri62rzzuPb0R7O63fxInRn9QWCrt3800LlA2VlsaUsY8/Bk48Mbzz19TQrxhoVVFYyN3S9u3a1C8KhGMQ5kVNiiTDGCqvigqv4go02rY/0N4e/rHG0FX8yiv0hMybxx1EfT31ys6d1BFjx9KYfO97jF2GQ1MT8MADdHVbehfgdSss5GJ3wQKed8qUEE/qcvHEDofvlbI/GhuZJvXee4wJWMyaRd+7lWJVUAAce2zo540VNTXB284CvHgHDoR/fmvSU7D23SK08moQIk7IBsEYsyuagiQLO3dyCtjOnd4gpzGcXfytb/lPm0x1OivbQIh4jy0vB15/na5oSwfMnk1d8vnn3krkuXPZlsJPM9SArFrFFkH+rovDQXkWLQrBINTVsTBsyRKe1BhO+znjDAY8AynKhgZW3e3Zw6CoNYDa5eKqeP164Kc/pd88UbG2VsFwuXrXjM9qNx0Mmy20XGYlbMJpXXGHr/uNMXdGTpzEZvt24He/486+c2Ws2816obvvBn72s/5pFCZO5GK5tdW/Lmht5Xs3cSL/XrqUi8nOC0KHg7sFK3ba0MDFeG/f0xUrgmckFRby+gXM+qys5MWvqfGmQBpD188f/kD//qWX+vdrP/cc3S3du+/Z7XyxlZX0jf/qVwkQ0PDDmDGUt6PDfz9yY/iFsC5yOBQXewM8gd4Dp5Nb8i1b+OHIzuYQjFB6pCsBCcdl1NTp90wAZ4JzlvsFLhfw8MNUGAUFXR+z2bjo27eP1bQ33RQfGeNJRgZd348/TmPZfWqZ00mPwDHHMLvS7WbsNFhqeW4uXdfNzYFT9P3R0hJcT4jwGvp1e7nd9Du1tHQto7b8TgMHctcwejS3ON2pqwsecS8u5htkpWklIgMGMFj89tv+28pWVfF96M1rGDmSRufgQf/1DM3NDCw/8QSzryyys5mF9dWvarC5D4TjMvpD579F5PdgRlC/YOtWftYDtVceOpTTvQ4e7J81NHPncpX98sv8TlqGs7aWC2mnkxXJVvLJxo3UtdOn++7pFgmGDgU2bQqc8NLRwYWv32O2bvW9urew2bhife01BlK7K6SdO7nqDeQbt/yPW7dGxiBYw7bffZd1A04nUzdPP52pWum97Dpz/vlsJPXZZ3RvWdvBjg76+nJyGOzpzcUUod/1N7+hFe/uh2xpoWEdMICPZWV5H2ttZU5zZSVw2WWJu8tKcPrS/jobLB7rF+zYEXzhYcXb9uwJbBA6Orjbra/n9/KII1Kj0EqEsdFjj+Xqf5OnC9XgwXytY8d2/Q6PG8dYwbp11JWdF98WDQ18L8OJ3XZmzhwW1AbyQlRUACefHCCR56OPgitQayuzb1/PWoHqan4oNm3iqjYjg2/GqFFd3xC7PbzovD+MoXFasIByFxfzw3noEPDXvzLu8T//07stV1YW8KMfAW+9xTzeqiqvMTvpJKaB9aVaedw44JZbgAcfpOFJT+e529poEHJygLKynlvQzEyuKt5+m49PmtR7Gfox4cQQNsDTxwisEygGcFc0hEpEgrk1Lawgs79zLF3K2p2mJu85RViAevHFXfVDsjJsGGsNAO4AbruNmVjdX9vo0TS0AwZw59B9DIAx1DdXXtl7L8CECdR/O3bw/N2vYV0dn/O00wKcpLExtBW1zdbVjQFwJf3oo1RuhYV8UYcOccuUnc1WrVaAxOmMTFD5ww8Zs+juuxs4kNu2HTtYaXzzzb1bSWdlMU/39NO5Ine7+doiNa5uwgTGZTZu5I7J5eLu7KOPuJLqbgwsbDYajLfeUoPQS8LZIZzZ6XcngArPbOV+wahRwRMbrKEuQ4f6fvy11/g9HT686w7C5QKWLaOO+NGPEiPlPFJ8/jldaL5W/3l5VNZbt/K9q6z06kOXi1lIEyb0bc6B3Q7ceCPwpz9RN2dlUW91dHDXkpPDBWnADKYhQ5hXHwjr4neumquupmIrLqby//xzCmSz8fjmZuD55+n7HjKEF/7IPpb1uN3ASy9xW+ZLcYrww7x+Pd/gUaN6HhMq6enRq5xOS6Nra9o0731PPBF891FYyP5Koa7glC6Es+663hizy3Pb66ksvidqkiUYkyfzu97U5P+YqioqMF+tVg4c4A7e10rZbuf9W7bQ1ZJKHDrk/3spwoXckUfSU7JjBxfSX3xBl/1XvsJFbF8NZG6ut2321Kk0CMOGsQbsnnuYoBKQY4/l6j1QymV1Na1b54KU5cv5whoaqPxtNiq69HS+KGtF/e67XA1ffHHfZyeXl9MHFqicW4QfutWr+/ZcscbtDr5VFOk6flQJi3B2CKcBuK3bffN93JeSpKVRgdx/Pxd83d2v1dVcIH7zm77/f/lyrz7whQh1yaJF9HunSqJEsNR1ESpky202YQL15YQJPbO5fFFdzZvDwSQef++v3U7j0ytPwsiR9EuvXu17Ek9LC7cb3/9+1/vffZd9fpYu5cXNy+M2sKOj6wVuaOAH6qSTeiFcNyzDE4yMDL5xycS4cTR4gXYJdXXRy1DoBwQ1CCLyPQDXAxgrIus7PZQL4L/REiwRmT6dK9bHHuMq1mbzLkaGD2cnT39V/Z9+2rN/T3dycph5WF8fmjJMBqyEGWs+si+sBJwzzgjdA7F7N9tdfPKJ9zpkZ9Ot/dWv9j6JxiciwNVXU5GvW8cnys31Dn2w2ZhZM2FC1/+rr6el6ujgbiAtjT+t4grAuyNIT+fFHzOmb7JmZ4e2Qm5vD/6BTDTmzWM73UGDfCt8Y3g9Lrss9rKlCKHsEP4PwBsAfgvgJ53ubzDGJNkSo+9Mm0a38KZNTCgRodIbNy4yq/pUW9gMGsTODB995N9dXVFBXRooTb8z27ax6DctjQbEet9bWxmj2bKFcYOIGoWsLBaYbNvGlf+uXVS+c+b4nwOQl0ffV+eLarMxpcxKK+vooGGx2Rhs6atBGDmSW9iGBv9uI2PoAisr69tzxZopU7gq++QTrrw6f+HcbmZyTZ3KqnGlVwQ1CMaYOgB1AC4RkYEADgcL0yAiMMYsi66IiYfDQb93OPG/CROYERfItdvURB3Sm26eicwll1B/7tpF372lqDs66EEpKAC+853QjGFbG7sv5+X1TFO3Mg/Xr+d7fcYZEX4hNhsvZPedgD9OOYVpnoF8Zs3NPF+wOoUQaGgAVq2yocl+DiYu/xuyJmRhyHBHz9jy3r1Urn0JKMcDh4M7saee8gbb7HZvtsesWcDll2vFch8IJ+30OwBuAmsP1gGYBeADAKdER7TUYvZsYPFiLsz8Zc1VVLAup496IeHIywNuv53xkXff9fZ1E2H+/5lncicRCuvXU/H5qxETodF54w2mksZVN5xwAvDMM1y5+sp6aWvjxR4xgtH3QFWPATCGmZbPPUfdmJV5HMpzKzB12UvYkZGJ8ScWY/BQO/3rVrfA7343ObejmZl0351zDt13dXX8gB11VP/sGRNhwgkq3wRgBoCVxpg5IjIBwG+iI1bqMXw48PWvMyNw5MiumTNuN2NlY8cmxkCsaJCbC1x0EdPXKyu9nZ/DTV3/+OPgRWpZWcz4OnAgzovgwkLgl79kj6OKCv7tcNAiNjfTGBx3HEu5Z8wI3Sp24513mJFZUmLtvgQVg89BxxFTMHzzO3D/Zw0ypriQP2UUM5mmT0/+3ObCwsQZJ5pChGMQWo0xrSICEckwxmwRkfFRkywFOfdcKrOXXmJMz/IkiFAfXH554hemNTQwHvDppzRkhx/OOoFQdZnVGLC3dE/Q8YfN1rXDdNwYP54Nnm65hZFwh4OrXCs/uaaGRSn+0tOC0NzMncGoUd1iJiKoLjwc1ScejkOHgKVFBr+4Q5JyU6DEjnAMQrmIFAB4CcBbIlIDQFtih4HNxmHzc+Yw7byujgu18eOTo/fRihXAv/5FRZuTQ0O2YQPw4otscXPGGdH3QowaxdY8gXC5aKx6ueCOPJMnMzVtwQL247YmBdXV0Zd47rm97l3yySc0koEW/IMGATu/EOzdy6detYq7tJwcNhuMVEKEkvyE09zuXM+vvxKR9wDkA1gcFalSnMxMfhGTibVr2e11+PCutVODBlEhPfMMldKpp0ZXjuOOY7ppoDTWigruuBKqP9SQIcD113NHYE1WGj68zxkEe/f6j0lZWLVaDz/McIbNxmvY3s64VmkpcMMNybEoUaJLyOsCId8SkTuMMUvBwPK0YP+nJD9uN0fpFhf7LqS10j+ff56pn5GkqYktJ7Zt44K6uJh1Bl984buVSG0tFeBZZ0VWjogxcCDdRePHRySdLCMjtJYqW7dyNzF6tHfC38iRNAYVFazYrq/vszhKkhOOy+hBAG4wq+hOAA0AFoCBZiWFCdSPyCIjg8bg008jk95eX8/RmkuXUuFZTQNnzfLOP37zTe8MepeLzz9wIJvphVrTkOyMHx+8Dq2ykklMJ5/s26U3ZAgN7LJlzPhS+i/hGIRjjTFHi8jHAGCMqRGRSJb+KAlKTU1ox9ntzO7pK3V1HE5WUcEUUit11OWi/3vDBk6mO/VUYOVKuk0yMlg0OHlylFJNnU5WI65fz3TRUaPol4pzte9hh9H4BZrBsW4ddwOB2iQNGUIDO39+6qU9K6ETjkHoEBE7PC2wRaQY3DEoKY7DEVqw2O2OTHXwM8/QsHRvA2Kl7FdW0h9+xx0xWtHu3An85S/s/ZOeTkGWL6egZ57JnPg4RWWtrhm/+Q0r54cM8Sr09nYW/tlswYt3rVTd5ubUK4xUQiccg/AnAAsBDBaRuwF8A8DPoyKVklCMG0eDEKwfkTF0YfSFmhqu+gP1NCoupo7etavXtVyhs28ftyuZmT19Zk4nI9wA54fGiZEjaRwXLuQOyuqubbez/U9xcXC3knX9dHfQvwmlud2TxpjLABQBuBXAXAAC4BxjTL+Zqdyfyc9nrcGKFf6b9x04wLnqffXd79zJn8GmTQIMNEfdILz0En/6cg05HDQSr71GB30c81yHDuVO4ZJLuNK35nxnZfH28suBB6RZBcwh1cE0NzPt7N13GcXPy+PrnzEjckNylLgQyj73GBEZDuAqABUAngYb3lWISKJkeitR5uKL6TbfubNrJlF7O1fqubnANdf0vQ4hWMaMRUwKz2pr2fI60PQcK+dz5cooCxMaBQUsFhw3zqvcTziB18VfBpjbTYMwf34I12/vXuDnPwf+8Q+vC62ujsV3P/0pPwxK0hKKQXgIwDsAJgBYA2C152b9rvQDBgxg9s5557Faec8ettuoqmIg97jjqDu3bOnbfBIrMBqoHxzA5/A3mS5iWNN9gsUHsrISWhEWFQFXXEHvV01N1/e2qYlGfs4c4Oijg5yosRH4/e+5CigtpfXJzOQWcvRonvjee2lIlaQklG6nfwLwJxH5mzHme9EQQkQuAPArABMBzDTGqKFJQLKzgbPP5kqyqor1AS++yFTTjRt5jNWj6NpruVINl9JSZhbV1fmfCdHSQlmmTOn1SwmNUB3qbnfCd9j8ylf4fr74oneWhzHU5VdcEeJQpo8+orL3l388aBDbc6xYAXztaxF+BUosCCWGIIb4NQbWMX2Q41MA5wF4uA/nUGJEejoXi48/TiXT3Y9fW8s47O23hzCeshsi7PhqzTvo7pJua2PmzDXXRHjegS+ssuzW1sA5my0tXWf/JihTp9KI7t/PXV5GBt2AIQeSlywJHicpLmbr1VAMwr59dLXt2cP3t6yMQkb9wir+CCXL6D0RWQDgZWPMbutOTw3CiQC+DeA9AP/qrRBWcFq081ZS4HYDjz5KY+CrPURBAY/517+Au+4KP64waRIn0z38MPPrMzK8PnCHg6NMZ85kRs3771O5DRzICZSTJwdv5RAy6elM01mwgFbP1wupr2dQNZzhGHHE6pjRK2pqus6M9kVmJgtIAs0/7ugAnnySF89mo9V3Omkc8vM53cgatafElFC+OqeDAeWnRWQMgFpwQI4dwBIA9xtjPo6eiF0RkWsBXAsAJf5SXpSosn07XUaBKpcHDqT3YOfO3n23jzoKuO8+FlVt3kz3xtixXETW1TGuWVXFzJmMDOqgtWuZgnnzzYHH7obFvHksSNu8mZrU6iLndtNatbcDt97aP1a1Awbw9QZKRerooFHwZwyMYa/upUtpZLsfV1vLPhq/+hV9h0pMCSWG0Aq2rXhQRNLA9NMWY0xYkSMReRuArzDgz4wxL4d6HmPMIwAeAYCysrK+uKmUXrJ/f/Cgr7WY3r+/94u9jAzg2GN5s6ivpzupo6Orq8oqptq3j7rkqqt4n3dGQC/JyKCFWbSIrpDWVioxl4v+l298o2/9vJOJk05i/mqghVhlZeA5BXv3cmfgyxgA3F42NzOV95pr+iyyEh5hba6NMR0A9vfmiYwxUe6DqcSKcFxAkfYCrljBHYIvHVxVxdqE8nLuTAYP5mJ2/nwu9Hsd983IYIvqM87gyZ1ObkH6W3vQE04AXn+dASRfRQ3NzVwpnHyy/3OsWEGfXqAI9pAhdB9dckng4gkl4kTK25o0VFdz1QrQAxDnVjRJiTWFzNdUSAur8jWSE8uMYb8dX5MS9+5lEkxmJnV1YyPdS62tHCCzYwdw/QVVcHz0X7b9dLmYBjV7NoUMxXJlZDDBv79SVAT84AfAAw/QKg8eTCvrdNJn53IB3/9+YFfPnj3Bi9esKHdtrRqEGJMQBkFEzgXwZwDFAF4XkXXGmHmRfI6KCrZnXrvW+903hsWVF1zQ/xZ7faG0lCv0Q4f8v29VVdS3gVpQhIvLRR3RPauprY1Dc3JyqJ9cLubXAzQQY0oNml99F1VvPEVdVVDAD8HSpcDbb7NL3iWXaN+GUJg6FbjzTs7tXLaMxsBu5+7h1FODrwBC6dcNMEYTsewAJVR69Y6LyFBjzIFICWGMWQj2SYoKBw4Ad99NxTFypHe36nLRQGzdyu6ZOqM7NEQ45/zuu+kyLi72Gtn2dhantbQAJ57IwHJJSWRcR3a7dyRxZ11RXt61FKB7WcCIfR/h6L3/wua8URg6M90rS24uPwRvvknf0vnn913IRMUYrs63bOG2afBgZkYFG1Dti+HDgcsu49jP1lYq+VCV94wZtN6BVmCNjdzmBctoUiJOb03wIgDB6hoTAmOYIul09ky3s7pn7t/PnPof/zg+MiYjJSXM9HnySfrtAb6PO3dSzx5xBNPWlyxhLcK11/b9+y3CiugPP+x6Lffv7zpCsrmZzw8AYtyYvOl5tBcMQUNL+pdFbV9it3O7s2gRJ++kYqvPgweBRx7hhbIqr10uRtvPO48Blt5YbLs9/N5F06ZxK2el63bHGG7nr7pK53rGgd6+40lTMFBezoraQMpo6FBW2x6I2J6nfzBqFIvP7r6b33Oru+bXv86up5Zrqbwc+O1vGb/pK3PnchfS3u69z+326rOODv5uJcIMrNmB7OZDaE+nL9pndpTDQQX5ySd9FzDRqK3lm797Ny/G6NG8cKWlXKU/9RTw6quxkyczk3UG9fX8wnXuc9LUxGDP8ccztqPEnN4ahL9HVIoosnNn4OAnwMdEvJ02ldAR4Yp7/Xpg+vSe+f8iNLh1dUxQCURzM/XWnj1dFX5nSkuBb3+bRqaykvpk4EC6AxsbqVPKyrwL14zWOhgRuFxccPodRu9wRGa6T6KxeDEt8dChPb8E6ek0EAsXchcRK444AvjlLxmPKC/nBd+9m0b5yiu5ndR4TlzorcvoxYhKEUWs8Yuh0JembP0Zq9FnoLTOYcMYgzz//J5u6+pqpp2//773GmRkAKedxh1H9+PnzKHL6PXXOT0tLY3G4PDDeeucOeaypwPGoLGRriu/rm6Xq3f+9ESmtRV4773AWT9WCugHH3BrFytGjWJGUl0ddzEOB42WGoK4kvIxhECdiztjjMawestnnwV3JaeleYt7O9c1VVZy2ldDA/WBZVRaWzmKYP16xna6n3/8eN4aG7mzWLiQKe7d3dLVgw5Dc3sasgZ0YOxYPxbLypGNere8GHPoEINnwSrzcnJ4EeNBfr7v/idKXEj5GML48VwxNjb6P6aujoajP6eY9wW7PXjlskXn3ZoxwEMPUfmPGtV1h5GZCYwZw67Szz/v/3w5OTTkV18NnH466xF276Z7urwc2L43C+UTT8VXxpQjK9OPkPv2sQlSr5v8JCjhbI21j5iCfhBDsNuByy/3zovtTmMje3ZdfrkmNfSWyZO9ef/+aGujG6jzLmzXLsYQA+3ihg/3NrALhMPBLMh77wUuugiYNYvupp/+FLjwqXOQNesoBok6n6ilhb2gi4oiM90n0SguZjqtv8k4Fo2NSdOcT4kuvXIZGWMejLQg0WT6dCY2PPooXRTWDrq9nSvMm2+mUlN6x4wZwP/9H/Wrv75n+/dzlkLnoO6WLd6Avj8cDu4kduwIPigeYFB7Xo+SxnRW2P73v0wv3e1p2jtgAHsRnXxyalbEpqVx2/T889xu+cJqITtzZmxlUxKSflMKeMwxdBGvX0/lAjAAOWVK/2hUGU0GDAC+8x3gr39lu/zOLmGnkx6Z0aN7Kuq2ttB3ZX0el5mWxuZsX/kKUx6NYc1BqlfDzp3L6sudO1l00/n11tczqHPddalZf6GETYp/G7qSkcHV7IwZ8ZYk9Zg5k37/p5+mK8ha9YuwYvmii3om8QwZEryLgRXvjdj8epvN/yi2VCQri1H5F15gmpeVdmcML8DNN3MLrSgIbWJaKF9Fd7jtsJXQsDoObN7MFXVxMV0niZgheeSRTC3fuZOppA4HW1/7Kki1jk9Pp+vO3y6tvp5Zk937FylhkJ3NINm553J73NHBTIsxYzRwpnQhlB3CPs8tUMTNDkCn1USYgwc5Neyzz7p2HEhLY8eB009PvDioCI1AKDMQsrPpwn/ySaaidq9jaGmhYfnxjxPvdSYlubmhBWJiSGOjN5anrtv4E4pB2GyMCbinFJGYTUzrL1gdB5qa6H/vrBDb2xnEbWsDzjknfjJGgtNO4+t50VPqOGAAd0WNjVQQN9zAXYeSOhjDWN7rr3OxY7MxG/Ckk/h50Hqg+CEmSAK5iGR6pqb16ZhoUFZWZlavXh3rp40JzzzDJpz+hnE5ncy5v/fe1GjdXV3Niuft26kcpkzp2oJCSQ2MofF/+WUmHwwaxMVORwd72jkcwG23+U+KUiKDiKwxxpR1vz/UEZr+TnqlMeaxeBiDVCbUjgMizKQ866zYyRYtBg3iQDIltfn4Y1agl5Z27VKRlsbW9LW1wP33c6xyZmbcxOy39DWi9OuISKF0IdSOA7m58es4oCjhYgx7Vg0c6L9lUUEBOwesWxdb2RQSSpbRen8PAQixU5ASDqEmfhijSSJK8lBbywy0kiDpJ7m53PnOmhUbuZKKxkbOtWhvp8/t8MMjWksTypmGAJgHoKbb/QLgvxGTRPmSoiJm4ASq/AXYhUE7DijJQns7FzDBMsbS0oK3Qul3tLay4nzp0q61JHl5TNU78cSIpOKFYhBeA5BjjOmxiROR//RZAqUHaWms6tWOA0oqYRVDu1yBu1w3NQETJ8ZGpqSgvR144AH2eulebd7cDPz971wdRiAIF9ThYIy52hiz3M9j3+yzBIpPTj2Vu8Fdu3q2baivZzuIK6/UjgNK8pCdTTdQRYX/Y4xhOrUOTOvEihXAxo30tXV3D2Vns1Xw888HfmNDRD3QCUpmJvDDH3IYzIED7Me2ezcNRHo6Ow4cf3y8pVSU8Jg/n922fXWvtaryJ07kYkgB36zXX2dxhj+XUFoafXErVvT56UIJKq81xgQchhPKMUr4WB0HzjsP+Pxz7hQKCrTjgJK8jBzJyvMHHmA2XUEBF71NTfR+TJzIQWr6+fZQX88inWCR+IICjg8877w+PV0oMYSJATKNAAaXdeRRFMnJSbiOA0osaWqiYkhLY3/vJO/jMWECCypXrWI2UWsrW53MmcNxyzpFsxOhTp4CIjIDOBSDMCGEY4L0rFQUJWz27wfeeINa0xh+4YcNA848k874JF5G5+TQAMyZE29JEpy8PAYKm5sDd7Ssq2Npfx8JpVJ5V+e/ReROz/+tA7DOGLOtz1IoitKVHTtYrut2c9i0NSmovh7429+YcXLFFUltFJQQsNvZxfKZZ/ynHLpc9CdHIBIf9qfJGHMHgAcA1AE4V0SSZpymoiQFbW3s35CV1TXNUITFSGPGsLfJBx/EV04lNpx0EmMI5eU93UIdHRwDO39+RGaCh2wQROQBETovjTEVxpg3jTH3GGOu6asQInKviGwRkfUislBE+tEEE0XpxiefcCfgb5CPzcbBGK++Gp6PWUlOsrOBW29lFeru3TQA1s+qKuDCC4ELLohZYZpFA4BXRORiY0yTiMwDcIcx5oQ+SwG8BeB2Y4xTRO4BcDuA2yJwXkVJPj78MPiM59xcKoWqKu0X3R/IzeVg+IoKTstqbWVHyClTIjotK2SDYIz5uYh8E8B/RKQdQCOAn0RCCGPMkk5/rgTwjUicV1GSEqsMPRDWxKT29tjIpCQGQ4bwFiXCcRnNBXANgCYARQBuNMa8HwWZrgLwRgA5rhWR1SKyuqqqKgpPryhxZsQINjELhDWMOl8zvpXIEU5Q+WcAfmGMORlcwT8rIqeE+s8i8raIfOrjdnanY34GWx6JlAAADjtJREFUwAngKX/nMcY8YowpM8aUFRcXhyG+oiQJJ5zArJFA8YGKCuDYY7V3iRJRwnEZndLp9w0iMh/AAgAhNVAwxpwa6HERuQLAmQDmmmBj3BQllSkpYdfCDz/kJJnuwcL6ehoLnSikRJheN9I2xuz3uJH6jIicDuBWACcZY5ojcU5FSVpEgKuvptJftYoVyjk53DU0NHCu6C23sA+EokSQPk1WMMa0REiOvwDIAPCWJ7N1pTHmugidW1GSj4wM4Prr2c3w/fc5QDsjgzuHo48OPChDUXpJ5Ebt9AFjzGHxlkFREg4RuoxKS+MtidJP0Lp3RVEUBYAaBEVRFMVDQriMFCVROXCALvyNGxnjnTCBPcSGD0/6LtSK0gM1CIriA2M4qGrBAir+gQN5/zvvAG++yYzPb3xDm40qqYUaBEXxwbJlwLPPsiQgLc17f24ui4RffZWZoFoKoKQSur5RlG50dAAvvEC3UGdjYGG3c675yy8DLZFKvFaUBEANgqJ0Y9s2thIKlOqfns6xBRs3xk4uRYk26jJSlG7U1YV2nAhQUxNdWRKGtjZg3ToGUaqr6S876SRgxozgrbqVpEENghJzqqqArVupYwoKgMmTgczMeEvlJT09tOOM6ScFw5WVwO9/z5/5+bxYdXXA448z6v6jH/kf76gkFWoQlJjR0AA88QSwejUnAYrwlp4OnHsuMG9eYqRyHnEEs4ecTv9jCSz5J0yIrWwxp6WFxqChoWvFdFYWjUNtLXDvvcBddwGFhXETU4kMGkNQYkJzM/XK2rUMyI4ZQ/0yejQHPz31FLBwYbylJHl5rDUoL/ffgXrvXqCsDCgqiq1sMWftWu4M/A1lKSjgQJ9ly2IrlxIV1CAoMWHpUo6AHTWqZ+5+RgYNwyuvsBAsEbjoIq7+d+7sOqumuRnYsYOv49vfjp98MeOdd/zPdrYYMgR46y2d75wCqEFQoo7TCSxeDAwd6v8Yh4PpnO9HYwZfL8jMBG6+GbjiCv69ezdv7e3At74F3HZbP4mlVlUFn9mbkUHXko7zTHo0hqBEnbo6uqCtal9/5OcDW7bERqZQyMgATjkFOPlkzqQBWJhmt8dVrNiSk0NFHyjS7nTyTfFVtKEkFbpDUKJOIgSK+4LNRq9JQUE/MwYAgymHDgU+prISOP547eORAugVVKJOfj5vTU2Bj6utZQqqkkAcdxy3Sg0Nvh9vbWVp96kBJ+QqSYIaBCXq2O3A/PmcC+8v7tjRwcdOPDG2silBKCgAbrqJPrPycl4ogA2d9u1jFsB3vsOmT0rSozEEJSbMns3xwNu3cxRwZ9dLSwuwfz8zewYPjp+Mih8mTgTuvBN4912mizmd9AMeeyxw2mlalJZCiEniVLGysjKzevXqeIuhhEhzM/DMM8CKFSzsssjJYSvp2bOTP96Q8jiddBNlZGgQOYkRkTXGmLLu9+sOQYkZ2dnAVVcB55/PBnLt7YwtjB+vuiVpcDj6Sb5t/0QNghJz8vPZE01RlMRCg8qKoigKADUIiqIoigc1CIqiKAoANQiKoiiKBzUIiqIoCoAEMQgicpeIrBeRdSKyRESGx1smRVGU/kZCGAQA9xpjjjTGTAPwGoA74i2QoihKfyMhDIIxpr7TnwMAJG/5tKIoSpKSMIVpInI3gMsB1AGYE+C4awFcCwAl2lBLURQlYsSsl5GIvA3A18ysnxljXu503O0AMo0xvwx2Tu1lpCiKEj5x72VkjAm1YfpTABYBCGoQFEVRlMiREDEEETm8059nA0igQYqKoij9g0SJIfxORMYDcAPYBeC6OMujKIrS70gIg2CMOT/eMiiKovR3EsJlpCiKosQfNQiKoigKADUIiqIoigc1CIqiKAoANQiKoiiKBzUIiqIoCgA1CIqiKIoHNQiKoigKADUIiqIoioeEqFRWoo/TCezYATQ1AVlZwLhxQFpavKVSFCWRUIOQ4hgDLF0KvPQSUFcHiPC+7GzgzDOBr34VsNvjLaWiKImAGoQUxhjgxRdpDIYNA0aP9j7W2go8/TRw4ADw7W8DNnUeKkq/R9VACrNzJ/DqqzQE2dldH8vMBEpLgffeAzZujIt4iqIkGGoQUpj33gPS0wGHn32gzQbk5gJvvhlbuRRFSUzUIKQw69YBhYWBjykq4g7B7Y6NTIqiJC5qEFIYtzu02IAxvCmK0r9Rg5DCjBsH1NYGPqa+Hhg1SjONFEVRg5DSnHYa0NDgf/VvDHDoEDB/fmzlUhQlMVGDkMJMmgSUlQG7dvWMERgD7NkDjB/PYxRFUbQOIYWx24Hvfhd46ing/fd5n8PBqmVjaAiuvJKZSIqiKGoQUpyMDOCqq4Czz2bWUU0NkJcHHHUUMGRIvKVTFCWRUIPQTygsBObOjbcUiqIkMhpDUBRFUQCoQVAURVE8qEFQFEVRAABikrhEVUSqAOyK4lMUATgYxfNHk2SVXeWOPckqe7LKDcRf9tHGmOLudya1QYg2IrLaGJOUWfrJKrvKHXuSVfZklRtIXNnVZaQoiqIAUIOgKIqieFCDEJhH4i1AH0hW2VXu2JOssier3ECCyq4xBEVRFAWA7hAURVEUD2oQFEVRFABqEIIiIneJyHoRWSciS0RkeLxlCgURuVdEtnhkXygiBfGWKVRE5AIR2SgibhFJuNS87ojI6SKyVUS2i8hP4i1PqIjIP0WkUkQ+jbcs4SAio0TkPRHZ5Pmc3BRvmUJBRDJF5CMR+cQj96/jLVN3NIYQBBHJM8bUe36/EcAkY8x1cRYrKCLyVQDvGmOcInIPABhjbouzWCEhIhMBuAE8DODHxpjVcRbJLyJiB7ANwGkAygGsAnCJMWZTXAULARGZDaARwBPGmCnxlidURGQYgGHGmLUikgtgDYBzEv09FxEBMMAY0ygiaQCWA7jJGLMyzqJ9ie4QgmAZAw8DACSFBTXGLDHGOD1/rgQwMp7yhIMxZrMxZmu85QiRmQC2G2N2GGPaATwD4Ow4yxQSxphlAKrjLUe4GGP2G2PWen5vALAZwIj4ShUcQxo9f6Z5bgmlT9QghICI3C0iewBcCuCOeMvTC64C8Ea8hUhRRgDY0+nvciSBckoVRKQUwHQAH8ZXktAQEbuIrANQCeAtY0xCya0GAYCIvC0in/q4nQ0AxpifGWNGAXgKwA3xldZLMLk9x/wMgBOUPWEIRXZFCYSI5ABYAOB/uu3kExZjjMsYMw3csc8UkYRy1emAHADGmFNDPPQpAIsA/DKK4oRMMLlF5AoAZwKYaxIsWBTGe57o7AUwqtPfIz33KVHE44NfAOApY8yL8ZYnXIwxtSLyHoDTASRMUF93CEEQkcM7/Xk2gC3xkiUcROR0ALcCOMsY0xxveVKYVQAOF5ExIpIO4GIAr8RZppTGE5x9FMBmY8wf4y1PqIhIsZXtJyJZYCJCQukTzTIKgogsADAezHrZBeA6Y0zCrwBFZDuADACHPHetTIbsKAAQkXMB/BlAMYBaAOuMMfPiK5V/ROQMAPcDsAP4pzHm7jiLFBIi8jSAk8FWzBUAfmmMeTSuQoWAiJwI4H0AG8DvJQD81BizKH5SBUdEjgTwOPg5sQF4zhhzZ3yl6ooaBEVRFAWAuowURVEUD2oQFEVRFABqEBRFURQPahAURVEUAGoQFEVRFA9qEBRFURQAahCUFEFESkWkxdMnxrqvR1tqEcnytDJvF5GiPj5nlogs9XQ8hYjcKCKbRSSsNiEiUiAi1/dFlhCeo0eraxFJF5FlIqIdCxQAahCU1OJzT58Yqy31XwHMBzAJwCUiMskY0+I5Zl8Enu8qAC8aY1yev68HcJox5tIwz1Pg+d+wEBLqd/hfYJuEL/F0Z30HwEXhPreSmqhBUJIGz1CU0zy//z8R+XOAw2PRlvpSAC975HkIwFgAb4jIzSLyLc8wlHUi8nCnXcRLIrLGMyDlWs95fgdgnOfYez27nc4r+R+LyK88v5d6dj1PgD1wRvl7rs4EaHX9kud1KIoaBCWp+CWAn4nIpWDL4/8JcGxU21J7+haNNcZ8AQCetiD7AMwBsBhcdZ/g2Y244FW6VxljjgFQBuBGESkE8BN4djfGmFtCePrDATxojJkMIDvAc4XCpwBmhHG8ksKo71BJGowxyzyNzX4I4GTLVSMid4HNzmJJEdhnyRdzARwDYBXFRRbY/x6gETjX8/soULkfCPO5d3WashXouYJijHF54im5nmEzSj9GDYKSNIjIVADDAByylJeIDAUnT3Un7LbUIvJ9ANd4/jwDwLmd/zbGdI47tADI9HcqAI8bY27vdv6TAZwK4DhjTLOI/MfPOZzounvvfkxTsOcKkwwArX34fyVFUJeRkhQI5+g+BcYBGj3tvQFgGoB1Pv4l7LbUxpi/etw204wx+7r/3e3YGgB2EfGl0N8B8A0RGeyRfZCIjAaQD6DGYwwmAJjlOb4BQG6n/68AMFhECkUkA5xp4Q9/zxUSHpfVQWNMR6j/o6QuahCUhEdEsgG8COBHxpjNAO6Cd0iRT4PgmSd9A4A3wZm7zxljNkZYtCUATvTx3JsA/BzAEhFZD+AtcGezGIBDRDaDgeSVnuMPAVjhmRh3r0c53wngI8//+u2ZH+C5uuBpdf0BgPEiUi4iV3semgPg9d68eCX10PbXSlIjIo+Cbp0SAK8ZY0IaSSgiXwAoM8Yc7MNzHw3gZmPMZb09R7wRkRcB/MQYsy3esijxR3cISlJjjLnaGOMGs2vyOxem+cIqTAPjDu5Ax4bw3GsBvOcrzTMZ8LjSXlJjoFjoDkFRFEUBoDsERVEUxYMaBEVRFAWAGgRFURTFgxoERVEUBYAaBEVRFMWDGgRFURQFgBoERVEUxYMaBEVRFAUA8P8BBYGhJSJXix0AAAAASUVORK5CYII=\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEKCAYAAAASByJ7AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAABOTElEQVR4nO2dd5iU5dX/v2dmtrINdpfOsoBKV9AFsQRFLGCMNbYYNWo0xhh9NTHGFJPoz8S8JrEkMWpeY9QYK4IdsQWEiFJEkKYIAkvZXdjed2bu3x/feZwtU3en7/lc11y7O/PsM2fmmTnnvk8VYwwURVEUxRZvARRFUZTEQA2CoiiKAkANgqIoiuJBDYKiKIoCQA2CoiiK4kENgqIoigIAcMRbgL5QVFRkSktL4y2GoihKUrFmzZoDxpji7vcntUEoLS3F6tWr4y2GoihKUiEiO33dry4jRVEUBYAaBEVRUhFjgPZ2oKMj3pIkFUntMlIUpQ84ncDGjcCbbwI7dgB2OzB9OjBnDjBmDCASbwnDp60NWLUKeP11YO9e3nfIIcDppwNHHMHXqPhFDYISVYwBGhv5MycHsOmeNDFobQUefBD45BMgNxcoLATcbuDDD4H33wfOPBM455zkMgpNTcB99wFbtwJFRcDo0by/shK4917g2GOBq68GHKr2/KHvjBIVOjqADz7gQq2ignolPx+YPx/42teArKx4S9jPeeIJYMMGoLS0q9IfPpw7h4ULqVRnz46biGHz+OPAF1/03N0MGgQMHAisXAkMGwacfXbcREx0Ema9JiKjROQ9EdksIhtF5MZ4y6T0jvZ24M9/Bv7+d/5eUsKbwwH861/A3XcDDQ3xlrIfU1lJaz1qlO8dgMMBDB0KLFoEuFwxF69XVFXRVTRypO/XJEJjt3gxd0eKTxLGIABwAviRMWYigFkAfiAik+Isk9ILFi4E1q8Hxo4F8vK8388BA3hfeTnw2GPxlbFf8/HH/BnIfzdgAFBTw9hCMrBhA/2SgV5TRgZXKJ9/Hju5koyEcRkZY/YB2Of5vUFENgMYAWBTXAVTwqKpCXj7bWDECP/u55EjqZMqKoAhQ2IrnwKgthZISwt+nM3GCxoramq4yt+7F8jMBKZOBSZMCC0Q3NQUesBYdwh+SRiD0BkRKQUwHcCHcRZFCZNt2+iCDqRvRLiY27gRGDyYXgm7Pbnil0lNXl5o6ZjGxCbY43LRPfXaa3zOzEx+iBYv5gfkhz+keysQBQWhubes7AbFJwlnEEQkB8ACAP9jjKn38fg1AK4BgJKSkhhLpwSjrS00xe5yAcuWAS+/zAWrwwEcfTRw8smMCSpRZNo04NlnqRz9XayWFmYfjR0bfXkWLABeeYVZQd0zgA4eZNDp9tsDbyenTuWOxlpd+MJ6TYceGjnZU4xEiiFARNJAY/CUMeZFX8cYYx4xxpQZY8qKi3u04lDiTF4e9UwgGhqANWuAjz4C0tOZ6DJ0KL0Fv/kN8O67MRG1/zJsGHDkkQzm+LpYbjfdNmecEf0Uzaoq4I03+CHw9VyFhVxlvPaa9z5j6PbpvCMoKADmzgV27vT9mpxOvqZzztG00wAkzDsjIgLgUQCbjTF/irc8Su845BCmlzY3A9nZPR93uYDly/n7tGk0CAC/o8OHM+b3z3/y9wkTYiV1P+Sqq5iz/9lnTMu0LPmBAywcOfVUKtho88EHXNkH8v8PHQqsWAGccgqDT0uWMGYgAsycyfvHjQMuuICrjQ8+4IevoICv6eBBusjOPhs48cTov6YkRkyw5VyMEJHjAbwPYAMAt+funxljXvf3P2VlZUab2yUe778PPPIIU027xxLKy4H33gPKyoCJE33/f1UVPRU/+lH0Ze3XtLcDq1dzhV5eTgU7dSqNwaRJsQnq3H8/A0+FhYGP27yZsQW7nXGFzEyuLioruYO49FIaMLebRu6tt1iTIAIcfjirr0eP1kCVBxFZY4wp635/wuwQjDHLAejVSgGOPx6orwdeeIGLv4ED+T2srfWmo44f7///CwsZcG5ooMtXiRLp6azePfZYKlKR2CtMh4PPHQinE/j0U+4GOscN7Xa6v9rbWZQ2ciQ/WBMm6PaylySMQVBSBxHg61+nm3r5choBY2goLAUfKF3cZuM5rBigEgPi1VNk+nQGjwLFA3fupNHwl2mUns7ModdfD7zSUIKiBkGJGsOGAeefz5vFX/4CbNpEl7U/3G4aEF8xCCXFmD6dF7qx0Xc6qDH8wBx6aGCjVVTElYe/8yghkVBZRkrqc8IJDDgHorISOOoo/V73C7KygOuuY+D3wIGuGUJtbcCXX9LnOG5c4PNY28q2tqiKm+qoQVBiysSJ3Pnv2+f78ZYWZhSefnps5VLiyJQpwC9+wQ/Gzp3A7t3Arl0MOp11Fj8M7e2Bz+Fy0SDotrJPqMtIiSkOB3DTTcAf/8jFX0EB2+Z0dHCBaLMB11+vxWkJx8GDbC3hcLAvSSitL8Jh3Djg1lvZz6S6ms9TUsL+Q598QndQoDhDZSUwa5a20e0jahCUmDNoEPDLXwLr1jGlvKKCWYRnnMHA8+DB8ZZQ+YqdO4EXX6RCttm8wZ1584DTTou8YRgypGdF8uTJ3m3lsGE9/6epiSuK006LrCz9kISpQ+gNWoegKFFk61bgnnuo9IuLvUHdlhYq52nT2Gco0kbBF9XV3FaWl3fdVh48yN3ED37AiWhKSCR8HYKiKAlEWxuHWuTl9UwJy8qiT2/dOuCdd7hbiDaDBrGf0SefsOisspLbynPPZR3FoEGhn6upiQbN7eZ2tKAgamInG2oQlLgQrzooJUTWraPiLCry/bgIW0q88QY7EsaiP1BGBovTZs7s3f83NAAvvQQsXdq1D9LMmexxpL3Y1SAosaO+ni1p3nqLHoDMTMYM5sxhnFJJID7+OHiANjubmQD797NKOJGpr2fX1H372CjLcnO5XOy0+OmnwM9+xsf6MZp2qsSEPXsYSH7uOS4mR49mi4qlS3n/ypXxllDpQnt7aANnRJJjzOYzzzB7YfTorjEPu52rEWOAhx4K3qo3xVGDoESd1lbGA10ufh+zs6lH0tL4XRw8GHj4YaahKglCSUnwaWkuFxXowIGxkam31NZyxRFo9V9UxPqHZBkZGiXUIChRZ+1auoj8uaMzM9mOZsmS2MoVFi4X5/b+3/8B994LPPkku2mm6orymGP42gKt/isqmPsfqA9JImCtNALteKxg1mefRV2cREZjCErUWbaMMxICMXgw8OGHwOWXM3aYUFRUsE2zNes3PZ39dd55hzny3/9+6vXZGDKEcwYWL+a2rrsyramhEv3GN+IjXziE6tKy2UIbLZrCqEFQok5tbXAlb7dzQdrSkmAGob4e+P3v6fcqLe36mDHs0//AA8BPfpJ6k7guvJBK8s03qfyzsqhcW1sZALrtNt+FYolGYaG3Y2KgtDa3OzleTxRJsU+wkogMHMigcqA2M06nV+ckFMuWcTU8enTPx0RYQbt1K3cMhx8ee/miid0OXHQR00pXruRFzMhgh9LJk5PHAI4ezetUW+s/3tHaytc2dWpMRUs0kuSKKsnMnDnAgw/6rx1qbGTau83GhfaIERzaNWVKnHWO283ARqBeGiJ0F731VuoZBIuiIvYVSVZEgG9/G/jd75jJ0N2919ZGY/fd7ybY9jT2aFBZiTpHHEGdWlnZ87GdO+mR2LWLk9Sys5nsce+9wP/+L41F3GhtZTFTsG1Lbi7jC0riMn48cPPN9Enu3MnaiYoKBpwPHACuuAKYPTveUsYd3SEoUScjg9/Fe+7h96+wkIp/717OXx4wgLuIoUO9xw8axFG7Dz3E2cpxqWi2tidud+DhLC4XA81KYnP44cCf/sT2F5s28bqOHcsB36mWFNBL1CAoMWHoUOCOOzgt8a23uDjbvJmDsKZMoVHojOWe//RTGpG4tMNOT6dw27cHbr188CD79vcCK2tVW3jEiL62v0hx1CAoMWPAAODEE3mrqmK8oKTEvzK0itc++KCXBqGxEdi4kZlCAwYAkyaF38hs3jxmGQ0c6Dug0dLC3cNxx4V8SqeTJQ1vvMFdkAgHB512GmO18RpvrChqEJS40NjIJJZgK+PMTLp4w8LpZA//JUv4u4XNxhmeF10UevBw0iSu/hctoq8rP59CG0PBGhtZh+Cv6q4bbW3A3/7GYr38fG8LoB076FKbPRv4zneSJ4FHSS30Y6fEhcxMb+eDQEahvT3MQli3G3jsMQYnSkq6alaXC3jvPbp4brghNK0rwhbLJSXAyy8z+m2z8XksY3HYYSGL9+yzzKgaM6br6y4upr1Ztow/zzkn9JesKJFCDYISF4YOZWuZ+vrAVczt7cDRR4dx4q1bgeXLWUTW3fditzMnfd06dvOcMSO0c4rw2LIypkq1tTEIGU4PfjAN/j//YWzElxG02bhjWLwYmD+fRlNRYol6K5W4IAKceSZjCZ29Op2xuiqPHx/Gid9+mylM/hzxIowHvP562DJDhC0dSkrCNgaAN7ElUEud9HQawS1bwhdPUfqKGgQlbsycSW/Mrl1ceLvdvL+piZlFubnAjTeGGWTdujV4982CAjrtY9y2uaEh9GyilpboyqIovkgol5GI/APAGQAqjTFT4i2PEl1E6CufOJHFaZ98wvsLCoBvfYsZN7W1QF0d3UshtbUIVePGIc8zLy/05qiB2nwoSrRIKIMA4J8A/gLgiTjLocQIK+Vy4kQu2J1OGoGXXuJMEyuhx+EATjqJHRQC1hBNnswYgVXl5ovqahZAhDIAJoJMnsyndDr9x7Pb2pgAFZabTFEiREK5jIwxywBUx1sOJT7Y7UwAuvNOFrCNGMEAbEkJszoXL2ZJQENDgJPMnUt/iz93kDHccsyfH5XXEIi8PBq13bt97xTcbqC8HPj61zWgrMSHhDIISv/GGOboG0Nj0HkBn57OxKE9e4Dnnw9wkkMOYR//L7/kcrszHR2MHcyaFbdGdBdcwNjJjh3egLrTycrtnTtpz04/PS6iKUrCuYyCIiLXALgGAEpKSuIsjRJJvviCK+RAl3X4cGDFCuCb3/RTnyDCAMTAgcArr3Q1CmlpTG0666yYu4s6i3DttSxAe/NN4PPPKfKkSbRjEyZoGwslfiSdQTDGPALgEQAoKytL0fmF/RMr1TKQQnQ4uIP48ssAi3ybjX6Xk0/mSRsbGaUdPz4horV2O9vu9/PW+0oCknQGQUldOjpCX7j7q13oQkYGe28rihISCRVDEJGnAXwAYLyIlIvIVfGWSYkdw4cHV/TGMPhaWBgbmRSlP5FQOwRjzMXxlkGJDPX13kajublMuQzUogKgCygjw5t6CQB2VzuG7FuHQ754E3n15WjpcODAmBkocZ8IoDTKr0JR+hcJZRCU5Kejg1lA77zDzE+rjsBmA04tq8a5w/6L9G2befDEicCxx37VBiIri3PdH3uM6aY5aMTRH9yHwurP0ZpZgJq0wWhtd2FOxgrIr9/jwfPnaxRWUSKEGgQlYrhcwN//Dnz4IRX6V8VXxqB021sYfu8z+GwgMOnoXLaj2LyZbaovvJBDlEUwZw7dRs89a3D4+oeR2bgD5dmlcLYJ0tKAmcfbUTBkBC3P009zNmdZWTxftqKkDGoQlIixaROwcmXP1s4lu97H9A1PonbIKHxWn47CdmDYMDA1tL0d+Ne/uD2YPRsitA3HjtiFhps3YN/Q0ci2CYqLWXz8lZFJS2O12qJFwFFH6S5BUSJAQgWVleRmyRLGCzrrZpurA5M3Po+G3OEwjnRkZnJK2FekpzOa/NxzXPV7yNm8CsNG2nHkUYLp09n1tEe7h7w8DmbWAfeKEhHUICgR47PPejYaLTqwBentDXCmsTNdVhZbCXVp3ZCVxVqBzZu991VXB+/fIMLgRFNT34VvbGTfjNbWvp8rllRXs+x5715vu1hF6SXqMlIihs3Ws0dPVks10tubMLhiA9LbG+CypUFsIwDXEMDRreigttb7e35+z9YT3TGGgYu+FJtt3gy89hr9XSK8HXssBxyPGNH780abL74AFi5kKpf1xhcVAd/4Buc762BmpReoQVAixqRJ1KtDhvBvh7MVh219GUP3f4y2jDy4bWlwdrgxybEH8nYmewp1Hnqflub9feZMTqEPNGOzoYFP1lvFvXgx8O9/089ljTFzOhkI+e9/gZtv5otKNNauBf78ZxrCUaO8yr+hAXjkEWD7duDSS9UoKGGjnxglLA4cAF59lU3oHn2Uuqm9nY+dcgq9N243AGMwfc3fkduwB+3pOWhPz0WHIwstMgDZwwuo6Fes6PQPYEtqi9JSNvbxFx9wOtkd7uyzexdQ3rqVWUqjRnGgsXUOh4Mxjfx84IEH2Bk1kairAx56iDIXF3dV+rm5jOi/8w6wZk38ZFSSFt0hKCHhcgEvvMBFtQgXpy4XZ9nn5wM//CFbBZ10EvDuu8DU7O0Yvm8NqgvHI7OtHgPq96Ee+cjPBwYMACBZrFrbto1lxzNm0OVhIQJ8//vAn/5EH3lhIRWe201D0NLCcWthDVzuxJtv8kV03pUANFQHD/I5KiqYR3vttUGGMMSQDz+kMfTnJrPZvCNCy8o0+0oJCzUIKUBVFZu9ud1Myy8tjbweeP556pjRo3v2G6qrA/73f4Ff/pKeisJCoPpPy1DXko5aCGrsR+BQZyOK0moxIDsH1sfOac+AWbcRLfPPRfa3Luv5YczPB267DVi9mu6jXbv45GVl7BN9yCG9e6FtbcDHH3N30P2FrFrFALPNxqynJ59kg7xzz419EVx9PeX56CNuw0pKgA0b/LR57URBAXtpNzQEP1ZROqEGIYk5cIAp/NboSREahZEjgcsuAw47LHLPs3ixb2MAUG+3tLAk4Ic/ZFyzY3U59m/KwfovgfqWDHwx7Hg0Nm3H4PIvcHB3ExwOoN3lQHN6CR6v+h7af52L+fOp57ss2jMzgeOP582KWPdVKbe3ezOULBobgeXLeb8V1+jo4HMOGUL3ktvNkW2xYN06+uXa2/kG2+3ABx8wvjF4MOMvDgeVfn09/ycnhwbACo6H1AFQUbyoQUhSDh4Efvtb6rHOcUVjmKzzu98Bt9wSmZjoypXUL4E6kQ4ezEV3TQ09Fh2SgR3bXHDbWFAmko7Wogn4rOlQ7N/RAmcbUDQyE6PT9qBoTC6k+jN8+f8+xPsP12D2WQPhOG5Wzx1ApFbnWVmsf2hv50+AuwCXq+uKuqODyjgtjavzF1+kYeocCI8G27YB999PF9qAAd77c3Nplbdu9b4XVVX8afUIGTiQFz0tjccr4eF0che2eLF3R3rkkcCcOdHZeicYGlROUp57jovD4cO7LnRFqBMGDgQefjgyi8Tdu4NndtpsvFV7BqD+p3kmbI31XQrV3G6gfJ8dzswc2PJy0FTZiIb0IpR99Decsvq3KGt+H87Nn2PvM8uAu+6ixYtGUNfh4FZk/37+3drKUWzd4wTt7cDYsfw9LY0K96OPIi9Pd158kW94Z2NgUVpK+deuBSorabAKCvgzP59B+rff5rCF7vERJTDNzcAf/wjcdx+TGQoLuUBYuRL4zW84cMnX7NMUQg1CElJTQ9dyoDnyeXnUpRs39v350tNDq3lyu71ejFf2z0B6XibS2xu/erypyTtg3i5u5HdUYe+BNOTX70JdQSka84ZDCguxsWYE3KNGMzDyxz9Gp1hszhy6o2pqqAi6u5Dq66lgBw/23pedzQB3NDlwgLURnQPsnRk0iDsZa/amrx1URgZHsaW48oo4//gHd4qlpTQGDoe3kn7kSAbSVq6Mt5RRJWyDICIDRCQ+8wcVAFzMAsGHyTgc3dpE9JLp0xkjCERLCxe0w4czOafNMQCrjr4emS01yGncDzHur2K12c56DGvdgQ0ZM7ClbQwa87x1BGlpXJg3NArrC3bu5Go40hQVAT/5CX/fs4dGp72dVqu2lhb1mGO6vsmWxYsmdXV8k/y5Jhobacjy83lsfT0NWmMj5U5L46S42trIXPz+wt69TNUtKfH93jscXBwsXJjSFeFBDYKI2ETkWyLymohUAtgCYJ+IbBSRe0Tk0GDnUCJLoFqtzlhB5r4ydapX//iTZ98+FvdaXgoRoGrwZCw94XZUDJ6CvPpyDGwqx7COnXDa0vHysO/hIxyN9nTf6ZxfyT1wIP250aC0FLj7buD6673B2KIiGoLZs3u2zmhpif7cy/T0wCv7+nrKWVzMWMHYsVzNDh9OuefO9cZBdu6MrqypxNq13mC8P3JzGbPZtSt2csWYUJY77wF4G8BtAD41xrgBQEQGAZgD4G4RWWiM+Vf0xFQ6M3QoFabbHbgYtaODOq+vpKUBN9wA/P731ImDB3uft6WFxuDww9ml1JLPSnKpKxiNVTOvR0ZrHco31WPr9jS4CgfDQJC3awWGDe3ah8jShV/pYquBXbTIyqL76JZb6CP294Y1NnILFO2RnMOHMybQ1OQ7hmCtBlwuBt07u7R8HauERnW1N8EgEJHqnZWghOIyOtkYc6cxZr1lDADAGFNtjFlgjDkPwLPRE1HpTnExFXBFhf9jmpvp8vY7iD5Mxo0Dbr+dk8/Ky3nbtYvPc8EFNBjW9yknhwvsznq8LTMf2eNH4YBjKFzGhg6nwGF3YXx2eZfnaWqiQcnK8tzhdPoMjrpcEd65n346V9s7d3bpugpj6Nc/eBC47jrvKLdoYbczb3f/ft8vMCeHVjg723+cwSKRezElGvn5Xa+7P4zp9OFMPYLuEIwxHSIyAcAIAB8aY76KEorIPGPMYmNMCO+kEkkuvBC4807uYIuKuu50m5poLH74w8jqr5Ejec6aGi6orC4PvpJZzjyT2Xt79nD2gc3GBe+4cYyZOhzAWRNqMah5LxrThwNgvZjbzY4VX1FVxWZtnsdXrWKN2p49POfUqdyZTJrUx4zArCzgxz+mj/g//6HFASjQIYcAF13En7Fg9mw2r1u6lBfXcme1t9NllJfHIhN/28P6erqRxo+PjbypwJFH8toH8sc2NXH3Nnp0TEWLJWKCbCtF5AYAPwCwGcA0ADcaY17yPLbWGHNktIX0R1lZmVm9enW8nj7ulJezs4LlKrZS0fPzWTEc70FiNTXAE0+wxsrC6jxhtwOTsnbgnA2/QU3WCLS505CRwZ52hYWegzs6qPl/8xs0DBqNe++lnhw0iDrRWrw3NjJ+cfHFEern1tTE7Y/TSWGGDYt9/rnbzYyW11/3Wj+Hg+6tww5j0Vp2ds9+4/X1fON//GNu55TQMIZtUjZv5sqnOy4Xv2hXXgmceGLMxYs0IrLGGNNDQ4RiEDYAOMYY0ygipQBeAPCkMeZ+EfnYGDM9KhKHQH83CAA/xzt2sMGly0XdNWlS9JNhwqGy0itfYSF72B04wMJbWfwGJnz8NPLHDETRuAI40sRbXVddDVxyCcxp83DvvUyh7d5tAuB5v/wSuOIK9lJKKaz3wuViUNPa8m3fztXA/v1dV7WDBlFpqTEIn4YG+Fx1VFXRNzpvHneKKVCc1heDsMkYM6nT3zmgUdgE4CRjzLQIyxoyahBSAGO4hVi0iKtym42r49JS4KyzgGnTUF4O/OIX3Kn7+y42N9Oj8oc/JJYxjCpuN1NLd+3i7yNG0N8WLB9Z8U9bm7d31p49/MBNm8ZWvhMmpIQxAPwbhFC+OvtFZJoxZh0AeHYKZwD4B4Ao5+ApKY8ICx2mTeNWwoqGDx781Zdv3brAqfkA/+XAAe4UYuXqjzs2G91HkWpapXAHdtxxvLndwVNRU4xQDMJlALo0QDDGOAFcJiIPR0Uqpf8h4p2s0426utC6MIgEL6BTlJDphwOGQskyKg/w2IrIiqMoPRk4MLSMQLfbd+p+0lFTwwZ2bW30Y0+cGHy+tKJEgP7ibVWSmCOPZDO/QBmBjY0MWEeiEC9uNDdzpOd//9u1BiEzk5PhTjutX7kvlNijBkFJSIxhYo3dzkK1WbM4LMxXqxmnk3UX116bxLv8tjZmuGzbxrTHzoHhtjbgqado9c47T42CEjVCNggiIgAuATDWGHOHiJQAGGqMiVg/YBGZB+B+AHYA/2eMuTtS51aSg/p6pqO+8QazLR0O1ibMnk19uGEDsy/z87mIPnCA2UXnnw8ce2y8pe8DK1bQTTRmTE+Fn5HBFKtXX6Vl9JUnrygRIJwdwoMA3ABOAnAHgAYACwDMiIQgng6qfwVwCoByAKtE5GVjzKZInF9JfPbv5yjOmhq25ygt5S5hzRrqy4svZvbfm28ym8huZz+3k04KnJKa8LjdLEDrlFnVA4eDt6VLgUsuia18Sr8hHINwtDHmSBH5GACMMTUiEkI3qJCZCWCbMWY7AIjIMwDOAusdlBSno4OFom1tXTsD2O0stmtvp3v9pz/1dq1OGRobWYRXUhL4uIICVtIqSpQIx+Pa4VnFGwAQkWJwxxApRgDY3envcs99XRCRa0RktYisrrLGByoJh8vFlX5NjbctUCA+/ZRlCMXFvh9PT2fCzauvRlZORVG8hLNDeADAQgCDReQuAN8E8IsIyuJrr9yjjNoY8wiARwBWKkfw+ZUI0NoKvP8+PSDW/IT8fGD+fN8jBiyWLw+eMlpYyIFWdXU8Z8qQk8NWCY2NPcd4dqa2lvMOEoGmJlb0bthAiz9uHP13XzWiigANDWzY5XazRiVYd1elz4RkEDwB5WUA1gCYCyrvs40xkdy/lgPo3KlmJIAoNsJXIk1TE90+n3/O76/lAWlqAv71L2YJ3Xyzb8VfXx+8M6tVNNrSkmIGwWYDvv514LHH+Ob4iiM4nVS8X/ta7OXrzpo1wCOP0L+Xk0P5P/kEWLCA6bFnntm3gE5dHc+1olOZk9vNWRQXXMAWu0pUCMllZNjwaJExZosx5q/GmL9E2BgAwCoAh4rIGE9s4iIAL0f4OZQo8vTTbLQ3dmxXpT9gAO/bsYPZk74YNCh4lbHbzXTUvhSftbZytEFDQ+/PERWOPZa9cnbt6ulja2tjp80zzoh/htHGjcCf/0yLXFrKVfugQbT+I0YAL7zAFLHeUlsL/Pa33DIOG8ZuhtZtyxbgjjuA3buDnkbpHeG4jFaKyAxjzKpoCGKMcYrI9QDeBNNO/2GMicCIeCUW1NSwnirQTJYRI5hSev75Pbs2z5pFN1NLC5Npiop6upeqqtjyKDc3fPn27WN20ooV1LfGcFzA178OTJmSABlKGRnATTfRqi5f7i1ME+Gshm9/2zuSLl4YQ/ny89k8qjsOBw3DwoXACSf0znI/+yxzibsH2G02FqQcPAg89BCHgSRt0UniEo5BmAPgeyKyE0AT6DYyxpgIzeQCjDGvA3g9UudTYsfWrdRhgRpt2u3UKVu30gAA/J8lS6hDdu/mY9nZ1IMlJVTWaWk0FC0tVODh8tln7IJqDF1ZDgd/37MHuOce1nr11csRFtbcUWMYRbdmIGdlsXX1uefyjWhvp/WbODH6k9pCYdcuvmmBsqHS0pgy9vHHwPHHh3f+mhr6FQOtKgoLuVvatk2b+kWBcAzC/KhJkWQYQ+VVUeFVXIFG2/YH2tvDP9YYuopffpmekNNO4w6ivp56ZccO6oixY2lMvv99xi7DoakJuP9+urotvQvwuhUWcrG7YAHPO2VKiCd1uXhih8P3StkfjY1Mk3rvPcYELGbNou/dSrEqKACOPjr088aKmprgbWcBXrz9+8M/vzXpKVj7bhFaeTUIESdkg2CM2RlNQZKFHTs4BWzHDm+Q0xjOLv72t/2nTaY6nZVtIES8x5aXA6+9Rle0pQNmz6Yu+eILbyXy3LlsS+GnGWpAVq1iiyB/18XhoDyvvx6CQairY2HYkiU8qTGc9nP66Qx4BlKUDQ2sutu9m0FRawC1y8VV8fr1wM9+Rr95omJtrYLhcvWuGZ/VbjoYNltoucxK2ITTuuJ2X/cbY+6InDiJzbZtwN13c2ffuTLW7Wa90F13AT//ef80ChMncrHc2upfF7S28r2bOJF/L13KxWTnBaHDwd2CFTttaOBivLfv6YoVwTOSCgt5/QJmfVZW8uLX1HhTII2h6+ePf6R//5JL/Pu1n3uO7pbu3ffsdr7Yykr6xn/96wQIaPhhzBjK29Hhvx+5MfxCWBc5HIqLvQGeQO+B08kt+ZYt/HBkZ3MIRig90pWAhOMyaur0eyaAM8A5y/0Clwt4+GEqjIKCro/ZbFz07d3Latobb4yLiHElI4Ou78cfp7HsPrXM6aRH4KijmF3pdjN2Giy1PDeXruvm5sAp+v5oaQmuJ0R4Df26vdxu+p1aWrqWUVt+p4EDuWsYPZpbnO7U1QWPuBcX8w2y0rQSkQEDGCx++23/bWWrqvg+9OY1jBxJo3PggP96huZmBpafeILZVxbZ2czCOvVUDTb3gXBcRn/s/LeI/AH9KC1061Z+1gO1Vx46lNO9DhzonzU0c+dylf3SS/xOWoaztpYLaaeTFclW8snGjdS106f77ukWCYYOBTZtCpzw0tHBha/fY7Zu9b26t7DZuGJ99VUGUrsrpB07uOoN5Bu3/I9bt0bGIFjDtt99l3UDTidTN+fNY6qW5bIKl/POYyOpzz+ne8vaDnZ00NeXk8NgT28upgj9rr/9La14dz9kSwsN64ABfCwry/tYaytzmisrgUsvTdxdVoLTl/bX2QASdCkTebZvD77wsOJtu3cHNggdHdzt1tfze3nYYalRaCXC2OjRR3P1v8nThWrwYL7WsWO7fofHjWOsYN066srOi2+Lhga+l+HEbjszZw4LagN5ISoqgBNPDJDI89FHwRWotZXZu7dnrUB1NT8UmzZxVZuRwTdj1Kiub4jdHl503h/G0DgtWEC5i4v54Tx4EPjrXxn3+J//6d2WKysL+NGPgLfeYh5vVZXXmJ1wAtPA+lKtPG4ccMstwIMP0vCkp/PcbW00CDk5QFlZzy1oZiZXFW+/zccnTfJ5eiUw4cQQNsDbSsIOoBjAndEQKhEJ5ta0sILM/s6xdClrd5qavOcUYQHqRRd11Q/JyrBhrDUAuAO49VZmYnV/baNH09AOGMCdQ/cxAMZQ31xxRe+9ABMmUP9t387zd7+GdXV8zlNOCXCSxsbQVtQ2W1c3BsCV9KOPUrkVFvJFHTzILVN2Nlu1WgESpzMyQeUPP2TMorvvbuBAbtu2b2el8U039W4lnZXFPN1587gid7v52iI1rm7CBMZlNm7kjsnl4u7so4+4kupuDCxsNhqMt95Sg9BLwtkhnNHpdyeACs9s5X7BqFHBExusoS5Dh/p+/NVX+T0dPrzrDsLlApYto4740Y8SI+U8UnzxBV1ovlb/eXlU1lu38r2rrPTqQ5eLWUgTJvRtzoHdDtxwA/DAA9TNWVnUWx0d3LXk5HBBGjCDacgQ5tUHwrr4navmqqup2IqLqfy/+IIC2Ww8vrkZeP55+r6HDOGFP7yPZT1uN7BoEbdlvhSnCD/M69fzDR41qucxoZKeHr3K6bQ0uramTfPe98QTwXcfhYXsrxTqCk7pQjjrruuMMTs9tz2eyuLfR02yBGPyZH7Xm5r8H1NVRQXmq9XK/v3cwftaKdvtvH/LFrpaUomDB/1/L0W4kDv8cHpKtm/nQvrLL+my/9rXuIjtq4HMzfW2zZ46lQZh2DDWgP3+90xQCcjRR3P1Hijlsrqa1q1zQcry5XxhDQ1U/jYbFV16Ol+UtaJ+912uhi+6qO+zk8vL6QMLVM4twg/d6tV9e65Y43YH3yqKdB0/qoRFODuEUwDc2u2++T7uS0nS0qhA7ruPC77u7tfqai4Qv/Ut3/+/fLlXH/hChLrk9dfp906VRIlgqesiVMiW22zCBOrLCRN6ZnP5orqaN4eDSTz+3l+7ncanV56EkSPpl1692vcknpYWbjd+8IOu97/7Lvv8LF3Ki5uXx21gR0fXC9zQwA/UCSf0QrhuWIYnGBkZfOOSiXHjaPAC7RLq6qKXodAPCGoQROT7AK4DMFZE1nd6KBfAf6MlWCIyfTpXrI89xlWszeZdjAwfzk6e/qr6P/20Z/+e7uTkMPOwvj40ZZgMWAkz1nxkX1gJOKefHroHYtcutrv45BPvdcjOplv71FN7n0TjExHgqquoyNet4xPl5nqHPthszKyZMKHr/9XX01J1dHA3kJbGn1ZxBeDdEaSn8+KPGdM3WbOzQ1sht7cH/0AmGqedxna6gwb5VvjG8HpcemnsZUsRQtkh/BvAGwB+B+Cnne5vMMYk2RKj70ybRrfwpk1MKBGh0hs3LjKr+lRb2AwaxM4MH33k311dUUFdGihNvzOffcai37Q0GhDrfW9tZYxmyxbGDSJqFLKyWGDy2Wdc+e/cSeU7Z47/OQB5efR9db6oNhtTyqy0so4OGhabjcGWvhqEkSO5hW1o8O82MoYusLKyvj1XrJkyhauyTz7hyqvzF87tZibX1KmsGld6RVCDYIypA1AH4GIRGQjgULAwDSICY8yy6IqYeDgc9HuHE/+bMIEZcYFcu01N1CG96eaZyFx8MfXnzp303VuKuqODHpSCAuC73w3NGLa1sftyXl7PNHUr83D9er7Xp58e4Rdis/FCdt8J+OOkk5jmGchn1tzM8wWrUwiBhgZg1SobmuxnY+LyvyFrQhaGDHf0jC3v2UPl2peAcjxwOLgTe+opb7DNbvdme8yaBVx2mVYs94Fw0k6/C+BGcHDNOgCzAHwA4KSoSJZizJ4NLF7MhZm/rLmKCtbl9FEvJBx5ecBttzE+8u673r5uIsz/P+MM7iRCYf16Kj5/NWIiNDpvvMFU0rjqhuOOA555hitXX1kvbW282CNGMPoeqOoxAMYw0/K556gbszKPQXluBaYuW4TtGZkYf3wxBg+1079udQv83veSczuamUn33dln031XV8cP2BFH9M+eMREmnKDyjQBmAFhpjJkjIhMA/CY6YqUew4cD3/gGMwJHjuyaOeN2M1Y2dmxiDMSKBrm5wIUXMn29stLb+Tnc1PWPPw5epJaVxYyv/fvjvAguLAR+9Sv2OKqo4N8OBy1iczONwTHHsJR7xozQrWI33nmHGZklJdbuS1Ax+Gx0HDYFwze/A/d/1iBjigv5U0Yxk2n69OTPbS4sTJxxoilEOAah1RjTKiIQkQxjzBYRGR81yVKQc86hMlu0iDE9y5MgQn1w2WWJX5jW0MB4wKef0pAdeijrBELVZVZjwN7SPUHHHzZb1w7TcWP8eDZ4uuUWRsIdDq5yrfzkmhoWpfhLTwtCczN3BqNGdYuZiKC68FBUH38oDh4ElhYZ/PJ2ScpNgRI7wjEI5SJSAGARgLdEpAY68zgsbDYOm58zh2nndXVcqI0fnxy9j1asAP75TyranBwasg0bgBdfZIub00+Pvhdi1Ci25gmEy0Vj1csFd+SZPJmpaQsWsB+3NSmoro6+xHPO6XXvkk8+oZEMtOAfNAjY8aVgzx4+9apV3KXl5LDZYKQSIpTkJ5zmdud4fv21iLwHIB/A4qhIleJkZvKLmEysXctur8OHd62dGjSICumZZ6iUTj45unIccwzTTQOlsVZUcMeVUP2hhgwBrruOOwJrstLw4X3OINizx39MysKq1Xr4YYYzbDZew/Z2xrVKS4Hrr0+ORYkSXUJeFwj5tojcboxZCgaWp0VLMCVxcLs5Sre42HchrZX++fzzTP2MJE1NbDnx2WdcUBcXs87gyy99txKpraUCPPPMyMoRMQYOpLto/PiIpJNlZITWUmXrVu4mRo/2TvgbOZLGoKKCFdv19X0WR0lywnEZPQjADWYV3QGgAcACMNCspDCB+hFZZGTQGHz6aWTS2+vrOVpz6VIqPKtp4KxZ3vnHb77pnUHvcvH5Bw5kM71QaxqSnfHjg9ehVVYyienEE3279IYMoYFdtowZX0r/JRyDcLQx5kgR+RgAjDE1IhLJ0h8lQampCe04u53ZPX2lro7DySoqmEJqpY66XPR/b9jAyXQnnwysXEm3SUYGiwYnT45SqqnTyWrE9euZLjpqFP1Sca72PeQQGr9AMzjWreNuIFCbpCFDaGDnz0+9tGcldMIxCB0iYoenBbaIFIM7BiXFcThCCxa73ZGpDn7mGRqW7m1ArJT9ykr6w2+/PUYr2h07gL/8hb1/0tMpyPLlFPSMM5gTH6eorNU147e/ZeX8kCFehd7ezsI/my148a6VqtvcnHqFkUrohGMQHgCwEMBgEbkLwDcB/CIqUikJxbhxNAjB+hEZQxdGX6ip4ao/UE+j4mLq6J07e13LFTp793K7kpnZ02fmdDLCDXB+aJwYOZLGceFC7qCs7tp2O9v/FBcHdytZ1093B/2bUJrbPWmMuRRAEYCfAJgLQACcbYzpNzOV+zP5+aw1WLHCf/O+/fs5V72vvvsdO/gz2LRJgIHmqBuERYv405dryOGgkXj1VTro45jnOnQodwoXX8yVvjXnOyuLt5deCjwgzSpgDqkOprmZaWfvvssofl4eX/+MGZEbkqPEhVD2uUeJyGgAVwKoAPA02PCuQkQSJdNbiTIXXUS3+Y4dXTOJ2tu5Us/NBa6+uu91CMEyZixiUnhWW8uW14Gm51g5nytXRlmY0CgoYLHguHFe5X7ccbwu/jLA3G4ahPnzQ7h+e/YAv/gF8H//53Wh1dWx+O5nP+OHQUlaQjEID4H1BhMArAGw2nOzflf6AQMGMHvn3HNZrbx7N9ttVFUxkHvMMdSdW7b0bT6JFRgN1A8O4HP4m0wXMazpPsHiA1lZCa0Ii4qA73yH3q+amq7vbVMTjfycOcCRRwY5UWMj8Ic/cBVQWkrrk5nJLeTo0TzxPffQkCpJSSjdTh8A8ICI/M0Y8/1oCCEi5wP4NYCJAGYaY9TQJCDZ2cBZZ3ElWVXF+oAXX2Sq6caNPMbqUXTNNVyphktpKTOL6ur8z4RoaaEsU6b09pWESKgOdbc74Ttsfu1rfD9ffNE7y8MY6vLvfCfEoUwffURl7y//eNAgtudYsQL4+tcjKr8SG0KJIYghfo2BdUwf5PgUwLkAHu7DOZQYkZ7OxeLjj1PJdPfj19YyDnvbbSGMp+yGCDu+WvMOuruk29qYOXP11RGed+ALqyy7tTVwzmZLS9fZvwnK1Kk0ovv2cZeXkUE3YMiB5CVLgsdJiovZejUUg7B3L11tu3fz/S0ro5BRv7CKP0LJMnpPRBYAeMkYs8u601ODcDyAywG8B+CfvRXCCk6Ldt5KCtxu4NFHaQx8tYcoKOAx//wncOed4ccVJk3iZLqHH2Z+fUaG1wfucHCU6cyZzKh5/30qt4EDOYFy8uTgrRxCJj2daToLFtDq+Xoh9fUMqoYzHCOOWB0zekVNTdeZ0b7IzGQBSaD5xx0dwJNP8uLZbLT6TieNQ34+pxtZo/aUmBLKV2ceGFB+WkTGAKgFB+TYASwBcK8xZl20BOyOiFwD4BoAKPGX8qJElW3b6DIKVLk8cCC9Bzt29O67fcQRwL33sqhq82a6N8aO5SKyro5xzaoqZs5kZFAHrV3LFMybbgo8djcsTjuNBWmbN1OTWl3k3G5aq/Z24Cc/6R+r2gED+HoDpSJ1dNAo+DMGxrBX99KlNLLdj6utZR+NX/+avkMlpoQSQ2gF21Y8KCJpYPppizGmNpwnEpG3AfgKA/7cGPNSqOcxxjwC4BEAKCsr64ubSukl+/YFD/pai+l9+3q/2MvIAI4+mjeL+nq6kzo6urqqrGKqvXupS668kvd5ZwT0kowMWpjXX6crpLWVSszlov/lm9/sWz/vZOKEE5i/GmghVlkZeE7Bnj3cGfgyBgC3l83NTOW9+uq+SqyESViba2NMB4B9vXkiY0yU+2AqsSIcF1CkvYArVnCH4EsHV1WxNqG8nDuTwYO5mJ0/nwv9Xsd9MzLYovr003lyp5NbkP7WHvS444DXXmMAyVdRQ3MzVwonnuj/HCtW0KcXKII9ZAjdRxdfHLh4Qok4kfK2Jg3V1Vy1AvQAxLkVTVJiTSHzNRXSwqp8jeTEMmPYb8fXpMQ9e5gEk5lJXd3YSPdSaysHyGzfDlx3fhUcH/2XbT9dLqZBzZ5NIUOxXBkZTPDvrxQVAT/8IXD//bTKgwfTyjqd9Nm5XMAPfhDY1bN7d/DiNSvKXVurBiHGJIRBEJFzAPwZQDGA10RknTHmtEg+R0UF2zOvXev97hvD4srzz+9/i72+UFrKFfrBg/7ft6oq6ttALSjCxeWijuie1dTWxqE5OTnUTy4X8+sBGogxpQbNr7yLqjeeoq4qKOCHYOlS4O232SXv4ou1b0MoTJ0K3HEH53YuW0ZjYLdz93DyycFXAKH06wYYo4lYdoASKr16x0VkqDFmf6SEMMYsBPskRYX9+4G77qLiGDnSu1t1uWggtm5l90yd0R0aIpxzftdddBkXF3uNbHs7i9NaWoDjj2dguaQkMq4ju907krizrigv71oK0L0sYMTej3Dknn9ic94oDJ2Z7pUlN5cfgjffpG/pvPP6LmSiYgxX51u2cNs0eDAzo4INqPbF8OHApZdy7GdrK5V8qMp7xgxa70ArsMZGbvOCZTQpEae3Jvh1AMHqGhMCY5gi6XT2TLezumfu28ec+h//OD4yJiMlJcz0efJJ+u0Bvo87dlDPHnYY09aXLGEtwjXX9P37LcKK6A8/7Hot9+3rOkKyuZnPDwBi3Ji86Xm0FwxBQ0v6V0VtX2G3c7vz+uucvJOKrT4PHAAeeYQXyqq8drkYbT/3XAZYemOx7fbwexdNm8atnJWu2x1juJ2/8kqd6xkHevuOJ03BQHk5K2oDKaOhQ1ltuz9ie57+wahRLD676y5+z63umt/4BrueWq6l8nLgd79j/KavzJ3LXUh7u/c+t9urzzo6+LuVCDOwZjuymw+iPZ2+aJ/ZUQ4HFeQnn/RdwESjtpZv/q5dvBijR/PClZZylf7UU8Arr8ROnsxM1hnU1/ML17nPSVMTgz3HHsvYjhJzemsQ/h5RKaLIjh2Bg58AHxPxdtpUQkeEK+7164Hp03vm/4vQ4NbVMUElEM3N1Fu7d3dV+J0pLQUuv5xGprKS+mTgQLoDGxupU8rKvAvXjNY6GBG4XFxw+h1G73BEZrpPorF4MS3x0KE9vwTp6TQQCxdyFxErDjsM+NWvGI8oL+cF37WLRvmKK7id1HhOXOity+jFiEoRRazxi6HQl6Zs/Rmr0WegtM5hwxiDPO+8nm7r6mqmnb//vvcaZGQAp5zCHUf34+fMocvotdc4PS0tjcbg0EN565w55rKnA8agsZGuK7+ubperd/70RKa1FXjvvcBZP1YK6AcfcGsXK0aNYkZSXR13MQ4HjZYagriS8jGEQJ2LO2OMxrB6y+efB3clp6V5i3s71zVVVnLaV0MD9YFlVFpbOYpg/XrGdrqff/x43hobubNYuJAp7t3d0tWDDkFzexqyBnRg7Fg/FsvKkY16t7wYc/Agg2fBKvNycngR40F+vu/+J0pcSPkYwvjxXDE2Nvo/pq6OhqM/p5j3Bbs9eOWyRefdmjHAQw9R+Y8a1XWHkZkJjBnDrtLPP+//fDk5NORXXQXMm8d6hF276J4uLwe27clC+cST8bUx5cjK9CPk3r1sgtTrJj8JSjhbY+0jpqAfxBDsduCyy7zzYrvT2MieXZddpkkNvWXyZG/evz/a2ugG6rwL27mTMcRAu7jhw70N7ALhcDAL8p57gAsvBGbNorvpZz8DLnjqbGTNOoJBos4namlhL+iioshM90k0iouZTutvMo5FY2PSNOdTokuvXEbGmAcjLUg0mT6diQ2PPkoXhbWDbm/nCvOmm6jUlN4xYwbw739Tv/rre7ZvH2cpdA7qbtniDej7w+HgTmL79uCD4gEGtU/rUdKYzgrb//6X6aW7PE17BwxgL6ITT0zNiti0NG6bnn+e2y1fWC1kZ86MrWxKQtJvSgGPOoou4vXrqVwABiCnTOkfjSqjyYABwHe/C/z1r2yX39kl7HTSIzN6dE9F3dYW+q6sz+My09LYnO1rX2PKozGsOUj1ati5c1l9uWMHi246v976egZ1rr02NesvlLBJ8W9DVzIyuJqdMSPekqQeM2fS7//003QFWat+EVYsX3hhzySeIUOCdzGw4r0Rm19vs/kfxZaKZGUxKv/CC0zzstLujOEFuOkmbqEVBaFNTAvlq+gOtx22EhpWx4HNm7miLi6m6yQRMyQPP5yp5Tt2MJXU4WDra18Fqdbx6el03fnbpdXXM2uye/8iJQyysxkkO+ccbo87OphpMWaMBs6ULoSyQ9jruQWKuNkB6LSaCHPgAKeGff55144DaWnsODBvXuLFQUVoBEKZgZCdTRf+k08yFbV7HUNLCw3Lj3+ceK8zKcnNDS0QE0MaG72xPHXdxp9QDMJmY0zAPaWIfBwheRQPVseBpib63zsrxPZ2BnHb2oCzz46XhJHhlFP4el70lDoOGMBdUWMjFcT113PXoaQOxjCW99prXOzYbMwGPOEEfh60Hih+iAmSQC4imZ6paX06JhqUlZWZ1atXx/ppY8Izz7AJp79hXE4nc+7vuSc1WndXV7Pieds2KocpU7q2oFBSA2No/F96ickHgwZxsdPRwZ52Dgdw663+k6KUyCAia4wxZd3vD3WEpr+TXmGMeSwexiCVCbXjgAgzKc88M3ayRYtBgziQTEltPv6YFeilpV27VKSlsTV9bS1w330cq5yZGR8Z+zN9jSj9JiJSKF0IteNAbm78Og4oSrgYw55VAwf6b1lUUMDOAevWxVIyxSKULKP1/h4CEGKnICUcQk38MEaTRJTkobaWGWglQdJPcnO58501KyZiJReNjZxr0d5On9uhh0a0liaUMw0BcBqAmm73C4D/RkwS5SuKipiBE6jyF2AXBu04oCQL7e1cwATLGEtLC94Kpd/R2sqK86VLu9aS5OUxVe/44yOSiheKQXgVQI4xZl33B0TkP32WQOlBWhqrerXjgJJKWMXQLlfgLtdNTcDEibGRKSlobwfuv5+9XrpXmzc3A3//O1eHEQjCBXU4GGOuMsYs9/PYt/osgeKTk0/mbnDnzp5tG+rr2Q7iiiu044CSPGRn0w1UUeH/GGOYTq0D0zqxYgWwcSN9bd3dQ9nZbBX8/POB39gQUQ90gpKZCdx8M4fB7N/Pfmy7dtFApKez48Cxx8ZbSkUJj/nz2W3bV/daqyp/4kQuhhTwzXrtNRZn+HMJpaXRF7diRZ+fLpSg8lpjTMBhOKEco4SP1XHg3HOBL77gTqGgQDsOKMnLyJGsPL//fmbTFRRw0dvURO/HxIkcpKafbw/19SzSCRaJLyjg+MBzz+3T04USQ5gYINMIYHBZRx5FkZychOs4oMSSpiYqhrQ09vdO8j4eEyawoHLVKmYTtbay1cmcORy3rFM0OxHq5CkgIjOAQzEIE0I4JkjPSkVRwmbfPuCNN6g1jeEXftgw4Iwz6IxP4mV0Tg4NwJw58ZYkwcnLY6CwuTlwR8u6Opb295FQKpV3dv5bRO4Am9mtA7DOGKOlUYoSabZvZ7mu281h09akoPp64G9/Y8bJd76T1EZBCQG7nV0sn3nGf8qhy0V/cgQi8WF/mowxtwN4AEADgPNEJGnGaSpKUtDWxv4NWVld0wxFWIw0Zgx7m3zwQVzFVGLECScwhlBe3tMt1NHBMbDz50dkJnjIBkFE7hOh89IYU2GMWWyMudsYc3VfhRCRe0Rki4isF5GFIlLQ13MqStLyySfcCfgb5GOzcTDGK6+E52NWkpPsbOAnP2EV6q5dNADWz6oq4IILgPPPj1lhmkUjgJdF5CJjTJOInArgV8aY4/osBfAWgNuMMU4R+T2A2wDcGoHzKkry8eGHwWc85+ZSKVRVab/o/kBuLgfDV1RwWlZrKztCTpkS0WlZIRsEY8wvRORbAP4jIm0AmgD8NBJCGGOWdPpzJYBvRuK8ipKUWGXogbAmJrW3x0YmJTEYMoS3KBGOy2gugKtBQ1AM4AZjzPtRkOlKAG8EkOMaEVktIqurqqqi8PSKEmdGjGATs0BYw6jzNeNbiRzhBJV/DuCXxpgTwRX8syJyUqj/LCJvi8inPm5ndTrm5wCcAJ7ydx5jzCPGmDJjTFlxcXEY4itKknDcccwaCRQfqKgAjj5ae5coESUcl9FJnX7fICLzASwAEFIDBWPMyYEeF5HLAZwBYK4JNsZNUVKZkhJ2LfzwQ06S6R4srK+nsdCJQkqE6XUjbWPMPo8bqc+IyDwwiHyCMaY5EudUlKRFBLjqKir9VatYoZyTw11DQwPnit5yC/tAKEoE6dNkBWNMS4Tk+AuADABveTJbVxpjro3QuRUl+cjIAK67jt0M33+fA7QzMrhzOPLIwIMyFKWXRG7UTh8wxhwSbxkUJeEQocuotDTekij9BK17VxRFUQCoQVAURVE8JITLSFESlf376cLfuJEx3gkT2ENs+PCk70KtKD1Qg6AoPjCGg6oWLKDiHziQ97/zDvDmm8z4/OY3tdmoklqoQVAUHyxbBjz7LEsC0tK89+fmskj4lVeYCaqlAEoqoesbRelGRwfwwgt0C3U2BhZ2O+eav/QS0BKpxGtFSQDUIChKNz77jK2EAqX6p6dzbMHGjbGTS1GijbqMFKUbdXWhHScC1NREV5aEoa0NWLeOQZTqavrLTjgBmDEjeKtuJWlQg6DEnKoqYOtW6piCAmDyZCAzM95SeUlPD+04Y/pJwXBlJfCHP/Bnfj4vVl0d8PjjjLr/6Ef+xzsqSYUaBCVmNDQATzwBrF7NSYAivKWnA+ecA5x2WmKkch52GLOHnE7/Ywks+SdMiK1sMaelhcagoaFrxXRWFo1DbS1wzz3AnXcChYXxklKJEBpDUGJCczP1ytq1DMiOGUP9Mno0Bz899RSwcGG8pSR5eaw1KC/334F6zx6grAwoKoqtbDFn7VruDPwNZSko4ECfZctiKpYSHdQgKDFh6VKOgB01qmfufkYGDcPLL7MQLBG48EKu/nfs6DqrprkZ2L6dr+Pyy+MnX8x45x3/s50thgwB3npL5zunAGoQlKjjdAKLFwNDh/o/xuFgOuf70ZjB1wsyM4GbbgK+8x3+vWsXb+3twLe/Ddx6az+JpVZVBZ/Zm5FB15KO80x6NIagRJ26OrqgrWpff+TnA1u2xEamUMjIAE46CTjxRM6kAViYZrfHVazYkpNDRR8o0u508k3xVbShJBW6Q1CiTiIEivuCzUavSUFBPzMGAIMpBw8GPqayEjj2WO3jkQLoFVSiTn4+b01NgY+rrWUKqpJAHHMMt0oNDb4fb21laffJASfkKkmCGgQl6tjtwPz5nAvvL+7Y0cHHjj8+trIpQSgoAG68kT6z8nJeKIANnfbuZRbAd7/Lpk9K0qMxBCUmzJ7N8cDbtnEUcGfXS0sLsG8fM3sGD46fjIofJk4E7rgDePddpos5nfQDHn00cMopWpSWQohJ4lSxsrIys3r16niLoYRIczPwzDPAihUs7LLIyWEr6dmzkz/ekPI4nXQTZWRoEDmJEZE1xpiy7vfrDkGJGdnZwJVXAuedxwZy7e2MLYwfr7olaXA4+km+bf9EDYISc/Lz2RNNUZTEQoPKiqIoCgA1CIqiKIoHNQiKoigKADUIiqIoigc1CIqiKAqABDEIInKniKwXkXUiskREhsdbJkVRlP5GQhgEAPcYYw43xkwD8CqA2+Msj6IoSr8jIQyCMaa+058DACRv+bSiKEqSkjCFaSJyF4DLANQBmBPguGsAXAMAJdpQS1EUJWLErJeRiLwNwNfMrJ8bY17qdNxtADKNMb8Kdk7tZaQoihI+ce9lZIwJtWH6vwG8BiCoQVAURVEiR0LEEETk0E5/ngkggQYpKoqi9A8SJYZwt4iMB+AGsBPAtXGWR1EUpd+REAbBGHNevGVQFEXp7ySEy0hRFEWJP2oQFEVRFABqEBRFURQPahAURVEUAGoQFEVRFA9qEBRFURQAahAURVEUD2oQFEVRFABqEBRFURQPCVGprEQfpxPYvh1oagKysoBx44C0tHhLpShKIqEGIcUxBli6FFi0CKirA0R4X3Y2cMYZwKmnAnZ7vKVUFCURUIOQwhgDvPgijcGwYcDo0d7HWluBp58G9u8HLr8csKnzUFH6PaoGUpgdO4BXXqEhyM7u+lhmJlBaCrz3HrBxY1zEUxQlwVCDkMK89x6Qng44/OwDbTYgNxd4883YyqUoSmKiBiGFWbcOKCwMfExREXcIbndMRFIUJYFRg5DCuN2hxQaM4U1RlP6NGoQUZtw4oLY28DH19cCoUZpppCiKGoSU5pRTgIYG/6t/Y4CDB4H582Mrl6IoiYkahBRm0iSgrAzYubNnjMAYYPduYPx4HqMoiqJ1CCmM3Q5873vAU08B77/P+xwOVi0bQ0NwxRXMRFIURVGDkOJkZABXXgmcdRazjmpqgLw84IgjgCFD4i2doiiJhBqEfkJhITB3brylUBQlkdEYgqIoigJADYKiKIriQQ2CoiiKAgAQk8QlqiJSBWBnFJ+iCMCBKJ4/miSr7Cp37ElW2ZNVbiD+so82xhR3vzOpDUK0EZHVxpikzNJPVtlV7tiTrLInq9xA4squLiNFURQFgBoERVEUxYMahMA8Em8B+kCyyq5yx55klT1Z5QYSVHaNISiKoigAdIegKIqieFCDoCiKogBQgxAUEblTRNaLyDoRWSIiw+MtUyiIyD0issUj+0IRKYi3TKEiIueLyEYRcYtIwqXmdUdE5onIVhHZJiI/jbc8oSIi/xCRShH5NN6yhIOIjBKR90Rks+dzcmO8ZQoFEckUkY9E5BOP3L+Jt0zd0RhCEEQkzxhT7/n9BgCTjDHXxlmsoIjIqQDeNcY4ReT3AGCMuTXOYoWEiEwE4AbwMIAfG2NWx1kkv4iIHcBnAE4BUA5gFYCLjTGb4ipYCIjIbACNAJ4wxkyJtzyhIiLDAAwzxqwVkVwAawCcnejvuYgIgAHGmEYRSQOwHMCNxpiVcRbtK3SHEATLGHgYACApLKgxZokxxun5cyWAkfGUJxyMMZuNMVvjLUeIzASwzRiz3RjTDuAZAGfFWaaQMMYsA1AdbznCxRizzxiz1vN7A4DNAEbEV6rgGNLo+TPNc0sofaIGIQRE5C4R2Q3gEgC3x1ueXnAlgDfiLUSKMgLA7k5/lyMJlFOqICKlAKYD+DDOooSEiNhFZB2ASgBvGWMSSm41CABE5G0R+dTH7SwAMMb83BgzCsBTAK6Pr7RegsntOebnAJyg7AlDKLInCeLjvoRa9aUqIpIDYAGA/+m2k09YjDEuY8w0cMc+U0QSylWnA3IAGGNODvHQfwN4DcCvoihOyASTW0QuB3AGgLkmwYJFYbzniU45gFGd/h4JYG+cZOk3eHzwCwA8ZYx5Md7yhIsxplZE/gNgHoCECerrDiEIInJopz/PBLAlXrKEg4jMA3ArgDONMc3xlieFWQXgUBEZIyLpAC4C8HKcZUppPMHZRwFsNsb8Kd7yhIqIFFvZfiKSBeBkJJg+0SyjIIjIAgDjwayXnQCuNcbsia9UwRGRbQAyABz03LUyGbKjAEBEzgHwZwDFAGoBrDPGnBZXoQIgIqcDuA+AHcA/jDF3xVei0BCRpwGcCLZirgDwK2PMo3EVKgRE5HgA7wPYAH4vAeBnxpjX4ydVcETkcACPg58TG4DnjDF3xFeqrqhBUBRFUQCoy0hRFEXxoAZBURRFAaAGQVEURfGgBkFRFEUBoAZBURRF8aAGQVEURQGgBkFJEUSkVERaPH1irPt6tKUWkSxPK/N2ESnq43NmichST8dTiMgNnpbMYbUJEZECEbmuL7KE8Bw9Wl2LSLqILBMR7VigAFCDoKQWX3j6xFhtqf8KYD6ASQAuFpFJxpgWzzGRaC9xJYAXjTEuz9/XATjdGHNJmOcp8PxvWAgJ9Tv8T7BNwld4urO+A+DCcJ9bSU3UIChJg2coyime3/+fiDwQ4PBYtKW+BMBLHnkeAjAWwMsicpOIfNszDGWdiDzcaRexSETWeAakXOM5z90AxnmOvcez2+m8kv+xiPza83upZxfyIIC1AEb5e67OBGh1vcjzOhRFDYKSVPwKwM9F5BKw5fFNAY6NaltqT9+iscaYLwHA0xZkL4A5ABaDq+7jPLsRF7xK90pjzFEAygDcICKFAH4Kz+7GGHNLCE8/HhxqMx1AdoDnCoVPAcwI43glhVHfoZI0GGOWeRqb3QzgRMtVIyJ3gs3OOhPtttRFYJ8lX8wFcBSAVRQXWWD/e4BG4BzP76MAHApgf5jPvbPTlK1AzxUUY4zLE0/J9QybUfoxahCUpEFEpgIYBuCApbxEZCh8f47DbkstIj8AcLXnz9MBnNP5b2NM5/9vAZDp71QAHjfG3Nbt/CeCHS6PMcY0e9of+zqHE113792PaQr2XGGSAaC1D/+vpAjqMlKSAuEc3afAOECTiFjdT6cDWOfjX8JuS22M+avHbTPNGLO3+9/djq0BYBcRXwr9HQDfFJHBHtkHichoAPkAajzGYAKAWZ7jGwDkdvr/CgCDRaRQRDLAmRb+8PdcIeFxWVUZYzpC/R8ldVGDoCQ8IpIN4EUAPzLGbAZwJ4Bfex6eBh8GwTNP+noAb4Izd58zxmyMsGhLABzv47k3AfgFgCUish7AW+DOZjEAh+e+O8FZ1zDGHASwwjMx7h6Pcr4DHAv5KgL0zA/wXF3wtLr+AMB4ESkXkas8D80BkNBto5XYoe2vlaRGRB4F3TolAF41xoQ0klBEvgRQZow50Ifnng7gZmPMpb09R7wRkRcB3GaM2RpvWZT4ozsEJakxxlxljHGD2TX5nQvTfGEVpgFIg3e4Sm+f+2MA7/lK80wGPK60RWoMFAvdISiKoigAdIegKIqieFCDoCiKogBQg6AoiqJ4UIOgKIqiAFCDoCiKonhQg6AoiqIAUIOgKIqieFCDoCiKogAA/j9ckqQf6WEScQAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": { - "tags": [], "needs_background": "light" - } + }, + "output_type": "display_data" } + ], + "source": [ + "plot_dataset(train_x, train_labels)" ] }, { @@ -820,29 +786,32 @@ "id": "SjPlpf2-wHl8" }, "source": [ - "## Обучение простейшего одноуровневого персептрона вручную\r\n", - "\r\n", - "Используем возможности tensorflow по вычислению градиента для обучения одноуровневого персептрона.\r\n", - "\r\n", - "Для начала, задаём архитектуру сети, в которой будет 2 входа и один выход. Соответственно, матрица весов $W$ будет иметь размерность $2\\times1$, а вектор сдвига $b$ -- $1$.\r\n", - "\r\n", - "Функция обучение будет такая же, как в прошлом примере, но функция ошибки будет представлять собой логистическую функцию ошибки. Для этого нам нужно получить на выходе сети значение **вероятности** класса 1, т.е. необходимо привести выход сети $z$ к диапазону [0,1] с помощью передаточной функции `sigmoid`: $p=\\sigma(z)$.\r\n", - "Далее, если для примера с номером класса $y_i\\in\\{0,1\\}$ был получен выход сети $p_i$, то ошибка вычисляется как $\\mathcal{L_i}=-(y_i\\log p_i + (1-y_i)log(1-p_i))$. \r\n", - "\r\n", - "В Tensorflow оба эти этапа (применение сигмоиды и взятие логистической функции ошибки) делается одним вызовом `sigmoid_cross_entropy_with_logits`. Поскольку мы делаем обучение по минибатчам, то необходимо усреднить ошибку по всем компонентам минибатча с помощью `reduce_mean`. " + "## Training One-Layer Perceptron\n", + "\n", + "Let's use Tensorflow gradient computing machinery to train one-layer perceptron.\n", + "\n", + "Our neural network will have 2 inputs and 1 output. The weight matrix $W$ will have size $2\\times1$, and bias vector $b$ -- $1$.\n", + "\n", + "Core model will be the same as in previous example, but loss function will be a logistic loss. To apply logistic loss, we need to get the value of **probability** as the output of our network, i.e. we need to bring the output $z$ to the range [0,1] using `sigmoid` activation function: $p=\\sigma(z)$.\n", + "\n", + "If we get the probability $p_i$ for the i-th input value corresponding to the actual class $y_i\\in\\{0,1\\}$, we compute the loss as $\\mathcal{L_i}=-(y_i\\log p_i + (1-y_i)log(1-p_i))$. \n", + "\n", + "In Tensorflow, both those steps (applying sigmoid and then logistic loss) can be done using one call to `sigmoid_cross_entropy_with_logits` function. Since we are training our network in minibatches, we need to average out the loss across all elements of a minibatch using `reduce_mean`: " ] }, { "cell_type": "code", + "execution_count": 57, "metadata": { - "trusted": true, - "id": "kdDxWeCqwHl8" + "id": "kdDxWeCqwHl8", + "trusted": true }, + "outputs": [], "source": [ - "W = tf.Variable(tf.random.normal(shape=(2,1)))\n", + "W = tf.Variable(tf.random.normal(shape=(2,1)),dtype=tf.float32)\n", "b = tf.Variable(tf.zeros(shape=(1,),dtype=tf.float32))\n", "\n", - "learning_rate = 0.1\n", + "learning_rate = 0.01\n", "\n", "@tf.function\n", "def train_on_batch(x, y):\n", @@ -853,9 +822,7 @@ " W.assign_sub(learning_rate * dloss_dw)\n", " b.assign_sub(learning_rate * dloss_db)\n", " return loss" - ], - "execution_count": 22, - "outputs": [] + ] }, { "cell_type": "markdown", @@ -863,52 +830,52 @@ "id": "zAAgw0h6KzUd" }, "source": [ - "Далее, разбиваем входные данные на минибатчи по 16 элементов, и по-очереди проводим обучение, подстраивая веса $W$ и $b$" + "We will use minibatches of 16 elements, and do a few epochs of training:" ] }, { "cell_type": "code", + "execution_count": 58, "metadata": { - "trusted": true, "colab": { "base_uri": "https://localhost:8080/" }, "id": "PfyqjVb2wHl8", - "outputId": "308850b8-fe17-4cda-ac27-8bcda210f113" + "outputId": "308850b8-fe17-4cda-ac27-8bcda210f113", + "trusted": true }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 0: last batch loss = 0.8960\n", + "Epoch 1: last batch loss = 0.7991\n", + "Epoch 2: last batch loss = 0.8121\n", + "Epoch 3: last batch loss = 0.9656\n", + "Epoch 4: last batch loss = 0.7388\n", + "Epoch 5: last batch loss = 1.0532\n", + "Epoch 6: last batch loss = 0.7494\n", + "Epoch 7: last batch loss = 0.8559\n", + "Epoch 8: last batch loss = 0.8226\n", + "Epoch 9: last batch loss = 0.7524\n", + "Epoch 10: last batch loss = 0.9427\n", + "Epoch 11: last batch loss = 0.6479\n", + "Epoch 12: last batch loss = 0.7756\n", + "Epoch 13: last batch loss = 0.7771\n", + "Epoch 14: last batch loss = 0.7996\n" + ] + } + ], "source": [ "# Create a tf.data.Dataset object for easy batched iteration\n", "dataset = tf.data.Dataset.from_tensor_slices((train_x, train_labels.astype(np.float32)))\n", - "dataset = dataset.batch(16)\n", + "dataset = dataset.shuffle(128).batch(16)\n", "\n", "for epoch in range(15):\n", " for step, (x, y) in enumerate(dataset):\n", - " loss = train_on_batch(tf.reshape(x,(-1,2)), tf.reshape(y,(-1,1)))\n", + " loss = train_on_batch(x, tf.expand_dims(y,1))\n", " print('Epoch %d: last batch loss = %.4f' % (epoch, float(loss)))" - ], - "execution_count": 25, - "outputs": [ - { - "output_type": "stream", - "text": [ - "Epoch 0: last batch loss = 0.5500\n", - "Epoch 1: last batch loss = 0.5502\n", - "Epoch 2: last batch loss = 0.5504\n", - "Epoch 3: last batch loss = 0.5506\n", - "Epoch 4: last batch loss = 0.5508\n", - "Epoch 5: last batch loss = 0.5510\n", - "Epoch 6: last batch loss = 0.5513\n", - "Epoch 7: last batch loss = 0.5515\n", - "Epoch 8: last batch loss = 0.5516\n", - "Epoch 9: last batch loss = 0.5518\n", - "Epoch 10: last batch loss = 0.5520\n", - "Epoch 11: last batch loss = 0.5522\n", - "Epoch 12: last batch loss = 0.5524\n", - "Epoch 13: last batch loss = 0.5525\n", - "Epoch 14: last batch loss = 0.5527\n" - ], - "name": "stdout" - } ] }, { @@ -917,42 +884,57 @@ "id": "s4_Atvn5K4K9" }, "source": [ - "Для демонстрации того, как сработало обучение, построим граничную прямую $W\\times x + b = 0.5$" + "To make sure our training worked, let's plot the line that separates two classes. Separation line is defined by the equation $W\\times x + b = 0.5$" ] }, { "cell_type": "code", + "execution_count": 49, "metadata": { - "trusted": true, "colab": { "base_uri": "https://localhost:8080/", "height": 283 }, "id": "PgRTHttLwHl9", - "outputId": "e4407e1b-edf5-48e5-fdc2-da28120a3c6b" + "outputId": "e4407e1b-edf5-48e5-fdc2-da28120a3c6b", + "trusted": true }, - "source": [ - "plot_dataset(train_x,train_labels,W.numpy(),b.numpy())" - ], - "execution_count": 26, "outputs": [ { - "output_type": "display_data", + "name": "stderr", + "output_type": "stream", + "text": [ + "C:\\Users\\dmitryso\\AppData\\Local\\Temp/ipykernel_66332/2721537645.py:17: UserWarning: Matplotlib is currently using module://matplotlib_inline.backend_inline, which is a non-GUI backend, so cannot show the figure.\n", + " fig.show()\n" + ] + }, + { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEKCAYAAAASByJ7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydd3hU1dbG3z2T3gtJgJCQQhFpAlFERJqCYPeq2CufBa9YsNBROna59nLtF70qliuo9A7SkY4ISYA0SkhPJpPZ3x8v46RMz7Qk+/c884hzzpyzZjKz115dSCmhUCgUCoXG2wIoFAqFwjdQCkGhUCgUAJRCUCgUCsU5lEJQKBQKBQClEBQKhUJxDqUQFAqFQgEA8PO2AI2hVatWMiUlxdtiKDzIieITyCvNw/lx5yPYP9jb4igUTZJt27adklLG1X++SSuElJQUbN261dtiKDxIYUUh0uanITUpFT/f/rO3xVEomiRCiCxzzzdphaBoeUQHR2N8//EYv3w81matxYD2A7wtUtPm1Clg40bgyBFAqwW6dwcyMoDQUG9L5hxSAllZwOrVwMGDgEYD9OwJDBgAtG7tbel8HtGUK5UzMjKkshBaHuXV5egwvwPSotOw9r61EEJ4W6Smh8EA/Pgj8NNPgBBAWBifKysDAgKAhx4Cevf2tpSOUVMDfPEFsHIl4O8PREVRQRQW8r3dfDMwYgTfbwtHCLFNSplR/3kVVFa4jaoq4PffgU8+AT7+mJu2srLGXzfEPwTTBk7D+mPrsejPRY2/YEtk0SJg4UIgMRFITgZiYoBWrYD27YHISGD+fGD/fm9L6Rg//AAsX873k5hIKycsDEhKAtq2BRYsANav97aUPo2yEBRuYdcu4L33gPJyICiIm7KKCm7c7rwTuOyyxm3Uqmuqcf7b5yPILwg7H9oJrUbrOuGbOyUlwJNPAgkJ/IOYo7AQiI4Gpk1rGjvqkhLgiSeANm0APwue8LIyQK8HXn6Z7rEWjM9bCEKIJCHESiHEPiHEXiHE496WSeEc+/YBr73GDVpKCl23CQn8d6tWwEcfNX6j5q/1x8zBM7GnYA8W7FngCrFbDjt20L1iSRkAdLdkZgLHj3tMrEbxxx98T5aUAcAv5NmzwOHDnpOrieEzCgGAHsA4KeX5AC4G8KgQ4nwvy6RwECmBL7/kemIuLhkYyE3cf/4D6HSNu9fNXW9Gr9a9MGXlFOhqGnmxlkRurnVlANAq0GhoKTQFzpyxb9cvBFBc7H55mig+oxCklLlSyu3n/l0CYD+ARO9KpXCUzEwgJ4duaEsEB9N9tGdP4+6lERrMGToHmWcz8f629xt3sZZEUBB30/ZgbcftanJygG++oXn51lsMQFVV2ffakBD73pOU3JUozOKTaadCiBQAvQD8bubYgwAeBIDk5GSPyqWwzalT3ITZcjsLAeTlNf5+w9KHYVDKIMxYMwP3XnAvwgLCGn/R5k7XrsD333NxtPSHqq6mMvBE4Wd1NfD558CaNbyncXHfupVB4bFjgY4drV/j/HPOBFvvyd8f6NDBtfI3I3zGQjAihAgD8B2AJ6SUDWw7KeX7UsoMKWVGXFyDQjuFl9Fq+Zu0hV4P/PUXMHs28PjjwHPPMQPy9GnH7ieEwJyhc1BQVoDXN73unNAtjfR0ZuKcOmX+uJTAiRPAkCFcnN3NF18Aq1ZRpnbtmPEUF8eMJz8/4MUXbccy2rQBevWi3OaQkte4/HLPvKcmik8pBCGEP6gMvpRSLvS2PArHMW4orVnv5eXMQlq1ilZCSIgpLX78eGD7dsfueXG7i3H9edfjpQ0v4VS5hUVOYUIIYMwYau9jx6idjVRUAEePAp06Addf735ZcnJoGaSkMGZRn8hIPv/TT7avdd99VAxHj5rym6VkzODIEeCCCzzznpowPqMQBKuLPgKwX0r5qrflUThHTAxw0UX8nZvDYGDdkFYLdOvG8wMDGYBOSmKm45tvMhbhCDMHz0SprhRz181t9HtoEbRuDUydCgwcSK187BiQnU1tfcstwFNPMdbgbjZs4IJvThkYiY+n+6ioiLL+9BPw/vvMXti/n18qAIiIACZMAO64g0ouO5uPoCAW2v3znyy6U1jEZ+oQhBCXAlgLYDeAc39hTJRSLrb0GlWH4JsUFQFz5gAFBdywGRNaamr4+927Fxg5EggPN//6vDx2G3jkEcfue9+P92HB7gX487E/kRSZ1Lg30ZIoK2OWjkbDxddWBpIref117t5jYqyfl5UFdOnCnGatlrsIvZ5B53btgMceq9uawmAASktNVdhNoZbCg/h8HYKUcp2UUkgpe0gpLzj3sKgMFL5LZCQwcSIwaBCVwrFjdN+eOMG1pm9fy8oAoPt461bHq5qfH/g8JCReWP1Co+RvcRjNs8REzyoDgLv32i4rc0gJHDgAbN7MOENSEhVX27ZAaipTY+fOrZsiq9HQYggPV8rAAXxGISiaFxERwN13cwM4fjzw7LPAq69ykxcba/21xnRyRxVC+6j2GJMxBh/v/BgHTh1wTnCFZ+nTh24qa+TkMA7QpYt511J8PI+vWOEeGVsQSiEo3EpoKDMGO3VisVp4uO3Ucilp8Tvjwp44YCJC/EMwecVk5wRWeJaePbl7KCoyf1xK+hhTU60XniUkAMuWMbVU4TRKISg8yqWX0rVrjdOnmVYeEeH49eNC4/B0v6fx3f7vsOXEFueEVHiOgADmHZeV0RKonZ5WWsr4QlQUdxTWCAzkTkNVITcKpRAUHqVHD7qMLKXAV1ezT9lVVzl/j6f6PYVWIa0wYfkE5y+i8Bzp6Wyil5FBpWDMeAKYSnrxxbaLW6Tkw5OV1c0Q9ekpPEpgIDMa583jbz4+3tRJoaCAm7w77jAVnjpDeGA4Jg+YjCd+ewLLjizD5WmXu+4NKNxDYiLw4IPAbbexAZ2fH78cWi13CV99Zb0fSnExs42cMSsVf6MsBIXHSUwEpk8HbryRXoGsLGYgXXABMHkyMGxY4+/xcMbDSI5Mxvhl4+ErqdUKOwgPZxZRmzammEHfvvx3RYX510hJk/Oqq1RGUSNRFoLCK0RFAVdfzd9wVRWzHV3Zoj7QLxDTB03HvT/ei+/2f4ebzr/JdRdvSRiLR5YtY+5wQAAX6P792cvcE0RFsbDszTdpAcTEmBb+8nJ2bx0wgBWRikbhM4VpzqAK0xTWqDHUoMe7PaA36LF3zF74adT+xyHKyrgI79vH4q6wMCoIYxHb6NH073uKAweA777jPAOtlqloERHcVQwZ0uKH3jiCpcI09QtRNFu0Gi1mD5mN67++Hp/s/ASje4/2tkhNBynZHuLgQfYZqu2KCQuj++add9hrpHNnz8h03nmseCwoYJpqQADjBiqQ7DJUDEHRrLm287W4uN3FeH7V86iotuCDVjQkK4sdCJOSzPvlg4Pp7//xR8/KJQRrDjp1oqJyVBlIyeZ3q1ezqVbtXkgKZSEoPI+Upnb71nqauQIhBOYOnYtBnw7CW1vewtOXPO3eGzYXNm7kH8hakDY2lgvq6dO2y899gWPHOL81M7Pu3IRWrZje2rWrV8XzBZRCUHiMwkJg7VpgyRK6p7Va4JJLgKFD2freXQxMGYgrO1yJ2WtnY3Tv0YgKinLfzZoLBQW0AqxhHLNZUuL7CuHECWDWLH7p2revq+iKi4GXXwbGjWML3haMchkpPEJ2NjBlCgd1hYXxN9m6NbBpE2uS1q517/1nD5mNwspCvLzhZffeqLkQHm67DYSxx0hTGEn55ZdUAnFxDa0eY+bShx/abrTXzFEKQeF2KiqAV17hZrJ9e1OPIj8/Nqxs04aW/F9/uU+GXm164dZut+K1Ta8hr9QFszubOxddBFRWWj+nuJh/vNptp32R3FxmSsXHWz4nPJwFcfv3e04uH0QpBIXb2baNa4ellveBgVQSv/3mXjlmDJ4BXY0OM9fMdO4Cej1THvfsYdC1Cads26RLF2rrggLzx2tqWAx2zTW+XwyWm8vdiC05tVr+XVswKoagcDtr1tjuKGCcgVBZ6b5BXR1iOmB0r9F4b9t7eKrfU0iLTrPvhQYDC7MWLaK/XAg+17o18I9/sAdPc0OrZdO5efO4SMbHM6YgJYPIxcXM//dkHYK7ac4K3k6UhaBwO8XFtt3MxpoiW62xG8uUgVPgr/HH1JVT7XuBwQB88gnw+ed8E8YBLcnJFHb+fEbJmyMJCQzw/OMfrAjOyuKjfXvgmWeAUaN83zoA2CvFGO+whsHANtstGGUhKNxObCyDyiEhls/R62nV20psaSxtw9vi8b6PY976eXjmkmfQs3VP6y/YtQtYtYoLRe0cWSFo9gQFAQsWAN2705/e3IiMZI+RkSMZDPLzaxpB5NokJDB76M8/Lcc7ior4RT3vPM/K5mMoC0HhdgYNoqfFGvn5bI/jiaLTZ/s/i8igSExaMcn2yb/8wkXRUsFEQACPrV7tWiF9DY2G046amjIwcscd/HLl59d1DUnJfOiiInZbbeHtL5RCULidnj25McvPN3/8+HEmgSxZAtx/P/Dcc9yU25qs6CzRwdEY3388Fv25CGuzrOS7VlUBhw7ZHgAfGwvs2OFaIRWupXVrYNIkuruys1mcZnSBhYUBEyZ4rgWHD6Oa2yk8wsmTrP3Jz2fzytBQrre7dvE32aOHaWRuaSnjlm3a0FVtaz12hvLqcnSY3wFp0WlYe99aCHO+8IoKYMwY21VzlZXM2X/lFdcLqnA9J05QKUhJd1JaWtOIhbgQS83tlIWg8AhxccDzzwP/93/sh1ZUxIxGvZ4u6m7daK0LwZTwlBQqhX/9yz2tZkL8QzBt4DSsP7Yei/5cZP6koCAKW1Zm/WJFRRRY0TRITAT69WOZfHp6i1MG1lAKQeExgoMZJ5g2jQt9aqppxro52rZlH7I//3SPPPf3uh8dYjpgwvIJqDHUNDxBCGDECMu5+AB3meXlwOVqKpui6aMUgsIr5OfTarc2FRFgzPb33xtxIykBnc5sjrm/1h8zB8/EnoI9WLBngfnX9+/PFssnTjS8hsFAf1dGhsP+5+pqhh2+/x744Qdg9+4W3zVB4QOotFOFV6ioMLmIrBEYyI4CDnPyJDN/li2jj9/fn+lOgwfT9DjHzV1vxrz18zBl5RTc0vUWBGgD6l4nJISBjHff5YAWjYZayqhkLr0UuOsuh9q2bt8O/PvfjJX4+fEyNTWMrTz8cIvPfFR4EaUQFF4hNJSLYO0uxOaorHSikeahQwzw6vWssA0M5JZ85Uo+Hn0U6NULAKARGswZOgdXfnkl3t/2Pv550T8bXi8qiqlP2dnc1hu7e2ZkWO+PY4adO4E33uDL6k+gLC4GXnyRCS8dOzr4nhUKF6BcRgqvEBfH5I7CQsvnGOcmONQd4exZ4LXXqHGSkkx58/7+dP3ExABvvcX+NucYlj4Mg1IGYcaaGSjVlZq/rhDMNrr+eloEI0c6rAz0ehY9x8VRvPpERDCg/umnqouCwjsohaDwCkIA111HhaDTmT/nxAm6T9LsbDkEANiwgfmsliLVxnLpVatqySIwZ+gcFJQV4PVNrztwM8c4cIAJSWFhls+JimJdRmam28RQKCyiFILCa/ToAdx7Lzfrx49zHdfrqSSOHuWGfswYB7MCV62y7WNKSOB5tbbhF7e7GNefdz1e2vASTpWfcuLd2CYvz/bO3zhzxlIRn0LhTpRCUHiVoUOBGTMY7y0tZSw4NpZu/okTbXdJbUBJCYO+1vD3p/apqZtqOnPwTJTqSjF33VwHb2ofGo19riAp3T9aVKEwh08FlYUQ/wZwNYACKWXLnmXXgmjXDrjzTj7qY1xA7bYSYmJMWUWWqKqi66he35qu8V1xd8+78ebmN/F438eRFJlk503tIyWF78NaIF1KPpKTXXprhcIufG0f8gmAK70thMK7VFdzNsKMGcADD7C6+Y032O/I5g576FDgzBnr5xQUAFdcYXZVfn7g85CQeGH1C86/AQukpjIufcqKRyovj41TfX0ImaJ54lMKQUq5BoCNX7OiOVNRAbz+OscMnDpF66FNG+DgQc5q+fxzG60sLrqI1W6nT5s/XlRE6+Gyy8webh/VHmMyxuDjnR/jwKkDjX9DtRACGD2a8ufl1X0fBgOD6EFBTGJSKLyBTykEheLzz2kJpKayjZBGQ89OfDx318uWAUuXWrlAWBjw9NN8UWYmk/urqxmgyMqiu+jpp60GnicOmIgQ/xBMXjHZ5e8vKQmYPBno0IFlDcbHsWO0DCZPdjibVaFwGT7X7VQIkQLgZ0sxBCHEgwAeBIDk5OQ+WS18Bmpz4uRJ4NlnuWhaCqpWVLB10KuvWg8ToLQU2LIFWL6caUsREaxS7tvXdr8MAC+segHPr34em0dvxoWJFzr3hmyQn09LQQj2W3O4AE+hcBJL3U6bnEKojWp/3bxYtgz48kvb3aazsqg4unRxnywlVSVIm5+Gngk9sezuZe67kULhBVT7a4XPU1xs38Q0Idw3PMdIeGA4Jg+YjOVHl2PZEaUQFC0Dn1IIQogFADYC6CyEOC6EeMDbMik8R2Qk3f32YK71g6t5OONhJEcmY/yy8fA1S1qhcAc+pRCklLdJKdtIKf2llO2klB95WyaF5+jZk7t/a1lEFRVUBunp7pcn0C8Q0wdNx7bcbfhu/3fuv6FC4WV8SiEomheFhWxJYasswEirVhw/YJxuaCSk/BQ6Hfwf+mx6C51Wv487O2+Bv6HKPULX484ed+L8uPMxecVk6A1qYIGieeNTlcqK5sGBA8BPPwH79zP7s6aG7Zyvuw7o2uYMkJPDE9u2bTAw+c47GUvYuROICq/BJbnfoeNfv6CySqBchmBAkgEdtmwADoQBjz3m9sHoWo0Ws4bMwg1f34BPd36KB3orL6ai+eJzWUaOoLKMfI/Vq4GPPmKWZ2ysqVWD7sRJdP7jG1wRvRWJ7c5VCBsMQJ8+wKhR7Al9Dr0e2LMHOPLiN2iz5SecCU9BfBst0tKoP4QAtUZxMTBliu20pEYipUS/j/rhRMkJHPrnIQT7B7v1fgqFu1FZRgq3c+IE+/0nJtL9Y+wMEVaWj+t2z8D5uh3YmtcOZ8OTWGyQlATs2sUeFbXae/r5ARckn8GNQb+g360puOpaLS680KRgAFDj+PsDCxe6/X0JITD38rk4Xnwcb2952+33Uyi8hVIICpexahVdRMaZNAAAKdFr58fQ1uhQHpUIjZ8GR4+eO6bRUHvodMDHH9cNHBgHKddrQFeHuDjgjz8st6lwIYNSBmF4+nDMXjcbRZVFbr+fQuENlEJQuIyNG+t4fgAA4SU5iD11AGWhCQCYIXT8eL0XJiQw8GCMLQDs5RAUZP2GGg0f9katG8nsobNxpuIMXt7wskfup1B4GqUQFC5Dp2tYWBZ1NpP/OOfrEcI0S/lvjH6g2mPCAgMbzCswi5T2VbPZorycaVFVlrOXerfpjVFdR+HVTa8iv9RHJticOcPPLSfHRtc/hcI2KstI4TJat+Z8mtqtgoQ0wE9fiZhThxCgK0WV9EdgeGsItAJQr/107QXtgguAlSut37CyEggOptvJWQ4eBH75hbEMIeiiGjCA7bHbtGlw+ozBM/Dtvm8xc81M/Gvkv5y/b2P56y/ghx8Yfddq+dnFxQFXX83cXTVhR+EE6lujcBnDh9f13mhqqpGUvQ5tc7Yi7tR+hJfmIvJMJroXrQdWrKD2MCJEXX9T165MKSosNH8zKTl7c/hw2xPSLLF0KTBrFnDoEAPcycl0X61ZA0ydSjdWPTrGdsTo3qPx3rb3cKTwiHP3bSw7dwIzZ1IpJCWxR3hyMtOz3n8f+OIL+0azKRT1UApB4RBSMpto9262qa7dU6hPH65NOTk8sefOTxF3ch/KQ+Kg9wtCGUJRExKOkLZR3N2vW8fS4+JiphB16mS6mJ8f6wwqK3nB2u6jigoOXe7WDbjSyXlKf/7JhbNdO/abNu6o/fz4XEQEp/IUFzd46dSBU6HVaDFt1TTn7t0YioqAt99mGldtuQEgPJx9w5cuBbZt87xsiiaPchkp7ObwYeCrr7gxFcI0EH7QIOCGGziV8umnOdymeHc24g6tQ15UKooiwpCStxGBIQJtkwOYOBQaysVt714qg6efbujmSE0Fpk1jldvmzaahxCEhwC23AMOGOW8d/Porr2Pu9VLy+dxcRsqHD69zuG14Wzze93G8uP5FPHPJM+iR0MM5GZzh999pCVhq5qTR0LJatIga2u7ZowqFE4VpQohQAJVSSjsifu5FFaZ5jj17OIMgNLRWcRjYjC4nh2v3M8/QpW8wALkvfY7in9fgdGAigoOB1KBcRGXugKZaZ1r4q6t58scfc9KZNYqK6I/SaunbtzoMwQZVVcDDDzccvGA0fw4fBs6e5cIbFARMnAgMGQJERf19amFFIdLmp+HS5Evxv9v+57wstqip4aAIvZ4Tg157jWm21mY6SMn+H2+8QUtHoaiHpcI0mxaCEEID4FYAdwC4EEAVgEAhxCkAiwC8J6U87GJ5FXZy9iywaRPdyno9J3ENGMCuEK7aHFZWAu+8Q0UQFlb3mL8/C4WPHGFs9sYbz5UX6DKReGE4yrRcm3bktIFBG482/vlICi9ESLDEaUM0KoqrcehgCtJj2LDOosyRkXYNtrELnc5k3hiRkh/i0aPUapGR/ECrq4Gff2ZcYfz4v4cdRwdH47n+z2HC8glYl70OlyZf6hrZjOj1vOeiRYyjGGXNz6dLy9pnYXxvetV7SeEY9riMVgJYBmACgD1SSgMACCFiAAwGME8I8b2U8gv3iakwx/r13FzX1HB90Gg4ZOa33zhr/rbbrNd12cuuXUBZWcMag9q0aQMsWQJcddW5wjR/f5zIrsHWv3g8OBgQWi0yq9pi+59todMBMdES8VXH8PMvWpQs54Z9zBizyT2uJTiYLiGdzuQyysykMoiOrmv+REYyYFtQQF/YjBl/f6hj+47F/N/nY/yy8Vh731oIV2lgvZ4aePNmBrmTk03PHzxIn92wYdTOubm0GKSkxm7blv/292+ovRX2U1PDqXtaLc3iFuJ6s0chXC6lbNClXkp5BsB3AL4TQjTCflc4wx9/AO+9x99/7fqtiAh+l3/7jQvzzTc3/l67d3MNtUZgINfX3FwgJQU4GpuBYzu+RGib6DplAsXF/J0JAaC0BDIxHgmpIeietwlxy1Zj9+KzCB8ZhbCrB7Eftq3iNGfw8+M4zSVLqIUMBmYahYXV/eHrdEBaGv8dH89RbYcO/T2qLcQ/BFMHTsUjix7B4j8X46pOV7lGvuXLqQzS0urK4+fHdNxly/gw1moYP+CsLH4xEhKA++5zPr7SkikpYUOu335jxoTBQIU8ciSQkeGaHZYPYzPLSEpZLYQ4TwgxVAhRZ8shhLjSeI67BFQ0RErg66+5ITS3Xmq1dOP88gtd741Fr7cvrd04y0BK4KsjF0Eb7I8gaUpD0uvpDg8KAoICJfzPnsTB6H4YsnIyLtz2LtoYTsBQXYPMDSeAd99l47rc3Ma/AXMMGcJd9NmzXAQqK+suoCUl1K61J977+wP1YlYP9HoA6dHpmLB8AgzSBYVhej2weDHNJHO70pgYPrKyaMFERVGRhYXx31LSf6diB45z+jQtwG+/ZcKBMRW5qAh4803gww/tK5Zswtj8mQshxgL4EcBjAPYIIa6rdXi2uwRTWCY7m4Fca25kPz8uztu3N/5+qam2R1Yaq49bteLg+IO5Edjd7yGElhUguJwuDWMGZ5CsQNuqo9gTciFOHzgJv+pKFEWloDI4BgFRIfjrTAyqWrfnTV96iSaFq4mL42Dmmhp+oDodH+XlVBJhYUC/fnV3hP7+9J3Vwl/rj5lDZmJ3wW4s2L2g8XLl5FAZhYRYPkdKppgWFtLkqqjgZ3T2LE25wYOB//3P/vFzCn6mb7/NxT8lxWQSC0FFm5ZGH+2SJV4V093YU4fwfwD6SCmvBzAIwBQhxOPnjrUMx5qPUVRkSvu0RkAAXd+NpW9f3sva+pKXB1x8MTemJSVcR/MS+2DdgIkoCW+DyKJshBYeQ9uabATKSiyLvx0rQq5GSYUW5aGm4ITREikvBxftwkK6T9xBWhowbx5wzz10v2g0jCFcfDEwcGBDP1llZV2L4Ry3dL0FF7S+AFNWToGuRtc4mfR663/YwkJ+OImJHDKRlkaZ27alAhs6lC6j0lKm9Crs4+hRxmbOJQ00QAhabYsXN2tFa08MQSOlLAUAKWWmEGIQgG+FEO2hFIJXsDfjUq+3vtG0l6goZg99/TUt6Pr3P3WKyue6c7ZjYKCpUPZ0bCesGzARoaX5OLytCHmn/VEanYQa4Yfw/ZvgF2nez/33mhgbS3/ukCGNfyPmCAkBrr2WebW5ubyfOaSkNXHxxQ0OaYQGc4bOwYgvR+CDbR/g0YsedV6eqCiadgaDeT+d0VTT66mczjvP/HWEcM1uoKWwcyd3MdaUcXAwfZ5ZWUzna4bYYyHkCyEuMP7POeVwNYBWALq7SzCFZVJTuSjrrGxGpeSa0q2ba+551VXA7bcz69HYS+3YMf47PByYMIEbU4BZkVFRdT09ZWEJ0Kd1QpYmFTXCjwJWVyM9sm7rar2ev8u/E2RCQjzS3hr/+AdNm4qKhsek5CJw8cXciZthePpwDGw/EDPWzECprhEurpgYoEcPLjzmMC5YUvKDtoQx00hhH6Wl9n1eQlhtgNjUsUch3A0gr/YTUkq9lPJuAJe5RSqFVYKCWDx74oTlljX5+czrT0lxzT2FAEaMAF5/HXjgAW7Yr7qKimDWrLprk1bLTXd+ft1+da1bM7ah0wGlZQIJQcVIDqjz1UJpKeX+OzNJp3NPplF9OncGHn2UxW9ZWfTNl5XRasjKYtXvffdZfLkQAnOGzkF+WT7e2PRG42S54Qa+79q9noxERvL59u0tp5UavxTp6Y2ToyURF2d9hwWYdlnNOGBv02Ukpazfvb72sfWuFUdhL1dfzd35jh3cmRvXhqoq+vNjYoBHHnF9+nRYGAvfbHHZZYzVLlvGNSwmhhuwbt3YwigyEujXqxJhOQUoiWwHg4FrcFRUPWv85MkGrSPOnqWbSqvlhr3OQHCVzBwAACAASURBVJ7GcOGF9Mtv3Ahs2cIFIiODvTk6dLD5YfZL6ofrOl+HFze8iIczHkZsiAX3ky1SUlj2PX8+raOwML7ZsjKaUP36WX/TBQWUNynJufu3RC68EPjvfy276gAG79q1s26ZNXHUTOUmTHU1sHYt41xnznC98vdn5+bLL3ddYa+zSMmitsWL2UsO4OaqSxcqs8qjubhxx2SUBraC3j8Y7dvz2N/ZnxUVXBBnzgRat0ZuLjs+b9liWpuN73fkSNu1Ep5gb8FedH+nO8b1G4eXhr3UuItVVFDjb91K5dS+PVtbBwcDc+ZQK7ZtazKnampo0YSGst2G0YensI9PPmHL9ZSUhsq/qoom+bhxrI9p4lhqXaEUQjPAYGDyicFAJeCL9UiVlaYgt7FH3fHjQPnqzYj79h2ExwTAPzGeO+GaGu5ydTq6cTIykJ3NNbCmhuucMRu0qorxjPR09sdzRRC9sdz7w734as9XODz2MNpFuGk3WVQE/Pgj21sYf8NS0nq44Qbm/yocQ6cD/v1vWohBQdy9GH9cQgB3383ss2ZAoxWCYF3+HQDSpJTThRDJAFpLKd2UE2gbpRCaCZmZzO/+/XcuakIwgDtsGNC+PQwGthGqqLCcBJSZSc/Sbbd5UnDzZJ7NROc3O+PuHnfjg2s/cO/NSktpFUhJTelts7CpIyXTT1etYiqqnx/Quzcts2akZF2hEN4BYAAwRErZRQgRDWCJlPJC14pqP0ohNDOqqrjqBwfX8ZHv389yAWsB8upqhhtef903rIQnfn0C/9r8L+wdsxfntbKQGqpQeAlLCsGRATl9pZSPAqgEACllIQAfdE4omiyBgYwq1wuY7t1re2yyvz9dUllZbpTPASYOmIgQ/xBMWTnF26IoFHbjiEKoFkJoAUgAEELEgRaDQuFWdDr7eooJ4TutZuJD4zGu3zh8u+9bbDmxxdviKBR24YhCmA/gewDxQohZANZB9TJSeIDERPtTxC3FGLzBU/2eQquQVpi4YqK3RVEo7MKuEZrnAsprAGwDMBRsWXG9lHK/G2VTKAAwpvf554wTWComPX2aJQRun6XgABGBEZg0YBKe/O1JLDuyDJenXW77RUVFnHi0di3bVMTHsz9Rz56+mT6maFY4ElTeLaX0qVYVKqjccli8GFiwwHwvpZISZgZOmuR7xbmV+kp0frMz4kPjsXn0ZutDdPbtYzFaVZWpkq+8nBV7iYnAU0/5lgmkaLK4Iqi8XQjh1owiIcSVQoiDQojDQojx7ryXwrfR602dnQG2zRg1ytRJ4uRJU1+lqioW9vqaMgCAIL8gvDDoBWzN2YqF+xdaPjEnh/OSQ0NZgBYezlz4mBimV506xePNuNOmwvs4YiEcANABQBaAMtBtJKWUPVwiCAPWhwBcAeA4gC0AbpNS7rP0GmUhND9OnwZWrODQMJ2OsYGuXakQzj+fHpXNm00p4t2705visvYVbqDGUIMe7/ZAjaEGe8bsgZ/GjKf200/pJrLWFiEzE3j8caBXL7fJqmgZWLIQ7IohnGO47VMaxUUADkspjwCAEOIrANcBsKgQFM2L7GzgxRfpJWndmi5zKbn4z5vHAtzrr2e9WlNCq9Fi1pBZuOHrG/Dpzk/xQO8H6p6g01EZWOrFbyQsjK0VlEJQuAm7XUZSyixzDxfKkgjgWK3/P37uuToIIR4UQmwVQmw9aalFsKLJUVXFojKNhnECY/xUCDaiTE4Gvv+evZGaItd1vg59E/vi+dXPo6K6Xott4+xeW8UWISGW22IrFC7AboUghJhq7uFO4cwhpXxfSpkhpcyIi4uz/QJFk2DXLgaGY2LMH/fzY83azz9bbvntywghMPfyuThefBxvb3m77kHjRCGDjbIenc5yy2uFwgU4ElQuq/WoATACQIoLZTkBoHa/3nbnnlM0MUpKGAd4+20+Vqww39q/Nhs2MJ5qjehotpk5e9Z1snqSQSmDMDx9OGavm42iyiLTgeBguoFs7f6LithX3FfQ6TglKTu7waxpRdPE7hiClPKV2v8vhHgZwG8ulGULgI5CiFRQEdwK4HYXXl/hAdasAT77jBXDxp5CW7YA//kPcNddlptFlpXZTrMXgi6lpjywavbQ2ejzfh+8vOFlzBgyw3TgyiuBbdu4yJr7IM6epXXQp4/nhLVERQWwdClHm1ZUmAZ8DxjAQR2uSI09c4a7hD/+4JepUyde38LEOoVrcCSoXJ8QcBfvEqSUeiHEP0ElowXwbymlmhLehNi0CfjgAybK1M/6qaoCPvyQz5sZS4z4eG42rQ2jMralCA93ncyepneb3hjVdRRe3fQq/nnRP5EQdm5mQadOwD33UJsGBvID0Wr5wRUUUEk8+6z3O/eVlwOvvsoBF23aMMADME94zRpg+3aO0bMVILfGqlX8HAB2bxWCCujXX9nS9pZbLA+xUTQKR2IIu4UQf5x77AVwEEAjZwXWRUq5WErZSUqZLqWc5cprK9yLXs/CsdatzaeABgby2IIFPLc+l13GmQnW4gMFBRxsZcu1ZAkpOeNk3z66nryV0j9j8AxU6aswc83MugeGDAGmTAEuuICCZmXRMhg5Epgxg8O0vc3ChfzwUlPrjjb18+OEtupq4J13nA/0bN3KmQQJCcwkiIzkLsE4qWzRIuCnn1zzXhQNcMRCuLrWv/UA8qWUZn7aipbIwYMsJGvf3vI5oaF0kx84wFGatUlIoNJYt44bz4SEupZAaSkVyciRzsm3Zw/w7bdcY40DekJDeb1hw+xrnucqOsZ2xOjeo/HetvfwZL8nkRadZjqYns7Zp6NHc3ENDPSscNYoKwNWr7butomL44d85IjjlYIGA8dYxsWZ31VotVQSP//MkYAqwO5yHLG7xtRKNz1xzsUzz22SKZoUp0/bvyk8c8b0b72ea8C4cVz0ja18Fi3i2nPyJGOWpaXAE09wPXCUDRtY31BYyNcnJfG/ISG0WN5/38NdUvV6TO3yMLRCi2nLJ5s/x9+fAvqKMgBYEFJTY7mhFGAK9OxzonzoyBH+wa35BI19zptq/rGP44iFcAWA5+o9N8LMcy2C8nIugkLQ3dvS+47ZSqE3d66UwBdfMAspOZnPp6ezS8ORI2xTsXs3R2MOHOjcMLAzZ+iBaNu2rocDYHJPaioVRq9e5mMbLqWyEli2DPjtN7QtK8PjUR3x4p4FeKa0J3pc/xDzan0Zvb7hrGFzGGMfjlJUZN/1/fzoP1S4HJs/YyHEIwDGAEgTQvxR61A4gA3uEsxXOXOGjdZWrzaljQcGMtY1bJhvDHr3BkbvgMFgOd5n/LyM5x49yvhhSorpNRoNFWx8vOmcoCDnJ0Nu2MD71lcGRoRgUsyiRUDfvjbWo5oa+ruWL6fZEhDAF11yiSm4aomKCgZjDx6kTyw2Fs8hDu/hT0za/yb+t7uAwVhfHtMYHc0P0zjm1BLV1c4Fle3dVdVOYVO4FHtcRv8BcA2An8791/joI6W8w42y+RwFBYztrVjBBSspiY+ICMbajG0XWiIJCdxl5+ZaPic3l32HEs4l1qxaxTXAWsJIfDzwyy+2a7YssX277Y13ZCRw/LiNWomKCjaXe/FFKgWtlimiP/7Igc8bbOyNvv0WOHyYJsm5XUM0gvEc+uPnkONYpztM35UvV90lJzOwa60QxNij3Jn2Gunp/FytRfulNDW4UrgcmwpBSlkkpcyUUt4GoBhAAoD2ALoJIXyoSsa9SAm89x7XhfotmIOC+DvPzAS++cZrInqdu+/mJjI7u+5vurqaz0VHsxbh9Gm6hQ4etL3zDw2lJ8HZuie93naGotHtbTGOICXzaffupTkTH89FPSyMX4a4OODddzn82RylpTQpExMb7KzHoi/ayDCMj90Beeggc299FSGAW2/lH8Tczkev5x/6hhucSwULCeHshxMnLCvG3Fz+2Pbto0X1yCPAc8+xJqKoyPxrFHbjSNrpaHBIzm8AXjj33+fdI5bvkZXFbDvj7tYciYlMxbZVldtciY7mTIIrrmBs8NgxPk6eZAygXz9g1iy2qn72WXpeDh+2Pg3NuCG0x7VsjpQUZj9Zo7KSSt1iLPP4cZoaycnmBTH6tBYuNL+QHTlCbWMm0BICf0zFQKwXx7A45IRlpeIs5eW0aPbuZb/wxtK1K/DYY1x8s7Ko2U+fpiLIyQFuvpmtaZ3lhht4jyNHqEiNVFZyx6XR8O+xYAF3GrGx/Gy/+gqYOJGvUziNI0HlxwFcCGCTlHKwEOI8tKARmgcPmgoyLeHnx/XgyBG6RloikZHAbbexK6kx7hcezg308uWm9HKA68iePfxv//7mMw2Ns2GcrT0YNMgU77FkKeTnU16LgfFNm3jQ2h8/JobFWidPmgIgRqqr+dqKCpNLpVaw6QH0wstyAyZEb8eIinKHUv8sUl5Od9aKFSZ/m8EAdOnCwq6UFOev3acP3WfbtpkqidPSGEux1IzKXgID2eJ77Vr6CrOz+dkFBTFIt2oVLYnoaNNrwsL4OHsWePll7jpqH1fYjSMKoVJKWSmEgBAiUEp5QAjR2W2S+RiVlfYXR5orvKrNmTP8LRUU8LvdsyetYGd3wb5IcLCpJuHTT7lW1n+PHTpwk1lcDOzY0TDLR0puQG+6yfnPJjWV69T69XWD18br5+dzDRs0yMpFCgosR6WNCEGlUVLSUCGcPMmF0+i/MhgYPO7cGYiLgz+0mIkhuC3gOyww7MIduMG5N2ukvBx46SXuqNu2Nfk3peRzM2fSzdKxo/P3CAlhK4kBAxonqzkCAug6GjyYucJScqexbBkVqqU6iKgoKpD169lCQ+EwjiiE40KIKAA/AFgqhCgEh+W0CBISbOeqGxtWWtqc6PXA11/zew3we6/XcyNnrEfy5SQTZyguphvNjPsc4eH0DvzxB70ApaWmWiOdjq7kjAwm8jiLEMD993PjuXo1/z8wkOuKXk8l8eijNmIZERHW/VpGzKUzrVgBfPklj/n78zrl5dzNZmbyzXXsiFt0HTFPxGJK9qe4uWYiArSNyGP+3/947fpVgsYc6aIi4M03uZu2VlPgbTSaun2Rli+3/QOJi+MPTCkEp3CkuZ1x2/K8EGIlgEgAv7pFKh+kRw8u4JZ6jwFc/Nq0Md9hQErulFev5u+0dr2RsaXCvHnsXGCtn09T488/rbf679CBC/Tvv9N9ZNz8+fsD117LhyM1Dubw9wfuvRe46io22isooAuqVy8qYpvWR0YGFyNrlJRwsWrTxvRcdjbw+efMzDlzBti4kW/G+IbKy9mfR6+HJiAAc0aOxYhD0/DBtg/w6EWPOvdmKyqohGrLUZ/ISCqMPXua1rCds2etvy+ACrmgoHGBpxaM3T81wengdwBIk1JOF0IkA7gAwGZ3CedLhIQAN97IzZ65Qe8VFfzNjxtn/nuYlcWdcn23BcDzW7fmb3TVKi6CzQVb/YKE4OdpMND7cMEFVLgdOri+piMuzsnWF507M784P998VkFNDRehBx+s+8dduZKLf24uo+tJSTzPYOCOwFh1u3IlMHEiho8ai4GfrcCMNTNwzwX3ICzAidYMx4/zQ7eV0x8YyEBzU1II4eHckVn7Yuh01PZKGTiFI/GrtwH0A3Dbuf8vAfCWyyXyYYyNFnNyuPkzpk9mZdHV+eijtCTMYU/OfevWzJ6zFYNoSjhSUNajB+OV3bv7WIGfRgOMHctdQVYWtT9gCnJkZTGz5tJL675u0yb6D/fsodkXHU2TJCGBO9mgIP67VSsgPx9CCMwZOgf5Zfl4Y5OTfSOtRc/rvydvdfdzlsGDbc+MKChgk0CFUzhijPeVUvYWQuwAAClloRCiRTVsEAK45hpmxGzcyDRUjYaN2jIyrPfaOnrU9uJotHZLSppPkkTHjnzfZWWWM4V0Om5Y6ze8s4Zez1T0Zcu4+TYWDV96acOYrkuIiwOmTWP2y2+/mVKozjuPyqB794a70qoqU7DC6Cby8+Mf1/gHlpK+xtxc4MgR9Evvh+s6X4cXN7yIhzMeRmyIg7MF4uKoFGwphspK650IfZH+/VlSXlJiPke4tJSWlzsC3S0ERxRCtRBCC0ACgBAiDoCT9aNNm5gY+qMdQau1XW1rzLlvTq3e/fyYJfTee/SY1Pdk6PVc0G+/3X6roKyMMdF9+6iEw8PptVm0iI/77+fa4XIiIviHHzGCVoJWaz37qHVrCmnNfWEciykEfY7p6Zg5ZCZ6vNMDc9fNxUvDXnJMxpgYpq3t32+5fUR1Nb9kGRmOXdvbxMayw+FrrzGeEB/PnYROR3eeRsOUVTVa12kcWXrmA/geQLwQYhaAdWhBdQiNpXdv26MfjUHp5hRUBrg433UXkJdHV1tREd/rsWN0eV9/PQeG2YOU7PBw8CDjMXFxXJNDQ6lw4uKofFxd31UHjYY3tJWKOnw4A8fW2lGUlzNgIsTfmQbd4rvhrp534c0tb+J48XHH5bvpJt6zsLDhMZ2Of4R//KNpftG6dGHa7MiR/EFlZVGRDh/O5x0xMxUNsKe53edSyrsAtALwLIChAASA66WU7vzZNSv69WMhq7Eqtj5Gd/RDDzW/eJgQrCnq3Zstf3bvprXUrx+te0f6oGVlATt3UhlYKxr+/nuuHV7lootMxRbGyV+1KSnhopyQQBdUmmkuwguDXsCC3QswffV0vH/N+zZvVVPDGPGSJcDRo4mIqxyPGzPfQkpYJsKj/SG0GioDrZbmmL0a2BeJj6dCu/FGvnGttvn9aLyEkDaaaQkh9gG4HMAvAAaByuBvpJRnzLzMI2RkZMitW7d66/YOs3YtW+LExNRdHyor6ULOyADGjPGtFvi+xldfMW7QzsrwVim5CX7xRR/wHpw6xXYOOTn0bfn50U+m1/NLcPHFPKdvX2Yp1eLxXx7HW1vewt4xe9G5leUa0KoqDinbsYPep6gofganC2rQ6uR+XJO+F317VUMktbMd7FK0CIQQ26SUDXyG9sQQ3gWwHEAagG21rwnGE9LMvUjRkAEDuCZ8+y0XLOPkrqAgWvkjRihlYIuTJ23HGozN6kpLfUAhtGoF/Oc/bN505Ah93mFh9G+Fh9OP1q4d+33UY9Jlk/DRjo8wZeUU/Pfm/1q8xRdfmLea2rTTQt+6G97O7oaqZPaTUiisYVMhSCnnA5gvhHhHSvmIB2Rq1lxwAWN+x47Rjx4QwB9yUxmwU13NLMoDB+j2SU/ne7LlTncV9hQNGyvGzfVG8gpt2gCvv84K4nXr+FxlpWkm6MiRZnft8aHxGNdvHKavmY6tOVuR0bZhEPjUKV7SUt89Pz+65L7/nhlYasOhsIY9LiMhbZxkzznuoKm5jJo6e/YwYFtaypoqIeiuCAwE7rmHPYPczf79rOi21putuJgyzZ7tg67l0lKaOcZqRBuatLiqGGlvpKF3m95YcteSBseXLWOxpK0M0uxsdotOTWWH2ZISfkYdOqhZMy2RxriMVgohvgPwo5Qyu9YFAwBcCuAeACsBfOIiWRU+yMGDwCuvMP5Rf/GpqGA3U43G/WMojUXDeXnmg9E1NVxvx4zxQWUAmDpz2klEYAQmDZiEp5Y8heVHlmNo2tA6xwsL7WvtISUr5d96yzSpUkoq9iuuAK67rulYqQr3YU/a6ZUAagAsEELkCCH2CSGOAPgTrFp+XUr5iRtlVHgZKdmSJyLCfD1QcDAX5y++cH/xq7FoOCyMrT6Mc1oMBiqCrCwWD7p9PrIHeeTCR5AUkYQJyyegviEeEWG76SJAF+WPP1IBtG9PF1P79gxx/Pwz8PbbzatCXuEc9kxMq5RSvi2l7A9OShsKoLeUsr2U8v+klDvcLqXCq2Rmsl7A2ijKkBB6Q/budd19S0rYp23+fOCNN1h0duaMqWh41Ci6rLKyuOClpTF2e/PNPmodOEmQXxBeGPQCtuRswfcHvq9zrGdP05hjSxQUsHni+ec3rBY3xrC2bWODQUXLxmYMwZdRMQTPsHkz0xrt8VM3dmCWkQ0bgI8/5q7VWMhbWsqF76abGIcVgothVRWDpc3Z5aE36NHjnR6QkNj9yG74aUx+ojff5EC3pKSGitBgYO+8kBDWfViiqIjnzJzZvJSpwjyNiSEoWjj2ZqYYm3g2lh07GJNo27ZuzDUmhi6pr77i4n/FFXQheaQRXkEBB69s3sw0p6Qk4PLLWf3mgdQdP40fZg2ZhRv/eyM+2/UZ7u91/9/H7ruPC/rBg6YaFynZfLGkhJZdZxujrCIiqNDLy52fTqdo+jSjrjkKd2HM6LHWi8loaHbo0Lh7GQwcl9uqlfkEHH9/pu1/+y0zNz3CunXA+PFMG9XrKcShQ5xK9tpr5gfOu4Hrz7seFyVehGmrpqFSb3rzoaGcU/3ww4zxGF1oqal0oXXtars/lnE8bBN2GChcgFMKQQjhQLMBRVMnNpZtJ3JzLZ9z6hRdSuaGAznCkSMMDltrsxMYSDfR7t2Nu5dd7N3L5knx8YzEGnsYxcdTU+7dC3z0kUdWUiEE5g6di+PFx/H2lrfrHAsIYNrvjBkU56OPgKefZmufTp1s99EqK6MloVJQWzbOWgiLXSqFwue58066Io4dq5vVYjCYFMWDDzbe/3zmjH3X0GhMHajdhpSs6IqKMm+uGKf7bNvGqLsHGJw6GMPSh2H22tkoqiwye45WW9ciGDyYCtSahVdQwEaudnXaNRhYmfjxxxzD+eGH7OpqT7qTwqdxViGosFMLIzoamDSJu9CcHCoG46NbN47+TExs/H38/e3bbHukErmggEMvrA2nEIKFAJs2uVkYE7OHzMbpitN4ZeMrdp2fksKZMZmZDVNLpaQuS0mxs7Dw7FlGnufO5Xs+dgzYupWNo55/noELRZPF2aDyBy6VQtEkiI4GHniAmUQnTnAxSUhgILOigq704ODGWQnp6dylGptYmsOoMGwFShtNSQmFsfWGgoJsT/JyIX3a9sEtXW/BqxtfxaMXPoqEMDNjPWshBC280FDgl19Mn60xXbV3bwambQbnq6pYnZiXZ75UPC+PimHaNOV7aqI4qxAWulIIIcTNAJ4H0AXARVJKlUvqw0REmHoKbdoELF7MtUAItu0ZOZLNO+vPnbb32v37m/rzmCM/n8rAWsdTlxAUZJ+5otOZr9hzIzMGz8B3+77DrLWzMH/EfJvna7VM1x0+HPjjD7rmQkJo3ZkbE22W7duZimQpUGQcDP777/RTKZocvhJD2APgRgBrXHxdhZuoqABefZXuY52OG8bkZD7/3nssJKuqcu7ao0bxWkeP1s0kqq7mehQaCvzf/3kgX75tW6Y7lZRYP0+n4+wDD9IpthMe6PUA3t36Lo4WHrX7deHhVLjXXAMMHeqAMgA4OtTWbNfYWODXXx24KOjHsjVOUOERfCKGIKXcL6U86MprKtzLf/7DvPfUVFNGkBAMPKelsRHefy13bLZKaCjTJW+4gY3qsrPpqj55krUHU6ZwnTbitgQfjQa49lrGEiwFTPPyqL06dnSTEJaZOnAqtBotpq2a5pkb5ubatoTCwmjC2VrgS0uBpUuBceOA0aP5ePdddt5TeI0mF0MQQjwI4EEASLbkU1C4lcJC1mi1a2d+ly4Ej61ezfGYznhTQkL42pEjqQik5ObT6Oc+eZIDh1as4NoSFcU6sf79bW9iHaJ/f2qkX3+l5ouN5RssL6eiiIsDHnvMK4OwEyMSMfaisXhpw0t45pJn0D2hu3tvGBjI3by1QryaGlMrXEucPMlYw8mT/PxSUvi6HTuAjRtpIo4YoUqmvYBT32Ip5du2z6qLEGKZEGKPmcd1Dt77fSllhpQyI87r009aJsZZCNbWBT8//sYPHGjcvQICmL3Urp1JGezZw4ynRYu4IW3fnmvVwoXA5Ml0NbkMITi85umnaQlkZ7Pyq7ISuOUWYOpUr07hee7S59gRdcUk99+sb1/bwfOCAp5naTGvqeFsiJISKgJjWbRWywBUu3YsRd+1y6WiK+zDY60rpJSXe+peCvdSUWHfeVI6H0ewRH4+4xORkXW7SAcHc70+e5YZkU8/zeynmBgXbDSFAHr04KOqiotaUJBXrIL6xATH4Ln+z2HiiolYn70e/ZP7u+9mgwZxAINxCEZ9qqt5bOjQhseM7N/PFDVLAy38/Wnu/fQTO/cpK8GjeP8brWhyWKsiro0Qrk++WbWK1om5kQIlJax0XrmSXpxnnmEG5ObNLowzBAbSn+UDysDI2L5j0TqsNcYvH9+gPbZLSUzkJKQTJ1iabryXlExbys6mNWWtXH3jRtv5rdHRNPNUTYPHsWkhCCFi7LiOQUppozje6j1uAPAvAHEAFgkhdkophzt7PYV76dqVrhydznKH0aoq/u7PO89195WSi725zJjTp9khFaBVUFrK/nNFRcC//gVceSVw+20S4vgxunyMRRQdO/rU4u4MoQGhmHrZVIxZPAa/HP4FIzuOdN/NBg1i246ffmJWgVZLiyk9nalf3W3EMYqKbLelNQ7FttcUVbgMe1xGOece1mw3LQCnI7xSyu8BfG/zxEZgrLZfupTfY4D94S+/nDntyjK1n+Bgpi3+97+0/OvHEmpqWP16xx2urSbW66lo6q8n1dWshwgI4P1qaujiF4Leh/BwYNPCHAxc+xHaVf5leqEQ9P/fey+/DE2Y0b1H45WNr2DC8gm4ssOV0Ag3Krnzz+fjzBk2QQoONgXbbREbC/z5p/VzjEOxHZgsp3AN9iiE/VLKXtZOEEL49JAcvR7497+ZGRMaaspC2bsX2LKFNTR33aUGkDvCVVcxJXTpUrp9Y2L4Oy4s5AJ99dXAsGGuvaefH1339S2T3Fze0xifrKmpq4giy3Mx7M9ZyMmSSBzZHkJTa+EqKmLX0nHjWKXVRPHX+mPG4Bm4feHt+GrPV7i9++3uv6kxSOMIl1zC9DNrnD7NtuIuTRdT2IM92wgrYzUcOsdrLFzIytf27Wnt+vvzU8U2mwAAG4NJREFUkZDA51asYMaKwn40GuD22+mjN/bAMRiYJBIfz/m9Tz8NfP219S6pjiAEe/Lk59d9/vjxugqgvJy1EEa67/4SQf41yNHHo7Kq3i42MpKL2ocfNvkZkqO6jULPhJ6YsnIKdDW6ugfz8oBvvqHie/RRYNYs7oZ0OvMXcxcdO/JhqRlgZSV3Gtc5lHyocBF2jdC0dEwIcZ+tc7xNaSmwZAkzUMy5ijUaLmKLFnmwv34zQQjGD++7j+meUVEM6lZWcp319+dnP2kSFYQrGDiQllzt4mG93vS3ragwpaoCQGhpPuIL9qAsLAFCWKgvCw+npbBvn2uE9BIaocGcoXNwpPAIPtz+oenAunXAxImspfDz4x8qPx946y1g9my+d48JqQH++U/+gTIzTSZlRQWrDwsK2DbX7Y2qFOZorKPxBZdI4UaMXXn9rDjHjAHS/fs9J1dzQkoOaT96lDGFmBh+psHBDOwmJJg6JDeW+HjgySep6LOzuY6EhtKVXVREWS65xGQxhJfkQAoNDFJACCsxDY2GweYmzpUdrsRl7S/D9NXTUaYrY8Dsgw/4wSUl8Y9i9PGlpHCn/uabnm0dERnJXcKYMYzhnD3LH+Dw4bRcLr3Uc7Io6mBPltEflg4BcKQTilcwzuG1h7Iy98rSXDlyhMq0fXvzccXAQK4BP/xgPXar03Ft0GjoPrYU0+nShRvb9euZdRQcTKXfvTvXPHOjC4xZR8403GtKCCEwZ+gc9P93f7zx+xuYuMmfwVlzHwrAnfqff7JlRKdOnhM0IID9nzzcA0phHXuCygkAhgMorPe8ALDB5RK5GEe68HpkNm8zZONG/r6tJZnExHDdOXWqbh8igO6fZcsYoK6qogKPimL3goEDzWcpxsayzdC11/L8119ncWt9C6AkIhH6KgNEkAEdOlgxiA0Gy8VSTYxLki7BtZ2vxbx1c/HQ4asQ287KQm80m9av96xCUPgk9riMfgYQJqXMqvfIBLDKrdK5gPPP507TWrywupoupS5dPCdXc+L0acsbUCNC8O9QWlr3+TNnOG/lxx9pRSQlMd6j1QKff86RxbZiO0JwnvAFF9Drk5ND99Hp08Dek/E4HtMDl3UusFwkV1xMDdSMvgCzhsxCia4U86L22jfP4dQpzwim8GnsCSo/IKVcZ+GYB3LbGkdEBGtpjh0z7zqSkseuuELN9HCWyEjbLSqkpFunthUmJd3bhYXcnNfe3YeGMmC9fz+nWNoiOBh4/HF2Qr3kErqmU1OpKEZ+cTsiY/0YsKz/JTh7lgKMHm090NTE6BbfDXd1vhn/CtuH49JG0Fin4x9R0eJpPr8AK4waxZ3o1q30TRvTm8+c4U6yXz+2WlY4R9++bClhjaIi7vzj403PHT9uij2YQwi6uFesoGvIWGdgCSFYMJueXv9IawYx//1vU1GUEFQOrVtTa7iypNpHeGH4XCw4+B2mVy7F+4E3WT6xstLO+ZmK5o49QeXtUsrejT3HmwQEMPX6jz84QtDYcr1zZ46E7N5dFaU1hs6dudjn5XF9rY9eT/fNXXfV9V7s38//t+bR8Pfn6//6i73lnKZNG6ZeHj9Ok9BgYPqTcWZnMyQlOhWPpN2Ct/76CuOq89DZ38wfJz+fn00zcpcpnMceC6GLlUwjgMFln7c3/fw4O7Z3b64FthYihf1oNMDYsSz4zcykuyY0lJ9zQQFTQ2+6iZ99bSoq7FPEQjDO02iEYJAiKckFF2saTLrhNXz02veYUvYr/nv6Sv5xtFqm1Bkj/E88oXZECgD2KQR7bGkL46R8k2a6IfQqrVrRf79pE60wY0p/Rgb7RXXq1FABx8XZVxwspXJxO0t8WALGXfospq+Zjq3J0cjYcYIfaHQ0O5NeconqGaT4G5sKQUpZp1pHCDH93Ot2AtgppTzkJtkUTYywMC7+Q4eaBmtZU749etByq662XB9QUkLFUbsVhcIxxl0yDm9teQsTo7ZhyYe/mio1lYmsqIfDe2Up5VQAbwAoAnCDEMJr4zRbCidPsrXz6tWMg7jEfeJGhOACb8sSCwtjsDg723xLiaoqupxuuUVZdY0hIjACkwZMwtIjS7E8c6XtEZeKFouwd6CGEOINAE9It07gcIyMjAy5detWb4vhNoqKgM8+A7Zvr5stGRbGRXLAgKb/uzYYgG+/BRYvpkUREcH3WlzM93b33SxOUzSOSn0lOv2rE1qHtcbvo3+HaOpfHEWjEEJsk1Jm1H/ekbTTEgA/CSFulVKWCSGGA5gqpXTjzL6WS2kpMG8ed8hJSXV3yBUV7A1UWen6FtOeRqMxKbd165gVqtHwffXrpzogu4ogvyC8MOgF3P/T/fj+wPe4scuNXpXn7FlWuG/YwO9x69Z0NXbr1qzKQZocdlsIACCEuB3AkwB0AEoBzJRSrnWTbDZpzhbCwoUcSmWpm4JOxzTPV15Ri6bCPvQGPXq80wMSErsf2Q0/jXdW3j/+YKNVnY4tTfz8mPRUUsIs4CeesH9Mq8I5LFkIdntmhRBDAfwfgDIArQCM9aYyaM7odOzr06aN5XMCAuha2bTJc3IpmjZ+Gj/MGjILB04dwGe7PvOKDFlZ7DsVEcGCxPBwVpm3asXK8uxsYP58zzZfVZhwJFQ3CcAUKeUgADcB+FoIMcQtUrVwTp2iUrA1fjI8XLXsVjjG9eddj4sSL8K0VdNQqff8AJBffmFM21LVeWIiC0eNY24VnsVuhSClHGLsaSSl3A1gBICZ7hJMoVC4HiEE5g6di+PFx/H2lrc9eu+yMmDz5rrtS+ojBHvtuWqgksIxnE7mk1LmAhjqQlkU52jVitaBrYZxJSWq44DCcQanDsaw9GGYvXY2iio9Ny2ttNTU9dYaISFMplCYITcXWL4c+PlnRuTrtw9uJI2KKkkpK1wliMJEQAC7r/74o/WgskbDTByFwlFmD5mNjA8y8MrGVzB98HSP3DMwkLEBKa2nS+t0qni6AUVFbM64axf/X6s1FRhedRULelzQfkSV+/goV1xBf+rx4w0DbOXlfP7229nGX6FwlD5t++CWrrfg1Y2vIr803yP3jIoCOnZkl2FrlJYC/VUyu4myMjYK27ePkfiUFOaip6SwQePChcDXX9s/GtIKSiH4KGFhwHPPAb16MfMiM9P0qKjgHPIhKqSvaAQzBs9Apb4Ss9bO8tg9r7mGm11LPazOnGEqas+eHhPJ91m9mh16ExMbmlb+/lQMS5YAJ040+laqBMSHiYhg2+5Tp5h5UV3NmoPOnZv/bGCF++kU2wkP9HoA7259F09e/CRSo1Pdfs9u3ViI+M03zDRq1Yquz6oqduIOCQGefNJ2hl2LoaYG+O03833ljWi1dB2tXg3ccUejbqcUQhOgVauGc4gVClcwdeBUfPbHZ5i2aho+u8H9tQlC0OWdns51btcu01jnq6/mdMOYGLeL0XQoLqYPzVb1aVQUcKjxfUaVQlAoWjCJEYkYe9FYvLThJTxzyTPontDd7fcUgtlxXbrQMqiuZnGaGslgBuNkP1tI6ZIOkCqGoFD4MtXV3EavXs00QzfkYz536XPsiLpiksuvbYvAQMbLlDKwQEQE3QMlJdbPO3uWox8bibIQFApfREpg7Vpmj5SV1c3V7N0buOcel00NigmOwbP9n8WkFZOwPns9+ierFB+fQaMBRo5kymlYmPl8XZ2O349LL2387Rp9BYVC4XqWLgU++ICLQEoKG/2kpHB49e7dbIXrwqKkx/s+joTQBExYPgE+1OFeAXCqXffubARVfxhKWRkzkEaNsl4CbidKISgUvkZhIfDVV8w1Dw6ue0yjYfphTg5TDV1EaEAopg6cirXZa/Hr4V9ddl2FCwgI4NDy4cPpMszKMj30euDhh13WB9+h9tfuQgjxEoBrwLbafwG4T0p51tbrmnP7a0ULZvFiTg1KTrZ8TlUVE/pff50LhgvQ1ejQ5a0uCA8Ix/aHtkMj1H7R5ygrYw66Tsf4QocOTgVgGt3+2s0sBdBNStkDwCEAE7wsj0LhPQ4cYCtbaxibXZ065bLbBmgDMGPwDOzK34Wv93ztsusqXEhoKKv2LryQBUkujsb7hEKQUi6RUhprFzcBaOdNeRQKr+Mly/3WbreiR0IPTF45GboanVdkUHgPn1AI9bgfwC+WDgohHhRCbBVCbD158qQHxVIoPESXLrbTDKuqaCW4uGJRIzSYM3QOjhQewUfbP3LptRW+j8cUghBimRBij5nHdbXOmQRAD+BLS9eRUr4vpcyQUmbExcV5QnSFwrP068fgsc7KDj0nhx0QXRQ/qM2IDiMwIHkApq+ZjjJdmcuvr/BdPKYQpJSXSym7mXn8CABCiHsBXA3gDukLkW6FwltERbGV7bFjbG1bG4OBzycluSyzpD5CCMwZOgd5pXmY//t8t9xD4Zv4RGGaEOJKAM8CGCilLLd1vkLR7BkyhC6hb74BTp6sW5h24YXAXXdZnkPpAvon98c1na7BvPXz8FDGQ4gJVg2GWgK+knZ6GEAggNPnntokpXzY1utU2qmi2aPXM+vo7Fl2tOzYEYiN9citd+fvRs93e+KZS57BvCvmeeSeCs9gKe3UJywEKWUHb8ugUPgkfn7sGe0Fuid0x5097sT8zfMxtu9YJEYkekUOhefwxSwjhULhI7ww6AXUGGowfbVnxmwqvItSCAqFwiKp0al4OONhfLTjIxw63fh++wrfRikEhUJhlUkDJiHILwhTVk7xtigKN6MUgkJhg8pKZnpmZ3OedUsjISwBT/V7Cv/d+19sy9nmbXEUbsQnsoycRWUZKdxJaSnwyy/AsmWmrsN+fsDgwcCIESwXaCkUVRYhfX46+rTtg9/u/M3b4igaia83t1MofIriYmDOHGDRIs74TU7mIy6OXadnzQLOnPG2lJ4jMigSEwdMxJK/lmDF0RXeFkfhJpRCUCjM8OWXQF4eZ9IEBpqe9/cH2rdn5+mPWlirnzEXjkG7iHZqiE4zRikEhaIeZ84AW7ZwDo0l2rQB9u1jS6GWQpBfEF4Y9AI2n9iMHw784G1xFG5AKQSFoh5//cVOERorvw4heM7hw56Tyxe4u+fdOK/VeZi0YhL0Br3tFyiaFEohKBT1qD+21hrWGpI2O/Ly4Lf4V8wSQ7H/1H58vuI1r81tULgHn2hdoWh5VFVxMQ0OZuaOLxHjQB83F48j8E0qK4FPPgE2bQI0GtwQ4I8LY1ph2uoXcNvWKgQ98hgQGeltKRUuwMd+iormzoEDwG+/ATt30u0SEAAMHcpUTl9ZXDt2BKKjmXYaFmb+nIoKICQEOP98z8rmcWpqgHfeAXbtYjRdCAgAczESQ8VneCfvZzz5chUwcSK1u6JJo1xGCo+xeDFTOQ8eZDv/5GTuxn/9FZg2DcjK8raERKvlOIKCAm6O66PTMZh8661umU/jW+zfT+19ThkYGYJUXCHTMCvqDxQfO0zrQdHkUQpB4RH27gW++gpo1w6IjzcFbAMCqBz8/IDXXjO/AHuDjAzgoYeA06eBzEzOsj99mkorPx+45x5gwABvS+kBli6lmVRLGRiZjaE4LSrwSttMVvCpeEKTR7mMFB5h0SIgIoJ5/OaIjubCu3MncPHFHhXNIv37Az16MAV13z6ud507A337tiCXeWYm/3BmyEBb3CzPxysB2/BoXjLidbq6RRuKJodSCAq3U1JCz0NysvXzIiKAtWt9RyEAQHg4h5cNGeJtSbyEnx/HdlpgBgZjIfZjVvRuvKHVelAwhTtQLiOF26mspIvIjNehDv7+QJma6e5b9O5NX5kFOqMV7q/sgnciDiGz9LgHBVO4A6UQFG7HmKlTU2P9vIoK9gpS+BADB/IPZ6k4w2DA1PzzoNX4YdqqaZ6VTeFylEJQuJ3gYLqB8vOtn1dezvVH4UO0aweMGsXe38XFdQPHZWXA0aNoN/g6PNb3MXy+63PsKdjjPVkVjUYpBIVHGDGCG83SUvPHT5wA0tKALl08K5fCDoYPB8aOZcA4O5vDIbKyaDXcd9//t3fvMVaUdxjHvw8gLKKgiHdAhKLWC6UCpoq0KErF1FrSNtVAjVFrDBov1FpwVQqUWELSNBCbaoKpGtJKIqWNRcQKYtECCgWLogStN6wX8ApCKfDrH/OuLutezpHdnTPL80k2ObPMOe+zu+z89p2Z83th7FgmDJtI105dqV5cnXda2we+qGytomdPGD8eZs3KTkl3755dr9y2LSsSffrADTdk7wGwCiPBkCHZvbivv5790KqqsvcmpB9Y987duWXoLVQvrubpN57mrF5n5RzavgwvkGOt6qOPYMUKWLYsu2Zw1FHZO5VPOaXhW1KtGLbt3Ea/mf044bATWHr5UtTUXQSWm4YWyPEMwVpVt24wcmT2YW1Ll45duONbd3DtgmtZuHEho/qPyjuSlcnXEMys2Vx1+lX0PbQvEx+fyJ5o+P0LVplcEMys2XRs35Gp50xl7TtreXDdg3nHsTK5IJhZs7rk1EsYcOQAbltyGzt3708LRhSfC4KZNat2asedI+7klQ9eYfbq/Wzh6YJzQTCzZjfqK6MY1nsYU56cwrad7kdSFC4IZtbsJHHniDt5e+vbzFwxM+84ViIXBDNrEUN7D+WiEy5i+lPTeX/7+3nHsRK4IJhZi5l27jQ+/u/HTF82Pe8oVoKKKAiSpkp6TtIaSYskHZN3JjPbd6cdeRpjB4xl5sqZbPp4U95xrAkVURCAGRExICIGAg8Dd+QdyMyax+Thk9m9ZzdTlk7JO4o1oSIKQkR8XGuzC1DcBktmtpfjDz2eawZfw+x/zmbDlg15x7FGVERBAJA0TdIbwBgamSFIulrSs5Kefe+991ovoJl9adXDqqnqUMXtS27PO4o1otUKgqS/SVpXz8fFABFRHRG9gDnAdQ29TkTcExGDI2Lw4V5ey6wQjjzoSMafOZ65z89l1Vur8o5jDai49teSegMLIuLUpvZ1+2uz4vhox0f0m9mPQccM4tGxj+YdZ7/WUPvrijhlJKl/rc2LgRfzymJmLaNbVTduHXYri15exOJ/L847jtWjIgoC8Kt0+ug5YCRwQ96BzKz5jRsyjp5dezLx8YlU2tkJq5CCEBHfj4hT062nF0WEb1g2a4OqOlQxefhkVm5ayfwX5+cdx+qoiIJgZvuPy752GSf1OInqxdXs2rMr7zhWiwuCmbWqDu06MO3caazfvJ4H1j6QdxyrxQXBzFrd6JNGM+SYIUx6YhI7du3IO44lFXfbaTkkvQe81oJD9AA2t+Drt6SiZnfu1lfU7EXNDflnPy4ivvBGrkIXhJYm6dn67tUtgqJmd+7WV9TsRc0NlZvdp4zMzAxwQTAzs8QFoXH35B1gHxQ1u3O3vqJmL2puqNDsvoZgZmaAZwhmZpa4IJiZGeCC0KSirvcsaYakF1P2P0k6JO9MpZL0Q0nPS9ojqeJuzatL0gWSXpK0UdKEvPOUStK9kt6VtC7vLOWQ1EvSEkkvpP8nhWiGKalK0kpJa1PuyXlnqsvXEJogqWvNEp+SrgdOjohrco7VJEkjgcURsUvSdICI+HnOsUoi6avAHuBu4OaIqNhFLyS1BzYA5wNvAs8Al0bEC7kGK4GkbwJbgftLWX+kUkg6Gjg6IlZLOhhYBXyv0r/nkgR0iYitkg4AlgE3RMTynKN9xjOEJhR1veeIWBQRNZ3DlgM988xTjohYHxEv5Z2jRGcAGyPilYjYCfyRbE2PihcRTwLv552jXBHxn4hYnR5/AqwHjs03VdMiszVtHpA+Kup44oJQglLXe65gVwCP5B2ijToWeKPW9psU4ODUVkjqA3wdWJFvktJIai9pDfAu8FhEVFRuFwSab73n1tZU7rRPNbCLLHvFKCW7WWMkHQQ8BNxYZyZfsSJid0QMJJuxnyGpok7Vdcg7QCWIiPNK3HUOsACY1IJxStZUbkmXA98BRkSFXSwq43te6TYBvWpt90yfsxaUzsE/BMyJiHl55ylXRHwoaQlwAVAxF/U9Q2hCUdd7lnQBcAvw3Yj4NO88bdgzQH9Jx0vqCFwC/CXnTG1aujg7G1gfEb/OO0+pJB1ec7efpM5kNyJU1PHEdxk1QdJDwIlkd728BlxThCU+JW0EOgFb0qeWF+HuKABJo4FZwOHAh8CaiPh2vqkaJulC4DdAe+DeiJiWc6SSSPoDMJysFfM7wKSImJ1rqBJIOhv4O/Avst9LgFsjYkF+qZomaQBwH9n/k3bA3IiYkm+qvbkgmJkZ4FNGZmaWuCCYmRnggmBmZokLgpmZAS4IZmaWuCCYmRnggmBthKQ+kranPjE1n/tCW2pJnVMr852SeuzjmJ0lLU0dT5F0vaT1kspqEyLpEEnj9iVLCWN8odW1pI6SnpTkjgUGuCBY2/Jy6hNT05b6LmAUcDJwqaSTI2J72uetZhjvCmBeROxO2+OA8yNiTJmvc0h6blmUKfV3+PdkbRI+k7qzPg78qNyxrW1yQbDCSIuinJ8e/1LSrEZ2b4221GOAP6c8vwP6Ao9IuknS2LQYyhpJd9eaRcyXtCotkHJ1ep1fAf3SvjPSbKf2X/I3S/pFetwnzXruJ+uB06uhsWprpNX1/PR1mLkgWKFMAqoljSFreXxjI/u2aFvq1Leob0S8CpDagrwFnAMsJPure2iajezm84PuFRExCBgMXC/pMGACaXYTET8rYfj+wG8j4hTgwEbGKsU6YEgZ+1sb5nOHVhgR8WRqbDYeGF5zqkbSVLJmZ62pB1mfpfqMAAYBz2Rx6UzW/x6yIjA6Pe5FdnB/u8yxX6u1ylZjYzUpInan6ykHp8VmbD/mgmCFIek04GhgS83BS9JRZCtP1VV2W2pJ1wI/SZsXAqNrb0dE7esO24Gqhl4KuC8iJtZ5/eHAecCZEfGppCcaeI1d7D17r7vPtqbGKlMnYMc+PN/aCJ8yskJQto7uHLLrAFtTe2+AgcCaep5SdlvqiLgrnbYZGBFv1d2us+8HQHtJ9R3QHwd+IOmIlL27pOOAbsAHqRicBHwj7f8JcHCt578DHCHpMEmdyNa0aEhDY5UknbLaHBH/K/U51na5IFjFk3QgMA/4aUSsB6by+SJF9RaEtJ70dcCjZGvuzo2I55s52iLg7HrGfgG4DVgk6TngMbKZzUKgg6T1ZBeSl6f9twBPpRXjZqSD8xRgZXpugz3zGxlrL6nV9T+AEyW9KenK9E/nAH/9Ml+8tT1uf22FJmk22Wmd3sDDEVHSkoSSXgUGR8TmfRj7dOCmiPjxl32NvEmaB0yIiA15Z7H8eYZghRYRV0bEHrK7a7rVfmNafWremEZ23WFPY/uWMPZqYEl9t3kWQTqVNt/FwGp4hmBmZoBnCGZmlrggmJkZ4IJgZmaJC4KZmQEuCGZmlrggmJkZ4IJgZmaJC4KZmQHwfyDHwTuSEudxAAAAAElFTkSuQmCC\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEKCAYAAAASByJ7AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAABSZklEQVR4nO2dd5jU5fX27zPbe2ELy1KWJioIFhCxgh00lkRji0lMMUYTjYmxJjFvEvOLMYnGWKKJpqjRJIolKlJUQBRUqoCAIrBLX2Bn+2yZ3ef9456vs2X6Tt/zua69YKc+MzvznOe0+4gxBoqiKIpii/UCFEVRlPhADYKiKIoCQA2CoiiK4kINgqIoigJADYKiKIriQg2CoiiKAgBIjfUCBkJJSYmpqqqK9TIUJSx8tP8j5Gfko6qwKtZLUZKcVatWHTTGlPa9PKENQlVVFVauXBnrZShKWMi+Jxtfn/Z1/O7s38V6KUqSIyLVni5PaIOgKMlCu7MdDqcDRZlF0X3igweB5cuBbduAlBTgqKOAqVOBnJzoriNcGANUVwNLlgBbtgA2GzBlCnDKKcDQobFeXdyjBkFR4gB7mx0AUJQVJYPQ3Q28/DLwyiuACJCby8tWrQL+9S/gO98Bjj02OmsJF11dwNNPA2+/DaSlAYWFNBBvvAG8/jpw6aXA7Nl8vYpH1CAoEaO9HVi7Fti0id/LMWMS+/AZSewOl0GIlofw2mvA3LnAqFFAao9toKQEaG0FHnwQuO024IgjorOecPDSS8CbbwJVVfQMLHJzgc5O4Nlngfx84OSTY7XCuEerjJSIsG4dcPPNwKOPAitWAB98APztb8APfkBvXiW0ehNVD6Gpid7ByJG9jYFFdjZQUAD8+9+J84dqaqIXMHJkb2NgkZbGkNELL9CTUDwSNwZBREaIyNsisklENorITbFekxIaH38M3H8/PYGqKn4Py8v5/5IS4IkngHffjfUq44uoeghr1nBTTEvzfpvCQmDHDmDXrsivJxx89BFfkycDZ5GTA9TXA1u3Rm1ZiUbcGAQATgA/MsYcAeAEADeIyJExXpMSJMYAzzzD/cRTaCgjA6ioYJi6oyPqy4tbouoh7N3r2xgAjLPbbIDdHvn1hIO6OibF/SECNDZGfj0JStzkEIwxewHsdf2/SUQ2AagE8HFMF6YExY4dwJ499Ny9kZUF1NYCGzYkXt4yUkTVQ8jMDDxs4uvEHW727KHruGsXkJ7OhNPRR/MU4Y/s7MBekzGBPd4gJW4MQk9EpArAMQDe93DdtQCuBYCRvnYdJSYcPMhDmL9CDhFg377orCkRsDyEwszCyD/ZxInAiy9yc/T2h+rspDGIRuNnZyfw1FPA0qV8TmtzX7mSCeEbbwTGj/f9GEe6ggn+XlNaGjBuXHjXn0TEU8gIACAiuQBeAPADY0w/384Y87gxZqoxZmppab9GOyXGpKQElod0OoHPPgN+/WvgpptY0PLKK8ChQ5FfYzxid9iRm56LtBQ/oZxwMHYsXbiDBz1fbwywezdw+uncnCPN008DixdzTcOHA8XFQGmpuwLqt7/1n8uoqACOOYbr9oQxfIwzz4zOa0pQ4sogiEgaaAyeMcbMjfV6lOCxDpS+vPfWVlYhLV5MLyE7210Wf/vtwOrV0VhpfGFvs0ev5FQEuP56Wu+dO2mdLRwOYPt24LDDgIsuivxa9uyhZ9C3VNSioICXv/KK/8e65hoahu3bgZYWXmYMcwbbtjH8FI3XlMDETchIRATAEwA2GWP+EOv1KKFRXAwcfzy9/REj+l/f3c2+oZQUYNIkt3efkcEkdGsr8NBDwM9+Fp1oRbxgb7NHrykNYOnXz37GUs2lS7lxGsM/wpe/zJN0Zmbk1/Hee9zwPRkDi7IyfqAaGmiwPviAJ4mcHCahJkzg/fPzgTvuAJYtA+bNA2pq+JoqKthoN326/2T6ICduDAKAkwBcDWC9iKx1XXanMeb12C1JCYUrr6R6QE0Nv4vWd7Cri01qzc3AnDmeQ73Z2cwnzpsHfPe70V13LLE7oughWJSUAF/9KvClL7FKx2bj5hvNTXPXLv+dilYc8rHHWNOcksIThNMJLFjAMNP3v08jl50NnH02DVpzs7sLW7uTAyJuDIIxZhkA/aslAQUFwJ13snH0nXfoFYjw37Q0HtTy8rzfv7SUB8KWlsHT1Wxvs2NccYySnTk5sXujMzN7h6w8YQyweTObz448sr83UVsL/OY3wN13A0Uuo2p5DEpQxI1BUJKL/Hz34XPPHn6ny8qoiNDQ4Pu+Vjn5oDIIsfAQ4oHjjmMruy/27GEe4MwzPYeWysrojr71Fj9wSsjEVVJZST5yclgxeNhhbFbLy6PGkS+MoTcRjRB2vBDVpHI8MWUKTw/eTgnGABs3AqNH+248Ky8HFi1iaakSMmoQlKhy8skM7fri0CFGBgaLx9/R1YHWztboJpXjhfR01h23tNAT6Fme1tzM6qDCQp4ofJGRwZOGdiEPCDUISlSZPBkYMsR7CXxnJ0PF550X3XXFkqgrncYbY8cy/j91Ko3Czp0MAQEsJT3hBP/NLVaVVDQ7q5MQffeUqJKRAfzwh8C99/I7X1bmVlKoreUh76qr3I2ng4Goz0KIRyorgWuvBa64ggJ0qan8cKSk8JTw3HOsVvBGYyOrjQaLWxkh1CAoUaeyEvjFL1gu/sYbwP79rEKaNg0466zBpyww6D2EnuTl9S9Bmz4d+O9/2YOQldX/PsbQ5bzsMi0vHSBqEJSYUFgInH8+Q0Pt7SxHDUSsMhmJaw/Bah5ZtMgtOjd9OnDSSexjiAaFhWwse+ghegDFxe6Nv7WV6q2nnMKOSGVAqEFQYorI4Kom8kTceggtLdyEP/6YzV25udQsf/ll4H//A771Lcb3o8G0aexCfuEFzjNISWEpmlXffPrpvrudlYBQg6AoMSYuPQRjgMcf56D6qqreoZjcXIZvHn2UjWATJkRnTYcfzo7H2lqWqaanM2+gieSwoe+kosSYuPQQqqupQDhqlOe4fFYWY/0vvwzcemv01iXCnoPy8tDubwyHdtTU0MMYOtSthaSoQVCijzFuuX39HtJDyEnLiY70daAsX84/kK8k7ZAhzC8cOsT/xzs7d3J+644dvecmlJSwvHXixJguLx5Qg6BEDbud2kYLFjA8nZICnHgicMYZPIgOVqKudBoItbWeK3p6Yo3ZbGqKf4Owezdwzz380PX1ehobgd/9DvjRjyjBO4jR85kSFWpqgJ/+lIO6cnP5nRw6lDI2d99NQzFYsTvsKM4qjvUyepOX518GwtIYSYSRlM88QyNQWtrf67Eql/76V/9Ce0mOGgQl4jgcwO9/z8PkqFHuqqLUVGDYMEpkP/EEJ6gNRuocdfGVPwBYwtnW5vs2jY384w0dGp01hcrevayUKivzfpu8PDbEbdoUtWXFI2oQlIizahX3jmIvh+CMDBqJ+fOju66gcTpZ8rhhA5OugcwKDYC4DBkdcQStdW2t5+u7utgM9oUvxH8z2N69PI34W2dKCv+ugxjNISgRZ+lS/4oC1gyEtrY47Evo7mZj1muvMV5uDXcYOpRyy1OnDujh41L6OiWFonP33stNsqyMOQVjmERubGRXYbT6EKJBmAx8IqMGQYk4jY3+w8xWl3J7e5wZhO5u4O9/59zPigr3ABZjaBwefBD4ylc4pStE4lb6urzcneCZP5/egjGsxjn33N4zUOOZykp3vsNXWVt3N2W2BzFqEJSIM2QIk8rZ2d5v43Tyu+qvsCXqrFsHLF7MjaLnZiJCtyczE3j2WeCoo2gwgiTupa8LCqgxMmcOk0GpqYmRRO5JeTmN16efes93NDTwg3r44dFdW5yhOQQl4sycycO0L/bvpzxO3DWdzpvHTdHbyTI9ndctWRLSw8dlU5onbDZOO0o0Y2Bx1VX8cO3f3zs0ZAzroRsaqLY6WAW1XMTb109JQqZM4cFs/37PDaa7drG4o7mZ+2p5OTB7NgtdfHkVEae9HfjkE/9NEkOGAGvWAJdfHvRTxKVsRTIydChw113AP/5BOQ6rMc0YhpS+9z2O9hvkqEFQIk56Ont+fvc7NokWFvKw2d7OiEx1NQfnjBvHg2hzM8P28+cDP/6x9+qkiNPdzU3DX5zcZgt5dGPCeAjJQEUFcPvtbFKrqaExKC8HxoxJjFxIFFCDoESF0lLg5z8HVq9mwc6BA0w2O50MUfesQrIk8ffsAf70Jza0xUTiIjOTSeSWFlowbzQ0hDzRRz2EGFBZyR+lH5pDUKJGVhbzBHffzY1+9Gj3jHVPDBsGbN/OXGBMEGHsylstPsBTZmsrcOaZIT2FeghKPKEGQYkJ+/fTa/c1FRFguOn99wfwRMZQwz/UGvOTTqLE8u7d/R+ju5vxrqlTg5aA7uxk2mHx+zQIe7cXDXbVBCUO0JCREhMcDhZ0+AvdZmRQUSBoDhxghnrRIna7paWx3GnWLLoegZKdzUTGn/8MbN7M2FV6utvInHwycPXVQcW0Vq8GnnySuZL1xXagAHjy4SK8VAhcd92gr3xUYogaBCUm5ORQ/aCnCrEn2tpCENL85BOKJzmd7LDNyOCR/O23+XPDDcAxxwT+eIWFwG230aVZs8at7jl1qm99HA+sXQv88Y+8W0kJsAV2pJkcjKlKQ2Mj8NvfcjCYFrwosUANghITSktZ3HHggPcqImtuQlDqCPX1wP330+L0TE6kpTH009oKPPww8MtfBtdIJsLy0wHodDudrJ4qLXXnqNtgRxaYP8jPp5H8xz+4PC18UaKN5hCUmCACXHghe4I6OjzfZvduhk/GjAnigd97j/Ws3jLVVmPD4sXBLDcsbN7MgqTcXPdlDtiRCXdCubCQfRk7dkR9eYqiBkGJHZMnA1//OsUod+3iPu500khs384D/fXXB3lSXrzYf4ypvJy3i7KY2b59/Z+yp4cAuGfO7N8f1aUpCgANGSkx5owzWKCzeDGriTo6WCJ+1VXA0UczfxsUTU3+O9nS0mh9urqiqpVhs/U3CA7YUYTeLpAxOlpUiQ1xZRBE5EkA5wOoNcYM7ll2g4jhwykY+pWv9L/O2kAD9hKKi91VRd5ob2foKMq6NVVVbrUE6/X09RCM4c/IkVFdmqIAiDODAODvAB4C8M8Yr0OJIZ2dlLSYN4+hI5uNYqJnncW5LT6NwxlnAE89xVZnb9TWsj06ylnb0aOZkz54kIlloH8OYd8+vtZ4H0KmJCdx5ZgaY5YCqIv1OpTY4XAADzzAMQMHD9J7qKigHtm993Kv7+728QDHH89ut0OHPF/f0EDv4dRTI7F8n4gA3/oW179vH9DZ3YlOaUEmitDdzSR6ZibbGhQlFsSVQVCUp57i+NvRoykjZLMxslNWxtP1okXAwoU+HiA3F7jlFt5pxw4KJnV2sgusuprholtuCaG5ITyMGAH85CcU8tu6i13KDnsRdu6kZ/CTnwTd2qAoYSPeQkZ+EZFrAVwLACM10JpUHDgALF/OTdNTNMdmo7fw6qvA6af7SBMMHw786lfAhx8Cb77JsqX8fOCKK4Dp0/3rZUSYykrapHe32PGf54ALzy7CtSfEzEYpyucknEEwxjwO4HEAmDp1qg5BTSLWreO/vipssrKYAti6lfkEr+TmUqZi1qywrjGcpOTQQ5hyWJEaAyUu0JCREjc0NgZWBSrChuNER5VOlXgjrgyCiDwLYDmACSKyS0S+Ges1KdGjoCDwOTO+xhMkCjoLQYk34sogGGOuMMZUGGPSjDHDjTFPxHpNSvSYMoWnf19VRA4HjcHYsdFbV6RQD0GJNxIuh6AkDnY7h41lZwc2BrOkhOMHli1jRZGVWM5uPYjhO5cjz16D2vo0TPrKMUjrngwgQQe+u1APQYk31CAoYWfzZuCVV4BNm1j92dVFOecLLwQmVtRxNibAuQR9LMVXvsJcwtq1QGFeF07c+wLGfzYPbe2CVpONU0Z0Y9yH7wGbc4Hvfz/owTTxRJ2jDjlpOUhPCVafQ1EigxoEJawsWQI88QSrPEeOdEs1NG07gA+v+S8Ki1aicrjr6N/dDRx3HHDZZZ+37mZmAjfeCGzYAGz77VxUrP8f9uVVoWxsCg4fQ/shAlqN++7jwOUBSFLHEnubXb0DJa5Qg6CEjd27qfdfWcmZNBa5Lftxyvp7YOtwYOW+4cg5wobCQtAgrFvHgTZ33UUVUrDS6OiRdTg6cx5weZVnzaH8fCYU5s4Fbr45Cq8u/Ngdds0fKHFFXCWVlcRm8WLu3T2NAYzBMWv/hpSuDrQWVsKWasP27a7rbDZaj44O4G9/6y0Fag1S9iVAV1oKfPSRd5mKOEc9BCXeUIOghI3ly92ibRZ5TXsw5OBmtOTw9J+Tw9kHvSgvZ+LByi0AwM6djB/5wmbjT11iyl+ph6DEG2oQlLDR0dG/saywfgf/4yoZEnHPUv4cq5yo55iwjAze0B/GhGemQWsry6La2wf+WAESFg+hro7v2549flT/FMU/mkNQwsbQoZxP01MqSEw3Up1tKD74CdI7mtFu0pCRNxSCEgB9BIt6bmhHHw28/bbvJ2xro5ZFZWXoi96yhTrb69bRMKWkAKecQq3tYGYuh8CAPITPPgNeeonZ95QUvnelpZT1PukknbCjhIQaBCVsnHMO8PjjboNg6+rEiJplGLZnJTrS89BtS0VmezfGdH8GvJVHqWprboFI73jTxIksKbLbKXvaF2M4e/NLXwphrJqLhQspr5qbS0U9m40zPJcuBd55B/jRjzjUOQJ0dnWipbMlNIOwdi3wxz/SGFrrBmiNH3+cQySuvjrq8x6UxEePEUpQGMNqovXrKVPdU1PouOMoNLpnD284Ze0/UHrgY7Rml8KZmokW5KArOw/Zwwp5ul+2jJVCjY2U+jzsMPeDpaayz6CtjQ/YM3zkcHDTmzQJOPfc0F7Ip58CTz/NBZeVuTfV1FRelp/PTbexMbTH90PITWkNDcAjj7CLr+e6ARrX0aNp6FatCuNqlcGCeghKwGzdCjz3HKMVIu6B8DNnAhdfzI7kW27hcJvG9TUo/WQZ9hWORkN+Lqr2LUdGtmDYyHQWDuXkcHPbuJHG4JZb+oc5Ro8G7r6bXW4ffOAeSpydDXz5y8DZZ4fuHbzxBh/H0/2N4eV79zJTfs45oT2HD0KWrXj/fXox3sScbDZ6Vq+9RgutXoISBEEbBBHJAdBmjAkg46ckCxs2AH/4A/chq+EMoBjdokU0Ej/+MaM7P/0psPe+pWjcn4buDBuyysqRN3E6Cnesga2l1b3xO5086f/sZ5wO44lhw4DrruMsg7o6xssrKnzPTPZHezuwejXDLT2x3J+tW4H6eq7vrruov3H66WDzRHgI2EPo6uKgCKeTb+4HH/if51BYyERzUxM9HUUJEL8GQURsAC4HcBWAaQDaAWSIyAEArwN43BjzaURXqXilvh5YsYJhZaeTk7hOOYX7aLgOh21twKOP8uCZm9v7urQ0Ngpv28bc7Be/6Gov6NiByml5aEkBamqANXsq0J1Shoq0/RiRZ0d2lsGh7iI4GjvxyZYqjC2mYJ3XNRcUhG+wTUeH272xMIZv4vbtjM0XFPAN7ezkRJ6lS4Hbbw/bsGO/HoKVy3jtNeZRrLXu38+Qlq/3wnptTmdY1qoMHgLxEN4GsAjAHQA2GGO6AUBEigHMAvAbEXnRGPN05JapeOLdd9nP1dXF/cFm42l9/nzOmr/iCt99XYGybh0PyX17DHpSUQEsWACcd56rMS0tDbtrurDyM16flQVISgp2tA/D6k+HoaMDKC4yKGvfiVfnpaDpTR7Yr78+4sU9XEx6Og2DFTLasYPGoKiot/tTUECXqLaWsbBf/jIsb6pPD8HppAX+4AP2aFiTAZ1OVkV99hnDZbm5DGsdOkSDVlzMk4AxtNR9rbcSOF1dHLuakkK3eJCE3gIxCGcaY/qp1Btj6gC8AOAFERmA/66EwkcfAY89xu9/z/6t/Hx+lufP58Z86aUDf67167mH+iIjg/vr3r1AVRWwfchU7FzzDHIqinq1CTQ28nsmAqC5CaayDOWjs3HUvhUoXbQE61+vR96cQuSeP5N62P6a00IhNZWT1BYsoBXq7qZ8Rm5u7y9+RwcwZgz/X1bGmcyffOJnVFtg+PQQ3nyTxmDMmN7rSU1lOe6iRfyxejWsN7i6mh+M8nLgmmtCz68MZpqaKMg1fz4rJrq7aZDnzAGmTg3PCSuO8VtlZIzpFJHDReQMEel15BCRc63bRGqBSn+MAf79bx4IPe2XKSkM48ybx7ztQHE6Aytrt2YZGAM8t+14pGSlIdO4y5CcTobDMzOBzAyDtPoD2FI0A6e//RNMW/VnVHTvRndnF3a8txv4859dyYi9A38BnrCGMtfXcxNoa+u9gVrx954T79PSgJUrw/L0Xj0EpxN4/XW6SZ5OpcXF/KmupgdTWEhDlpvL/xvD+J3mDoLn0CF6gM8/z4KDESNoDBoagIceAv7618CaJRMYv19zEbkRwMsAvg9gg4hc2OPqX0dqYYp3ampYiekrjJyays159eqBP9/o0f5HVlrdxyUlwL59wJa9+Vg/4zvIaalFVitDGlYFZ6ZxYFj7dmzInoZDmw8gtbMNDYVVaMsqRnphNj6rK0b70FF80vvuo0sRbkpLgVtv5cJraugNdHTwOevrucHOmNH7RJiWxthZGLA77MhOy+4vfb1nD41Rdrb3OxvDElO7nS6Xw8H3qL6ertysWcD//hf4+DmF7+kjj3Dzr6pyu8QiNLRjxjBGu2BBLFcZcQLpQ/g2gOOMMRcBmAngpyJyk+u6wRFYizMaGtxln75IT2foe6BMn87n8rW/7NsHnHACD6ZNTdxH91Ueh2Wn3ImmvAoUNNQgx74Tw7pqkGHasKjsSryVfT6aHClozXEnJyxPpLUV3LTtdoZPIsGYMcC99wJf+xrDLzYbcwgnnACcdlr/OFlbW2+PYQDY27x0KTudvv+wdjvfnMpKDpkYM4ZrHjaMBuyMMxgyam5mSa8SGNu3MzfjrWhAhF7b668ntaENJIeQYoxpBgBjzA4RmQngeREZBTUIMSHQikun0/dBM1AKC1k99O9/04Pu+/wHD9L4XOjyHTMy3FpFh4YchmWn3Imc5v3YuqoB+w6lobloBLokFXmbViC1wHOc+/M9ccgQxnNPP33gL8QT2dnABRewrnbvXj6fJ4yhN3HCCWF5Wq86RoWFdO26uz3H6SxXzemkcfLWSS0SntPAYGHtWp5ifBnjrCzGPKurWc6XhATiIewTkaOtX1zG4XwAJQC8FI8rkWT0aG7KHR3eb2MM95RJk8LznOedB1x5JaseLS21nTv5/7w84I47Ph9ngOHDua/1jPS05JbDOeYwVNtGo0tSucDOTowt6C1d7XTye/l5gUx2dnTkrb/0Jbo2Dkf/64zhJnDCCTyJhwGvOkbFxcDkydx4PGFtWMbwjfaGVWmkBEZzc2Dvl0hUBRCjTSAewlcB9CpoNsY4AXxVRB6LyKoUn2Rmsnn25ZcZ7vR0qNm/n3X9VVXheU4RYPZs9jisWcP+rfR0Sg6NH9/7MJuSwkP3E09wP7euGzqUuQ2G6wXjMhsxMn0fDNxeQnMzp2J+XpnU0RGZSqO+TJgA3HADtYCsJrCUFMbo29vZ9XvNNWF7OnubHVWFVZ6vvPhiJjebmtxaTxYFBbx80iTvZaWWezZ2bNjWm/SUlvo+YQHuU1YSJ+z9GgRjTF/1+p7XvRve5SiBcv75PJ2vWcOTubU3tLcznl9cDHz3u+Evn87NpVHwx6mnMle7aBH3sOJiHsAmTaKEUUEBMOOYNuTuqUVTwXB0d3PvLSzs440fONBPOqK+nmGqlBQe2HsN5BkI06bRui1fDnz4ITeIqVOpzTFuXFjfTLvDjmOGHuP5yqoqtn0/+CC9o9xcvtiWFhqrGTN8v+jaWq63bye24p1p04D//Md7qA5g8m74cN+eWYKjWkYJSno6td/eeYd5rpoa7ldpaTQWZ54ZvsbeULDZKLg5eTLX96mrl72ggLJFO3YA67efiPEtr6PV6YAzLQujR7PE/3PP3eHgizr1VAAM8b/0Evdqa29OS6NS9Zw5/nslAqKwkK7Q7NlheDDveE0qWxx+OPD739Pir1xJ4zRqFKWts7KA//s//tGHDXO7U11dfJNycoBvf3vQNFOFhZISFhK8/bZnt7u9ndIp11yT1O+rGoQEJi2NudaZM1l80t3NDTde+pFE2Ed19NEs0LGS3JZG3a5dFWhd8h1Mev5R5BWnI62yjCfhri6ecjs6GMYZOhQ1NdwDu7pYYGNVg7a3U/tu40YamnAk0SNNZ1cnmjua/esYZWUBJ57In77cdRdjhkuXukNExtB7uPhibnBKcFx5JT+oy5czTJmfzy+V3c4P8ze/yWbJJCZggyAiAuoZjTHG/EJERgIYaoyJUE2gEig2m/fimHihbxpAxBXR+MrxwMllrO9+/31uaiJM4J59NjBqFLq72ReUmupOXFtkZDDJvm0b98crrojaSwqZ+rZ6ACEonfakoAD46ldZ/rV3L9+38vLYuoWJTno68J3v0L1evJilqKmp9FBPOmlQGNlgPIRHAHQDOB3ALwA0gdIV0yKwLmUwUVUFXHst+wEcDp6Me8TIt2yhw+ArQV5ZSW//wgvj30sIeRaCJ3JzmfdQwoMI8y9JWlbqj2AG5Ew3xtwAoA0AjDF2AHESnFCSgowMxvD7JEw3bvQ/NjktjSGp6urILS9chDwLQVEiTDAGoVNEUgAYABCRUtBjUJSI0tERmKaYSGJIzVgeQnFWcYxXoii9CcYgPAjgRQBlInIPgGVQLSMlClRWBl4iHu+5FACoc9QBCFPISFHCSEA5BFdCeSmAVQDOACUrLjLGbIrg2hQFAHDsscBTT1FCxlsz6aFDDKVHfJZCGPAZMmpo4MSjd96hTEVZGfWJpkyJn/IxJWkJyCAYY4yIvGSMOQ7A5givSVF6kZfHYppnn/WspdTUxL3zsstis75g8ZpU/vhjNqO1t7OTLz2dGiEPP0w36Yc/TAwXSElYggkZrRCRiFYUici5IrJFRLaKyO2RfC4lvnE63crOAPvELruMFZbV1WxgtnSV2tvZ2JsoSg0epa/37AHuv59NZaNG0QpmZtIwVFWxNfv++5NaaVOJPcGUnc4C8B0RqQbQAoaNjDFmcjgW4kpYPwzgLAC7AHwoIq8YYz4Ox+MricGhQ8Bbb3FoWEcHcwMTJ9IgnHcecPLJVMO2SsSPOorRlLDJV0QBj13KCxe65xx4oqKC1m/DBuAYL5IXijJAgjEIke3lB44HsNUYsw0AROQ5ABcCUIMwSKipAX77W4Z/hg5lxMQYbv733ssG3IsuYr9aItNP+rqjgzkDb1r8Frm5bLZQg6BEiIANgjEm0hXelQB29vh9F4DpfW8kItcCuBYARlrDx5WEp70deOABdl33/LOKUIiyqAh48UV2JR99dKxWGR76SV9bs3v9NVtkZ3uXxVaUMBCMdMXPPF1ujPlFmNbiSTHKeHi+xwE8DgBTp07td72SmKxbR8mYUaM8X5+ayp61V19liCiR9cX6SV9bE4V8KW0C9CRKS71frygDJJiQUc9hspngkJxwlp3uAtBTr3c4gD1hfHwlSjQ1UZF0s6se7fDDqS7sLTwOAO+9x3yqL4qKOOWwvp7/T1T6SV9nZTEMtGlTf7GmnjQ0cJBPvNDRwcy+Max+8vcHVOKeYEJGv+/5u4j8DsArYVzLhwDGi8hoALsBXA7gyjA+vhIFli4F/vlPdgxbmkIffgj861+Uwz7tNM/3a2nxX2YvwgN0og+s8phUPvdcYNUqbrKe3oj6euYQjjsuKmv0icPBJPj8+W6JchEOyjj//PCUxtbV8ZTw0Uf8MB12GB8/TBPrFM8MRP46G8CYcC3EGOMUke8BmA8gBcCTxhidEp5ArFgB/OUvnB/St+qnvR346195uaexxGVlHMnpaxiVJUvhy9OId7xKXx92GMX9/vlPvkllLinw9nYq+6WnA7feGnvlvtZW4A9/4ICLigp3CMvp5Glg9WrOU/WXIPfF4sV8HwCqt4rQAL3xBoclffnLvkNrSsgEk0NYD3dMPwVAKYBfhnMxxpjXAbwezsdUooPTycaxoUM9l4BmZPC6Z5/lELK++dNTTwXefdetfu2J2lqGnkKNTBjDcv+GBq7HU5NbpPEpfX366UyiLFhAt8oY9iLMmUPXKh7kl+fOZdxu9Ojel6emUs+8thZ49FHg5z8PLdGzciXw5JNsxOv5QcrP54ngtdf4nlx00UBeheKFYDyE83v83wlgv2u2sqJgyxY2knlLCgPcyA8cYG5h0qTe15WX8/u/bBkPnuXlvT2B5mYanTlzQlvfhg3A88+zqc0a0JOTw8c7++zAxPPCgV/p67FjOfv0W99iE1pGRvQW54+WFmDJEt9hm9JSvsnbtgXfKdjdzTGWpaWeTxUpKbTir77KmQXeZkorIROMQbjeGHNbzwtE5N6+lymDk0OH3IO7/FFX5/6/08lD5/z5HFbV0MD9JDWVhuHIIxmmTk8HfvCD3iWpgfLee8Cf/8xE9MiR7oOrw0GPZccOjmOIxr77uY6RA3RXiov7Tw8C6LpE233xx/btPKX7WpeV6Pn44+ANwrZtPDH4OlVYOufr1nFojRJWgjEIZwHou/nP9nDZoKC1lZugCMO9g113zF8JvafbGgM8/TQ7k0eO5OVjx1KlYds2ylSsX8/RmKedFtowsLo6RiCGDeu/72ZlMfLx3nss8vGU2wgrbW2wL3kDAFD092eB9re4wc2axaRyYWGEFzBAnM7AwkBW7iNYGhoCe/zUVIamlLDj92ssIt8FcD2AMSLyUY+r8gC8F6mFxSt1dRwav2QJPVyA3u055zD0EJZB7wmIdRj0VUpvvV/WbbdvZ/6wqsp9H5uNBraszH2bzMzQJ0O+9x6f19MhHOD+M2QIQ9PTp/vZj7q6GO968022Vaen804nnui/P8DhAP7wB9h3LwLKgaLyKgClrCqaP585gzvuiI88gTeKivhm+kr0AAx1hZJUDvRU1bOETQkrgaTq/wXgC2CJ6Rd6/BxnjLkqgmuLO2prgV/+kifasjLm0EaMYL5r7ly37MJgpLycp+y9e73fZu9eNpVZpfaLF3MP8FUwUlYGzJvnNibBsnq1/4N3QQGwaxf7J7zicFBc7re/pVFISeFm/vLLwO230/L44vnnga1bYS9lYqQIrpNDejpDJC0twOOPBx53iwUjR7KErL7e+20sjfJQ5DXGjuX76kvAzxi3wJUSdvwaBGNMgzFmhzHmCgCNAMoBjAIwSUROjfQC4wVjgMce477QtzolM5Ohhx07gP/+N2ZLjDlf/SoPkTU1vb/TnZ28rKiIvQiHDjEstGWL/5N/Tg4jCS0tvm/nDafTf4WiFfb2Om3NGNbTbtxId6asjK5gbi4/DKWlTFJs8tKn2dxMl7KyEnZpAwAUoY/LUl4OfPIJa2/jFRHg8sv5B/F08nE6+Ye++OLQSsGyszn7Yfdu74Zx715+2T7+mB7Vd78L3HYbvayGhuCfU+lFwMW8IvItcEjOfAD/z/XvzyOzrPijuprVdr4aSSsrWYrt86SZxBQVAXfdBZx1FnODO3fy58AB5gBmzADuuYdS1bfeysjL1q2+p6FZB8JQpSqqqlj95Iu2Nhp1r/0Nu3bR1eiZke6JFdOaO9fzRrZtG61NairsaEOWSUVG32it1dzlzaiESmsrPZqNG9lVPFAmTgS+/3139v/gQVr4mhomyS+9lNK0oXLxxXyObdtoSC3a2njistn493j2WZ40hgzhe/vcc8Cdd/J+SsgEk1S+CcA0ACuMMbNE5HDQMAwKtmxxf2e9kZrK/WDbNoZGBiMFBcAVV7BM3Mr75eXxAP3mmzSoVqXQoUMsBz10iAUjnioNGxtpaEPtPZg5053v8eYp7N/P9XpNjK9YwSt9/fGLi9msdeCAOwFi0dnJ+zocsKc2oig107NyV0oKN75w0NrKcNZbb7njbd3dwBFHsLGrqir0xz7uOIbPVq1ydxKPGcNcSvEA50RnZAA33UT113nzaGhEaHTPPptxxuzs3tolubn8qa8Hfvc7njoSWdskhgRjENqMMW0iAhHJMMZsFpEJEVtZnNHWFnhzpNNPd0ZdHb9LtbX8bE+ZQi84kQXb+pKV5a4e/Mc/uFf2fY3jxvGQ2dgIrFnTv8rHGB5AL7kk9Pdm9GjuU+++2zt5bT3+/v3cw2bO9PEgtbXes9IWIjQaTU39DcKBA9w4nU7Yp+9CUa4TWLcMmDChdzK6q6v/fUOhtRW47z6eqIcNc8c3jeFlv/oVwyzjx4f+HNnZlJI45ZSBr7cv6ekMHc2aRcVDY3jSWLSIxtVbH0RhIQ3Iu+9SQkMJmmAMwi4RKQTwEoCFImLHIBKfKy/3EWN2YQlWejucOJ3Av//NzzXAz73TyYOc1Y8Uz0UmodDYyDBaZWX/TT0vj9GBjz5iFKC52d1r1NHBUPLUqSzkCRUR4Bvf4MFzyRL+npHBfcXppJG44QY/uYz8fN9xLQtP5UxvvQU88wyvS0uDPc2JojYwVrZjB1/c+PF8/NTU8LiW//sfH7tvPb9VI93QADz0EE/T8dbr0BObrbcu0ptv+v+ClJbyC6YGISSCEbe72PXfn4vI2wAKALwRkVXFIZMncwP3pj0GcPOrqOjf1Q/QWPzjH9yURo3q3QRlDDe/e+8FfvpT33o+icann/qW+h83jhv0++8zfGQd/tLSgAsu4E8wPQ6eSEsDvv51Tlz78EMe+HNyWAgzdmwA3sfUqdyMfNHUxM2qosJ9WU0N8NRTrMypqwOWL4f9hDaMbEnlB6m1lfo8Tic/VFdcMfBySoeDRqjnOvpSUJCY09fq632/LoAGubZ2YImnQUwwWkYC4CoAY4wxvxCRkQCOBvBBhNYWV2Rnc9D7M8941sBxOPid/9GPPH8Oq6t5Uu4btgB4+6FD+R1dvJibYLLgbwSwCN/P7m5GH44+mnvjuHHh7+koLQ1R+mLCBNYX79/vuaqgq4ub0LXX9v7jvv02rdnevcyujxgBe8YGTDnkSkZZXbdvv82E6ECSsRa7dvFN91fTn5HBRHMiGYS8PBpSXx+Mjg5aezUGIRGMZOAjAGYAuML1exM4A3nQYAkt7tnDw59VPlldzVDnDTfQk/BEIDX3Q4eyes5fDiKRCKahbPJk5iuPOirOGvxsNuDGG3kqqK6m9QfcSY7qam7mJ5/c+34rVjB+uGED3b6iItizBEUpuTzJZmbSwJSUhKcCCPA/ZKfna/JnreONWbP8T4yrraVIoBISwTjj040xx4rIGgAwxthFZFAJNogAX/gCK2KWL2cZqs1GobapU31rbW3f7n9ztLzdpqbkKZIYP56vu6XFe6VQRwcPrH0F73zhdLIUfdEiHr6tpuGTTw5PXrYfpaXA3Xez+mX+fHcJ1eGH0xgcdVT/U2l7uztZkZoKpxg0pXahKCWHHgdAo9LYSC8iFEE4T+vs7vZvGNrafGsGxSMnncSW8qYmzzXCzc2MxUYi0T1ICMYgdIpIClwS2CJSCiDE/tHEpriY8ehgSEnx321r1dwnk9R7aiqrhB57jHtg30iG08kN/corA/cKWlqYE/34YxrhvDy3MvJrrzGJHBHds/x8/uFnz6aXkJLiu/po6FAu0mUo6lPp+hV39og3dnTwRYgw5jhQg1BczMT0pk3e5SM6O/khmzp1YM8VbYYMocLh/fczn1BWxpOENbnNZmPJqo4ZDZlgtp4HAbwIoExE7gGwDMCvI7KqJOTYY313/APupHQyJZUBbs5XXw3s28dQW0MDX+vOnQx5X3QRtd0CwRgqPGzZwnxMaSn35BzXobu0lMYn3P1dvbDZ+IT+SlHPOYeJY1ezWl0qQzRFzh7nsNZWJkxEwie3esklfE67vf91HR38I3zpS4n5QTviCJbNzpnDL1R1NQ3pOefw8mDcTKUfgYjbPWWMuRpACYBbAZwBttVcZIyJ5NcuqZgxg42sVldsX6xw9He+k3z5MBH2FB17LCV/1q+ntzRjBr37YHTQqquBtWtpDHw1Db/4IveOmHL88e5mi4IC2NPoIXxuEJqauCmXlzMENSb0AYRdXcwRL1gAbN9eidK22/HFHQ+jKncH8orSICk2GoOUFLpjgVrgeKSsjAbti1/kC09JSb4vTYwIJGR0nIiMAvANAP8E8Kx1hYgUG2PqvN5T+ZwhQ4BrrqEkTnGxezIgQCOxdy/3jxkzYrvOSFJS4i4lDZUVK1icE2jTcEyjB1lZwC9+QXdlzx7YsxkzLGrs5Om2oIDdeAcO8N8Q5a/b2zmkbM0aRp8KC4GOgtF4OO1elBzYhC/kb8T0YzohI4b7T3YlElYzoBI2Ank3/wz2G4wBsKrH5QLmE8I2VznZOeUUxruff55euzW5KzOTXv7s2fEzHCteOXDAf67BEqtrbo6DcHJJCfCvfwG33gp76/sAgKKiCuCwI/lh2LePfQpXXOHngbzz9NOevaaK4SlwDp2ER2omoX0k9aQUxRd+DYIx5kEAD4rIo8aY70ZhTUnN0Ucz57dzJ+Po6en8IifKgJ3OTlZRbt7MsM/YsXxN/sLp4SKQpmGrY9yTNlJMqKgAHngA9n/dDNR9iqLCCrqF1kzQOXNCPrUfPMixo95091JTGZJ78UVWYOmBQ/FFIDkEMcSrMbBuE96lJS9WM1aisWEDE7bNze6wzcKF3Hi/9jVqBkWa449nH5cvmpq4Cfprao0qQ4bAPmUC8DZQdOvPgNRMLnKAlnTtWv7rqzItO9s9hW70aKpmNDXx7zZunM6aUdwEEjJ6W0ReAPCyMabGutDVg3AygK8BeBvA3yOyQiUu2LIF+P3vGZ/vW77ucFDN1GaL/BhKq2l43z7PyeiuLoaVrr8+/vKM9jY7slKzkDHu8PA9pj2wMLox7JR/+GH3pEpjaNjPOgu48MLE8VKVyBFI2em5ALoAPCsie0TkYxHZBuBTsGv5fmPM3yO4RiXGGENJnvx8z/1AWVncnJ9+OvLNr1bTcG4upT6sOS3d3TQE1dVsHoz4fOQQsDvsKMoKb8dhfr5/0UWAIcqXX6YBGDWKHuqoUUxxvPoq8MgjydUhr4RGIDmENlC24hERSQPLTx3GmPoIr02JE3bsYL+Ar8bW7GxWTm7cyJxCOGhqohjdhg00SuPGsQrLahpetozacJaawaRJLEefODH+vAOAHkJRZngNwpQpnBXjS8uttpbiiRde2D9CZeWwVq2iwGBEGvqUhCGomi1jTCcAH1NzlWTkwAH/w4EAnt737g2PQXjvPeBvf+Op1WrkXbeOFVqXXMI87OzZNADt7UyWxnvIw94Wfg9h6FBWkq5ezVBa379Rdzf7PkaO9J6uEKGn8PrrzAPFozFVooMW8Sp+CbQypbs7PFUsa9YwJzFsWO9NrLiYIannnuPmf9ZZNEJREcKrreXglQ8+YJnTiBHAmWey+y3AF2132DGyIPzVBNdcw7zAli3uHhdjKL7Y1MS+hAl+Rlnl57MUurU19Ol0SuKjBkHxizVt0ZdemlVjNm7cwJ6ru5shkJISzyfatDSW7T//PPs6olLuumwZ8OST/P+QIVzEJ5/Qch11FDPYAZTq2NvsmDI0/LNVc3I4p3rlSk6drK7mKX/iRDYkP/20fx0tywPUWsHBTUgGQUSGGmP2hXsxSnwyZAhlJz76iJPPPHHwIHMMnoYDBcO2bQxR+cpXZGQwTLR+PTBt2sCezy8bN1I8qa+7kpnJ3XPjRuCJJ4Dvfc9vrMXuCH8OwSI9neGeE09kktlqzgOAww5jh7cvm9XSQk9CS1AHN6Hqar4e1lUocc9XvsJQxM6dvatauruZNwA4H2ag8ee6usAew2ZzK1BHDGPY0VVY6NkVsRpKVq1i1t0Hzm4nmjqaImYQepKS0tuTmzWLBtSXl1BbSyHXgJR2u7vZmfi3v3EM51//SlXXQMqdlLgm1JCRpp0GGUVFwF13UaDvvffcl3d303u49NLgROq8kZYWWNgiKp3ItbUceuGri9DS01mxwj3jwAP1bfUAEPakciBUVXFmzJtv8qX07FuwxrdWVQXYWFhfDzz4IF25jAwayq1bGVYbMYLy1D3nICsJRagG4S9hXYWSEBQVAd/8Jjf/3bu5mZSXM5HpcDAhmZU1MC9h7FieUi0RS09YBsNfonTANDVxMf5eUGam30ledgelqKPhIfRFhB5eTg5zDNZ7293N9/LYY5mY9pucb29nd+K+fe7EUk/27QN++1vWBGvsKSEJ1SDMDeciRORSAD8HcASA440xK8P5+Ep4yc93awqtWMFyxX37uPFUVLAkdPr0/nOnA33sk05y6/N4Yv9+GoPhwwf2Ovxi5Qn80dHhuWOvB/Y2l0GIgYcA0ABccgnLdD/6iKG57Gz2bngaE+2R1atZiuQtUWQNBn//fcaplIQjVIPwOoBjw7iODQC+COCxMD6mEkEcDuCPf2TouKSEB0ZrGuRjj9FQfP/7oYV1LruM+8727TQwVvi+s5P5iqIi4NvfjkK9/LBhfHHeRjZadHRQZMkHsfQQepKXN4Dms/nz/c92HTKE3YLBGASnk55YMo0KTFDiIodgDdoR7YhJGP71L9a9jx7t3phFmHjOz2d38X/+w0lpwZKTA9x6K4e9LFjAUL41UOyss9iQ1nN0gK8u3QFhs3F4w2OP8TjtKYa1bx9dmfHjfT5UrD2EsLB3r/+B1bm5rHv1N9O5uZmDyd94gw0TNhuN6plnDrx2WQmZhMshiMi1AK4FgJGJKBmaBNjt7NEaPtzzRizC65Ys4XhMP9EUj2Rn875z5jA8bwwPn1ac+8ABzrt/6y3uLYWF3EtOOsn/ITYoTjqJ7sobb9DSDRnCF9jaSktVWkpXyM/pNl48hAGRkcHTvK9GvK4u/xOMDhxgrsGaYFRVxfutWUMjcdlltPp6QIw6IRkEY8wjwd5HRBYB8FSHcpcx5uUgnvtxAI8DwNSpU7WNJgZYsxB87QupqfyOb948sF6B9PT+vQ8bNrDQpauL+8mQIRwvMHcuk6a33DLwfojPEeHwmkmTaBSsYc15ecCXv+yeeuSHpPAQpk+n9riv5E1tLW/nbTPv6gIeeIBhuJ6J6ZQUxgetVvRhw8IniqUETNQ6lY0xZ0bruZTI4nAEdjtjWJgSTvbvZ+6ioKD3TJmsLEZu6uuB3/yGRqG4mD8DPmiKAJMn86e9nZtaZmZQMW+7w47M1ExkpkZpklAkmDkTWLSI74Gn5FBnJ6874wzvj7Fpk7vO1RNpaXT3XnmFyn3qJUQVla5QgiY/P7DbiYQWLvLF4sX0TjwNGGtqYnn85s0MYw8bxsPs+efTSwnL3hJi80MklE6jTmUlJyE9+WTv8JkxjCPa7cBVV/l2z5Yv91/fWlTEioJDh5jUV6JGIBPTigN4nO6ByGGLyMUA/gSgFMBrIrLWGHNOqI+nRJaJExnK6ejwrjDa3s7v/eHhmwUDYxix8FQmeeiQu2GuuJh5hREjKPr2pz9R0+fKKwxk105aC6uJYvz4qFS3RELpNCbMnMnE8iuvsKogJYUe09ixLP066ijf929o8C9La+luBOqKKmEjEA9hj+vH1/kqBUDIGV5jzIsAXgz1/oFgddsvXMjPMQAceSQTkRMmqGcaDFlZHELzn//Q8++bS+jqopLDVVeFt5vY6aSh6bufdHayzDU9nc/X1cWcggijD3l5wIq5e3DaO09geNtn7juKMAnx9a/zwxBBIqljFHWOPJI/dXUUQcrKcnsL/hgyBPj0U9+3sYZihzhnWgmdQAzCJmPMMb5uICJrwrSeiOB00st9912WNFpVKBs3cgDLrFksj9QB5IFz3nnsOVi4kGHf4mJ35KCzk2Gas88O73OmpjJ039cz2buXz2nJNnd19TZEBa17cfan92BPtUHlnFEQW4+Nq6EBuO8+4Ec/YuI4Qtjb7BieH+lOuihjJWmC4cQTWX7mi0OHKCse1nIxJRACMQgzwnSbmDF3Ljtfq6p6RwfKy3kQeestfq4vuCBmS0w4bDbgyiv5/V682F15NHw4D41Ll7Jhdfp04NRTwzPwXoSaPG+80Vs2aNeu3gagtZX5X4uj1j+DzLQu7HYORVt7nxB2QQFfzF//SqG2QAYUh4DdYcfk8sn+bxgp9u1jne6KFXSfhg2jxZ4yJbqThcaP58/27Z6rldraeNK48MLorUn5HL/BU9cITY+IyDX+bhNrmpvZ3DRypOdQsc3Gz+Vrr/GzqASOCPOH11wD/OQnDM9s28b3saCAnsOCBRTFW7o0PM952mn05Jqa3JdZja4Aw849S1VzmvejrHYDWnLLIeJFkDMvj57Cxx+HZ5EeqHPUxS5ktGwZcOedtKSpqfxD7d8PPPww8Otf87VHC5uNUuGVlZS5sFxKh4NSurW1lM2NuFCV4omBZtP+X1hWEUEsVV5fBz8rQWqVmCvBYQyHtG/fTi+suJjvaVYWT/Ll5W6F5IFSVgbcfDMNfU0N95GcHHolDQ1cy4knuj2GvKY9MGJDtxGI+Mhp2GxMNkeAaEpf92PLFuAvf+EbN2IE/yhWjK+qiu7VQw/5n6ATTgoKeEq4/nrmcOrr+QU85xzgnnuAk0+O3lqUXgRSZfSRt6sABCqLFTOamwOfAtXSEtm1JCvbttGYjhrlOa+YkcE94KWXfOduOzq4N9hsDB97y+kccQQPtu++y6qjrCwa/aOO4p7naXSBVXUUiuDeQIml9DVeeYXJWW+j5SormeTdupWTdKJFejqlKvxoQCnRJZCAaTmAcwDY+1wuAN7rf/P4IhgV3qjM5k1Cli/n99tXkUlxMfedgwf7l5Y3NbHfaeFCVhEZw6jG7NkMEXkKcQ8ZwpzPBRfw9g88AKxb198DaMqvhLO9G5LZjXHjfDjE3d3em6UGSMxkK+x2umX+5jlkZNC6RtMgKHFJIAbhVQC5xpi1fa8QkcXhXlC4OfJInjSdTu9ho85OXnfEEdFdW7Jw6JD/2caWOF1zc2+DUFcH3HsvQ9oVFe4NvaUFeOopKi7fdJPvxxcBrruOGnRr19KA5OTwb76jqQxjiyfjjBGfIC/PywSfxkZaoAh9AGImW9HcHPg8h4MHo7MmJa4JJKn8TWPMMi/XXRn+JYWX/Hz20uzc6Tl0ZAyvO+ssnekRKgUF/iUqjGFYp6cXZgzD23Y7D+c9T/c5OUxYb9rEKZb+yMqi4fjpT5lDKC3l/a+7Dpjz9JUoGJLKhGXfD0F9PRfwrW9FtMIIiIGHkJXF1+svZtrRwT+iMugZFNIVl13Gk+jKlYxNW+XNdXVMRM6YAVx8cWzXmMhMn87SU180NDBy0VM9edcud+7BEyIMcb/1FkNDVp+BN0TYMDt2bN9rhjKJ+eST7qYoS3Jh6FBajXC2VPchZh7CkCG0igcP+q7pb2sLcH6mkuwEklRebYzxOQwnkNvEkvR04IYbOClq3jzmzwBWtn3zm0xGalNa6EyYwM1+3z7Pc5WdToaVrr66d/Ri0yb+7iuikZbG+3/2We/egqCpqGDp5a5ddAm7u1n+ZM3sjCAx8xBE2FL++98zsewpo27F6jReqiAwD+EIH5VGAJPLce9vpqZyduyxx3Iv8LcRKYFjswE33siG3x07GK7JyeH7XFvL0tBLLuF73xOHIzBDLMI8z4ARYalRz662KBBT6espU4DLLwf++1/mCkpL+aa3tLgz/D/4gZ6IFACBGYRAfGlP7T5xi07qCz8lJYzfr1hBL8wq6Z86lXpRhx3W3wCXlvL07w9jEjvEHVPpaxHqjEyYwFKuDz7gG1pUxDkPJ56omkHK5/g1CMaYXt06IvILUMxuLYC1xhg/SlXKYCE3l5v/GWe4B2v5Mr6TJ9Nz6+z03h/Q1ETDMWZMZNYcDeJC+nrcOP5ce627U1NdZKUPQZ+VjTE/A/AggCYAXxKRmI3THCwcOEBp5yVLmAcJS/gkgohwg/fnieXmMllcU+NZUqK9nSGnL385sb26uJK+ttn8j7hUBi0BVxmJyAMAbjZkP4A3XD9KhGhoAP75T9bi96wczM11T29M9O/1+eezyOX11+lR5OfztTY28rV961sMOyUySSV9rSQ1wZSdNgN4RUQuN8a0iMjZAO42xpwUobUNapqb2bBVW8scaM8TssNBbaC2tvBLTEcbm81t3JYtY1WozcbXNWNGciggJ6X09QCpr2eH+3vv8XM8dChDjZMmRawdRAmAgN96Y8xPRORKAItFpB1AC4DbI7ayQc6CBcCePZ7VFLKyqND63HMcDZkMm2ZFBXDppbFeRWSwO+w4qszPJLFBxEcfUWi1o4OSJqmpFEa8/35WAf/gB4GPaVXCS8CRWRE5A8C3QUNQCuBGY8w7kVrYYKajg7o+vmYIpKcztLJiRfTWpYRGXCSV44TqaupO5eezITEvjweckhL20NXUAA8+GF3xVcVNMKm6uwD81BgzE8AlAP4tIqdHZFWDnIMHaRT8jZ/My1PJ7ninq7sLje2N8ZNUjjHz5jGn7a3rvLKSjaPWmFslugRsEIwxp1uaRsaY9QBmA/hVpBamKMnA59LX6iGgpYVtED3lS/oiwv65cA1UUoIj5PSNMWavK4ykhJmSEnoH7e2+vYSmJlUciHdi2qUcZzQ3u1VvfZGdzWIKxQN791LS3OFgAmby5LA2Fg4on2+McYRrIYqb9HSqr778sneJ/o4OVuPMiOtp1krMdIzikIwM5gaM8V0u3dGhzdP9aGigOOO6dfw9JcXdYHjeeWzoCYP8iBZ4xSlnnUV11l27OA+9Z9lpayuF5L72Ncr4K/GLeghuCguB8eN5yB0yxPvtmpuBk7SY3U1LC4XC9u/vP5awsxOYO5cewxVXDLgxKYH7P5Ob3FzgttuAY45h5cWOHe4fh4MKBKdrSj/uUQ+hN1/4Ag+73jSs6uoYCZkyJbrrimuWLKFCb2Vl/w0/LY1hhAULgN27B/xU6iHEMfn5lO0+eJCVF52d7DmYMCE2s4GV4FEPoTeTJrER8b//ZaVRSQm93/Z2HoCzs4Gbb/ZfYTdo6OoC5s/3rCtvkZLC0NGSJcBVVw3o6dQgJAAlJf3nECuJgXoIvbHEV8eO5T63bp17rPP553O6YXFxrFcZRzQ2Mobmr/u0sBD45JMBP50aBEWJIPY2OzJSMpCVluX/xoMEEVbHHXEEPYPOTjan6UgGD1iT/fxhTFgUINUgKEoEsTsGqHTa2ckyw/p6xgnHjfNdyJ9gZGRoeMgn+fkMDzQ1sRPVG/X1YcnEq0FQlAhib7OjOCuEGIgxwDvvAP/+N6tMetZqHnssS8wSeWqQEhg2GzBnDktOc3M9VxF1dPDzcfLJA3+6AT+CoiheqXPUhZY/WLgQ+MtfuAlUVVHop6qKw6vXr6cUbnNzuJerxCMnnsjB79XV/YehtLSwAumyy8LiOapBUJQIEtJwHLudUrYjRjC43hObjeWHe/aw1FBJftLTObT8nHPYwl1d7f5xOoHrrgubDn5chIxE5D4AXwDQAeAzANcYY+pjuihFCQMhSV8vX85/09O936aigjOSzz/f9+2U5CAjg41nF1zAGvSODuYXxo0LazY+XjyEhQAmGWMmA/gEwB0xXo+ihIWQpK83b/adQATcYlcHD4a+OCXxyMlh1960aWxICnNpVlwYBGPMAmOM1bu4AoCOl1ISngFJXwdSaqgoYSYuDEIfvgFgnrcrReRaEVkpIisPHDgQxWUpSnCELH19xBEsM/SFJYWrHYtKGImaQRCRRSKywcPPhT1ucxcAJ4BnvD2OMeZxY8xUY8zU0tLSaCxdUUIiZNmKGTOYPO7o8H6bPXuogKj5AyWMRC2pbIw509f1IvI1AOcDOMMY9ZeVxCdk2YrCQuDKK4G//53J4+xs93Xd3RQxGzEibJUlimIRL1VG5wK4DcBpxpjWWK9HUcLBgITtTj+dIaH//hc4cKB3Y9q0acDVV3ufQ6koIRIXBgHAQwAyACwUfuhXGGOui+2SFGVgDEjYToSdpyecwKqj+noqWo4f73uYgKIMgLgwCMaYcbFeg6KEm7BIX6emUjNaUaJAPFYZKUpSoNLXSqKhBkFRIoRKXyuJhhoERYkQA5a+VpQoExc5BEWJZ9ra3IU+paX99ea8EZJshaLEEDUIiuKF5mZg3jxqyFmqw6mpwKxZwOzZbBfwRUhKp4oSQ9QgKIoHGhs5cmD3bvaGWVO9OjupOr16NXDHHb7n/9oddgzLGxadBStKGNAcgqJ44JlngH37OJOm54jHtDRg1CigoQF44gnfj6EegpJoqEFQlD7U1QEffsg5NN6oqOCo4z17vN/G7tAcgpJYqEFQlD589hkTyDYf3w4R3mbrVs/Xd3V3oaG9QQ2CklBoDkFR+tB3bK0vvAmSNrQ3ABhgl3K8sW8fsHIlh/IUFADHHssZz54GvysJiRoEJSa0t3Mzzcpi5U484StR3Bdv4wiSqku5rY3KqytW0G1KT6fVfPllzm647joaCCXhibOvopLsbN4MzJ8PrF3Lg2V6OnDGGSzljJdZL+PHA0VFLDvNzfV8G4eDqtRHHun5+rDoGMUDXV3Ao48C69Yxm97TGzAG+PRT4He/A+68M/AGDSVu0RyCEjVefx34v/8DtmyhnP/IkTyNv/EGcPfdQHV1rFdIUlI4jqC2lofjvnR0MJl8+eXe59MkjYewaROtd19jAPD34cOBmhp6D0rCowZBiQobNwLPPcf9o6zMnbBNT6dxSE0F7r/f8wYcC6ZOBb7zHeDQIWDHDobNDx2i0dq/H/ja14BTTvF+/6TxEBYupJvkK09QWsoOPp1rlfBoyEiJCq+9BuTns47fE0VF3HjXruUIgHjgpJOAyZNZgvrxx9zvJkwApk/3HzJPGg9hxw7+4XyRm8vbdXT0btpQEg41CErEaWpi5GHkSN+3y88H3nknfgwCAOTlcXjZ6acHdz/LQyjOCiJDHY+kpnJspy+saW4pKdFZkxIxNGSkRJy2NoaI/FUnpqUBLS3RWVOkqXPUJYf09bHHMlbmi4MHmV2Pt3IxJWjUICgRx6rU6eryfTuHg+HoZCBppK9PO41/OG/NGd3ddAHPPTe661IighoEJeJkZTEMtH+/79u1tnL/SQaSRvp6+HDgsstYSdTY2Dtx3NICbN8OnHmmjvlMEtTHU6LC7NmsTPRW2797NzBmDPuckoGkErY75xxgyBBg7lwaBpuNnkFBAXDNNcDMmdqtnCSoQVCiwvDhwA9/CPzpTwxJFxcz5NzSQiNRVQXcdFPy5CWTSvpaBJg2jbW4NTX8o2VmsjchWf5gCgA1CEoUmTiRMwbefx9Ytow5g9Gj2ak8caL3ktRExN5mx8SyibFeRngRoRFQkhY1CEpUKSgAzj6bP8mMSl8riYgmlRUlzKj0tZKoqEFQlDCTlNLXyqBADYKihJmkka1QBh1qEBQlzCSNsJ0y6FCDoChhRj0EJVFRg6AoYUY9BCVRUYOgKGFGPQQlUVGDoChhRj0EJVGJC4MgIr8UkY9EZK2ILBCRJOn5VwYjdocd6SnpyEpNcOlrZdARFwYBwH3GmMnGmKMBvArgZzFej6KEjKV0Kir4piQYcWEQjDGNPX7NAaDDWZWEJamUTpVBRdxoGYnIPQC+CqABwCwft7sWwLUAMNLfTEZFiQGqY6QkKlHzEERkkYhs8PBzIQAYY+4yxowA8AyA73l7HGPM48aYqcaYqaXJMl5LSSrUQ1ASlah5CMaYMwO86b8AvAbg7gguR1Eiht1hxxElSTLpRxlUxEUOQUTG9/j1AgCbY7UWRRkoSTM+Uxl0xEsO4TciMgFAN4BqANfFeD2KEhLdphsNbQ0aMlISkrgwCMaYL8V6DYoSDhraGmBg1ENQEpK4CBkpSrKgXcpKIqMGQVHCiOoYKYmMGgRFCSPqISiJjBiTuE3BInIATEJHihIAByP4+JEkUdeu644+ibr2RF03EPu1jzLG9GvkSmiDEGlEZKUxZmqs1xEKibp2XXf0SdS1J+q6gfhdu4aMFEVRFABqEBRFURQXahB883isFzAAEnXtuu7ok6hrT9R1A3G6ds0hKIqiKADUQ1AURVFcqEFQFEVRAKhB8EuiznsWkftEZLNr7S+KSGGs1xQoInKpiGwUkW4RibvSvL6IyLkiskVEtorI7bFeT6CIyJMiUisiG2K9lmAQkREi8raIbHJ9Tm6K9ZoCQUQyReQDEVnnWvf/i/Wa+qI5BD+ISL414lNEbgRwpDEm7tVYReRsAG8ZY5wici8AGGNui/GyAkJEjgCVbx8DcIsxZmWMl+QVEUkB8AmAswDsAvAhgCuMMR/HdGEBICKnAmgG8E9jzKRYrydQRKQCQIUxZrWI5AFYBeCieH/PhUO2c4wxzSKSBmAZgJuMMStivLTPUQ/BD4k679kYs8AY43T9ugLA8FiuJxiMMZuMMVtivY4AOR7AVmPMNmNMB4DnAFwY4zUFhDFmKYC6WK8jWIwxe40xq13/bwKwCUBlbFflH0OaXb+muX7iaj9RgxAAInKPiOwEcBWAn8V6PSHwDQDzYr2IJKUSwM4ev+9CAmxOyYKIVAE4BsD7MV5KQIhIioisBVALYKExJq7WrQYB4Zv3HG38rdt1m7sAOMG1xw2BrD1BEA+XxdWpL1kRkVwALwD4QR9PPm4xxnQZY44GPfbjRSSuQnVxMSAn1iTqvGd/6xaRrwE4H8AZJs6SRUG85/HOLgAjevw+HMCeGK1l0OCKwb8A4BljzNxYrydYjDH1IrIYwLkA4iaprx6CHxJ13rOInAvgNgAXGGNaY72eJOZDAONFZLSIpAO4HMArMV5TUuNKzj4BYJMx5g+xXk+giEipVe0nIlkAzkSc7SdaZeQHEXkBQK95z8aY3bFdlX9EZCuADACHXBetSITqKAAQkYsB/AlAKYB6AGuNMefEdFE+EJE5AB4AkALgSWPMPbFdUWCIyLMAZoJSzPsB3G2MeSKmiwoAETkZwDsA1oPfSwC40xjzeuxW5R8RmQzgH+DnxAbgP8aYX8R2Vb1Rg6AoiqIA0JCRoiiK4kINgqIoigJADYKiKIriQg2CoiiKAkANgqIoiuJCDYKiKIoCQA2CkiSISJWIOFw6MdZl/WSpRSTLJWXeISIlA3zOLBFZ4lI8hYjc6JJkDkomREQKReT6gawlgOfoJ3UtIukislREVLFAAaAGQUkuPnPpxFiy1A8DmA3gSABXiMiRxhiH6zbhkJf4BoC5xpgu1+/XA5hjjLkqyMcpdN03KIQE+h3+OyiT8DkuddY3AVwW7HMryYkaBCVhcA1FOcv1/1+JyIM+bh4NWeqrALzsWs+fAYwB8IqI3CwiX3ENQ1krIo/18CJeEpFVrgEp17oe5zcAxrpue5/L2+l5kr9FRH7u+n+Vywt5BMBqACO8PVdPfEhdv+R6HYqiBkFJKO4GcJeIXAVKHt/s47YRlaV26RaNMcbsAACXLMgeALMAvAGeuk9yeSNdcG+63zDGHAdgKoAbRWQIgNvh8m6MMT8O4OkngENtjgGQ7eO5AmEDgGlB3F5JYjR2qCQMxpilLmGzHwKYaYVqROSXoNhZTyItS10C6ix54gwAxwH4kMtFFqh/D9AIXOz6/wgA4wHsC/K5q3tM2fL1XH4xxnS58il5rmEzyiBGDYKSMIjIUQAqABy0Ni8RGQrPn+OgZalF5AYA33b9OgfAxT1/N8b0vL8DQKa3hwLwD2PMHX0efyaocDnDGNPqkj/29BhO9Pbe+96mxd9zBUkGgLYB3F9JEjRkpCQEwjm6z4B5gBYRsdRPjwGw1sNdgpalNsY87ArbHG2M2dP39z63tQNIERFPG/qbAC4RkTLX2otFZBSAAgB2lzE4HMAJrts3Acjrcf/9AMpEZIiIZIAzLbzh7bkCwhWyOmCM6Qz0PkryogZBiXtEJBvAXAA/MsZsAvBLAD93XX00PBgE1zzp7wGYD87c/Y8xZmOYl7YAwMkenvtjAD8BsEBEPgKwEPRs3gCQ6rrsl+CsaxhjDgF41zUx7j7X5vwLcCzkq/Chme/juXrhkrpeDmCCiOwSkW+6rpoFIK5lo5XoofLXSkIjIk+AYZ2RAF41xgQ0klBEdgCYaow5OIDnPgbAD40xV4f6GLFGROYCuMMYsyXWa1Fij3oISkJjjPmmMaYbrK4p6NmY5gmrMQ1AGtzDVUJ97jUA3vZU5pkIuEJpL6kxUCzUQ1AURVEAqIegKIqiuFCDoCiKogBQg6AoiqK4UIOgKIqiAFCDoCiKorhQg6AoiqIAUIOgKIqiuFCDoCiKogAA/j/oQ7KLrYAZXwAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": { - "tags": [], "needs_background": "light" - } + }, + "output_type": "display_data" } + ], + "source": [ + "plot_dataset(train_x,train_labels,W.numpy(),b.numpy())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's see how our model behaves on the validation data." ] }, { "cell_type": "code", + "execution_count": 23, "metadata": { "colab": { "base_uri": "https://localhost:8080/", @@ -961,43 +943,47 @@ "id": "oEQswfCGrmHw", "outputId": "3cf61882-60e1-4baa-8e51-0c31ea80875c" }, - "source": [ - "pred = tf.matmul(valid_x,W)+b\r\n", - "fig,ax = plt.subplots(1,2)\r\n", - "ax[0].scatter(valid_x[:,0],valid_x[:,1],c=pred[:,0]>0.5)\r\n", - "ax[1].scatter(valid_x[:,0],valid_x[:,1],c=valid_labels)" - ], - "execution_count": 27, "outputs": [ { - "output_type": "execute_result", "data": { "text/plain": [ - "" + "" ] }, - "metadata": { - "tags": [] - }, - "execution_count": 27 + "execution_count": 23, + "metadata": {}, + "output_type": "execute_result" }, { - "output_type": "display_data", "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd5xU9b3/8ddnZnZnC0WaIqKCXSxE3UiIGsSKFTXq1dxr7MRekvvLtXujN15bOpZwNZY8jDUxoqLYg8aoLIoCIoqFACIsSN02Mzuf3x+7moWdhZWZnTMz5/18PHiw8z2H832rn/l4OHPmfM3dERGR0hcJOoCIiOSHGr6ISEio4YuIhIQavohISKjhi4iERCzoAOvTv39/HzJkSNAxpERNmzZtqbsPyPe8qmvpTuur64Ju+EOGDKG2tjboGFKizGxeEPOqrqU7ra+udUlHRCQk1PBFREJCDV9EJCTU8EVEQkINH2hY3ciMV2czf87CoKOI5NSiTxYz49XZrF6+JugoUgAK+i6dfHj0FxO558oHcRxPO4N33IKbn7+avgP7BB1NZKOtWVHP1UffyOw3PsIihqed4y4+nLNvPgUzCzqeBCTUZ/hTJ0/nrsseIJlIkUq00JJKM2/WfC7e56qgo4lk5boTfsHM1z6gJdVCKpGiJdXCo794kifGPxt0NAlQqBv+PVc+SLol3WH8i0+X8NE7nwSQSCR7q75czfSXZmbcdu81D+U5jRSSUDf8RZ8u7nRb7eTpeUwikjtrVtTT2ToX9Ssb8pxGCkmoG/5mW3f+rfpB2w7MYxKR3Nl0q/6dbovGQv2WD71Q/9c//fqTIMPnV2XxMkYcsVf+A4nkQCwWY5d9dsq4bd/vfyfPaaSQhLrhjzhiL75/6ZFEYxHMjEg0QllFGddP/C8qquJBxxPZaNf99acMHLppW21DrCzKVjsP5pI7xgUdTQIU+tsyz7n1VMaeN4baydOpqK7gu2NrqO5dHXQskaz06teT+z76HW+/MIMFcz5nq2GD+dboXYhEQn2OF3qhb/gAm2+zGUede2jQMURyKhKJUHPIcGoOGR50FCkQ+t+9iEhIqOGLiISEGr6ISEio4YuIhIQavohISKjhi4iERE4avpn9wcyWmFnGJzaZ2f5mttLMprf9uiYX84p0J9W1lJpc3Yd/LzAeuH89+7zq7kfmaD6RfLgX1bWUkJyc4bv7FODLXBxLpFCorqXU5PMa/kgze9fMnjGzXTrbyczGmVmtmdXW1dXlMZ7IRlFdS9HIV8N/G9ja3YcDvwP+2tmO7j7B3WvcvWbAgM4fXyxSAFTXUlTy0vDdfZW7r2n7eRJQZmadP7RbpAiorqXY5KXhm9lAa1s52cz2bpt3WT7mFukuqmspNjm5S8fMHgT2B/qb2QLgWqAMwN3vBI4HzjWzFNAInOSdrcEmUiBU11JqctLw3f3kDWwfT+vtbSJFQ3UtpUbftBURCYmSafirvlzN3OmfUr+yPugoIjnj7iz4aBGfzZpPOp0OOo4UuaJf8SqVTPHrcyfw0gOvURaPkUqkOPr8MZx9039oOTcpav/8YCH/fdwtLJlXh0WMql5VXPGnixk+qtPb/UXWq+g74t1X/IlXHvw7yeYkDasaSTQlefKO53j8t5OCjiay0ZKJJD/Z/xoWzFlIc2OCpvpmvly0nKuO/F+WLVoedDwpUkXd8NPpNE/d+RzNjYm1xpsbmnn0F08GlEoke29NeodEY5J17/lJt6R5/v5XAskkxa+oG34q2UJinWb/ldXLVuc5jUjufLloOS2plg7jiaYkS+brVn/ZOEXd8MvjZQzabmDGbTt+e7s8pxHJnV333Qms43hljwr2GL1r/gNJSSjqhg9w4fiziFeV0/aFRyIRo6I6zjm/PDXgZCIbb+huW/OdI2uoqI5/PVZeWc7gHQcx8uiaAJNJMSv6u3T2PGh3fvm36/jTDX9h3vsL2GGvbfnBFcey9bAtg44mkpXLH7iIyfe8wtO/f45kIsUBP9iPYy48jFhZ0b9tJSBWyN8Er6mp8dra2qBjSIkys2nunvfTZdW1dKf11XXRX9IREZGuUcMXEQkJNXwRkZBQwxcRCQk1fBGRkMhJwzezP5jZEjOb2cl2M7PfmtlcM3vPzPbMxbwi3Ul1LaUmV2f49wJj1rP9MGD7tl/jgDtyNK9Id7oX1bWUkJw0fHefAny5nl3GAvd7qzeATcxs81zMLdJdVNdSavJ1DX8LYH671wvaxkSKmepaikrBfWhrZuPMrNbMauvq6oKOI5ITqmspBPlq+AuB9g+3Gdw21oG7T3D3GnevGTBgQF7CiWwk1bUUlXw1/InAD9vuavgOsNLdF3X3pJ9//AX/eLKW+XMyvgdFshVIXbe0tPDelPd5c9Lb1K9q6O7ppITk5LF7ZvYgsD/Q38wWANcCZQDuficwCTgcmAs0AKfnYt7OJJqT/PykX1E7eTqx8hipZAu77bcz//2X/0dFVXzDBxCh8Ooa4ON3P+Pyw35OU30TZkYq2cL5vzmdw886qLunlhKQk4bv7idvYLsD5+dirq64/9qHqX3uXRJNSRJNSQBmTHmfO39yH5fcMS5fMaTIFVpdt6RauOyQ61lRt2qt8dsvuYcdarZlu28NzVcUKVIF96FtLjz9fy90WPow0ZTk+fteoZAfBy2yPu+8NJNEc7LDeLI5xdMTng8gkRSbkmz4zQ3NGceTzSnS6XSe04jkRv3KBshwvpJuSbNq2Zr8B5KiU5INf/fvDcMyrAe604jtiEaj+Q8kkgO7jxpGKpnqMF5RHWffY0cEkEiKTUk2/PN+cwZVvaooi5cBECuPUdmjgotuPzvgZCIbr8+mvfmPa04gXhX/+oSmojrOtt8ayn7fV8OXDSvJxTG32mkL7n7/10y87VnmTJ3LNsOHcMwFY9h0K93/LMXt5MuOZdjIHXh6wgvUr2xg1AkjGX3yPlrnVrqkZKuk3+Z9OP1/1nuThUhRGj5qF4aP2iXoGFKESvKSjoiIdKSGLyISEmr4IiIhoYYvIhISavgiIiGhhi8iEhJq+CIiIaGGLyISEmr4IiIhoYYvIhISOWn4ZjbGzOaY2VwzuyzD9tPMrM7Mprf9OisX84p0N9W2lJKsn6VjZlHgNuBgYAEw1cwmuvv76+z6sLtfkO18Ivmi2pZSk4sz/L2Bue7+ibsngIeAsTk4rkjQVNtSUnLR8LcA5rd7vaBtbF3fN7P3zOwxM9uys4OZ2TgzqzWz2rq6uhzEE9loOatt1bUUgnx9aPskMMTddweeB+7rbEd3n+DuNe5eM2CAnl8vBa9Lta26lkKQi4a/EGh/VjO4bexr7r7M3b9aaPYuYK8czCvS3VTbUlJy0fCnAtub2VAzKwdOAia238HMNm/38mhgdg7mFeluqm0pKVnfpePuKTO7AJgMRIE/uPssM7sOqHX3icBFZnY0kAK+BE7Ldt5cSSaS/OXXT/PMXS+SSrUw+qR9OPny46jqWRl0NAlYsdf2go8Wcf+1DzPj1dn0G9SHky8/jn2O2TvoWBIgc/egM3SqpqbGa2tru+347s7lY/6Hma99QHNjAoCyeBlbbD+QO6bdrHVCS5yZTXP3mnzP2911DfD5x19w7l4/pWlNE+l063u8oirO6TeczHEXHdGtc0uw1lfXof6m7QdvzWXW63O+bvYAyeYkiz+r4/UnpgaYTCQ7f7zuUZrqm79u9gBNDc3ce9VDJJqTASaTIIW74b/5ES2pdIfxxjVNzPz7BwEkEsmNma99QLqlY21jsOiTxfkPJAUh1A1/wJb9iJVHO4zHK8sZOHTTABKJ5MZmW2e+9TOVbKHPZr3znEYKRagb/ogj9qSyRyUWsbXGo2VRDvr37wWUSiR7J19+LPGq+Fpj5RVl7DP22/Tq2zOgVBK0UDf8svIyfv3q9Wy/51DK4mWUV5QxeMdB3PLitfTqpzeFFK+9Dh7OhePPpEefaiqq45TFy9j3uBH85O7zgo4mAQr9bSibb7MZt711E8sXryCVbGHA4H5BRxLJiUNPG81B//E9lsxfSq9+PanuVRV0JAlY6Bv+V/pstknQEURyLhqLsvnQzYKOIQUi1Jd0RETCRA1fRCQk1PBFREJCDV9EJCTU8EVEQkINX0QkJHRbZh698dQ0nhj/DKtX1LPf90dw9LmHUtlDj2GW4ubp1XjDH6HpRYj0wapPxeL7BR1LMlDDz5N7rnmIv/zqKZrqWxdH+nTGP3nu3r9x29QbqVjnK/AixcLTa/Blx0DLEqC1tj0xFe9xPpEe44INJx3okk4eLF+8gkdvnfh1swdINCZYPK+O5+//W4DJRLLjDQ9DSx1fNftWjbDmd3h6VVCxpBM5afhmNsbM5pjZXDO7LMP2uJk93Lb9TTMbkot5i8X7//iQsvKOf5lqbmjmrUlvB5BIukq1vQHNLwFNHcetDJIz8h5H1i/rhm9mUeA24DBgGHCymQ1bZ7czgeXuvh3wK+CmbOdtz92ZU/sxbzw1jS+/WJ7LQ+dE7wG98HTHlcUi0Qh9N+8TQCLpikKo7cb6JqY++w7TX55JKpnK5aFzI7opYBk2tEBEtV1ocnENf29grrt/AmBmDwFjgffb7TMW+O+2nx8DxpuZeQ7WV1z6+Zdcdsj1LJ5XRyQaIdmcYuwFYxh38ymYZSrE/Bs2cgd69e9JU0PzWo2/LB7j6PMODTCZbECgtf23R17n1jNvJxJtPS+LlkW5/onL2OW7O2Z76Jyxqh/iTS+y9ll+BCJbQGznoGJJJ3JxSWcLYH671wvaxjLu4+4pYCWQ8bGUZjbOzGrNrLaurm6Dk193/K3Mn/M5TfXNNKxqJNmc5Kk7n+OVh1/fmH+WbhGJRLjlhWsZvMMgKqrjVPWqpKpnJZdOOIdthw8JOp50Lme1/U3retEni7n59Nu+ruuGVY2sXraGKw7/OY31GS6hBMTK94BeV4NVgvUAKiG2Pdb37oI54ZJ/Kbi7dNx9AjABWhd7Xt++S+Yv5ePpn3VYyq2pvpnHf/s0o0/ap/uCfkObb7MZd8/6FfPeX0DDqga223MbyuNlOTn26uVr+OCtuWwyoBfb7TFUb7QC9E3qGuC5+18hnWrJdBzefGoa+/9b4dR2pOoEvPIoSL4Pkd5YbNucHdtTc6Hlc4jtjEUzr+IlXZeLhr8Q2LLd68FtY5n2WWBmMaA3sCzbiRtWNRKNRYGOizKvWdGQ7eFzzswYssuWG97xG3jopsf5488epSxeRkuqhU236s//PnMlm26lN0cOBFbbq7+sJ5Xs2PDTLWnqVxZibVdA+Z45O56nV+LLx0FydusHwN6MV/0b1vMqndBkIReXdKYC25vZUDMrB04CJq6zz0Tg1LafjwdeysU1zi13HEQsw90vZeUx9jnm29kevuBNnTydB/7nzySaktSvbKCpvpkFHy7iqqNuDDpaqQistkccvgcVPSo6jHva2ePA3bI9fMHzFT+F5EygCXw1kICGx/CGR4KOVtSybvht1y0vACYDs4FH3H2WmV1nZke37XY30M/M5gI/Bjrc3rYxorEoP/6/c4hXlX/9wVa8spw+m2/CCf959Ab+dPF7/LdPr3VvP7SeAX7+8RfMm70goFSlI8ja3uuQ4ez+vZ2pqP7Xl/IqquMcdd6hDNp2YC6mKFieXgWJ1+j4N/dGaLgviEglIyfX8N19EjBpnbFr2v3cBJyQi7nWte+xI/jdP27gr+OfZfG8OmoOGc5hZx0YiuXcVi5dnXE8GouyZnl9ntOUpqBqOxKJcN0T/8WUR9/gpQdfpTxexpgzD6TmkOG5nqrweD2dnov6yrxGKTUF96Htxhi629Zc+vsfBR0j7/Y5Zm8+m/FPEk1rnwml0852ewzJ6VzuSWiZB9YHi2rd33yIRqOMPmmfgrr5IC8iAyGyCaQXr7MhCvH9cz6dp5dDehlEt6L1yl3p0qMVitjY88fQf3A/4pWtRWpmxKvinPfr04hX5u75POmGJ/AlI/Flx+N1o0h/eZa+Ni/dxsyw3jcAFUC0bbQcrDfW48KczePeRHrFJfiS/Vpre8kI0vX35+z4hagkzvDDqrpXFXdMu5lJ//cCbzw1jX6D+nDMhYez84jtczaHJ6bCqqtZ64s1iX/gKy7E+up6qnQPi+8H/f6MN9wLqXlQPgKr/ncs0jdnc/jKK1qf8EkCPNE6uPoXeHQQVnFQzuYpJGr4Ra6qZyXH//gojv/xUd1yfK+/m47PSklC4m28ZSEWXfd7SCK5YWXbY71/3i3H9vRqaHoOSKyzpRGvv7NkG35JXtJZvXwNX3y2hHQ6veGdZf1aPs88bmXQsjS/WULO3Vk8r46VS3U5LWvp5fzrctE6Wtb97KB0lNQZ/urla7jp1PG8/fx7RKJGVa8qLv39jxh5VE3Q0YpX+Xch9TEdbpHzFOTwG5WyftNfnsnNp41n1dLVpNNpho3ckSv+dDF9B+oBZRslOqjtC12N62yIQHnpfoenpM7wrz3mZqY99y7J5iTNDQmWf7GCn5/8K+a+82nQ0YqWVZ/R9oyU9ucGldDjAizSI6hYobLok8VcfdSN1M1fRnNjgmRzipmvfcBPD76OHHzHK5TMYtDzclo/GP5KBKwK63FxULG6Xck0/AUfLeLD2o9JJdZ+hGyiKcljv3wyoFTFz6KbYv2fgMoTILollO2BbXKLVjPKoyduf7bDo5FbUi0smbeU2W9+FFCq4hep+j7W53Yo2xuig6HiaKzf41hs66CjdZuiuqSzdOEyJt4+mbnvfMoOe23DUeeNoV/b8+Tr5i8lVh6juXHtD2E87Sz6pHSvyeWDRQdivX8WdIyS9u4rs5h014s01Tcx6sTvMuqEkW3PiYJFcxdnfK6OmVE3fxl8J99pS4fF98Xi+wYdI2+KpuF/OvOfXLLvVSSbkySbU0x/eRZ/Hf8sv3n952y982C2HT6EZHPHh6iVxWN8a/SuASQW6Zr7f/YIj9wykURjM+7w9gvvMfnel7lh0hVEo1G+dcAuTHvhXZob1j6ZSSVT7FCzTUCppRgVzSWd351/V9vz7lv/aptsTtKwqoHbLvoDAL369WTsBWPWevZINBahqmclx1x0eCCZRTakbsEyHr7przQ3tDZ7aH289/v/+JA3n2pd/vKQ00bTq19PYmX/uqskXhVn/5P2YfOhmwURW4pUUTR8d2fm3z/IMA7v/W3W16/PvukULrrtbIbuvhX9t+jLIaeN5o63b6HPpr3zGVeky6a/PJNIrOPtgU1rmnh94lSg9Qt2t9fexJHnHMKALfux1c6DGXfLKfzkrnPzHVeKXFFc0jEzyivKaW5o7rCt/SMEzIyDfziKg384Kp/xRDZada8qIhme7x6NRejZ9193QW0yoDfn/+YMzv/NGfmMJyWmKM7wAQ49bX/KK9ZeIaq8oowxZx4QUCKR7NUcOvzrR3u3Fy2LMeb00QEkklJWNA3/7JtPYbfvDSNeWU5Vr0rKK8sZPnpXzrzhB0FHE9lo5RXl/O+zV9Krf0+qelZS1auSeGU5F912FlsPy+3qaCJFcUkHoKIqzo3PXsW82QtYMOdzttp5C7bcUc9xkeK3097b8/DCCcx4dTbNDQl2HzWMqp6VQceSEpRVwzezvsDDwBDgM+BEd1+eYb8WYEbby3+6+0YvR7X1zoPZeufBG/vHRbok37UdK4uxxwGlv3ShBCvbSzqXAS+6+/bAi3S+vFuju3+r7Vfprz0opUC1LSUn24Y/Fvjqoej3AcdkeTyRQqHalpKTbcPfzN0Xtf38BdDZt0AqzKzWzN4ws/W+ccxsXNu+tXV1dVnGE9loOa1t1bUUgg1ewzezF4CBGTZd2f6Fu7uZdfbovq3dfaGZbQO8ZGYz3P3jTDu6+wRgAkBNTY0eBSjdaQczm5lhPOe1rbqWQrDBhu/unS79YmaLzWxzd19kZpsDSzo5xsK23z8xs1eAPYCMDV8kjz5094yLJai2pRRle0lnInBq28+nAk+su4OZ9TGzeNvP/YF9gPeznFeku6m2peRk2/BvBA42s4+Ag9peY2Y1ZnZX2z47A7Vm9i7wMnCju+tNIYVOtS0lJ6v78N19GXBghvFa4Ky2n18HdIOxFBXVtpSionm0goiIZEcNX0QkJNTwRURCQg1fRCQk1PBFREJCDV9EJCTU8EVEQkINX0QkJNTwRURCQg1fRCQk1PBFREJCDV9EJCTU8EVEQkINX7qdN79OetnxpBfvRXrpsXjzlKAjiWTNPUF6zXjSS/YjvfjbpFf8FG9ZHHSs9VLDl27lzVPw5edA8j3w1ZCahS+/gHTj80FHE8mKr7gY1kyA9GLwldD0JL7sODy9Ouhoncqq4ZvZCWY2y8zSZpZxqbi2/caY2Rwzm2tml2UzpxQXX3Uj0LTOaBOsuTGIOF2m2pb18dRcaP47a9d2C6RX441/DirWBmV7hj8TOA7o9O/oZhYFbgMOA4YBJ5vZsCznlWLR8mkn4/NxT+c3yzej2pbOJWeDRTNsaILE23mP01VZNXx3n+3uczaw297AXHf/xN0TwEPA2GzmlSISGZB53PpiVrhXFFXbsl7RLQHPsKEcYtvmO02X5eMdtwUwv93rBW1jGZnZODOrNbPaurq6bg8n3azH+WCV6wxWQo9zAomTY12ubdV1iSkbDtGt6LBKrMWwqn8LJFJXbLDhm9kLZjYzw69uOZNx9wnuXuPuNQMGdHJ2KEXDKk+EHpeC9QTKW3/vcT5WdWrQ0QB2yFdtq65Li5lhfe+D+CigDIhBbAes7x+x6MCg43Vqg4uYu/tBWc6xENiy3evBbWMSAmaGVZ+GV50CvgqsJ2YbLLt8+dDdO/1AtgtU2yFmkT5YnztwbwRPYZGeQUfaoHxc0pkKbG9mQ82sHDgJmJiHeaWAmEVb3yCF0+xzQbUtmFUWRbOH7G/LPNbMFgAjgafNbHLb+CAzmwTg7ingAmAyMBt4xN1nZRdbpHuptqUUZXW65e6PA49nGP8cOLzd60nApGzmEskn1baUosK9L05ERHJKDV9EJCRK6hM06bqlC5cx5bE3SDQlGXHEngzddaugI4lkzd2Z8epsZrw6mz6b9mbUiSOp7l0ddKyCoYYfQi8//HduPf12wGlJpfnjdY9y9HmH8qNbfhh0NJGNlkqmuGbsTcx4dTaJxgTlleX8/j/v56bnr2anvbcPOl5B0CWdkFmzop5fnHE7iaYEiaYkLakWEo0JnrrzOWa9vqEnCYgUrmfufon3psymqb6ZdNppqm+mYXUjPzv+VtwzPQYhfNTwQ+atZ94hEuv4n725IcGLD+g59VK8Jt/zMs0NzR3G16xo4LOZ/wwgUeFRww8bnelI6Kjmv6KGHzLfPmwPWlIdH0tcXlnOAT/YL4BEIrlx6OmjiVfFO4z36F3NEN2UAKjhh07PPj34yd3nUl5RRnlFGdFYlPLKco4852B23WenoOOJbLTDzjyA3fbdiYrqOBYxKqrjVPWs5JrH/hMzCzpeQdBdOiF0wEn7svt+OzPl0TdINCVab8vcbeugY4lkJVYW44ZnruS9Ke8z89UP6LOZbstclxp+SPXfoh/HXXJE0DFEcsrMGD5qF4aP2iXoKAVJl3REREJCDV9EJCTU8EVEQkINX0QkJNTwRURCItsVr04ws1lmljazTtcGNbPPzGyGmU03s9ps5hTJB9W2lKJsb8ucCRwH/L4L+45296VZzieSL6ptKTnZLnE4G9C32KTkqLalFOXrGr4Dz5nZNDMbt74dzWycmdWaWW1dXV2e4olstC7VtupaCsEGz/DN7AVgYIZNV7r7E12cZ193X2hmmwLPm9kH7p7xWbzuPgGYAFBTU6PH3El32sHMZmYYz3ltq66lEGyw4bv7QdlO4u4L235fYmaPA3sDevh6EXN3SH0AnoCyYZiVBR1pY3zo7p1+INsVqu3S4+k1kPoQIv2xWGk9ZbPbn6VjZtVAxN1Xt/18CHBdd88r3ceTH+LLfwS+HDAgBpvcisVHBR0tr1TbpSe95m5Y8xuwGHgSL9sF63M7FukbdLScyPa2zGPNbAEwEnjazCa3jQ8ys0ltu20GvGZm7wJvAU+7+7PZzCvBcU/gX54C6YXgDeD14Cvx5RfiLQuDjpczqu3w8aaXYc1vgSbwNUAzJN/Dl18YdLScyfYunceBxzOMfw4c3vbzJ8DwbOaRAtI8BUhk2NCCN/wZ63lRvhN1C9V2+HjDPUDjOqOp1qbfsgiLbh5ErJzSN23lm0kvA++4YhYkIa27T6SItXRSvxaD9PL8ZukmavjyzZTvDWRo+FaFxffNexyRnImPAjq5+SC2XV6jdBc1fPlGLDYUKscCle1GKyC2A8QPDCqWSNas+myI9AbK241WQM8rMSvv7I8VFa14Jd+Y9boeykfiDQ8BTVBxNFZ1ImYqJyleFu0H/Z/C6+9r/awqOhCrPgMr/3bQ0XJG71D5xswMKo/AKrVEopQWi/TFel4KPS8NOkq30CUdEZGQUMMXEQkJNXwRkZBQwxcRCQk1fBGRkFDDFxEJCTV8EZGQUMMXEQkJNXwRkZBQwxcRCQk1fBGRkMh2xatbzOwDM3vPzB43s0062W+Mmc0xs7lmdlk2c4rkg2pbSlG2Z/jPA7u6++7Ah8Dl6+5gZlHgNuAwYBhwspkNy3Jeke6m2paSk1XDd/fn3D3V9vINYHCG3fYG5rr7J+6eAB4CxmYzr0h3U21LKcrlNfwzgGcyjG8BzG/3ekHbWEZmNs7Mas2stq5OS+ZJQci6tlXXUgg22PDN7AUzm5nh19h2+1wJpIAHsg3k7hPcvcbdawYMGJDt4UTWZ4d81bbqWgrBBhdAcfeD1rfdzE4DjgQOdHfPsMtCYMt2rwe3jYkE7UN3r+lso2pbSk22d+mMAX4KHO3uDZ3sNhXY3syGWuvCkCcBE7OZV6S7qbalFGV7DX880BN43symm9mdAGY2yMwmAbR98HUBMBmYDTzi7rOynFeku6m2peRktaatu2/XyfjnwOHtXk8CJmUzl0g+qbalFGkRcyk57mlIvIY3vQSRXljlMVhsm6BjiWTN06vwxschNRdiu2CVR2GR6i7/eTV8KSnuLfiK8yHxBngDEMPr78V7XU+kSrfIS/Hy1N54wcoAAAOCSURBVKf4shPBE0AjUInXj4d+j2HRgV06hp6lI6Wl+YV2zR5a76hsglVX4+k1QSYTyYqvvAp8Fa3Nntbf08vwVTd2+Rhq+FJSvPGpds2+HYu2/o9ApAi5JyE5DVj37uAWSLzc5eOo4UtpsQrAMm0Ai+c7jUiOROi8XZd9o6OIlAyrPB6oyLAlAuUj8h1HJCfMohA/kI4fu5ZDRdc/m1LDl5Ji8RFQfRoQByrBqsGqsT530vrdKJHiZL1/BtEhrTVNBVgVlA3Dev64y8fQXTpSciI9L8UrT4DE661vjvhoLFIVdCyRrFikL/R/ChJvQss8iO0AZXtglukSZmZq+FKSLDYYYicGHUMkp8wiEB8JjNyoP69LOiIiIaGGLyISEmr4IiIhoYYvIhISavgiIiFhmRfyKQxmVgfM68Yp+gNLu/H4uVAMGaE4c27t7nlfb1B1/TXlzJ0u1XVBN/zuZma161virhAUQ0ZQzkJSLP+Mypk7Xc2oSzoiIiGhhi8iEhJhb/gTgg7QBcWQEZSzkBTLP6Ny5k6XMob6Gr6ISJiE/QxfRCQ01PBFREIi1A3fzE4ws1lmljazgrvtyszGmNkcM5trZpcFnScTM/uDmS0xs5lBZ+mMmW1pZi+b2ftt/70vDjpTdyvk2i6GuobSrO1QN3xgJnAcMCXoIOsysyhwG3AYMAw42cyGBZsqo3uBMUGH2IAU8BN3HwZ8Bzi/QP9d5lJB1nYR1TWUYG2HuuG7+2x3nxN0jk7sDcx190/cPQE8BHR9LbM8cfcpwJdB51gfd1/k7m+3/bwamA1sEWyq7lXAtV0UdQ2lWduhbvgFbgtgfrvXCyjxJpUPZjYE2AN4M9gkoaW67iZdqe2SX/HKzF4ABmbYdKW7P5HvPBIcM+sB/Bm4xN1XBZ0nW6pt+UpXa7vkG767HxR0ho20ENiy3evBbWOyEcysjNY3xAPu/peg8+RCkda26jrHvklt65JO4ZoKbG9mQ82sHDgJmBhwpqJkras83w3MdvdfBp0n5FTXOfRNazvUDd/MjjWzBbSuCPy0mU0OOtNX3D0FXABMpvWDmEfcfVawqToysweBfwA7mtkCMzsz6EwZ7AOcAhxgZtPbfh0edKjuVKi1XSx1DaVZ23q0gohISIT6DF9EJEzU8EVEQkINX0QkJNTwRURCQg1fRCQk1PBFREJCDV9EJCT+P2uLgQUAFhskAAAAAElFTkSuQmCC\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAixElEQVR4nO3deZxcVZn/8c9TVd3VSzohG4QQlgAJENSItMEIGKIsAREEhQmOiqBmQED05/yQRXCE0UHRcWMzI7LMT9lGkQiBsGpABdJggISwRJZJSEyakK3X6qp6fn9UA013ddJJ3apbVff7fr361V3n3r7nCTz15ObUueeYuyMiItUvFnYAIiJSGir4IiIRoYIvIhIRKvgiIhGhgi8iEhGJsAPYkjFjxvgee+wRdhhSpZ588sk33H1sqftVXksxbSmvy7rg77HHHrS0tIQdhlQpM3stjH6V11JMW8prDemIiESECr6ISESo4IuIRIQKvohIRKjgAx2bO3n2kWWseOH1sEMRCdTql9fw7CPL2Ly+LexQpAyU9SydUrj9R/O4/qKbcRzPOhP22YUf3H8xo8aNDDs0ke3WtqGdi4+7nGWPvYTFDM86J557DF/+wecws7DDk5BE+g5/0YLF/PL8X9OTSpNOZciks7y2dAXnHvytsEMTKcilJ/2IJY8+TyadIZ1Kk0lnuP1Hf+DOK+8NOzQJUaQL/vUX3Uw2kx3Q/o9X1vLS314OISKRwm16czOLH1qS99gNl9xS4miknES64K9+Zc2gx1oWLC5dICIBatvQzmD7XLRv7ChxNFJOIl3wd9p98Kfqx+81roSRiARnx93GDHosnoj0Wz7yIv1//7TLZkOez69qkjUc9PEDSx+QSAASiQT7H7xv3mOHfOpDJY5GykmkC/5BHz+QT339WOKJGGZGLB6jpq6Gy+Z9k7qGZNjhiWy3S39/HuMm7tib25CoibPbfhP42jVzwg5NQhT5aZln/PBUjv/KLFoWLKausY4PH99M44jGsMMSKcjw0U3c+NLPeeqBZ1n5wip2mzKB98/cn1gs0vd4kRf5gg+w85478Ykzjwo7DJFAxWIxmo+cSvORU8MORcqE/roXEYkIFXwRkYhQwRcRiQgVfBGRiFDBFxGJCBV8EZGICKTgm9mvzGytmeVdscnMDjOzjWa2uPfrkiD6FSkm5bVUm6Dm4d8AXAnctIVzHnH3YwPqT6QUbkB5LVUkkDt8d18IvBnEtUTKhfJaqk0px/Cnm9nTZnaPme0/2ElmNsfMWsyspbW1tYThiWwX5bVUjFIV/KeA3d19KvBz4PeDnejuc9292d2bx44dfPlikTKgvJaKUpKC7+6b3L2t9+f5QI2ZDb5ot0gFUF5LpSlJwTezcda7c7KZTevtd10p+hYpFuW1VJpAZumY2c3AYcAYM1sJfBuoAXD3a4FPA2eaWRroBGb7YHuwiZQJ5bVUm0AKvrufspXjV5Kb3iZSMZTXUm30pK2ISERUTcHf9OZmli9+hfaN7WGHIhIYd2flS6t5dekKstls2OFIhav4Ha/SPWl+cuZcHvr1o9QkE6RTaY47axZf/v5ntZ2bVLT/ff51/u3EK1j7WisWMxqGN3Dhb85l6oxBp/uLbFHFV8TrLvwNf7z5z/R099CxqZNUVw9/uOY+7vjZ/LBDE9luPakevnHYJax84XW6O1N0tXfz5ur1fOvY/2Dd6vVhhycVqqILfjab5a5r76O7M/Wu9u6Obm7/0R9CikqkcE/M/xupzh76z/nJZrLcf9MfQ4lJKl9FF/x0T4ZUv2L/ls3rNpc4GpHgvLl6PZl0ZkB7qquHtSs01V+2T0UX/NpkDeP3Hpf32D4f3LvE0YgE5z2H7As2sL1+WB0HzHxP6QOSqlDRBR/gnCu/RLKhlt4HHonFjLrGJGf856khRyay/Sa+d3c+dGwzdY3Jt9tq62uZsM94ph/XHGJkUskqfpbOBw5/H//5p0v5zfd+x2vPrWTygXvxmQtPYPcpu4YdmkhBLvj1V1lw/R+5+xf30ZNK89HPHMonzzmaRE3Fv20lJFbOT4I3Nzd7S0tL2GFIlTKzJ9295LfLymsppi3ldcUP6YiIyNCo4IuIRIQKvohIRKjgi4hEhAq+iEhEBFLwzexXZrbWzJYMctzM7GdmttzMnjGzDwTRr0gxKa+l2gR1h38DMGsLx48GJvV+zQGuCahfkWK6AeW1VJFACr67LwTe3MIpxwM3ec5jwA5mtnMQfYsUi/Jaqk2pxvB3AVb0eb2yt02kkimvpaKUquDnWQaKvI/4mtkcM2sxs5bW1tYihyVSEOW1VJRSFfyVQN/FbSYAq/Kd6O5z3b3Z3ZvHjh1bkuBEtpPyWipKqQr+PODzvbMaPgRsdPfVxe501d//wV//0MKKF14vdlcSTaHkdSaT4ZmFz/H4/Kdo39RR7O6kigSy7J6Z3QwcBowxs5XAt4EaAHe/FpgPHAMsBzqA04LodzCp7h6+O/vHtCxYTKI2Qbonw3sP3Y9/+93/pa4hufULiFB+eQ3w96df5YKjv0tXexdmRronw1k/PY1jvnR4sbuWKhBIwXf3U7Zy3IGzguhrKG769q203Pc0qa4eUl09ADy78Dmu/caNfO2aOaUKQypcueV1Jp3h/CMvY0Prpne1X/2165ncvBd7v39iqUKRClWVT9re/V8PDNj6MNXVw/03/pFyXg5aZEv+9tASUt09A9p7utPcPff+ECKSSlOVBb+7oztve093mmw2W+JoRILRvrEj7xygbCbLpnVtpQ9IKk5VFvz3fWQKlmfC3L4H7U08Hi99QCIBeN+MKaR70gPa6xqTHHLCQSFEJJWmKgv+V356Og3DG6hJ1gCQqE1QP6yOr1795ZAjE9l+I3ccwWcvOYlkQ/LtG5q6xiR7vX8ih35KBV+2rio3x9xt31247rmfMO+qe3lh0XL2nLoHnzx7FjvupvnPUtlOOf8EpkyfzN1zH6B9YwczTprOzFMO1j63MiRVmyWjdx7Jaf++xUkWIhVp6oz9mTpj/7DDkApUlUM6IiIykAq+iEhEqOCLiESECr6ISESo4IuIRIQKvohIRKjgi4hEhAq+iEhEqOCLiESECr6ISEQEUvDNbJaZvWBmy83s/DzHDzOzjWa2uPfrkiD6FSk25bZUk4LX0jGzOHAVcAS5TZ0Xmdk8d3+u36mPuPuxhfYnUirKbak2QdzhTwOWu/vL7p4CbgGOD+C6ImFTbktVCaLg7wKs6PN6ZW9bf9PN7Gkzu8fMBl3qz8zmmFmLmbW0trYGEJ7Idgsst5XXUg6CKPh59pYasBHbU8Du7j4V+Dnw+8Eu5u5z3b3Z3ZvHjtX69RKqwHJbeS3lIIiCvxLYtc/rCcCqvie4+yZ3b+v9eT5QY2ZjAuhbpJiU21JVgij4i4BJZjbRzGqB2cC8vieY2Tiz3KZsZjatt991AfQtUkzKbakqBc/Scfe0mZ0NLADiwK/cfamZndF7/Frg08CZZpYGOoHZ7t7/n8ah6En18Luf3M09v3yQdDrDzNkHc8oFJ9LQVB92aBKySs/tlS+t5qZv38qzjyxj9PiRnHLBiRz8yWlhhyUhsjLJzbyam5u9paWlaNd3dy6Y9e8sefR5ujtTANQka9hl0jiuefIH2ie0ypnZk+7eXOp+i53XAKv+/g/OPPA8utq6yGZz7/G6hiSnfe8UTvzqx4vat4RrS3kd6Sdtn39iOUv/8sLbxR6gp7uHNa+28pc7F4UYmUhh/vvS2+lq73672AN0dXRzw7duIdXdE2JkEqZoF/zHXyKTzg5o72zrYsmfnw8hIpFgLHn0ebKZgbmNweqX15Q+ICkLkS74Y3cdTaI2PqA9WV/LuIk7hhCRSDB22j3/1M90T4aRO40ocTRSLiJd8A/6+AeoH1aPxd493TpeE+fwf/5ISFGJFO6UC04g2ZB8V1ttXQ0HH/9Bho9qCikqCVukC35NbQ0/eeQyJn1gIjXJGmrrapiwz3iuePDbDB+tN4VUrgOPmMo5V36RYSMbqWtMUpOs4ZATD+Ib130l7NAkRJGfhrLznjtx1RPfZ/2aDaR7MoydMDrskEQCcdQXZnL4Zz/C2hVvMHx0E43DG8IOSUIW+YL/lpE77RB2CCKBiyfi7Dxxp7DDkDIR6SEdEZEoUcEXEYkIFXwRkYhQwRcRiQgVfBGRiFDBFxGJCE3LLKHH7nqSO6+8h80b2jn0Uwdx3JlHUT9MyzBLZfPsZrzjv6HrQYiNxBpPxZKHhh2W5KGCXyLXX3ILv/vxXXS1dwPwyrP/y303/ImrFl1OXb9H4EUqhWfb8HWfhMxaIJfbnlqEDzuL2LA5ocYmA2lIpwTWr9nA7T+c93axB0h1pljzWiv33/SnECMTKYx33AqZVt4q9jmd0PZzPLsprLBkEIEUfDObZWYvmNlyMzs/z3Ezs5/1Hn/GzD4QRL+V4rm/vkhN7cB/THV3dPPE/KdCiEiGSrm9Fd0PAV0D260Gep4teTiyZQUXfDOLA1cBRwNTgFPMbEq/044GJvV+zQGuKbTfvtwd73kW73oYz7QGeelAjBg7HM8O3FksFo8xaueRIUQkQ1EOud3Z3sWie//G4oeXkO5JB3npYMR3BCzPgQzElNvlJogx/GnAcnd/GcDMbgGOB57rc87xwE29e30+ZmY7mNnO7r660M49swZ/8zTIrgJi4Cm84bNY0zfp3Vs6dFOmT2b4mCa6OrrfVfhrkgmO+8pRIUYmWxFqbv/ptr/wwy9eTSyeuy+L18S57M7z2f/D+xR66cBYw+fxrgd5911+DGK7QGK/sMKSQQQxpLMLsKLP65W9bdt6DgBmNsfMWsyspbV163frvuEcyLwC3gHeBqSg42bounub/hDFFIvFuOKBbzNh8njqGpM0DK+noamer889g72m7hF2eDK4wHJ7W/N69ctr+MFpV9HV3k3Hpk46NnWyeV0bFx7zXTrb8wyhhMRqD4DhF4PVgw0D6iExCRt1XdnccMk7grjDz/d/tf/4xVDOyTW6zwXmQm6z5y117JnV0LMMyPQ70ol33ITVH7ulXy+pnffcieuW/pjXnltJx6YO9v7AntQmawK5tmc3Qs8zEBsFiSl6owUnsNzelrwGuO+mP5JN98/r3PDl43c9yWH/dPDWLlEysYaT8PpPQM9zEBuBJfYK7NqeXg6ZVZDYD4vn38VLhi6Igr8S2LXP6wnAqu04Z9tl28Di+f/qKMMZAmbGHvvvuvUTt0G2bS60/Tz3IZlnIb4zjLoOi48PtJ+ICi23N7/ZTrpnYMHPZrK0b+wo9PKBM6uD2uA+r/bsRnz9nNwNndWAd+MN/4Q1fUs3NAUIYkhnETDJzCaaWS0wG5jX75x5wOd7ZzR8CNgYxBgniYlAbZ4DtVB3RMGXL3fe/Qi0XQV09w5ndUDmFXz9v4QdWrUILbcPOuYA6obVDWj3rHPAx95b6OXLnm84D3qWAF3gm8kN1f4P3nFb2KFVtIILvrungbOBBcAy4DZ3X2pmZ5jZGb2nzQdeBpYD/wUEss+aWQKGfxeoA97ajLwOYmOwxi8G0UVZ8/Ybgc5+rVlIv5b7p7AUJMzcPvDIqbzvI/tR1/jOQ3l1jUk+8ZWjGL/XuCC6KFue3QSpR4Gefkc6oePGMEKqGoE8aevu88klft+2a/v87MBZQfTVX6z+CDxxO97x/yDzOtQegjWcjMWGFaO78pJdn7/dEmU5pFWJwsrtWCzGpXd+k4W3P8ZDNz9CbbKGWV/8GM1HTg26q/Lj7Qx6L+obSxpKtamKpRWsZh9sxGVhh1F6dYdD24u8+ylHgCzU9J8uXhj3Hsi8BjYSi2vf31KIx+PMnH0wM2eXzwe0JREbB7EdILum34E4JA8LvDvProfsOojvRm7krnppaYUKZg2fg/g4ckNakJswUgdN38p9iBaQbMed+Nrp+LpP460zyL75JT02L0VjZtiI7/HuodpasBHYsHMC68e9i+yGr+FrD83l9tqDyLbfFNj1y1FV3OFHlcWGweg7ch9kdT8M8R2xhs9jtcH9s99Ti2DTxbzrwZrUX/EN52CjNJ4qxWHJQ2H0b/GOGyD9GtQehDX+MxYbFVgfvvHC3AqfpMBTucbNP8Lj47G6wwPrp5yo4Fc4iw3Dhp0Ow04vyvW9/ToGrpXSA6mn8MzrWDzv83MiBbOaSdiI7xbl2p7dDF33Aal+Rzrx9murtuBX5ZCOZzfi6ZW4Z8MOpfJlBplSbjWQeaO0sUScu7PmtVY2vqHhtIJl1/POcFE/mf6fHVSPqrrD9+zG3Pzd1J/JrecxDIb/O1b30bBDq1y1H4b03xkwRc7TEOATlbJlix9ewg++cCWb3thMNptlyvR9uPA35zJqnBYo2y7x8b0PdPWf1hyD2g+GElIpVNUdvq8/s3f+bgroguwb+Iav4T3Pbe1XZRDWeHrvGil97w3qYdjZ0Zj6WgZWv7yGiz9xOa0r1tHdmaKnO82SR5/nvCMuJTcrVLaVWQKaLuCdCQ8AMbAGbNi5YYVVdFVT8D39au+Tef0f1kjh7b8KIaLqYPEdsTF3Qv1JEN8Vag7AdrhCuxmV0J1X3ztgaeRMOsPa195g2eMvhRRV5Ys1fAobeTXUTIP4BKg7Dht9B5bYPezQiqaihnQ88w+849e59TVq3oM1fAaL75g7mFnd+0+0/h8wZiGzYsC1ZOgsPg4b8Z2ww6hqT/9xKfN/+SBd7V3MOPnDzDhpOvFEbox59fI1edfVMTNaV6yDD5U62uphyUOw5CFhh1EyFVPwvedF/M3ZvdOnUpB6LLdx8uhbscTeULPvO1Or3qUWavWOkPJ103du47Yr5pHq7MYdnnrgGRbc8DDfm38h8Xic9390f5584Gm6O96d3+meNJOb9wwpaqlEFTOk45u+885695D77m34ptwTthYbCQ2fBer7/FYcrBFr+HyJoxUZmtaV67j1+7+nuyNX7AG62rt57q8v8vhdue0vj/zCTIaPbiJR886skmRDksNmH8zOE3cKI2ypUBVR8N0dep7MdwRST7z9yprOw0b8GyT2yT2eXf8pbMydWgpAytbih5cQSwycHtjV1sVf5i0CoHF4A1e3fJ9jzziSsbuOZrf9JjDnis/xjV+eWepwpcJVxJCOmeEkGbgyJNBnCQEzg/oTsPoTShecSAEahzcQy7O+ezwRo2nUO7Ogdhg7grN+ejpn/bQ4D9hJNFTEHT4A9ScCyX6NydzsEZEK1XzU1Lf3rO0rXpNg1mkzQ4hIqlnFFHwbfl7vAxF1vfPCk7n1NZq+EXZoItuttq6W/7j3IoaPaaKhqZ6G4fUk62v56lVfYvcpwe6OJlIRQzoAZvXYqF/lNvZIvwKJvbCEZihI5dt32iRufX0uzz6yjO6OFO+bMYWGpvqt/6LINiqo4JvZKOBWYA/gVeBkdx+wK4eZvQpsJrfbeNrdm7e7z8TekNh7e39dZEhKnduJmgQHfLT6ty6UcBU6pHM+8KC7TwIe7H09mJnu/v5Cir1ICSm3peoUWvCPB95aFP1G4JMFXk+kXCi3peoUWvB3cvfVAL3fdxzkPAfuM7MnzWyLi7CY2RwzazGzltbW1gLDE9lugea28lrKwVbH8M3sAWBcnkMXbUM/B7v7KjPbEbjfzJ5394X5TnT3ucBcgObmZi0FKMU02cyW5GkPPLeV11IOtlrw3X3QrV/MbI2Z7ezuq81sZ2DtINdY1ft9rZndAUwD8hZ8kRJ6cbBxd+W2VKNCh3TmAaf2/nwqcGf/E8ys0cya3voZOBLId1clUk6U21J1Ci34lwNHmNlLwBG9rzGz8WY2v/ecnYBHzexp4Angbne/t8B+RYpNuS1Vp6B5+O6+DvhYnvZVwDG9P78MTC2kH5FSU25LNaqYpRVERKQwKvgiIhGhgi8iEhEq+CIiEaGCLyISESr4IiIRoYIvIhIRKvgiIhGhgi8iEhEq+CIiEaGCLyISESr4IiIRoYIvIhIRKvhSdN79F7LrPk12zYFk3zgB79b+IFL53FNk264ku/ZQsms+SHbDeXhmTdhhbZEKvhSVdy/E158BPc+Ab4b0Unz92WQ77w87NJGC+IZzoW0uZNeAb4SuP+DrTsSzm8MObVAFFXwzO8nMlppZ1szybhXXe94sM3vBzJab2fmF9CmVxTddDnT1a+2CtsvDCGfIlNuyJZ5eDt1/5t25nYHsZrzzt2GFtVWF3uEvAU5kC3t4mlkcuAo4GpgCnGJmUwrsVypF5pVB2lfgni1tLNtGuS2D61kGFs9zoAtST5U8nKEqqOC7+zJ3f2Erp00Dlrv7y+6eAm4Bji+kX6kgsbH5220UZuU7oqjcli2K7wp4ngO1kNir1NEMWSnecbsAK/q8XtnblpeZzTGzFjNraW1tLXpwUmTDzgKr79dYD8POCCWcgA05t5XXVaZmKsR3Y8AusZbAGv4plJCGYqsF38weMLMleb6Geidjedry/dWYO+A+192b3b157NhB7g6lYlj9yTDs62BNQG3u+7CzsIZTww4NYHKpclt5XV3MDBt1IyRnADVAAhKTsVH/jcXHhR3eoLa6ibm7H15gHyuBXfu8ngCsKvCaUiHMDGv8At7wOfBNYE2YbTXtSuVFdx/0A9khUG5HmMVGYiOvwb0TPI3FmsIOaatKMaSzCJhkZhPNrBaYDcwrQb9SRsziuTdI+RT7ICi3BbP6iij2UPi0zBPMbCUwHbjbzBb0to83s/kA7p4GzgYWAMuA29x9aWFhixSXcluqUUG3W+5+B3BHnvZVwDF9Xs8H5hfSl0gpKbelGpXvvDgREQmUCr6ISERU1SdoMnSe+Qd0LQDvhuRhWM3ksEMSKZi78+wjy3j2kWWM3HEEM06eTuOIxrDDKhsq+BGU7bwbNr617EsG2q7EGz5DbLiWgpHKle5Jc8nx3+fZR5aR6kxRW1/LL/71Jr5//8XsO21S2OGVBQ3pRIxnN8HGC4Du3q800AWdN+NlvAaIyNbcc91DPLNwGV3t3WSzTld7Nx2bO/nOp3+I+6DPekaKCn7UdC/Mv+iTd+GdmkIulWvB9Q/T3dE9oL1tQwevLvnfECIqPyr4kaM7HYka5fxbVPCjJvkR8PTAdqvD6j9R+nhEAnLUaTNJNiQHtA8b0cge79kthIjKjwp+xFhsBIz4DyDZ+xUH6qB+NlZ7YLjBiRTg6C9+lPcesi91jUksZtQ1JmloqueS//lXzPKtcxc9mqUTQbH6Y/HaZui6t8+0zH3CDkukIImaBN+75yKeWfgcSx55npE7aVpmfyr4EWXxcdD4hbDDEAmUmTF1xv5MnbF/2KGUJQ3piIhEhAq+iEhEqOCLiESECr6ISESo4IuIREShO16dZGZLzSxrZoPuDWpmr5rZs2a22MxaCulTpBSU21KNCp2WuQQ4EfjFEM6d6e5vFNifSKkot6XqFLrF4TJAT7FJ1VFuSzUq1Ri+A/eZ2ZNmNmdLJ5rZHDNrMbOW1tbWEoUnst2GlNvKaykHW73DN7MHgHF5Dl3k7ncOsZ+D3X2Vme0I3G9mz7v7wnwnuvtcYC5Ac3OzlrmTYppsZkvytAee28prKQdbLfjufnihnbj7qt7va83sDmAakLfgS2Vwd0g/D56CmimY1YQd0vZ40d0H/UB2KJTb1cezbZB+EWJjsER1rbJZ9LV0zKwRiLn75t6fjwQuLXa/Ujze8yK+/l/A1wMGJGCHH2LJGWGHVlLK7eqTbbsO2n4KlgDvwWv2x0ZejcVGhR1aIAqdlnmCma0EpgN3m9mC3vbxZja/97SdgEfN7GngCeBud7+3kH4lPO4p/M3PQfZ18A7wdvCN+Ppz8MzrYYcXGOV29HjXw9D2M6ALvA3ohp5n8PXnhB1aYAqdpXMHcEee9lXAMb0/vwxMLaQfKSPdC4FUngMZvOO3WNNXSx1RUSi3o8c7rgc6+7Wmc0U/sxqL7xxGWIHSk7aybbLrwLN5DvRAVrNPpIJlBslfS0B2fWljKRIVfNk2tdOAPAXfGrDkISUPRyQwyRnAIJMPEnuXNJRiUcGXbWKJiVB/PFDfp7UOEpMh+bGwwhIpmDV+GWIjgNo+rXXQdBFmtYP9WkXRjleyzWz4ZVA7He+4BeiCuuOwhpMxUzpJ5bL4aBhzF95+Y+6zqvg4rPF0rPaDYYcWGL1DZZuZGdR/HKv/eNihiATKYqOwpq9D09fDDqUoNKQjIhIRKvgiIhGhgi8iEhEq+CIiEaGCLyISESr4IiIRoYIvIhIRKvgiIhGhgi8iEhEq+CIiEaGCLyISEYXueHWFmT1vZs+Y2R1mtsMg580ysxfMbLmZnV9InyKloNyWalToHf79wHvc/X3Ai8AF/U8wszhwFXA0MAU4xcymFNivSLEpt6XqFFTw3f0+d0/3vnwMmJDntGnAcnd/2d1TwC3A8YX0K1Jsym2pRkGO4Z8O3JOnfRdgRZ/XK3vb8jKzOWbWYmYtra3aMk/KQsG5rbyWcrDVgm9mD5jZkjxfx/c55yIgDfw63yXytPlg/bn7XHdvdvfmsWPHDuXPILK9Jpcqt5XXUg62ugGKux++peNmdipwLPAxd8+X7CuBXfu8ngCs2pYgRYrkRXdvHuygcluqTaGzdGYB3wSOc/eOQU5bBEwys4mW2xhyNjCvkH5Fik25LdWo0DH8K4Em4H4zW2xm1wKY2Xgzmw/Q+8HX2cACYBlwm7svLbBfkWJTbkvVKWhPW3ffe5D2VcAxfV7PB+YX0pdIKSm3pRppE3OpOu5ZSD2Kdz0EseFY/SexxJ5hhyVSMM9uwjvvgPRySOyP1X8CizUO+fdV8KWquGfwDWdB6jHwDiCBt9+AD7+MWIOmyEvl8vQr+LqTwVNAJ1CPt18Jo/8Hi48b0jW0lo5Ul+4H+hR7yM2o7IJNF+PZtjAjEymIb/wW+CZyxZ7c9+w6fNPlQ76GCr5UFe+8q0+x78Piub8IRCqQew/0PMnAxzwykHp4yNdRwZfqYnXkfx7KwJKljkYkIDEGL9c123QVkaph9Z8G6vIciUHtQaUORyQQZnFIfoyBH7vWQt3QP5tSwZeqYsmDoPELQBKoB2sEa8RGXkvu2SiRymQjvgPxPXI5TR1YA9RMwZr+z5CvoVk6UnViTV/H60+C1F9yb47kTCzWEHZYIgWx2CgYcxekHofMa5CYDDUHYJZvCDM/FXypSpaYAImTww5DJFBmMUhOB6Zv1+9rSEdEJCJU8EVEIkIFX0QkIlTwRUQiQgVfRCQiLP9GPuXBzFqB14rYxRjgjSJePwiVECNUZpy7u3vJ9xtUXr9NcQZnSHld1gW/2MysZUtb3JWDSogRFGc5qZQ/o+IMzlBj1JCOiEhEqOCLiERE1Av+3LADGIJKiBEUZzmplD+j4gzOkGKM9Bi+iEiURP0OX0QkMlTwRUQiItIF38xOMrOlZpY1s7KbdmVms8zsBTNbbmbnhx1PPmb2KzNba2ZLwo5lMGa2q5k9bGbLev9/nxt2TMVWzrldCXkN1ZnbkS74wBLgRGBh2IH0Z2Zx4CrgaGAKcIqZTQk3qrxuAGaFHcRWpIFvuPt+wIeAs8r0v2WQyjK3KyivoQpzO9IF392XufsLYccxiGnAcnd/2d1TwC3A0PcyKxF3Xwi8GXYcW+Luq939qd6fNwPLgF3Cjaq4yji3KyKvoTpzO9IFv8ztAqzo83olVV6kSsHM9gAOAB4POZSoUl4XyVByu+p3vDKzB4BxeQ5d5O53ljqebZBv3zLNoS2AmQ0Dfgt8zd03hR1PoSo0t5XXRTDU3K76gu/uh4cdw3ZaCeza5/UEYFVIsVQ8M6sh94b4tbv/Lux4glChua28Dti25LaGdMrXImCSmU00s1pgNjAv5JgqkuV2eb4OWObu/xl2PBGnvA7QtuZ2pAu+mZ1gZivJ7Qh8t5ktCDumt7h7GjgbWEDug5jb3H1puFENZGY3A38F9jGzlWb2xbBjyuNg4HPAR81sce/XMWEHVUzlmtuVktdQnbmtpRVERCIi0nf4IiJRooIvIhIRKvgiIhGhgi8iEhEq+CIiEaGCLyISESr4IiIR8f8B/BEmpoDipmoAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": { - "tags": [], "needs_background": "light" - } + }, + "output_type": "display_data" } + ], + "source": [ + "pred = tf.matmul(valid_x,W)+b\n", + "fig,ax = plt.subplots(1,2)\n", + "ax[0].scatter(valid_x[:,0],valid_x[:,1],c=pred[:,0]>0.5)\n", + "ax[1].scatter(valid_x[:,0],valid_x[:,1],c=valid_labels)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To compute the accuracy on the validation data, we can cast boolean type to float, and compute the mean:" ] }, { "cell_type": "code", + "execution_count": 24, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -1005,23 +991,32 @@ "id": "HUjdeIefsIsg", "outputId": "f267f505-8ba4-43ef-9ebe-df124c3c05a1" }, - "source": [ - "tf.reduce_mean(tf.cast(((pred[0]>0.5)==valid_labels),tf.float32))" - ], - "execution_count": 28, "outputs": [ { - "output_type": "execute_result", "data": { "text/plain": [ "" ] }, - "metadata": { - "tags": [] - }, - "execution_count": 28 + "execution_count": 24, + "metadata": {}, + "output_type": "execute_result" } + ], + "source": [ + "tf.reduce_mean(tf.cast(((pred[0]>0.5)==valid_labels),tf.float32))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's explain what goes on here:\n", + "* `pred` is the values predicted by the network. They are not quite probabilities, because we have not used an activation function, but values greater than 0.5 correspond to class 1, and smaller - to class 0.\n", + "* `pred[0]>0.5` creates a boolean tensor of results, where `True` corresponds to class 1, and `False` - to class 0\n", + "* We compare that tensor to expected labels `valid_labels`, getting the boolean vector or correct predictions, where `True` corresponds to the correct prediction, and `False` - to incorrect one.\n", + "* We convert that tensor to floating point using `tf.cast`\n", + "* We then compute the mean value using `tf.reduce_mean` - that is exactly our desired accuracy " ] }, { @@ -1030,15 +1025,14 @@ "id": "_95qF9lY2kHp" }, "source": [ - "## Используем оптимизаторы TensorFlow\r\n", - "\r\n", - "Tensorflow достаточно плотно интегрирован с библиотекой Keras, которая содержит в себе множество полезного. Например, мы можем использовать оптимизаторы, реализующие немного другие алгоритмы обучения, чем градиентный спуск.\r\n", - "\r\n", - "Также попробуем выводить точность на всех этапах обучения." + "## Using TensorFlow/Keras Optimizers\n", + "\n", + "Tensorflow is closely integrated with Keras, which contains a lot of useful functionality. For example, we can use different **optimization algorithms**. Let's do that, and also print obtained accuracy during training." ] }, { "cell_type": "code", + "execution_count": 52, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -1046,79 +1040,78 @@ "id": "ups7nlV22ofp", "outputId": "aa4dff06-82b9-4b2f-ca00-33970ea2b989" }, - "source": [ - "optimizer = tf.keras.optimizers.Adam(0.01)\r\n", - "\r\n", - "learning_rate = 0.05\r\n", - "\r\n", - "W = tf.Variable(tf.random.normal(shape=(2,1)))\r\n", - "b = tf.Variable(tf.zeros(shape=(1,),dtype=tf.float32))\r\n", - "\r\n", - "@tf.function\r\n", - "def train_on_batch(x, y):\r\n", - " vars = [W, b]\r\n", - " with tf.GradientTape() as tape:\r\n", - " z = tf.sigmoid(tf.matmul(x, W) + b)\r\n", - " loss = tf.reduce_mean(tf.keras.losses.binary_crossentropy(z,y))\r\n", - " correct_prediction = tf.equal(tf.round(y), tf.round(z))\r\n", - " acc = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))\r\n", - " grads = tape.gradient(loss, vars)\r\n", - " optimizer.apply_gradients(zip(grads,vars))\r\n", - " return loss,acc\r\n", - "\r\n", - "for epoch in range(40):\r\n", - " for step, (x, y) in enumerate(dataset):\r\n", - " loss,acc = train_on_batch(tf.reshape(x,(-1,2)), tf.reshape(y,(-1,1)))\r\n", - " print('Epoch %d: last batch loss = %.4f, acc = %.4f' % (epoch, float(loss),acc))" - ], - "execution_count": 30, "outputs": [ { + "name": "stdout", "output_type": "stream", "text": [ - "Epoch 0: last batch loss = 6.1202, acc = 0.6667\n", - "Epoch 1: last batch loss = 5.9872, acc = 0.6667\n", - "Epoch 2: last batch loss = 5.8651, acc = 0.6667\n", - "Epoch 3: last batch loss = 5.7551, acc = 0.8333\n", - "Epoch 4: last batch loss = 5.6570, acc = 0.8333\n", - "Epoch 5: last batch loss = 5.5700, acc = 0.8333\n", - "Epoch 6: last batch loss = 5.4931, acc = 0.8333\n", - "Epoch 7: last batch loss = 5.4255, acc = 0.8333\n", - "Epoch 8: last batch loss = 5.3660, acc = 0.8333\n", - "Epoch 9: last batch loss = 5.3140, acc = 0.8333\n", - "Epoch 10: last batch loss = 5.2685, acc = 0.8333\n", - "Epoch 11: last batch loss = 5.2289, acc = 0.8333\n", - "Epoch 12: last batch loss = 5.1945, acc = 0.8333\n", - "Epoch 13: last batch loss = 5.1649, acc = 0.8333\n", - "Epoch 14: last batch loss = 5.1394, acc = 0.8333\n", - "Epoch 15: last batch loss = 5.1177, acc = 0.8333\n", - "Epoch 16: last batch loss = 5.0993, acc = 0.8333\n", - "Epoch 17: last batch loss = 5.0840, acc = 0.8333\n", - "Epoch 18: last batch loss = 5.0714, acc = 0.8333\n", - "Epoch 19: last batch loss = 5.0612, acc = 0.8333\n", - "Epoch 20: last batch loss = 5.0532, acc = 0.8333\n", - "Epoch 21: last batch loss = 5.0472, acc = 0.6667\n", - "Epoch 22: last batch loss = 5.0430, acc = 0.6667\n", - "Epoch 23: last batch loss = 5.0404, acc = 0.6667\n", - "Epoch 24: last batch loss = 5.0391, acc = 0.6667\n", - "Epoch 25: last batch loss = 5.0391, acc = 0.6667\n", - "Epoch 26: last batch loss = 5.0403, acc = 0.8333\n", - "Epoch 27: last batch loss = 5.0424, acc = 0.8333\n", - "Epoch 28: last batch loss = 5.0454, acc = 0.8333\n", - "Epoch 29: last batch loss = 5.0491, acc = 0.8333\n", - "Epoch 30: last batch loss = 5.0535, acc = 0.8333\n", - "Epoch 31: last batch loss = 5.0584, acc = 0.8333\n", - "Epoch 32: last batch loss = 5.0638, acc = 0.8333\n", - "Epoch 33: last batch loss = 5.0696, acc = 0.8333\n", - "Epoch 34: last batch loss = 5.0756, acc = 0.8333\n", - "Epoch 35: last batch loss = 5.0818, acc = 0.8333\n", - "Epoch 36: last batch loss = 5.0882, acc = 0.8333\n", - "Epoch 37: last batch loss = 5.0946, acc = 0.8333\n", - "Epoch 38: last batch loss = 5.1010, acc = 0.8333\n", - "Epoch 39: last batch loss = 5.1073, acc = 0.8333\n" - ], - "name": "stdout" + "Epoch 0: last batch loss = 1.4725, acc = 1.0000\n", + "Epoch 1: last batch loss = 5.4883, acc = 0.6667\n", + "Epoch 2: last batch loss = 3.4839, acc = 0.8333\n", + "Epoch 3: last batch loss = 6.0828, acc = 0.6667\n", + "Epoch 4: last batch loss = 7.4671, acc = 0.3333\n", + "Epoch 5: last batch loss = 4.8935, acc = 0.8333\n", + "Epoch 6: last batch loss = 4.5684, acc = 0.8333\n", + "Epoch 7: last batch loss = 4.8099, acc = 0.6667\n", + "Epoch 8: last batch loss = 6.8351, acc = 0.5000\n", + "Epoch 9: last batch loss = 4.1826, acc = 0.8333\n", + "Epoch 10: last batch loss = 4.8398, acc = 0.8333\n", + "Epoch 11: last batch loss = 5.0882, acc = 0.8333\n", + "Epoch 12: last batch loss = 3.6824, acc = 1.0000\n", + "Epoch 13: last batch loss = 2.9054, acc = 1.0000\n", + "Epoch 14: last batch loss = 3.8756, acc = 1.0000\n", + "Epoch 15: last batch loss = 5.0647, acc = 1.0000\n", + "Epoch 16: last batch loss = 3.7667, acc = 1.0000\n", + "Epoch 17: last batch loss = 2.2130, acc = 1.0000\n", + "Epoch 18: last batch loss = 3.9063, acc = 0.8333\n", + "Epoch 19: last batch loss = 5.5129, acc = 0.6667\n", + "Epoch 20: last batch loss = 6.1930, acc = 0.6667\n", + "Epoch 21: last batch loss = 5.8449, acc = 0.6667\n", + "Epoch 22: last batch loss = 2.4126, acc = 1.0000\n", + "Epoch 23: last batch loss = 2.6007, acc = 0.8333\n", + "Epoch 24: last batch loss = 2.1278, acc = 0.8333\n", + "Epoch 25: last batch loss = 2.1106, acc = 1.0000\n", + "Epoch 26: last batch loss = 5.0858, acc = 0.5000\n", + "Epoch 27: last batch loss = 5.0222, acc = 0.6667\n", + "Epoch 28: last batch loss = 5.5569, acc = 0.6667\n", + "Epoch 29: last batch loss = 6.2620, acc = 0.6667\n", + "Epoch 30: last batch loss = 2.6355, acc = 0.8333\n", + "Epoch 31: last batch loss = 3.6728, acc = 0.8333\n", + "Epoch 32: last batch loss = 4.3967, acc = 0.6667\n", + "Epoch 33: last batch loss = 4.1097, acc = 0.8333\n", + "Epoch 34: last batch loss = 0.6812, acc = 1.0000\n", + "Epoch 35: last batch loss = 4.7926, acc = 0.6667\n", + "Epoch 36: last batch loss = 3.1601, acc = 0.8333\n", + "Epoch 37: last batch loss = 1.8902, acc = 1.0000\n", + "Epoch 38: last batch loss = 7.0491, acc = 0.5000\n", + "Epoch 39: last batch loss = 1.2064, acc = 1.0000\n" + ] } + ], + "source": [ + "optimizer = tf.keras.optimizers.Adam(0.01)\n", + "\n", + "learning_rate = 0.05\n", + "\n", + "W = tf.Variable(tf.random.normal(shape=(2,1)))\n", + "b = tf.Variable(tf.zeros(shape=(1,),dtype=tf.float32))\n", + "\n", + "@tf.function\n", + "def train_on_batch(x, y):\n", + " vars = [W, b]\n", + " with tf.GradientTape() as tape:\n", + " z = tf.sigmoid(tf.matmul(x, W) + b)\n", + " loss = tf.reduce_mean(tf.keras.losses.binary_crossentropy(z,y))\n", + " correct_prediction = tf.equal(tf.round(y), tf.round(z))\n", + " acc = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))\n", + " grads = tape.gradient(loss, vars)\n", + " optimizer.apply_gradients(zip(grads,vars))\n", + " return loss,acc\n", + "\n", + "for epoch in range(40):\n", + " for step, (x, y) in enumerate(dataset):\n", + " loss,acc = train_on_batch(tf.reshape(x,(-1,2)), tf.reshape(y,(-1,1)))\n", + " print('Epoch %d: last batch loss = %.4f, acc = %.4f' % (epoch, float(loss),acc))" ] }, { @@ -1127,9 +1120,9 @@ "id": "dvAiaj_JndyP" }, "source": [ - "**Задание 1**: Постройте графики ошибок на обучающей и тестовой выборке в процессе обучения\r\n", - "\r\n", - "**Задание 2**: Попробуйте решить задачу классификации на датасете MNIST с помощью этого кода. Подсказка: используйте `softmax_crossentropy_with_logits` или `sparse_softmax_cross_entropy_with_logits` в качестве функции ошибки. При этом в первом случае на выход сети необходимо подавать целевые значения в формате *one hot encoding*, а во втором - в виде целочисленного номера класса." + "**Task 1**: Plot the graphs of loss function and accuracy on training and validation data during training\n", + "\n", + "**Task 2**: Try to solve MNIST classificiation problem using this code. Hint: use `softmax_crossentropy_with_logits` or `sparse_softmax_cross_entropy_with_logits` as loss function. In the first case you need to feed expected output values in *one hot encoding*, and in the second case - as integer class number." ] }, { @@ -1138,22 +1131,32 @@ "id": "995iCprDrgYQ" }, "source": [ - "## Keras\r\n", - "### Deep Learning for Humans\r\n", - "\r\n", - "* Раньше работал поверх Tensorflow, CNTK или Theano, сейчас включен в состав Tensorflow\r\n", - "* Оперирует нейросетями на уровне слоёв\r\n", - "* Включает упрощённый \"обучатель\", средства работы с типовыми данными (картинками, ...)\r\n", - "* Много готовых примеров\r\n", - "* Functional API vs. Sequential API\r\n", - "\r\n", - "Keras даёт более высокоуровневое API для реализации нейросетей, позволяя определять нейросети как комбинации слоёв и оперировать понятиями \"модель\", \"слой\", \"алгоритм обучения\".\r\n", - "\r\n", - "Книжка от создателя Keras: [Deep Learning with Python](https://www.manning.com/books/deep-learning-with-python)" + "## Keras\n", + "### Deep Learning for Humans\n", + "\n", + "* Keras is a library originally developed by Francois Chollet to work on top of Tensorflow, CNTK and Theano, to unify all lower-level frameworks. You can still install Keras as a separate library, but it is not advised to do so. \n", + "* Now Keras is included as part of Tensorflow library\n", + "* You can easily construct neural networks from layers\n", + "* Contains `fit` function to do all training, plus a lot of functions to work with typical data (pictures, text, etc.)\n", + "* A lot of samples\n", + "* Functional API vs. Sequential API\n", + "\n", + "Keras provides higher level abstractions for neural networks, allowing us to operate in terms of layers, models and optimizers, and not in terms of tensors and gradients. \n", + "\n", + "Classical Deep Learning book from the creator of Keras: [Deep Learning with Python](https://www.manning.com/books/deep-learning-with-python)\n", + "\n", + "### Functional API\n", + "\n", + "When using functional API, we define the **input** to the network as `keras.Input`, and then compute the **output** by passing it through a series of computations. Finally, we define **model** as an object that transforms input into output.\n", + "\n", + "Once we obtained **model** object, we need to:\n", + "* **Compile it**, by specifying loss function and the optimizer that we want to use with our model\n", + "* **Train it** by calling `fit` function with the training (and possibly validation) data" ] }, { "cell_type": "code", + "execution_count": 26, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -1161,71 +1164,71 @@ "id": "QJWplVfy34Eo", "outputId": "9be976f2-4f9a-495c-bddc-a7f9ec30989a" }, - "source": [ - "inputs = tf.keras.Input(shape=(2,))\r\n", - "z = tf.keras.layers.Dense(1,kernel_initializer='glorot_uniform',activation='sigmoid')(inputs)\r\n", - "model = tf.keras.models.Model(inputs,z)\r\n", - "\r\n", - "train_x_norm = train_x-np.min(train_x) / (np.max(train_x)-np.min(train_x))\r\n", - "\r\n", - "model.compile(tf.keras.optimizers.Adam(0.1),'binary_crossentropy',['accuracy'])\r\n", - "model.summary()\r\n", - "h = model.fit(train_x_norm,train_labels,batch_size=8,epochs=15)" - ], - "execution_count": 32, "outputs": [ { + "name": "stdout", "output_type": "stream", "text": [ - "Model: \"model_1\"\n", + "Model: \"functional_1\"\n", "_________________________________________________________________\n", "Layer (type) Output Shape Param # \n", "=================================================================\n", - "input_2 (InputLayer) [(None, 2)] 0 \n", + "input_1 (InputLayer) [(None, 2)] 0 \n", "_________________________________________________________________\n", - "dense_1 (Dense) (None, 1) 3 \n", + "dense (Dense) (None, 1) 3 \n", "=================================================================\n", "Total params: 3\n", "Trainable params: 3\n", "Non-trainable params: 0\n", "_________________________________________________________________\n", "Epoch 1/15\n", - "9/9 [==============================] - 0s 1ms/step - loss: 1.1443 - accuracy: 0.3283\n", + "9/9 [==============================] - 0s 5ms/step - loss: 1.2681 - accuracy: 0.1857\n", "Epoch 2/15\n", - "9/9 [==============================] - 0s 2ms/step - loss: 0.6759 - accuracy: 0.5823\n", + "9/9 [==============================] - 0s 1ms/step - loss: 0.7420 - accuracy: 0.5286\n", "Epoch 3/15\n", - "9/9 [==============================] - 0s 1ms/step - loss: 0.4717 - accuracy: 0.7888\n", + "9/9 [==============================] - 0s 2ms/step - loss: 0.5167 - accuracy: 0.7714\n", "Epoch 4/15\n", - "9/9 [==============================] - 0s 1ms/step - loss: 0.3932 - accuracy: 0.9044\n", + "9/9 [==============================] - 0s 1ms/step - loss: 0.4352 - accuracy: 0.8429\n", "Epoch 5/15\n", - "9/9 [==============================] - 0s 1ms/step - loss: 0.4254 - accuracy: 0.8513\n", + "9/9 [==============================] - 0s 2ms/step - loss: 0.4198 - accuracy: 0.8571\n", "Epoch 6/15\n", - "9/9 [==============================] - 0s 1ms/step - loss: 0.3866 - accuracy: 0.8534\n", + "9/9 [==============================] - 0s 2ms/step - loss: 0.4091 - accuracy: 0.8429\n", "Epoch 7/15\n", - "9/9 [==============================] - 0s 1ms/step - loss: 0.4578 - accuracy: 0.8344\n", + "9/9 [==============================] - 0s 1ms/step - loss: 0.4118 - accuracy: 0.8286\n", "Epoch 8/15\n", - "9/9 [==============================] - 0s 1ms/step - loss: 0.4432 - accuracy: 0.8192\n", + "9/9 [==============================] - 0s 1ms/step - loss: 0.4058 - accuracy: 0.8429\n", "Epoch 9/15\n", - "9/9 [==============================] - 0s 1ms/step - loss: 0.4172 - accuracy: 0.8408\n", + "9/9 [==============================] - 0s 1ms/step - loss: 0.4072 - accuracy: 0.8571\n", "Epoch 10/15\n", - "9/9 [==============================] - 0s 1ms/step - loss: 0.3778 - accuracy: 0.8819\n", + "9/9 [==============================] - 0s 1ms/step - loss: 0.4125 - accuracy: 0.8714\n", "Epoch 11/15\n", - "9/9 [==============================] - 0s 1ms/step - loss: 0.4008 - accuracy: 0.8800\n", + "9/9 [==============================] - 0s 1ms/step - loss: 0.4186 - accuracy: 0.8429\n", "Epoch 12/15\n", - "9/9 [==============================] - 0s 3ms/step - loss: 0.3769 - accuracy: 0.9019\n", + "9/9 [==============================] - 0s 1ms/step - loss: 0.4092 - accuracy: 0.8429\n", "Epoch 13/15\n", - "9/9 [==============================] - 0s 1ms/step - loss: 0.3995 - accuracy: 0.8336\n", + "9/9 [==============================] - 0s 2ms/step - loss: 0.4078 - accuracy: 0.8429\n", "Epoch 14/15\n", - "9/9 [==============================] - 0s 1ms/step - loss: 0.3816 - accuracy: 0.8719\n", + "9/9 [==============================] - 0s 1ms/step - loss: 0.4095 - accuracy: 0.8429\n", "Epoch 15/15\n", - "9/9 [==============================] - 0s 1ms/step - loss: 0.4211 - accuracy: 0.8244\n" - ], - "name": "stdout" + "9/9 [==============================] - 0s 1ms/step - loss: 0.4087 - accuracy: 0.8429\n" + ] } + ], + "source": [ + "inputs = tf.keras.Input(shape=(2,))\n", + "z = tf.keras.layers.Dense(1,kernel_initializer='glorot_uniform',activation='sigmoid')(inputs)\n", + "model = tf.keras.models.Model(inputs,z)\n", + "\n", + "train_x_norm = train_x-np.min(train_x) / (np.max(train_x)-np.min(train_x))\n", + "\n", + "model.compile(tf.keras.optimizers.Adam(0.1),'binary_crossentropy',['accuracy'])\n", + "model.summary()\n", + "h = model.fit(train_x_norm,train_labels,batch_size=8,epochs=15)" ] }, { "cell_type": "code", + "execution_count": 27, "metadata": { "colab": { "base_uri": "https://localhost:8080/", @@ -1234,36 +1237,32 @@ "id": "K2Kf60IrZcqs", "outputId": "b60b868d-3562-4715-f5d5-1f9764e45f09" }, - "source": [ - "plt.plot(h.history['accuracy'])" - ], - "execution_count": 37, "outputs": [ { - "output_type": "execute_result", "data": { "text/plain": [ - "[]" + "[]" ] }, - "metadata": { - "tags": [] - }, - "execution_count": 37 + "execution_count": 27, + "metadata": {}, + "output_type": "execute_result" }, { - "output_type": "display_data", "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAbKElEQVR4nO3dfXRU933n8fdXT0iAAAkE2EJIGAtsbKd+EPhpm+MmddZJeiDd7Db41E+JU7a7cZJts921d3u8OW67p/vUdM+ptz3eOMa1U7Neb9LQLq3tk6Yn3fgBCRs/AJYgWAIBGgQSYiTQ43z3jxnhsRBoECPdufd+Xudw0Ny5zHyA0Ud3fvd3f2PujoiIhF9R0AFERCQ/VOgiIhGhQhcRiQgVuohIRKjQRUQioiSoJ16yZIk3NDQE9fQiIqG0a9euE+5eM9l9gRV6Q0MDLS0tQT29iEgomVnHhe7TkIuISESo0EVEIkKFLiISESp0EZGIUKGLiESECl1EJCJU6CIiERHYPHSZvv6hUX74VidXLKzglvoqquaVBR1JRAqACj1kegeGeeiZnbzT2XduW+PS+TQ1VNFUX836hmrqqiswswBTikgQVOgh0tU3yP1Pv0lHzxn+9NdvpnpeGS0dvTS39/DX7x7jhZ2HAaipnMP6rIK/9opKSoo1uiYSdSr0kGg/McB9T79J78Awz355A7evXgzArVelf0+lnLbjSZrbe9nV3kNzey873usCYG5ZMTetXERTfTVNDVXctLKK+XP0Xy8SNRbUR9A1NTW51nLJzb5jp7n/6Z2MpVI8+5UNfGLFopz+3NFTZ2np+KjgP+g6TcqhyGDdlQvOFfz6hmqWLSif4b+FiOSDme1y96ZJ71OhF7ZdHb18+ZmdzC0r4bmHN9C4rHLaj5UcHOHtQ6doyRT87sOnODsyBkBddQVN9enhmeKi/A/PLK2cQ1NDFVcsrMj7Y+fL4MgY7x3pY+/R04ym9Fm7M6Gs2LhhxSKuu3IBpQU6DOjuHO45y65DPfQMjMzIc9yxejHXXrFgWn/2YoWu990F7B/2d7Plz3exbMEcnnv4Vuqq517W41WWl/LJNTV8ck165c2RsRR7j56mub2HXR29/MP+E/zw7SP5iH5BtYsq0idwG6pZ31DFmqWVFBUFcwK3d2CYXR29NHf00NLey3udfQyPpQLJEjflpUXcWLeI9Q3V3FJfxc31VSwoLw0ky+hYin3HkrRkXgfN7T0cTw7N6HP+/heun3ahX4yO0AvU37x3jG9se5vVNfP584c3sLRy5odE3J3k0Ch5f0k4HOo5c+4HR/Y3TGV5CbfUp4d9muqr+IW6RZSXFuc5QPrvdqjnDC3tvbR0pN+hHDjeD0BpsXFD7UKaZjiDwMDQKLsPn6K5PV2ee4+dZizlmME1yxewvqHq3OvhykUz825uYGiUtw+dOvd6fOtQL2eG0+9Usw84muqrZixDeWkRc0qm9xrTkEvIvNh8mEd/8C43raziew+tZ2FFMEcuM2X8Le14sba097A/q1yvr11IU/1H31SL58+55OcYHUux99jp9EnizPN0Z/0QyX58FXhwsgt+V0cvb3X0MpBVrulyr+KW+mrWLq+keBrv5hKnB88debd09LDvWPLcD5Frly+YlQLPJxV6iPzPnx7kD3bs45Nraviz+25mblk8RsVOnckMf2QK+J3DHw1/XLVkXtYwTTUNi+eeN8++f2iUtw/1nvsBsfvwqXNHXSuqKs4V+PqGahqXzg9smEcubnQsxQddyfR5no70/2XidOYH8ZwSbq6vOvd/eWPdIirKPv6DOJVyDnT3p39AtKeH0w73nAXSR8U31WV+QDRUc9PKRYEN81wOFXoIuDv/7ZU2/uQnB/j8DVfwnS/dSFlJYZ40mg2DI2O8f6SPlsw3dUtHL6fOpE9QLZ5XRlNDFTfWVZE4PUhzew/7jn00g+faKxacG5st9BOxcnHuTmfvR+/mdrX30ppIAlBSZFxXu5D1mauld3X0squjl76z6dfJkvlzMuWfHsJZV8AnYi+FCr3ApVLOf9i+h+fe6GDz+jr+4FdvmNZbyyhLpZyDJ/ppzrx13tXRS8fJM1SUZubYZ06yao599PWdGeGtQ73nxuF3d55ieDTF6pp56XMxmeGT+kneyUWBCr2AjYyl+Nf/+x1+tPso//yTV/HoZ6+J5ItwJvQODDO/vCQSR10yfUOjYwwOp1g4N3zDJ9OhaYsFanBkjK99/y1+/MFxfucfr+Vf3rVaZX4JtCiZAMwpKZ72jJGoUaEHJDk4wlefbWFnew+/94Xruf+2+qAjiUjIqdADcLJ/iAef2ckHx5L88ZduZNONtUFHEpEIUKHPsqOnznL/02/S2XuWpx64hU9dsyzoSCISESr0WfThiQHu++6bnD47wnMP38qGVdVBRxKRCFGhz5I9R/t48Hs7cYcXttzG9bULg44kIhGjQp8Fze09fGVrM5VzSnjuq7eyumZ+0JFEJIJU6DPs71uP85vP7+LKhRU899VbqQ3BWhEiEk4q9BnUOzDMv3j+La5akl4xcck0FpkSEcmVLrGbQduaD3N2ZIzvfOlGlbmIzLicCt3M7jGzVjM7YGaPTnL/SjP7iZm9bWbvmtnn8h81XEbHUjz3ejt3rF7M2uXT/5QhEZFcTVnoZlYMPAl8FlgH3Gtm6ybs9rvAi+5+E7AZ+B/5Dho2r+5NcLRvkIfuaAg6iojERC5H6BuAA+5+0N2HgW3Apgn7ODD+eUoLgaP5ixhOW19rZ0VVBZ++VhcOicjsyKXQa4HDWbc7M9uyfRu4z8w6gR3A1/OSLqT2Hj3Nmx/28MDt9VoGV0RmTb5Oit4LbHX3FcDngOfM7LzHNrMtZtZiZi3d3d15eurC8+xr7VSUFvOlppVBRxGRGMml0I8AdVm3V2S2ZXsYeBHA3V8HyoElEx/I3Z9y9yZ3b6qpqZle4gLXMzDMX+4+wq/eXBub9ZlFpDDkUujNQKOZrTKzMtInPbdP2OcQ8GkAM7uWdKFH9xD8IrY1H2JoNKWToSIy66YsdHcfBR4BXgb2kZ7NssfMnjCzjZndvgX8hpm9A7wAPORBfRRSgEbHUjz/egd3rF7MmmWaqigisyunK0XdfQfpk53Z2x7P+novcGd+o4XP+FTFb2+8LugoIhJDulI0j57RVEURCZAKPU/2HO1j54c9PHh7g6YqikggVOh5Mj5V8dea6qbeWURkBqjQ8yA9VfEo/0RTFUUkQCr0PNjWfIjh0RQPaqqiiARIhX6Z0qsqdnDn1ZqqKCLBUqFfplf2JjjWN8hDd6wKOoqIxJwK/TJt/Vk7ddUVfOqapUFHEZGYU6Ffhj1H+9jZrqmKIlIYVOiXYXyq4j/TVEURKQAq9Gn62FTFCk1VFJHgqdCn6YWd6amKWlVRRAqFCn0aRsZSPP9GB//o6iU0aqqiiBQIFfo0vLJnfKpiQ9BRRETOUaFPw9bXPqSuuoJf0lRFESkgKvRL9P6RPprbezVVUUQKjgr9EmmqoogUKhX6JTjZP8SP3jnKF2/RVEURKTwq9EuwrflwelXF2xuCjiIich4Veo5GMqsqaqqiiBQqFXqOXtmToOu0piqKSOFSoedo62sfsrJ6rqYqikjBUqHnYHyq4gO312uqoogULBV6Dra+1s7cMk1VFJHCpkKfwon+IbZrVUURCQEV+hS27TzE8JimKopI4VOhX0R6VcVD/GKjpiqKSOHLqdDN7B4zazWzA2b26CT3f8fMdmd+tZnZqfxHnX0v7+nSVEURCY2SqXYws2LgSeBuoBNoNrPt7r53fB93/62s/b8O3DQDWWfd1p+1U794Lr+0VlMVRaTw5XKEvgE44O4H3X0Y2AZsusj+9wIv5CNckN7r7KOlo5cHbm+gSFMVRSQEcin0WuBw1u3OzLbzmFk9sAr4uwvcv8XMWsyspbu7+1KzzqqPpiquCDqKiEhO8n1SdDPwkruPTXanuz/l7k3u3lRTU5Pnp86fE/1D/NU7R/nizStYUK6piiISDrkU+hEg+4qaFZltk9lMBIZbzk1VvKM+6CgiIjnLpdCbgUYzW2VmZaRLe/vEnczsGqAKeD2/EWfXyFiK597o4Bcbl3D1Uk1VFJHwmLLQ3X0UeAR4GdgHvOjue8zsCTPbmLXrZmCbu/vMRJ0df/t+F4nTQ3z5zoago4iIXJIppy0CuPsOYMeEbY9PuP3t/MUKztbX0lMV71qjqYoiEi66UjTLe5197NJURREJKRV6Fk1VFJEwU6FnjE9V/Ke3aKqiiISTCj3jhTfTUxUf0KqKIhJSKvSMH39wnKb6Kq5eOj/oKCIi06JCB1IpZ38iyfW1C4OOIiIybSp04MipswwMj7FGa56LSIip0IG2RBKAtcs13CIi4aVCB1ozha5PJRKRMFOhA21dSa5cWK7piiISaip0oDXRz5rlOjoXkXCLfaGPjqX4+fF+1mq4RURCLvaF3tFzhuGxlMbPRST0Yl/obV2ZGS4qdBEJudgXemsiiRm6QlREQi/2hd6WSFJfPZeKsuKgo4iIXJbYF3prV1JXiIpIJMS60AdHxmg/eYa1mrIoIhEQ60I/2D3AWMp1hC4ikRDrQv9oDRcVuoiEX6wLvTWRpLTYaFg8L+goIiKXLdaF3taV5Kol8ykrifU/g4hERKybrDWR1BouIhIZsS30/qFROnvPsnaZLigSkWiIbaHv1xroIhIxMS70fkBruIhIdORU6GZ2j5m1mtkBM3v0Avv8mpntNbM9ZvYX+Y2Zf62JJOWlRdRVzw06iohIXpRMtYOZFQNPAncDnUCzmW13971Z+zQCjwF3unuvmS2dqcD50pZI0ri0kuIiCzqKiEhe5HKEvgE44O4H3X0Y2AZsmrDPbwBPunsvgLsfz2/M/NMaLiISNbkUei1wOOt2Z2ZbtjXAGjP7mZm9YWb35CvgTOgdGOZ4coi1yzXDRUSiY8ohl0t4nEbgLmAF8FMzu8HdT2XvZGZbgC0AK1euzNNTX7rxS/51hC4iUZLLEfoRoC7r9orMtmydwHZ3H3H3D4E20gX/Me7+lLs3uXtTTU3NdDNfNq3hIiJRlEuhNwONZrbKzMqAzcD2Cfv8Jemjc8xsCekhmIN5zJlXrYkkleUlLF9QHnQUEZG8mbLQ3X0UeAR4GdgHvOjue8zsCTPbmNntZeCkme0FfgL8jrufnKnQl6utq5+1yyox0wwXEYmOnMbQ3X0HsGPCtsezvnbgtzO/Cpq705pI8vlPXBF0FBGRvIrdlaLHk0P0nR3RFaIiEjmxK/TWLs1wEZFoil2hfzRlUXPQRSRaYlnoS+aXsXj+nKCjiIjkVewKvTXRr+EWEYmkWBV6KuXsT2gNFxGJplgV+pFTZzkzPKYrREUkkmJV6JrhIiJRFq9C1wwXEYmwWBV6WyJJ7aIKKstLg44iIpJ3sSr09Ida6OhcRKIpNoU+MpbiYPcAa3RCVEQiKjaF3nFygOGxlNZwEZHIik2ht3b1A5rhIiLRFZ9CTyQpMrh6qcbQRSSaYlPobV1J6hfPo7y0OOgoIiIzIj6FflwzXEQk2mJR6IMjY7SfGNAJURGJtFgU+s+7+0k5mrIoIpEWi0If/1ALHaGLSJTFotBbu/opLTYalswLOoqIyIyJRaG3JZKsrplPaXEs/roiElOxaLj0Gi4abhGRaIt8oScHRzhy6qw+1EJEIi/yhb7/uC75F5F4iHyht3VphouIxEPkC701kaSitJgVVRVBRxERmVE5FbqZ3WNmrWZ2wMweneT+h8ys28x2Z359Nf9Rp6ctkb7kv6jIgo4iIjKjSqbawcyKgSeBu4FOoNnMtrv73gm7/i93f2QGMl6W1q5+7lpbE3QMEZEZl8sR+gbggLsfdPdhYBuwaWZj5UfPwDAn+oc0fi4isZBLodcCh7Nud2a2TfRFM3vXzF4ys7rJHsjMtphZi5m1dHd3TyPupRm/5F9ruIhIHOTrpOhfAQ3u/gngVeDZyXZy96fcvcndm2pqZn4YRGu4iEic5FLoR4DsI+4VmW3nuPtJdx/K3PwucEt+4l2e1q4kC8pLWLZgTtBRRERmXC6F3gw0mtkqMysDNgPbs3cwsyuybm4E9uUv4vS1JZKsXV6JmWa4iEj0TVno7j4KPAK8TLqoX3T3PWb2hJltzOz2DTPbY2bvAN8AHpqpwLlyd63hIiKxMuW0RQB33wHsmLDt8ayvHwMey2+0y5M4PcTpwVGt4SIisRHZK0Vbx2e46AhdRGIisoU+voaLCl1E4iKyhd6aSFJTOYfqeWVBRxERmRWRLfS2RFLzz0UkViJZ6KmU05ZI0rhsftBRRERmTSQL/XDvGQZHUjpCF5FYiWShtyUyn1KkKYsiEiMRLfT0DJfGpRpyEZH4iGSht3YlqV1UQWV5adBRRERmTSQLfXwNFxGROIlcoY+Mpfh5d78uKBKR2IlcobefGGBkzFm7XOPnIhIvkSt0reEiInEVuUJv60pSZLC6RkfoIhIvkSv01kSShiXzKC8tDjqKiMisilyhtyX6dYWoiMRSpAp9cGSM9pMDGj8XkViKVKEfON6Pu06Iikg8RarQWzMfaqEpiyISR5Eq9LbjScqKi6hfPC/oKCIisy5ahd6V5KqaeZQWR+qvJSKSk0g1X1uiX2u4iEhsRabQk4MjHDl1VidERSS2IlPo4x9qoTnoIhJXESr08RkuKnQRiafIFHprV5K5ZcXULqoIOoqISCByKnQzu8fMWs3sgJk9epH9vmhmbmZN+YuYm7ZEksZllRQV2Ww/tYhIQZiy0M2sGHgS+CywDrjXzNZNsl8l8E3gzXyHzEVbIsnaZbqgSETiK5cj9A3AAXc/6O7DwDZg0yT7/R7wn4DBPObLyYn+IU70D2uGi4jEWi6FXgsczrrdmdl2jpndDNS5+//NY7ac6YSoiEgeToqaWRHwR8C3cth3i5m1mFlLd3f35T71OW1d+pQiEZFcCv0IUJd1e0Vm27hK4Hrg782sHbgN2D7ZiVF3f8rdm9y9qaamZvqpJ2hN9LOwopSllXPy9pgiImGTS6E3A41mtsrMyoDNwPbxO929z92XuHuDuzcAbwAb3b1lRhJPYn8iydpllZhphouIxNeUhe7uo8AjwMvAPuBFd99jZk+Y2caZDjgVd6c1kWSNlswVkZgryWUnd98B7Jiw7fEL7HvX5cfKXdfpQZKDo7rkX0RiL/RXirbqhKiICBCBQh+fsqhCF5G4C32ht3b1s7RyDlXzyoKOIiISqNAXelsiqQuKREQIeaGPpZz9x5MabhERIeSFfrjnDIMjKc1wEREh5IXeOn5CVEMuIiLhLvTxNVwal+qiIhGRUBd6ayLJiqoK5s3J6fooEZFIC3Wht2XWcBERkRAX+vBoioPdAxo/FxHJCG2ht58cYDTlOkIXEckIbaFrDRcRkY8LbaG3JZIUFxlX1cwLOoqISEEIbaG3diVpWDyX8tLioKOIiBSE0Ba61nAREfm4UBb62eExOnrOaPxcRCRLKAv9wPF+3NEMFxGRLKEsdK3hIiJyvlAWelsiSVlJEfXVc4OOIiJSMEJZ6K1dSVbXzKekOJTxRURmRCgbMb2Gi1ZYFBHJFrpC7zs7wrG+QY2fi4hMELpCP3A8fUJUM1xERD4udIXe2tUPaA0XEZGJQlfoS+aXcfe6ZdQuqgg6iohIQQndR/185rrlfOa65UHHEBEpODkdoZvZPWbWamYHzOzRSe7/TTN7z8x2m9n/M7N1+Y8qIiIXM2Whm1kx8CTwWWAdcO8khf0X7n6Du98I/Gfgj/KeVERELiqXI/QNwAF3P+juw8A2YFP2Du5+OuvmPMDzF1FERHKRyxh6LXA463YncOvEnczsa8BvA2XApyZ7IDPbAmwBWLly5aVmFRGRi8jbLBd3f9LdVwP/FvjdC+zzlLs3uXtTTU1Nvp5aRETIrdCPAHVZt1dktl3INuALlxNKREQuXS6F3gw0mtkqMysDNgPbs3cws8asm58H9ucvooiI5GLKMXR3HzWzR4CXgWLge+6+x8yeAFrcfTvwiJn9MjAC9AIPzmRoERE5n7kHMyHFzLqBjmn+8SXAiTzGmWlhyhumrBCuvGHKCuHKG6ascHl569190pOQgRX65TCzFndvCjpHrsKUN0xZIVx5w5QVwpU3TFlh5vKGbi0XERGZnApdRCQiwlroTwUd4BKFKW+YskK48oYpK4Qrb5iywgzlDeUYuoiInC+sR+giIjKBCl1EJCJCV+hTrc1eKMyszsx+YmZ7zWyPmX0z6Ey5MLNiM3vbzP466CwXY2aLzOwlM/vAzPaZ2e1BZ7oYM/utzOvgfTN7wczKg86Uzcy+Z2bHzez9rG3VZvaqme3P/F4VZMZxF8j6XzKvhXfN7IdmtijIjOMmy5p137fMzM1sSb6eL1SFnuPa7IViFPiWu68DbgO+VsBZs30T2Bd0iBz8d+Bv3f0a4Bco4MxmVgt8A2hy9+tJX3G9OdhU59kK3DNh26PAj929Efhx5nYh2Mr5WV8Frnf3TwBtwGOzHeoCtnJ+VsysDvgMcCifTxaqQieHtdkLhbsfc/e3Ml8nSRdObbCpLs7MVpBei+e7QWe5GDNbCHwSeBrA3Yfd/VSwqaZUAlSYWQkwFzgacJ6PcfefAj0TNm8Cns18/SwFsujeZFnd/RV3H83cfIP0IoKBu8C/K8B3gH9Dnj87ImyFPtna7AVdkgBm1gDcBLwZbJIp/THpF1kq6CBTWAV0A89khoe+a2bzgg51Ie5+BPivpI/GjgF97v5KsKlysszdj2W+7gKWBRnmEnwF+JugQ1yImW0Cjrj7O/l+7LAVeuiY2Xzg/wD/asInOxUUM/sV4Li77wo6Sw5KgJuBP3X3m4ABCmc44DyZsedNpH8QXQnMM7P7gk11aTw9v7ng5zib2b8nPdz5/aCzTMbM5gL/Dnh8Jh4/bIV+qWuzB8rMSkmX+ffd/QdB55nCncBGM2snPZT1KTN7PthIF9QJdLr7+Duel0gXfKH6ZeBDd+929xHgB8AdAWfKRcLMrgDI/H484DwXZWYPAb8C/LoX7gU2q0n/YH8n8722AnjLzJbn48HDVuhTrs1eKMzMSI/x7nP3gv/QbHd/zN1XuHsD6X/Xv3P3gjyKdPcu4LCZrc1s+jSwN8BIUzkE3GZmczOvi09TwCdxs2zno6WwHwR+FGCWizKze0gPF2509zNB57kQd3/P3Ze6e0Pme60TuDnzmr5soSr0zEmP8bXZ9wEvuvueYFNd0J3A/aSPdHdnfn0u6FAR8nXg+2b2LnAj8B8DznNBmXcSLwFvAe+R/r4rqEvVzewF4HVgrZl1mtnDwB8Cd5vZftLvMv4wyIzjLpD1T4BK4NXM99qfBRoy4wJZZ+75CvediYiIXIpQHaGLiMiFqdBFRCJChS4iEhEqdBGRiFChi4hEhApdRCQiVOgiIhHx/wHO1VhtMoQ9YwAAAABJRU5ErkJggg==\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD7CAYAAAB68m/qAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAfbUlEQVR4nO3deXCU953n8fdXFwJdXBKgg8NYNgjbHJbtOKfXHhw8yYQc9g6e3JMU8e44m92a3QnZqczWbqp2k83s7sxsnCVU4nhmkwoVJ86YyRAb1jmcSRwPQsLYEmBjDJIQEgJ0g+7v/tGNaISEGmjp6X7686pS0U/3j9aHLvTR079+fs9j7o6IiKS+jKADiIhIYqjQRURCQoUuIhISKnQRkZBQoYuIhIQKXUQkJOIqdDPbZGZHzOyomW2b4PF5ZvYTMztoZv9sZrclPqqIiFzNlIVuZpnAE8BDQBXwqJlVjRv2H4ED7n4H8AngrxMdVEREri4rjjF3A0fd/RiAme0ENgMNMWOqgP8G4O6HzWy5mS1y97bJnnThwoW+fPny6w4uIpKO9u/ff8bdiyd6LJ5CLwOaYrabgXvGjXkF+DDwT2Z2N7AMKAcmLfTly5dTU1MTx7cXEZGLzOzEZI/FM4duE9w3/nwBXwXmmdkB4PNAHTA8QZCtZlZjZjXt7e1xfGsREYlXPHvozUBFzHY50BI7wN27gU8DmJkBb0W/GDduB7ADoLq6WieRERFJoHj20PcBlWa2wsxygC3ArtgBZjY3+hjAZ4EXoyUvIiIzZMo9dHcfNrPHgeeBTOBJd683s8eij28HVgN/Z2YjRD4s/cw0ZhYRkQnEM+WCu+8Gdo+7b3vM7ZeAysRGExGRa6GVoiIiIaFCFxEJibimXEQkuZwfHObQqR4aTnUzKzODTbcvpjA3O+hYEjAVukiSO9M7QH1LNw0t3dS3dNFwqpu3zvQRe/XILz/7GptuW8wjd1bw9pULyMiYaPmIhJ0KXSRJjI46jefO03AqWtwt3dS3dHO6Z2BsTPm82VQtKWTz2jKqSgupKi3kbO8AT9c08+yBkzx7oIWyubP5yIYyHr6zgqUL5gT4L5KZZkFdJLq6utq19F/S1cDwCG+09dLQ0j1W4IdO9dA7EFlgnZlhVJbkR0p7SSFrSouoWlJI0ZzJp1X6h0bY29DG0/ub+fUb7bjDPSvm80h1Bb9/+2Lm5Gj/LQzMbL+7V0/4mApdAHr6hzh0qodTXRdYWZxP5aJ8ZmVlBh0rFLr7hyLFHd3jbjjVzRttPQyPRn728nIyWb0ksre9prSQqiVFVC7KJzf7+l//U10XeKb2JE/XNHH87HnycjJ53x1LeKS6gupl84gs6JZUpEKXMe7O6Z6By97SN5zq5sTZ85eNy840bi4piO4dFo69vdcHb5Nzd1q7+y+9ri3d1J/qounchbExxQWzLntN15QWsWz+nGmb83Z3ak508HRNE/948BR9gyOsWJjHw3eW8+ENZSwpmj0t31emjwo9TY2MOm+d6btsTrahpZuzfYNjY5YtmBPdK4yUy5K5uRw93RvzIVw3Z3ovzeFWzJ/NmiVFl/YmSwtZXJibdnt8I6POsfbe6GvbPTZ1ci7mtV2xMC9myiTyWpUU5AaWuW9gmJ+91srTNU28/NY5MgzeWVnMI3eWs7Fq0Q29I5CZo0JPA/1DIxxp7YnucXdR39LN4VM9XBgaASJ73Lcsit3jLmL1kgIK4tjjPt3Tf6m0osX11pm+scfn5+WM2+ssZMXCfDJDcqTFhcERDrdeejdT39LNkdZu+odGAcjJzODWxdHXtixS4KuWFJI/K3nnrE+c7ePH+5v5ce1JTnZeoGh2Nh9YW8oj1eXcXlaUdr+gU4kKPYRqGzvYf7xj7DC2N9v7GInOyRbMymL12HxsZM/75pJ8crISt46sd2CYw6cun1p4vbWXwZFIyeVmZ7Bqcey8cCGrFhcyOye59wLP9Q1eMR11rL2X6EtLYW5WdK+7iDWlkQJfWZxPdmZqrtEbHXV+++ZZnt7fxHOvtTIwPMqtiwp4pLqcWxcXBB0vtMrnzWHFwrzr+rsq9BBxd775yzf5+vNHAFhcmHvZnnHVkiIq5s8OZA9raGSUo6d7x8rw4i+bnv7IkRsZBjcV51/2i6aqtJD5eTlTPHPiuTtN5y6MvZu5mLm1u39sTGlRbvSzg6KxdyDl84J5bWdC14UhfnqwhadrmjnQ1Bl0nFB77D0r2fbQquv6uyr0kHB3vvbcEbb/6k0+uK6UL7+/igX5s4KOdVXuTnPHhWhpdo1NWZzqulScS4pyr/igMJHFOTQyyhttvWO/YOpbujnU0k3PwKVfNCsv/qKJ/lIM6hdNsjh+pu+yz04ksRYV5lIx//rWCKjQQ2B01PlPu+r5v787wUfvWcpXNt+W0qsBz/UNRufjL01vvBkztVGQm0XVkksFX7WkkMpFU09t9PQPcbi1h/qTl8r7jbZLU0GzszNZtaTgsncIqxYX6ANBSRkq9BQ3PDLKn/3oIM/UneRz776JbQ+tCuXb/v6hEQ639ly2xP3Qqcs/fKxcdGnKpqq0iL6B4cuO4jkec/jlgrycscMtLxb4ioV5ofmwVtKTCj2FDQyP8IUfHOC5+lb+dOMtPH7/zaEs88lcPPTyYsFf3JuPPTwQYOn8yOGXsdM2JQWz0uq1kvRwtUJP3uOqhAuDI3zue/t58fV2vvz+Kj7zzhVBR5pxmRnGzSX53FySz+Z1ZUBkXr6te4BDp7qZk5PJai14EgFU6Emrp3+IzzxVw74T5/jaR27nD+9aGnSkpGFmLC7KZXFRcIt0RJKRCj0JdfQN8snv/jMNLd38zZb1/MHa0qAjiUgKUKEnmdPd/XzsOy9z/Ox5vvXxO3lg9aKgI4lIiohreZuZbTKzI2Z21My2TfB4kZn9g5m9Ymb1ZvbpxEcNv+aO8zzyrZdo7rjAU5++S2UuItdkykI3s0zgCeAhoAp41Myqxg37E6DB3dcC9wH/w8zSd1XGdTjW3ssj21+io2+Q7332Ht6+cmHQkUQkxcSzh343cNTdj7n7ILAT2DxujAMFFjlGLB84BwwnNGmINbR08y+/9RKDw6Ps3HovG5bOCzqSiKSgeAq9DGiK2W6O3hfrG8BqoAV4FfiCu48mJGHI1TZ2sGXHS2RnZvDDx+6lqrQw6EgikqLiKfSJVmaMX430XuAAUAqsA75hZlc0k5ltNbMaM6tpb2+/xqjh89s3z/Cxb7/MvLwcfvi5e1lZnB90JBFJYfEUejNQEbNdTmRPPNangWc84ijwFnDFqcTcfYe7V7t7dXFx8fVmDoWfH27j09/dR/m82Tz9uXuv+0Q9IiIXxVPo+4BKM1sR/aBzC7Br3JhG4AEAM1sE3AocS2TQMPnpwRa2/t1+bllUwM6t91JSqAUyInLjpjwO3d2Hzexx4HkgE3jS3evN7LHo49uBrwBPmdmrRKZovujuZ6Yxd8r64b4mtj1zkDuXzeM7n7pLS9ZFJGHiWljk7ruB3ePu2x5zuwV4MLHRwue7v3mL//wPDbyrciE7Pl6d9FfvEZHUopWiM8DdeeIXR/nLPa/z3jWL+JtH1zMrS2UuIomlQp9m7s5XnzvMt351jA+vL+O/P3wHWSl6/UkRSW4q9Gn21Z8d5lsvHuNjb1vKf/lAal9lSESSmwp9GvUODPPd3xznQ+vL+Mrm23SxBRGZVnrvP41+daSdwZFRttxVoTIXkWmnQp9GexpamZ+Xw53LdG4WEZl+KvRpMjQyys8Pn+b+VSX6EFREZoSaZpq8fOwcPf3DPFilc5qLyMxQoU+TvQ2t5GZn8K7K9D5njYjMHBX6NHB39ja08a7KYq0GFZEZo0KfBvUt3bR09bNR0y0iMoNU6NNgT30rGQYPrCoJOoqIpBEV+jTY09BG9bL5LMifFXQUEUkjKvQEazp3nsOtPTy4RtMtIjKzVOgJtqehDUDz5yIy41ToCbanvpVbFxWwbEFe0FFEJM2o0BOoo2+QfcfPae9cRAKhQk+gFw6fZtTR/LmIBEKFnkB7G1pZXJjL7WVFQUcRkTSkQk+Q/qERXnz9DBurFulUuSISiLgK3cw2mdkRMztqZtsmePw/mNmB6NdrZjZiZvMTHzd5/dMbZ7gwNKL5cxEJzJSFbmaZwBPAQ0AV8KiZVcWOcfevu/s6d18HfAn4lbufm4a8SWtPQysFs7J4200Lgo4iImkqnj30u4Gj7n7M3QeBncDmq4x/FPhBIsKlipFR54VDp7lvVQk5WZrFEpFgxNM+ZUBTzHZz9L4rmNkcYBPw4xuPljpqGzs42zeoc5+LSKDiKfSJPuHzScb+AfCbyaZbzGyrmdWYWU17e3u8GZPe3oY2sjON+27Vuc9FJDjxFHozUBGzXQ60TDJ2C1eZbnH3He5e7e7VxcXhKD93Z099K/euXEhBbnbQcUQkjcVT6PuASjNbYWY5REp71/hBZlYEvAd4NrERk9vR070cP3teR7eISOCyphrg7sNm9jjwPJAJPOnu9Wb2WPTx7dGhHwL2uHvftKVNQmMn41qtQheRYE1Z6ADuvhvYPe6+7eO2nwKeSlSwVLGnoY215UUsLsoNOoqIpDkdY3cD2rr7eaWpkwfXLA46ioiICv1G7NW5z0UkiajQb8CehjaWL5hDZUl+0FFERFTo16unf4iX3tTJuEQkeajQr9Mvj7QzNOKaPxeRpKFCv057G9pYkJfDhqXzgo4iIgKo0K/L4PAovzh8mgdWl5CZoekWEUkOKvTr8PJbZ+kZGObBKk23iEjyUKFfhz31bczOzuSdlQuDjiIiMkaFfo3cnb0Nbbz7loXkZmcGHUdEZIwK/Rq9erKL1u5+Nmq6RUSSjAr9Gu1taCPD4IFVJUFHERG5jAr9Gu2pb+Ou5fOZl5cTdBQRkcuo0K/BibN9HGnr0WIiEUlKKvRrcPFkXLp2qIgkIxX6NdjT0MaqxQVUzJ8TdBQRkSuo0ON0rm+QmuPntHcuIklLhR6nFw61Mepo/lxEkpYKPU57GtooLcplTWlh0FFERCakQo/DhcERfv1Gu859LiJJLa5CN7NNZnbEzI6a2bZJxtxnZgfMrN7MfpXYmMH69Rvt9A+NanWoiCS1rKkGmFkm8ASwEWgG9pnZLndviBkzF/gmsMndG80sVMso9za0UZCbxT03zQ86iojIpOLZQ78bOOrux9x9ENgJbB435o+AZ9y9EcDdTyc2ZnBGRp0XDp/m/lUlZGdqhkpEklc8DVUGNMVsN0fvi3ULMM/Mfmlm+83sExM9kZltNbMaM6tpb2+/vsQzbP+JDs71Derc5yKS9OIp9Ik+BfRx21nAncD7gPcCXzazW674S+473L3a3auLi4uvOWwQ9tS3kpOZwXtuTY28IpK+ppxDJ7JHXhGzXQ60TDDmjLv3AX1m9iKwFng9ISkD4u7sPdTG229eQP6seF4qEZHgxLOHvg+oNLMVZpYDbAF2jRvzLPAuM8sysznAPcChxEadea+39XLi7Hk2anWoiKSAKXc73X3YzB4HngcygSfdvd7MHos+vt3dD5nZc8BBYBT4tru/Np3BZ8LehlYANq5WoYtI8otrHsHddwO7x923fdz214GvJy5a8PY0tLGuYi4lhblBRxERmZKOw5vEqa4LHGzu4sE12jsXkdSgQp/E/9O5z0UkxajQJ7GnoY2bFuaxsjg/6CgiInFRoU+gu3+I3x07q5NxiUhKUaFP4JdH2hkacc2fi0hKUaFPYE99Kwvzc1hXMS/oKCIicVOhjzMwPMIvj7Tze6sXkZmh6RYRSR0q9HF+d+wcvQPDWh0qIilHhT7O3oZW5uRk8o6bFwYdRUTkmqjQY4yOOnsb2nh3ZTG52ZlBxxERuSYq9BivnuyirXtAR7eISEpSocfY09BKZoZx/6pQXUFPRNKECj3G3oY27l4+n7lzcoKOIiJyzVToUcfP9PF6W6+ObhGRlKVCj9obPRmXCl1EUpUKPeqlY2e5uSSfivlzgo4iInJdVOhErh1a19jBhqVzg44iInLdVOjA8bPn6Tg/xIalOneLiKQuFTpQ19gBwHoVuoikMBU6UNfYSf6sLG4u0cUsRCR1xVXoZrbJzI6Y2VEz2zbB4/eZWZeZHYh+/UXio06f2sYO1lXM1dkVRSSlZU01wMwygSeAjUAzsM/Mdrl7w7ihv3b3909Dxml1fnCYw609/Ov7VgYdRUTkhsSzh343cNTdj7n7ILAT2Dy9sWbOq81djIw663WEi4ikuHgKvQxoitlujt433r1m9oqZ/czM1kz0RGa21cxqzKymvb39OuImXm1jJwDrdXUiEUlx8RT6RBPLPm67Fljm7muB/w38/URP5O473L3a3auLi4uvKeh0qWvsYMXCPObl6fwtIpLa4in0ZqAiZrscaIkd4O7d7t4bvb0byDazpL9ChLtT19TJ+oq5QUcREblh8RT6PqDSzFaYWQ6wBdgVO8DMFpuZRW/fHX3es4kOm2jNHRdo7xlg/TJNt4hI6pvyKBd3Hzazx4HngUzgSXevN7PHoo9vBx4G/pWZDQMXgC3uPn5aJunUNXUCaA9dREJhykKHsWmU3ePu2x5z+xvANxIbbfrVNXaQm53BqsUFQUcREblhab1StLaxkzvK55KVmdYvg4iERNo2Wf/QCA0tXTohl4iERtoWen1LN0MjWlAkIuGRtoV+6QyLc4MNIiKSIGlc6J2Uz5tNSUFu0FFERBIijQu9Q+c/F5FQSctCb+3qp6WrX8efi0iopGWhX5w/36AVoiISIulZ6E2d5GRlULWkMOgoIiIJk56F3tjBbaWF5GSl5T9fREIq7RptcHiUg81aUCQi4ZN2hX64tZuB4VEd4SIioZN2hV538QpFWlAkIiGTdoVe29jB4sJcSufODjqKiEhCpV2h1zV2au9cREIprQr9TO8AjefOq9BFJJTSqtAvzZ/rA1ERCZ80K/QOsjKM28uKgo4iIpJwaVbonVSVFpKbnRl0FBGRhIur0M1sk5kdMbOjZrbtKuPuMrMRM3s4cRETY3hklFeaO3VCLhEJrSkL3cwygSeAh4Aq4FEzq5pk3NeA5xMdMhFeb+vl/OCITsglIqEVzx763cBRdz/m7oPATmDzBOM+D/wYOJ3AfAlT1xS9QlGFCl1EwimeQi8DmmK2m6P3jTGzMuBDwPbERUus2hOdLMjLoWK+FhSJSDjFU+g2wX0+bvuvgC+6+8hVn8hsq5nVmFlNe3t7nBETo64pcoUis4n+OSIiqS+eQm8GKmK2y4GWcWOqgZ1mdhx4GPimmX1w/BO5+w53r3b36uLi4utLfB06zw9yrL1PC4pEJNSy4hizD6g0sxXASWAL8EexA9x9xcXbZvYU8FN3//vExbwxB5o6AZ2QS0TCbcpCd/dhM3ucyNErmcCT7l5vZo9FH0/aefOLahs7yTBYWz436CgiItMmnj103H03sHvcfRMWubt/6sZjJVZdYwe3Li4kb1Zc/1wRkZQU+pWio6POgSadYVFEwi/0hf5mey89/cO65JyIhF7oC11XKBKRdBH+Qm/qoGh2NisW5AUdRURkWoW+0GtPdLKuYi4ZGVpQJCLhFupC7+kf4vXTPZo/F5G0EOpCP9jchbvmz0UkPYS60GtPRM6wuFbnQBeRNBDqQq9r6qSyJJ+i2dlBRxERmXahLXR3p66xQ9MtIpI2Qlvox8+ep+P8EOv1gaiIpInQFnpdY2T+XEe4iEi6CHGhd5I/K4ubS/KDjiIiMiPCW+hNHaytKCJTC4pEJE2EstDPDw5z6JQWFIlIegllob/a3MXIqOsIFxFJK6Es9LroJefWVWgPXUTSRygLvfZEB8sXzGF+Xk7QUUREZkzoCt3dqWvq1Py5iKSd0BX6yc4LtPcMaP5cRNJOXIVuZpvM7IiZHTWzbRM8vtnMDprZATOrMbN3Jj5qfGrHrlCkPXQRSS9ZUw0ws0zgCWAj0AzsM7Nd7t4QM+wFYJe7u5ndAfwQWDUdgadS19hBbnYGqxYXBPHtRUQCE88e+t3AUXc/5u6DwE5gc+wAd+91d49u5gFOQOoaO7mjfC5ZmaGbTRIRuap4Wq8MaIrZbo7edxkz+5CZHQb+EfjjxMS7Nv1DI9S3dGn+XETSUjyFPtHa+Sv2wN39J+6+Cvgg8JUJn8hsa3SOvaa9vf2agsajvqWboRHXES4ikpbiKfRmoCJmuxxomWywu78IrDSzhRM8tsPdq929uri4+JrDTuXiGRbX6wpFIpKG4in0fUClma0wsxxgC7ArdoCZ3WxmFr29AcgBziY67FTqGjspmzubksLcmf7WIiKBm/IoF3cfNrPHgeeBTOBJd683s8eij28HPgJ8wsyGgAvAH8Z8SDpj6ho7uHP5/Jn+tiIiSWHKQgdw993A7nH3bY+5/TXga4mNdm1au/pp6erns5puEZE0FZpj+8bmz3WEi4ikqfAUelMnOVkZrCktCjqKiEggwlPojR3cVlpITlZo/kkiItckFO03NDLKweYunb9FRNJaKAr90KluBoZHNX8uImktFIVeFz3DolaIikg6C0mhd7CocBZLirSgSETSVygKvbaxk/UV84guVhURSUspX+hnegdoPHeeDcvmBh1FRCRQKV/oB3SFIhERIASFXtvYQVaGcXuZFhSJSHpL+UKva+ykqrSQ3OzMoKOIiAQqpQt9ZNR5pblT5z8XESHFC/1Iaw/nB0c0fy4iQooXel1T5AyLWlAkIpLqhd7YyYK8HCrmzw46iohI4FK60GsbO1i/dK4WFImIkMKF3nl+kGPtfZo/FxGJStlCP9DUCegKRSIiF6Vsodc1dpJhcEf53KCjiIgkhbgK3cw2mdkRMztqZtsmePyjZnYw+vVbM1ub+KiXq23s4JZFBeTPius61yIioTdloZtZJvAE8BBQBTxqZlXjhr0FvMfd7wC+AuxIdNBYo6POgaZONizT/LmIyEXx7KHfDRx192PuPgjsBDbHDnD337p7R3Tzd0B5YmNe7tiZXnr6h7VCVEQkRjyFXgY0xWw3R++bzGeAn91IqKnUnugEdIZFEZFY8UxAT3SQt0840OxfECn0d07y+FZgK8DSpUvjjHiluqYOimZnc9PCvOt+DhGRsIlnD70ZqIjZLgdaxg8yszuAbwOb3f3sRE/k7jvcvdrdq4uLi68nLxA5wmVdxVwyMrSgSETkongKfR9QaWYrzCwH2ALsih1gZkuBZ4CPu/vriY95SU//EEfaenT8uYjIOFNOubj7sJk9DjwPZAJPunu9mT0WfXw78BfAAuCb0WX4w+5ePR2BDzZ34a4TcomIjBfXQdzuvhvYPe6+7TG3Pwt8NrHRJpaTlcH9q0pYqyNcREQuk3Krcu5aPp+7PjU/6BgiIkknZZf+i4jI5VToIiIhoUIXEQkJFbqISEio0EVEQkKFLiISEip0EZGQUKGLiISEuU944sTp/8Zm7cCJ6/zrC4EzCYwz3VIpbyplhdTKm0pZIbXyplJWuLG8y9x9wrMbBlboN8LMaqbrXDHTIZXyplJWSK28qZQVUitvKmWF6curKRcRkZBQoYuIhESqFvq0XoR6GqRS3lTKCqmVN5WyQmrlTaWsME15U3IOXURErpSqe+giIjJOyhW6mW0ysyNmdtTMtgWdZzJmVmFmvzCzQ2ZWb2ZfCDpTPMws08zqzOynQWe5GjOba2Y/MrPD0df43qAzXY2Z/bvo/4PXzOwHZpYbdKZYZvakmZ02s9di7ptvZnvN7I3on0lxmbBJsn49+n/hoJn9xMzmBhjxMhPljXns35uZm9nCRHyvlCp0M8sEngAeAqqAR82sKthUkxoG/tTdVwNvA/4kibPG+gJwKOgQcfhr4Dl3XwWsJYkzm1kZ8G+Aane/jcilHLcEm+oKTwGbxt23DXjB3SuBF6LbyeAprsy6F7jN3e8AXge+NNOhruIprsyLmVUAG4HGRH2jlCp04G7gqLsfc/dBYCewOeBME3L3U+5eG73dQ6RwyoJNdXVmVg68D/h20FmuxswKgXcD3wFw90F37ww01NSygNlmlgXMAVoCznMZd38RODfu7s3A30Zv/y3wwZnMNJmJsrr7Hncfjm7+Diif8WCTmOS1BfhfwJ8BCfsgM9UKvQxoitluJslLEsDMlgPrgZcDjjKVvyLyH2w04BxTuQloB74bnR76tpnlBR1qMu5+EvhLIntip4Aud98TbKq4LHL3UxDZQQFKAs4Trz8GfhZ0iKsxsw8AJ939lUQ+b6oVuk1wX1IfpmNm+cCPgX/r7t1B55mMmb0fOO3u+4POEocsYAPwf9x9PdBH8kwHXCE697wZWAGUAnlm9rFgU4WTmf05kenO7wedZTJmNgf4c+AvEv3cqVbozUBFzHY5SfbWNZaZZRMp8++7+zNB55nCO4APmNlxIlNZ95vZ94KNNKlmoNndL77j+RGRgk9Wvwe85e7t7j4EPAO8PeBM8WgzsyUA0T9PB5znqszsk8D7gY96ch+PvZLIL/dXoj9v5UCtmS2+0SdOtULfB1Sa2QozyyHywdKugDNNyMyMyBzvIXf/n0HnmYq7f8ndy919OZHX9efunpR7ke7eCjSZ2a3Rux4AGgKMNJVG4G1mNif6/+IBkvhD3Bi7gE9Gb38SeDbALFdlZpuALwIfcPfzQee5Gnd/1d1L3H159OetGdgQ/X99Q1Kq0KMfejwOPE/kB+KH7l4fbKpJvQP4OJE93QPRr98POlSIfB74vpkdBNYB/zXYOJOLvpP4EVALvErk5y6pVjaa2Q+Al4BbzazZzD4DfBXYaGZvEDka46tBZrxokqzfAAqAvdGfte2BhowxSd7p+V7J/c5ERETilVJ76CIiMjkVuohISKjQRURCQoUuIhISKnQRkZBQoYuIhIQKXUQkJFToIiIh8f8BXPL7vZXYSYQAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": { - "tags": [], "needs_background": "light" - } + }, + "output_type": "display_data" } + ], + "source": [ + "plt.plot(h.history['accuracy'])" ] }, { @@ -1272,13 +1271,14 @@ "id": "iJruFXmb_dur" }, "source": [ - "Выше мы использовали **функциональный** способ задания модели, когда мы сначала описываем входную переменную, затем - происходящие с ней преобразования, и потом определяем объект `Model`.\r\n", - "\r\n", - "Мы можем также задавать модель как последовательности слоёв с помощью `Sequential`:" + "### Sequential API\n", + "\n", + "Alternatively, we can start thinking of a model as of a **sequence of layers**, and just specify those layers by adding them to the `model` object:" ] }, { "cell_type": "code", + "execution_count": 28, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -1286,81 +1286,78 @@ "id": "iWc_kSr8_YXt", "outputId": "345dbe65-629d-468f-ed75-1d412c966340" }, - "source": [ - "model = tf.keras.models.Sequential()\r\n", - "model.add(tf.keras.layers.Dense(5,activation='sigmoid',input_shape=(2,)))\r\n", - "model.add(tf.keras.layers.Dense(1,activation='sigmoid'))\r\n", - "\r\n", - "test_x_norm = test_x-np.min(train_x) / (np.max(train_x)-np.min(train_x))\r\n", - "# это не ошибка, мы нормируем тестовые данные так же, как нормировали обучающие!\r\n", - "# (поэтому min и max считаем для обучающих данных)\r\n", - "\r\n", - "model.compile(tf.keras.optimizers.Adam(0.1),'binary_crossentropy',['accuracy'])\r\n", - "model.summary()\r\n", - "model.fit(train_x_norm,train_labels,validation_data=(test_x_norm,test_labels),batch_size=8,epochs=15)" - ], - "execution_count": 45, "outputs": [ { + "name": "stdout", "output_type": "stream", "text": [ - "Model: \"sequential_6\"\n", + "Model: \"sequential\"\n", "_________________________________________________________________\n", "Layer (type) Output Shape Param # \n", "=================================================================\n", - "dense_14 (Dense) (None, 5) 15 \n", + "dense_1 (Dense) (None, 5) 15 \n", "_________________________________________________________________\n", - "dense_15 (Dense) (None, 1) 6 \n", + "dense_2 (Dense) (None, 1) 6 \n", "=================================================================\n", "Total params: 21\n", "Trainable params: 21\n", "Non-trainable params: 0\n", "_________________________________________________________________\n", "Epoch 1/15\n", - "9/9 [==============================] - 1s 19ms/step - loss: 0.6962 - accuracy: 0.5596 - val_loss: 0.5533 - val_accuracy: 0.7333\n", + "9/9 [==============================] - 0s 26ms/step - loss: nan - accuracy: 0.4571 - val_loss: nan - val_accuracy: 0.4000\n", "Epoch 2/15\n", - "9/9 [==============================] - 0s 5ms/step - loss: 0.5057 - accuracy: 0.8035 - val_loss: 0.4924 - val_accuracy: 0.8000\n", + "9/9 [==============================] - 0s 4ms/step - loss: nan - accuracy: 0.5000 - val_loss: nan - val_accuracy: 0.4000\n", "Epoch 3/15\n", - "9/9 [==============================] - 0s 5ms/step - loss: 0.4173 - accuracy: 0.8726 - val_loss: 0.5273 - val_accuracy: 0.6667\n", + "9/9 [==============================] - 0s 4ms/step - loss: nan - accuracy: 0.5000 - val_loss: nan - val_accuracy: 0.4000\n", "Epoch 4/15\n", - "9/9 [==============================] - 0s 4ms/step - loss: 0.4405 - accuracy: 0.8145 - val_loss: 0.5641 - val_accuracy: 0.6667\n", + "9/9 [==============================] - 0s 4ms/step - loss: nan - accuracy: 0.5000 - val_loss: nan - val_accuracy: 0.4000\n", "Epoch 5/15\n", - "9/9 [==============================] - 0s 5ms/step - loss: 0.3523 - accuracy: 0.9039 - val_loss: 0.5498 - val_accuracy: 0.7333\n", + "9/9 [==============================] - 0s 4ms/step - loss: nan - accuracy: 0.5000 - val_loss: nan - val_accuracy: 0.4000\n", "Epoch 6/15\n", - "9/9 [==============================] - 0s 5ms/step - loss: 0.3239 - accuracy: 0.9012 - val_loss: 0.5666 - val_accuracy: 0.6667\n", + "9/9 [==============================] - 0s 4ms/step - loss: nan - accuracy: 0.5000 - val_loss: nan - val_accuracy: 0.4000\n", "Epoch 7/15\n", - "9/9 [==============================] - 0s 5ms/step - loss: 0.4043 - accuracy: 0.8568 - val_loss: 0.5883 - val_accuracy: 0.6667\n", + "9/9 [==============================] - 0s 4ms/step - loss: nan - accuracy: 0.5000 - val_loss: nan - val_accuracy: 0.4000\n", "Epoch 8/15\n", - "9/9 [==============================] - 0s 5ms/step - loss: 0.3351 - accuracy: 0.8903 - val_loss: 0.5906 - val_accuracy: 0.6667\n", + "9/9 [==============================] - 0s 4ms/step - loss: nan - accuracy: 0.5000 - val_loss: nan - val_accuracy: 0.4000\n", "Epoch 9/15\n", - "9/9 [==============================] - 0s 5ms/step - loss: 0.3624 - accuracy: 0.8761 - val_loss: 0.6198 - val_accuracy: 0.6667\n", + "9/9 [==============================] - 0s 4ms/step - loss: nan - accuracy: 0.5000 - val_loss: nan - val_accuracy: 0.4000\n", "Epoch 10/15\n", - "9/9 [==============================] - 0s 4ms/step - loss: 0.4015 - accuracy: 0.8622 - val_loss: 0.6087 - val_accuracy: 0.6667\n", + "9/9 [==============================] - 0s 4ms/step - loss: nan - accuracy: 0.5000 - val_loss: nan - val_accuracy: 0.4000\n", "Epoch 11/15\n", - "9/9 [==============================] - 0s 5ms/step - loss: 0.3387 - accuracy: 0.8846 - val_loss: 0.6604 - val_accuracy: 0.6667\n", + "9/9 [==============================] - 0s 4ms/step - loss: nan - accuracy: 0.5000 - val_loss: nan - val_accuracy: 0.4000\n", "Epoch 12/15\n", - "9/9 [==============================] - 0s 5ms/step - loss: 0.3579 - accuracy: 0.8694 - val_loss: 0.6177 - val_accuracy: 0.7333\n", + "9/9 [==============================] - 0s 4ms/step - loss: nan - accuracy: 0.5000 - val_loss: nan - val_accuracy: 0.4000\n", "Epoch 13/15\n", - "9/9 [==============================] - 0s 5ms/step - loss: 0.3304 - accuracy: 0.8830 - val_loss: 0.6274 - val_accuracy: 0.6667\n", + "9/9 [==============================] - 0s 4ms/step - loss: nan - accuracy: 0.5000 - val_loss: nan - val_accuracy: 0.4000\n", "Epoch 14/15\n", - "9/9 [==============================] - 0s 4ms/step - loss: 0.3193 - accuracy: 0.8807 - val_loss: 0.6472 - val_accuracy: 0.6667\n", + "9/9 [==============================] - 0s 4ms/step - loss: nan - accuracy: 0.5000 - val_loss: nan - val_accuracy: 0.4000\n", "Epoch 15/15\n", - "9/9 [==============================] - 0s 4ms/step - loss: 0.3552 - accuracy: 0.8546 - val_loss: 0.6596 - val_accuracy: 0.6667\n" - ], - "name": "stdout" + "9/9 [==============================] - 0s 3ms/step - loss: nan - accuracy: 0.5000 - val_loss: nan - val_accuracy: 0.4000\n" + ] }, { - "output_type": "execute_result", "data": { "text/plain": [ - "" + "" ] }, - "metadata": { - "tags": [] - }, - "execution_count": 45 + "execution_count": 28, + "metadata": {}, + "output_type": "execute_result" } + ], + "source": [ + "model = tf.keras.models.Sequential()\n", + "model.add(tf.keras.layers.Dense(5,activation='sigmoid',input_shape=(2,)))\n", + "model.add(tf.keras.layers.Dense(1,activation='sigmoid'))\n", + "\n", + "test_x_norm = test_x-np.min(train_x) / (np.max(train_x)-np.min(train_x))\n", + "# This is not an error: we use min/man on training data when normalizing our test data\n", + "# Because we want to use the same normalization during both training and testing\n", + "\n", + "model.compile(tf.keras.optimizers.Adam(0.1),'binary_crossentropy',['accuracy'])\n", + "model.summary()\n", + "model.fit(train_x_norm,train_labels,validation_data=(test_x_norm,test_labels),batch_size=8,epochs=15)" ] }, { @@ -1369,38 +1366,22 @@ "id": "BmHNhUU8bqEX" }, "source": [ - "## Функции ошибки при классификации\r\n", - "\r\n", - "При использовании Keras для классификации важно правильно указать функцию ошибки и передаточную функцию на последнем слое. Основные правила:\r\n", - "* Если у сети один выход, то передаточная функция - сигмоида, если несколько - softmax\r\n", - "* Если на ожидаемый выход подается в виде one-hot-encoding, то функция ошибки - cross entropy loss (categorical cross-entropy), если номер класса - sparse categorical cross-entropy, для бинарной классификации с одним выходом - binary cross-entropy (она же log loss)\r\n", - "\r\n", - "В целом бинарную классификацию можно рассматривать как частный случай мультиклассовой, подавая на выход one hot encoded вектор.\r\n", - "\r\n", - "| Классификация | Формат входных данных | Передат.функция | Функция ошибки |\r\n", - "|---------------|-----------------------|-----------------|----------|\r\n", - "| Бинарная | Вероятность 1-го класса | sigmoid | binary crossentropy |\r\n", - "| Бинарная | One-hot encoding (2 выхода) | softmax | categorical crossentropy |\r\n", - "| Мультикласс | One-hot encoding | softmax | categorical crossentropy |\r\n", - "| Мультикласс | Номер класса | softmax | sparse categorical crossentropy" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "yX6hqiafwHl9" - }, - "source": [ - "## Выводы\n", + "## Classification Loss Functions\n", "\n", - "* Tensorflow позволяет более гибко определять структуру графа вычислений, описывать свои функции и конфигурации.\n", - "* Есть более удобные средства для работы с данными (`td.Data`), со слоями (`tf.layers`)\n", - "* Для массового использования нейросетей Google рекомендует **Keras**, который позволяет собирать нейросети как конструктор\n", - "* При этом возможно реализовать свой слой для Keras, и потом использовать его в своих моделях.\n", - "* Для типовых задач имеет смысл использовать Keras\n", - "* Также стоит посмотреть на PyTorch, это \"восходящая звезда\"\n", + "It is important to correctly specify loss function and activation function on the last layer of the network. The main rules are the following:\n", + "* If the network has one output (**binary classification**), we use **sigmoid** activation function, for **multiclass classification** - **softmax**\n", + "* If the output class is represented as one-hot-encoding, the loss function will be **cross entropy loss** (categorical cross-entropy), if the output contains class number - **sparse categorical cross-entropy**. For **binary classification** - use **binary cross-entropy** (same as **log loss**)\n", + "* **Multi-label classification** is when we can have an object belonging to several classes at the same time. In this case, we need to encode labels using one-hot encoding, and use **sigmoid** as activation function, so that each class probability is between 0 and 1.\n", "\n", - "Хороший Notebook про Keras и Tensorflow 2.0 от создателя Keras - [тут](https://t.co/k694J95PI8)" + "| Classification | Label Format | Activation Function | Loss |\n", + "|---------------|-----------------------|-----------------|----------|\n", + "| Binary | Probability of 1st class | sigmoid | binary crossentropy |\n", + "| Binary | One-hot encoding (2 outputs) | softmax | categorical crossentropy |\n", + "| Multiclass | One-hot encoding | softmax | categorical crossentropy |\n", + "| Multiclass | Class Number | softmax | sparse categorical crossentropy |\n", + "| Multilabel | One-hot encoding | sigmoid | categorical crossentropy |\n", + "\n", + "> Binary classification can also be handled as a special case of multi-class classification with two outputs. In this case, we need to use **softmax**.\n" ] }, { @@ -1409,25 +1390,67 @@ "id": "gZ-kWx84bMDH" }, "source": [ - "**Задание 3**: \r\n", - "Используйте Keras для обучения классификатора на сети MNIST. При этом:\r\n", - "* Обратите внимание, что в keras заложены типовые датасеты, включая MNIST. Для обращения к нему достаточно пары строчек кода (см, например, [тут](https://www.tensorflow.org/api_docs/python/tf/keras/datasets/mnist))\r\n", - "* Попробуйте несколько конфигураций сети с несколькими полносвязными слоями, передаточными функциями, и разным количеством нейронов\r\n", - "\r\n", - "Какой точности вам удалось достичь?\r\n" + "**Task 3**: \n", + "Use Keras to train MNIST classifier:\n", + "* Notice that Keras contains some standard datasets, including MNIST. To use MNIST from Keras, you only need a couple of lines of code (more information [here](https://www.tensorflow.org/api_docs/python/tf/keras/datasets/mnist))\n", + "* Try several network configuration, with different number of layers/neurons, activation functions.\n", + "\n", + "What is the best accuracy you were able to achieve?" ] }, { - "cell_type": "code", + "cell_type": "markdown", "metadata": { - "trusted": true, - "id": "MG64NKzawHl-" + "id": "yX6hqiafwHl9" }, "source": [ - "" - ], - "execution_count": null, - "outputs": [] + "## Takeaways\n", + "\n", + "* Tensorflow allows you to operate on tensors at low level, you have most flexibility.\n", + "* There are convenient tools to work with data (`td.Data`) and layers (`tf.layers`)\n", + "* For beginners/typical tasks, it is recommended to use **Keras**, which allows to construct networks from layers\n", + "* If non-standard architecture is needed, you can implement your own Keras layer, and then use it in Keras models\n", + "* It is a good idea to look at PyTorch as well and compare approaches. \n", + "\n", + "A good sample notebook from the creator of Keras on Keras and Tensorflow 2.0 can be found [here](https://t.co/k694J95PI8)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [] } - ] -} \ No newline at end of file + ], + "metadata": { + "celltoolbar": "Slideshow", + "colab": { + "collapsed_sections": [], + "name": "IntroKerasTF.ipynb", + "provenance": [] + }, + "interpreter": { + "hash": "0cb620c6d4b9f7a635928804c26cf22403d89d98d79684e4529119355ee6d5a5" + }, + "kernelspec": { + "display_name": "Python 3.8.12 64-bit (conda)", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.12" + }, + "livereveal": { + "start_slideshow_at": "selected" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/3-NeuralNetworks/05-Frameworks/IntroPyTorch.ipynb b/3-NeuralNetworks/05-Frameworks/IntroPyTorch.ipynb index b37041ac..9ffddec2 100644 --- a/3-NeuralNetworks/05-Frameworks/IntroPyTorch.ipynb +++ b/3-NeuralNetworks/05-Frameworks/IntroPyTorch.ipynb @@ -1,35 +1,4 @@ { - "nbformat": 4, - "nbformat_minor": 0, - "metadata": { - "celltoolbar": "Slideshow", - "kernelspec": { - "name": "python3", - "display_name": "Python 3", - "language": "python" - }, - "language_info": { - "mimetype": "text/x-python", - "nbconvert_exporter": "python", - "name": "python", - "file_extension": ".py", - "version": "3.7.4-final", - "pygments_lexer": "ipython3", - "codemirror_mode": { - "version": 3, - "name": "ipython" - } - }, - "livereveal": { - "start_slideshow_at": "selected" - }, - "colab": { - "name": "IntroPyTorch.ipynb", - "provenance": [], - "collapsed_sections": [] - }, - "accelerator": "GPU" - }, "cells": [ { "cell_type": "markdown", @@ -37,37 +6,25 @@ "id": "En2vX4FuwHlu" }, "source": [ - "# Введение в нейронные сети\n", + "## Introduction to PyTorch\n", "\n", - "## Эпизод 2b: Многослойный персептрон на PyTorch\n", + "> This notebook is a part of [AI for Beginners Curricula](http://github.com/microsoft/ai-for-beginners). Visit the repository for complete set of learning materials.\n", "\n", - "Дмитрий Сошников | dmitri@soshnikov.com\n", + "### Neural Frameworks\n", "\n", - "http://github.com/shwars/NeuroWorkshop\n", - "-> Notebooks -> IntroPyTorch.ipynb" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "fVjlmWBwwHl2" - }, - "source": [ - "## Нейросетевые фреймворки\n", + "We have learnt that to train neural networks you need:\n", + "* Quickly multiply matrices (tensors)\n", + "* Compute gradients to perform gradient descent optimization\n", "\n", - "Мы видели, что для обучения нейросетей нужно:\n", - "* Быстро умножать матрицы (тензоры)\n", - "* Считать производные для вычисления градиента для метода обратного распространения ошибки\n", + "What neural network frameworks allow you to do:\n", + "* Operate with tensors on whatever compute is available, CPU or GPU, or even TPU\n", + "* Automatically compute gradients (they are explicitly programmed for all built-in tensor functions)\n", "\n", - "Что позволяют делать нейросетевые фреймворки:\n", - "* Оперировать с тензорами, как на CPU, так и на GPU\n", - "* Автоматически вычислять производные (они вручную прописаны для всех элементарных функций)\n", - "\n", - "Опционально:\n", - "* Конструктор для нейросетей (описание сети как набора слоёв)\n", - "* Простые функции для обучения (`fit`, как в Scikit Learn)\n", - "* Набор алгоритмов оптимизации\n", - "* Набор абстракций для работы с данными" + "Optionally:\n", + "* Neural Network constructor / higher level API (describe network as a sequence of layers)\n", + "* Simple training functions (`fit`, as in Scikit Learn)\n", + "* A number of optimization algorithms in addition to gradient descent\n", + "* Data handling abstractions (that will ideally work on GPU, too)" ] }, { @@ -76,48 +33,50 @@ "id": "8cACQoFMwHl3" }, "source": [ - "## Основные фреймворки\n", + "### Most Popular Frameworks\n", "\n", - "* Tensorflow 1.0 - первый, получивший широкое распространение (Google). Позволял определять статический computation graph, и затем в явном виде выполнять вычисления\n", - "* PyTorch - Facebook\n", - "* Keras - надстройка над Tensorflow/PyTorch для унификации (Francois Chollet)\n", - "* Tensorflow 2.0 + Keras - динамический вычислительный граф, код получается похожим на обычные вычисления в numpy\n", + "* Tensorflow 1.x - first widely available framework (Google). Allowed to define static computation graph, push it to GPU, and explicitly evaluate it\n", + "* PyTorch - a framework from Facebook that is growing in popularity\n", + "* Keras - higher level API on top of Tensorflow/PyTorch to unify and simplify using neural networks (Francois Chollet)\n", + "* Tensorflow 2.x + Keras - new version of Tensorflow with integrated Keras functionality, which supports **dynamic computation graph**, allowing to perform tensor operations very similar to numpy (and PyTorch)\n", "\n", - "Мы рассмотрим PyTorch. Для начала рекомендуется установить PyTorch [по инструкции на сайте](https://pytorch.org/get-started/locally/). Либо можно выполнять код в [Google Colab](https://colab.research.google.com/), в котором PyTorch уже установлен." + "In this Notebook, we will learn to use PyTorch. You need to make sure that you have recent version of PyTorch installed - to do it, follow the [instructions on their site](https://pytorch.org/get-started/locally/). It is normally as simple as doing\n", + "```\n", + "pip install torch torchvision\n", + "```\n", + "or\n", + "```\n", + "conda install pytorch -c pytorch\n", + "```" ] }, { "cell_type": "code", + "execution_count": 1, "metadata": { - "tags": [], "colab": { "base_uri": "https://localhost:8080/", "height": 35 }, "id": "xwqVx9-bwHl3", - "outputId": "38564a63-0567-4406-ee1a-1d3618f27351" + "outputId": "38564a63-0567-4406-ee1a-1d3618f27351", + "tags": [] }, + "outputs": [ + { + "data": { + "text/plain": [ + "'1.8.2'" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "import torch\n", "torch.__version__" - ], - "execution_count": 168, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "application/vnd.google.colaboratory.intrinsic+json": { - "type": "string" - }, - "text/plain": [ - "'1.8.0+cu101'" - ] - }, - "metadata": { - "tags": [] - }, - "execution_count": 168 - } ] }, { @@ -126,14 +85,14 @@ "id": "6tp2xGV7wHl4" }, "source": [ - "## Основные понятия в PyTorch\n", + "## Basic Concepts: Tensor\n", "\n", - "**Тензор** - это многомерный массив произвольной размерности. Удобно использовать при обучении нейросетей, например:\n", - "* 400x400 - чёрно-белая картинка\n", - "* 400x400x3 - цветная картинка\n", - "* 16x400x400x3 - minibatch из 16 картинок, используемый для одного шага обучения\n", - "* 25x400x400x3 - секунда видео\n", - "* 8x25x400x400x3 - minibatch из 8 1-секундных видео" + "**Tensor** is a multi-dimensional array. It is very convenient to use tensors to represent different types of data:\n", + "* 400x400 - black-and-white picture\n", + "* 400x400x3 - color picture \n", + "* 16x400x400x3 - minibatch of 16 color pictures\n", + "* 25x400x400x3 - one second of 25-fps video\n", + "* 8x25x400x400x3 - minibatch of 8 1-second videos" ] }, { @@ -142,45 +101,47 @@ "id": "qG2bsaR7wHl4" }, "source": [ - "### Простые тензоры" + "### Simple Tensors\n", + "\n", + "You can easily create simple tensors from lists of np-arrays, or generate random ones:" ] }, { "cell_type": "code", + "execution_count": 2, "metadata": { - "trusted": true, "colab": { "base_uri": "https://localhost:8080/" }, "id": "ybpnk08HwHl4", - "outputId": "54e2c89b-b373-4389-b285-49b0510be931" + "outputId": "54e2c89b-b373-4389-b285-49b0510be931", + "trusted": true }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "tensor([[1, 2],\n", + " [3, 4]])\n", + "tensor([[-0.7488, 0.5776, 0.0520],\n", + " [-0.5258, -1.6104, 0.4226],\n", + " [ 2.0568, 0.4692, 0.5858],\n", + " [-1.2225, -1.6223, -0.3628],\n", + " [-2.7874, 1.0577, 0.9720],\n", + " [-1.6521, 1.4820, -1.1416],\n", + " [-1.0172, -0.3259, -1.4338],\n", + " [-0.7763, -1.1454, -0.5500],\n", + " [-2.1639, -0.2575, -0.8015],\n", + " [-0.1890, -0.7774, -0.2899]])\n" + ] + } + ], "source": [ "a = torch.tensor([[1,2],[3,4]])\n", "print(a)\n", "a = torch.randn(size=(10,3))\n", "print(a)" - ], - "execution_count": 169, - "outputs": [ - { - "output_type": "stream", - "text": [ - "tensor([[1, 2],\n", - " [3, 4]])\n", - "tensor([[ 0.1288, -0.8088, -0.2092],\n", - " [ 0.0789, -1.4706, -0.8880],\n", - " [-0.0601, 0.6659, -3.2120],\n", - " [ 0.8023, -0.4045, -1.0164],\n", - " [-1.5578, -1.3014, -0.8372],\n", - " [ 1.3626, -1.0825, -2.7018],\n", - " [ 0.2258, -0.7675, -0.4102],\n", - " [-1.9663, 0.0386, -2.6183],\n", - " [ 0.7666, -1.3057, -0.4486],\n", - " [-0.9006, 0.7092, 0.8018]])\n" - ], - "name": "stdout" - } ] }, { @@ -189,11 +150,12 @@ "id": "AXFMsV3r09Ux" }, "source": [ - "С тензорами можно производить обычные вычисления, которые производятся поэлементно (как в numpy). При этом тензоры автоматически дополняются до нужной размерности. Можно извлечь numpy-массив из тензора при помощи `.numpy()`:" + "You can use arithmetic operations on tensors, which are performed element-wise, as in numpy. Tensors are automatically expanded to required dimension, if needed. To extract numpy-array from tensor, use `.numpy()`:" ] }, { "cell_type": "code", + "execution_count": 3, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -201,29 +163,28 @@ "id": "e5Nu5Xgj1DnQ", "outputId": "c1fbcd86-dde6-40b6-8edf-7a37f9d60901" }, - "source": [ - "print(a-a[0])\n", - "print(torch.exp(a)[0].numpy())" - ], - "execution_count": 170, "outputs": [ { + "name": "stdout", "output_type": "stream", "text": [ "tensor([[ 0.0000, 0.0000, 0.0000],\n", - " [-0.0499, -0.6618, -0.6788],\n", - " [-0.1889, 1.4747, -3.0028],\n", - " [ 0.6734, 0.4044, -0.8072],\n", - " [-1.6867, -0.4926, -0.6280],\n", - " [ 1.2338, -0.2737, -2.4927],\n", - " [ 0.0970, 0.0413, -0.2010],\n", - " [-2.0951, 0.8475, -2.4091],\n", - " [ 0.6377, -0.4969, -0.2395],\n", - " [-1.0295, 1.5180, 1.0110]])\n", - "[1.1375165 0.44538715 0.81125516]\n" - ], - "name": "stdout" + " [ 0.2230, -2.1880, 0.3706],\n", + " [ 2.8056, -0.1084, 0.5339],\n", + " [-0.4737, -2.1999, -0.4148],\n", + " [-2.0386, 0.4801, 0.9200],\n", + " [-0.9033, 0.9044, -1.1936],\n", + " [-0.2685, -0.9035, -1.4857],\n", + " [-0.0275, -1.7230, -0.6019],\n", + " [-1.4151, -0.8351, -0.8535],\n", + " [ 0.5598, -1.3550, -0.3418]])\n", + "[0.47294793 1.7817812 1.0533323 ]\n" + ] } + ], + "source": [ + "print(a-a[0])\n", + "print(torch.exp(a)[0].numpy())" ] }, { @@ -232,13 +193,14 @@ "id": "uQ5zN6cVyrG7" }, "source": [ - "## In-place и out-of-place операции\n", + "## In-place and out-of-place Operations\n", "\n", - "Обычно операции с тензорами возвращают новые тензоры. Однако для большинства операций сущетвуют аналогичные варианты, которые модифицируют исходный тензор:" + "Tensor operations such as `+`/`add` return new tensors. However, sometimes you need to modify the existing tensor in-place. Most of the operations have their in-place counterparts, which end with `_`:" ] }, { "cell_type": "code", + "execution_count": 171, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -246,22 +208,21 @@ "id": "Mjkbcw3-ACKS", "outputId": "ca021008-9ab6-4b09-c5a5-bbe854cd1493" }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Result when adding out-of-place: tensor(8)\n", + "Result after adding in-place: tensor(8)\n" + ] + } + ], "source": [ "u = torch.tensor(5)\n", "print(\"Result when adding out-of-place:\",u.add(torch.tensor(3)))\n", "u.add_(torch.tensor(3))\n", "print(\"Result after adding in-place:\", u)" - ], - "execution_count": 171, - "outputs": [ - { - "output_type": "stream", - "text": [ - "Result when adding out-of-place: tensor(8)\n", - "Result after adding in-place: tensor(8)\n" - ], - "name": "stdout" - } ] }, { @@ -270,11 +231,12 @@ "id": "DLPUcVsXACKT" }, "source": [ - "Например, вот там можно \"наивно\" посчитать сумму строк тензора `a`:" + "This is how we can compute the sum or all rows in a matrix in a naive way:" ] }, { "cell_type": "code", + "execution_count": 4, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -282,22 +244,21 @@ "id": "7pu0UZ-_yqfB", "outputId": "bd2e8c6a-39e1-4f29-990b-9591e866936c" }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "tensor([-9.0260, -2.1524, -2.5472])\n" + ] + } + ], "source": [ "s = torch.zeros_like(a[0])\n", "for i in a:\n", " s.add_(i)\n", "\n", "print(s)" - ], - "execution_count": 172, - "outputs": [ - { - "output_type": "stream", - "text": [ - "tensor([ -1.1197, -5.7273, -11.5398])\n" - ], - "name": "stdout" - } ] }, { @@ -306,11 +267,12 @@ "id": "rIh1EHcezlNo" }, "source": [ - "Умный способ:" + "But it is much better to use" ] }, { "cell_type": "code", + "execution_count": 5, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -318,23 +280,20 @@ "id": "aQIdWZ1kzn6P", "outputId": "89000bb4-f45e-493b-a7b0-39fa4e7d92c1" }, - "source": [ - "torch.sum(a,axis=0)" - ], - "execution_count": 176, "outputs": [ { - "output_type": "execute_result", "data": { "text/plain": [ - "tensor([ -1.1197, -5.7273, -11.5398])" + "tensor([-9.0260, -2.1524, -2.5472])" ] }, - "metadata": { - "tags": [] - }, - "execution_count": 176 + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" } + ], + "source": [ + "torch.sum(a,axis=0)" ] }, { @@ -343,7 +302,7 @@ "id": "5UzUmEZhACKT" }, "source": [ - "Подробнее про тензоры в PyTorch смотрите [в официальном руководстве](https://pytorch.org/tutorials/beginner/basics/tensorqs_tutorial.html)" + "You can read more on PyTorch tensors in the [official documentation](https://pytorch.org/tutorials/beginner/basics/tensorqs_tutorial.html)" ] }, { @@ -352,21 +311,32 @@ "id": "U-auwezDwHl6" }, "source": [ - "## Вычисляем производные\n", + "## Computing Gradients\n", "\n", - "Для обратного распространения ошибки, нам нужно уметь вычислять градиенты. Мы можем пометить любой тензор в PyTorch атрибутом `requires_grad`, и впоследствии автоматически будут вычисляться все градиенты при операциях с этим тензором. Для вычисления производной необходимо вызвать метод `backward()`:\n" + "For back propagation, you need to compute gradients. We can set any PyTorch Tensor's attribute `requires_grad` to `True`, which will result in all operations with this tensor being tracked for gradient calculations. To compute the gradients, you need to call `backward()` method, after which the gradient will become available using `grad` attribute:\n" ] }, { "cell_type": "code", + "execution_count": 6, "metadata": { - "trusted": true, "colab": { "base_uri": "https://localhost:8080/" }, "id": "m8vFOXr7wHl6", - "outputId": "7054c2b1-0b61-4938-937d-813f75f0b195" + "outputId": "7054c2b1-0b61-4938-937d-813f75f0b195", + "trusted": true }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "tensor([[-0.2004, -0.0751],\n", + " [-0.2429, 0.1520]])\n" + ] + } + ], "source": [ "a = torch.randn(size=(2, 2), requires_grad=True)\n", "b = torch.randn(size=(2, 2))\n", @@ -375,17 +345,6 @@ "c.backward() # call backward() to compute all gradients\n", "# What's the gradient of `c` with respect to `a`?\n", "print(a.grad)" - ], - "execution_count": 179, - "outputs": [ - { - "output_type": "stream", - "text": [ - "tensor([[0.0227, 0.1661],\n", - " [0.1980, 0.2253]])\n" - ], - "name": "stdout" - } ] }, { @@ -394,11 +353,12 @@ "id": "nPj3rtrtACKU" }, "source": [ - "На самом деле, PyTorch может таким образом вычислять \"накапливаемые\" градиенты. Если при вызове `backward` указать `retain_graph=True`, то граф вычислений будет сохраняться, и градиенты - накапливаться. Чтобы начать их вычислять заново, нужно в явном виде обнулить поле `grad`: " + "To be more precise, PyTorch automatically **accumulates** gradients. If you specify `retain_graph=True` when calling `backward`, computational graph will be preserved, and new gradient is added to the `grad` field. In order to restart computing gradients from scratch, we need to reset `grad` field to 0 explicitly by calling `zero_()`: " ] }, { "cell_type": "code", + "execution_count": 8, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -406,6 +366,18 @@ "id": "z_VIw8MoACKU", "outputId": "36a28b11-6919-47ab-c3f9-c7f1d8500423" }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "tensor([[-0.6012, -0.2252],\n", + " [-0.7286, 0.4559]])\n", + "tensor([[-0.2004, -0.0751],\n", + " [-0.2429, 0.1520]])\n" + ] + } + ], "source": [ "c = torch.mean(torch.sqrt(torch.square(a) + torch.square(b)))\n", "c.backward(retain_graph=True)\n", @@ -414,19 +386,6 @@ "a.grad.zero_()\n", "c.backward()\n", "print(a.grad)" - ], - "execution_count": 180, - "outputs": [ - { - "output_type": "stream", - "text": [ - "tensor([[0.0681, 0.4983],\n", - " [0.5939, 0.6758]])\n", - "tensor([[0.0227, 0.1661],\n", - " [0.1980, 0.2253]])\n" - ], - "name": "stdout" - } ] }, { @@ -435,11 +394,12 @@ "id": "HM9sUkVgCiG9" }, "source": [ - "Для вычисления градиентов PyTorch создаёт и поддерживает **граф вычислений**. Для каждого тензора, который вычисляется с использованием тензоров с установленным флагом `requires_grad`, устанавливается специальная функция `grad_fn`, представляющая собой функцию для вычисления производной по правилу дифференциирования сложной функции:" + "To compute gradients, PyTorch creates and maintains **compute graph**. For each tensor that has the `requires_grad` flag set to `True`, PyTorch maintains a special function called `grad_fn`, which computes the derivative of the expression according to chain differentiation rule:" ] }, { "cell_type": "code", + "execution_count": 9, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -447,18 +407,17 @@ "id": "PcxHb-7jC7Vv", "outputId": "3b3fa138-6d09-4636-8a71-f4a4051c7827" }, - "source": [ - "print(c)" - ], - "execution_count": 181, "outputs": [ { + "name": "stdout", "output_type": "stream", "text": [ - "tensor(1.0358, grad_fn=)\n" - ], - "name": "stdout" + "tensor(1.3326, grad_fn=)\n" + ] } + ], + "source": [ + "print(c)" ] }, { @@ -467,11 +426,13 @@ "id": "rvLfNiblACKV" }, "source": [ - "На самом деле, PyTorch может вычислять градиенты только для скалярных функций. Если речь идет о вычислении производной тензора по тензору, то PyTorch позволяет нам вычислять произведение якобиана на вектор.\n", + "Here `c` is computed using `mean` function, thus `grad_fn` point to a function called `MeanBackward`.\n", "\n", - "Например, пусть есть векторная функция $\\vec{y}=f(\\vec{x})$, где\n", - "$\\vec{x}=\\langle x_1,\\dots,x_n\\rangle$ и\n", - "$\\vec{y}=\\langle y_1,\\dots,y_m\\rangle$, тогда градиент $\\vec{y}$ по $\\vec{x}$ задаётся **якобианом**:\n", + "In most of the cases, we want PyTorch to compute gradient of a scalar function (such as loss function). However, if we want to compute the gradient of a tensor with respect to another tensor, PyTorch allows us to compute the product of a Jacobian matrix and a given vector.\n", + "\n", + "Suppose we have a vector function $\\vec{y}=f(\\vec{x})$, where\n", + "$\\vec{x}=\\langle x_1,\\dots,x_n\\rangle$ and\n", + "$\\vec{y}=\\langle y_1,\\dots,y_m\\rangle$, then a gradient of $\\vec{y}$ with respect to $\\vec{x}$ is defined by a **Jacobian**:\n", "\n", "$$\n", "\\begin{align}J=\\left(\\begin{array}{ccc}\n", @@ -481,12 +442,13 @@ "\\end{array}\\right)\\end{align}\n", "$$\n", "\n", - "Вместо вычисления якобиана, PyTorch вычисляет произведение $v^T\\cdot J$ на некоторый вектор\n", - "$v=(v_1 \\dots v_m)$. Для этого необходимо вызвать ``backward``, передав `v` в качестве аргумента. Размер `v` должен совпадать с размером исходного тензора, по которому мы вычисляем производную.\n" + "Instead of giving us access to the whole Jacobian, PyTorch computes the product $v^T\\cdot J$ of Jacobian with some vector\n", + "$v=(v_1 \\dots v_m)$. In order to do that, we need to call ``backward`` and pass `v` as an argument. The size of `v` should be the same as the size of the original tensor, with respect to which we compute the gradient.\n" ] }, { "cell_type": "code", + "execution_count": 10, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -494,21 +456,20 @@ "id": "VUNYiQCOACKV", "outputId": "e3127c21-fce6-420d-f347-ec40cc827e7e" }, - "source": [ - "c = torch.sqrt(torch.square(a) + torch.square(b))\n", - "c.backward(torch.eye(2))\n", - "print(a.grad)" - ], - "execution_count": 182, "outputs": [ { + "name": "stdout", "output_type": "stream", "text": [ - "tensor([[0.1135, 0.1661],\n", - " [0.1980, 1.1263]])\n" - ], - "name": "stdout" + "tensor([[-1.0020, -0.0751],\n", + " [-0.2429, 0.7599]])\n" + ] } + ], + "source": [ + "c = torch.sqrt(torch.square(a) + torch.square(b))\n", + "c.backward(torch.eye(2)) # eye(2) means 2x2 identity matrix\n", + "print(a.grad)" ] }, { @@ -517,7 +478,7 @@ "id": "dGHlkVlvACKV" }, "source": [ - "Подробнее про вычисление градиентов в PyTorch читайте [в официальном руководстве](https://pytorch.org/tutorials/beginner/basics/autogradqs_tutorial.html)" + "More on computing Jacobians in PyTorch can be found in [official documentation](https://pytorch.org/tutorials/beginner/basics/autogradqs_tutorial.html)" ] }, { @@ -526,29 +487,29 @@ "id": "FnVvj4LkD15r" }, "source": [ - "# Пример 0: Оптимизация функции методом градиентного спуска\r\n", - "\r\n", - "Попробуем использовать автоматическое дифференциирования для оптимизации простой функции двух переменных $f(x_1,x_2)=(x_1-3)^2+(x_2+2)^2$. Пусть тензор `x` представляет собой текущие координато точки. Мы начнем с некоторой начальной точки $x^{(0)}=(0,0)$, и будет вычислять следующую точку по формуле:\r\n", - "$$\r\n", - "x^{(n+1)} = x^{(n)} - \\eta\\nabla f\r\n", - "$$\r\n", - "Здесь $\\eta$ - т.н. **learning rage** (назоём его `lr` в коде), а $\\nabla f = (\\frac{\\partial f}{\\partial x_1},\\frac{\\partial f}{\\partial x_2})$ - градиент функции $f$.\r\n", - "\r\n", - "Для начала определим стартовое значение `x` и функцию `f`:" + "# Example 0: Optimization Using Gradient Descent\n", + "\n", + "Let's try to use automatic differentiation to find a minimum of a simple two-variable function $f(x_1,x_2)=(x_1-3)^2+(x_2+2)^2$. Let tensor `x` hold the current coordinates of a point. We start with some starting point $x^{(0)}=(0,0)$, and compute the next point in a sequence using gradient descent formula:\n", + "$$\n", + "x^{(n+1)} = x^{(n)} - \\eta\\nabla f\n", + "$$\n", + "Here $\\eta$ is so-called **learning rage** (we will denote it by `lr` in the code), and $\\nabla f = (\\frac{\\partial f}{\\partial x_1},\\frac{\\partial f}{\\partial x_2})$ - gradient of $f$.\n", + "\n", + "To begin, let's define starting value of `x` and the function `f`:" ] }, { "cell_type": "code", + "execution_count": 11, "metadata": { "id": "nDw5mV9KEeOa" }, + "outputs": [], "source": [ - "x = torch.zeros(2,requires_grad=True)\r\n", - "f = lambda x : (x-torch.tensor([3,-2])).pow(2).sum()\r\n", + "x = torch.zeros(2,requires_grad=True)\n", + "f = lambda x : (x-torch.tensor([3,-2])).pow(2).sum()\n", "lr = 0.1" - ], - "execution_count": 184, - "outputs": [] + ] }, { "cell_type": "markdown", @@ -556,11 +517,12 @@ "id": "Wt815LWdEj77" }, "source": [ - "Теперь проделаем 15 итераций градиентного спуска. На каждой итерации, мы будем обновлять координаты `x` и печатать их, чтобы убедиться, что мы достигаем минимума (3,-2):" + "Now let's do 15 iterations of gradient descent. In each iteration, we will update `x` coordinates and print them, to make sure that we are approaching the minimum point at (3,-2):" ] }, { "cell_type": "code", + "execution_count": 12, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -568,38 +530,37 @@ "id": "KfwMf555EyWJ", "outputId": "67e2199c-61ff-4ad1-9c48-b4a646bf8bbd" }, - "source": [ - "for i in range(15):\r\n", - " y = f(x)\r\n", - " y.backward()\r\n", - " gr = x.grad\r\n", - " x.data.add_(-lr*gr)\r\n", - " x.grad.zero_()\r\n", - " print(\"Step {}: x[0]={}, x[1]={}\".format(i,x[0],x[1]))" - ], - "execution_count": 186, "outputs": [ { + "name": "stdout", "output_type": "stream", "text": [ - "Step 0: x[0]=1.2000000476837158, x[1]=-0.800000011920929\n", - "Step 1: x[0]=1.5600000619888306, x[1]=-1.0399999618530273\n", - "Step 2: x[0]=1.8480000495910645, x[1]=-1.2319999933242798\n", - "Step 3: x[0]=2.078400135040283, x[1]=-1.385599970817566\n", - "Step 4: x[0]=2.2627201080322266, x[1]=-1.5084799528121948\n", - "Step 5: x[0]=2.4101760387420654, x[1]=-1.6067839860916138\n", - "Step 6: x[0]=2.5281407833099365, x[1]=-1.685427188873291\n", - "Step 7: x[0]=2.6225125789642334, x[1]=-1.7483417987823486\n", - "Step 8: x[0]=2.698009967803955, x[1]=-1.798673391342163\n", - "Step 9: x[0]=2.7584080696105957, x[1]=-1.8389387130737305\n", - "Step 10: x[0]=2.8067264556884766, x[1]=-1.8711509704589844\n", - "Step 11: x[0]=2.845381259918213, x[1]=-1.8969208002090454\n", - "Step 12: x[0]=2.876305103302002, x[1]=-1.9175366163253784\n", - "Step 13: x[0]=2.9010441303253174, x[1]=-1.9340293407440186\n", - "Step 14: x[0]=2.920835256576538, x[1]=-1.947223424911499\n" - ], - "name": "stdout" + "Step 0: x[0]=0.6000000238418579, x[1]=-0.4000000059604645\n", + "Step 1: x[0]=1.0800000429153442, x[1]=-0.7200000286102295\n", + "Step 2: x[0]=1.4639999866485596, x[1]=-0.9760000705718994\n", + "Step 3: x[0]=1.7711999416351318, x[1]=-1.1808000802993774\n", + "Step 4: x[0]=2.0169599056243896, x[1]=-1.3446400165557861\n", + "Step 5: x[0]=2.2135679721832275, x[1]=-1.4757120609283447\n", + "Step 6: x[0]=2.370854377746582, x[1]=-1.5805696249008179\n", + "Step 7: x[0]=2.4966835975646973, x[1]=-1.6644556522369385\n", + "Step 8: x[0]=2.597346782684326, x[1]=-1.7315645217895508\n", + "Step 9: x[0]=2.677877426147461, x[1]=-1.7852516174316406\n", + "Step 10: x[0]=2.7423019409179688, x[1]=-1.8282012939453125\n", + "Step 11: x[0]=2.793841600418091, x[1]=-1.8625609874725342\n", + "Step 12: x[0]=2.835073232650757, x[1]=-1.8900487422943115\n", + "Step 13: x[0]=2.868058681488037, x[1]=-1.912039041519165\n", + "Step 14: x[0]=2.894446849822998, x[1]=-1.929631233215332\n" + ] } + ], + "source": [ + "for i in range(15):\n", + " y = f(x)\n", + " y.backward()\n", + " gr = x.grad\n", + " x.data.add_(-lr*gr)\n", + " x.grad.zero_()\n", + " print(\"Step {}: x[0]={}, x[1]={}\".format(i,x[0],x[1]))" ] }, { @@ -608,29 +569,30 @@ "id": "8sfjBMBu59B5" }, "source": [ - "## Пример 1: Линейная регрессия\r\n", - "\r\n", - "Попробуем с помощью полученных знаний решить классическую задачу линейной регрессии. Для этого сгенерируем небольшой синтетический датасет:" + "## Example 1: Linear Regression\n", + "\n", + "Now we know enough to solve the classical problem of **Linear regression**. Let's generate small synthetic dataset:" ] }, { "cell_type": "code", + "execution_count": 13, "metadata": { - "trusted": true, - "id": "j723455WwHl7" + "id": "j723455WwHl7", + "trusted": true }, + "outputs": [], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "from sklearn.datasets import make_classification, make_regression\n", "from sklearn.model_selection import train_test_split\n", "import random" - ], - "execution_count": 187, - "outputs": [] + ] }, { "cell_type": "code", + "execution_count": 14, "metadata": { "colab": { "base_uri": "https://localhost:8080/", @@ -639,41 +601,37 @@ "id": "WJNK_J6v6I-Z", "outputId": "09e6386e-a6d4-4b81-c8d2-153f0acf9696" }, - "source": [ - "np.random.seed(13) # pick the seed for reproducability - change it to explore the effects of random variations\n", - "\n", - "train_x = np.linspace(0, 3, 120)\n", - "train_labels = 2 * train_x + 0.9 + np.random.randn(*train_x.shape) * 0.5\n", - "\n", - "plt.scatter(train_x,train_labels)" - ], - "execution_count": 188, "outputs": [ { - "output_type": "execute_result", "data": { "text/plain": [ - "" + "" ] }, - "metadata": { - "tags": [] - }, - "execution_count": 188 + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" }, { - "output_type": "display_data", "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAb2ElEQVR4nO3df4xlZXkH8O+zwwUuP8pVmVoYXZemzZIKZZedEJo1RtZYbFHcgFEb0xbbZpM2bYVQ2tE/ivYfJjFWbW1qJpQWW6pLXdhQUJFkN6FuFDvDLgKyNChuZbBllB0QZsrODE//uPcsd86c97zvuec957zvvd9PsnF25+yZ9+yV5773eZ/3eUVVQURE4drU9ACIiCgfAzURUeAYqImIAsdATUQUOAZqIqLAnVLFTc8991zdsmVLFbcmIhpKc3NzP1HV8azvVRKot2zZgtnZ2SpuTUQ0lETkmOl7TH0QEQWOgZqIKHAM1EREgWOgJiIKHAM1EVHgKqn6ICIaJfsPz+NT9z+JZxeXcX6njZuu3Ird2ye83Z+BmoiohP2H5/Gxux7F8soaAGB+cRkfu+tRAPAWrJn6ICIq4VP3P3kySCeWV9bwqfuf9PYzGKiJiEp4dnG50J8PgoGaiKiE8zvtQn8+CAZqIiJ0c807pw/ggqn7sHP6APYfnnf6ezdduRXt1ti6P2u3xnDTlVu9jY2LiUQ08sosCCbfZ9UHEVGF8hYEXQLu7u0TXgNzGlMfRDTy6lgQLMMaqEVkq4gc6fv1oohcX8fgiIjqUMeCYBnWQK2qT6rqNlXdBmAHgCUAd1c+MiKimtSxIFhG0Rz1OwF8X1WNDa6JiGJTZEGwf7v4Oe0WRIDFpZVKFhEToqruF4vcBuBhVf18xvf2ANgDAJs3b95x7BhjORENl3R1SFq7NYZbrrl4oGAtInOqOpn1PefFRBE5FcDVAP4t6/uqOqOqk6o6OT6eeewXEVHw8uqps6pD+vneOp4okvr4DXRn0//rfRRERAGw1VO7VIFUUSlSpDzvtwB8yfsIiIgCYWuw5FIFUkWliFOgFpEzAbwLwF3eR0BEFAhbPXVWdUi/qipFnFIfqvoygDd4/+lERAE5v9PGfEawTmbJ6eqQuqo+uIWciKjnpiu3bqjqSM+Sq94unoWBmoiop44GS4NgoCYi6tPEjNmGTZmIiALHQE1EFDgGaiKiwDFHTUQjob+ZUiiLhK4YqIloKOQF4kGP2kruOb+4jDERrKliooEgz0BNRIWEODPNC8QAcOOdj2At1SnUdtRW+p7J3y9ynqIvhdqcupqcnNTZ2Vnv9yWiZmW1+SzT2tOXndMHMncUdtotvLL6am7Hu4lOO/NNx3TP/r93aGpX+cH3eGlzSkRka1rUFFOPjsXlldwgLejOkBWvzZSTtqa2Lnh1nqfIQE1EzkI9BHaQjnUCIJ1PKNIpr87zFBmoiciZKTid024Zm+3XwXTm4evOaGVePyayIUgnXDrl1X2eIgM1ETnLCl6tTYKXT6waUwhVSJ/CAgC3XHMxJjptCLr541uuuRg3v/etmQH80x+4BBOWk8d3b584eU+gG9zRd29WfRBRkLKaFi2dWMXxpZV119kqKsowVXjccs3FxsU9U5VKiJ3ysrDqg4hKuWDqvsw0ggB4evoq7z/PVI0xSBVGSKWGeVUfnFETUSm2Zvu++VzQDGXGbMMcNRGVYlrIq2qxzfQGUGcVRt0YqImolP5Ft/6FvKpmqnW/MYSAqQ8iKq3OFEKop7BUySlQi0gHwK0ALkK3Rvz3VPVbVQ6MiMgkltyyL64z6s8B+Lqqvl9ETgVwRoVjIiLKFFKVRp2sgVpEzgHwdgDXAYCqngBwotphERGtN2ir0mHgsph4AYAFAP8oIodF5FYROTN9kYjsEZFZEZldWFjwPlAialZ6N2Dd28RDbQhVB5dAfQqASwH8vapuB/AygKn0Rao6o6qTqjo5Pj7ueZhE1KRkNlvnNvH+n53XcrTphlB1cMlRPwPgGVV9qPf7ryAjUBPR8DLNZj9xz+O5p6qUzSdn9b9OG+b66YQ1UKvq/4jIj0Rkq6o+CeCdAL5X/dCIKBR5/Z4Xl7t9PtKnqvjIJ2e9QfQb9vrphGvVx58AuKNX8fEDAB+pbkhEFBrTNvG0/pyxKZ9cJFDnpTWaOLuwKU6BWlWPAMhsFkJEw++mK7daUxCJvOBaNJ9seoPwfQxW6LiFnIissraJm5ryn99pO/XjcKkiGcXt4lm4hZyInKR3A5oOuk2CaN73XGqik8XI5ZU1jIlgTXWk0h39GKiJRliZygyXnhum7+XVRO/ePrEhkK+pngz0oxakAR4cQDSyTDPiOo6ZMh02AHTTKqaFy6zc9LBsK887OIA5aqIR1eROP1MOW4Dc6pL0YmSTG3HqxEBNNKJ8npRSVNYioQDGWXYiHeBHZVs5AzXRkDNVV5hmtQpU3ssjXUXSabesQTqr2qPJN5s6cTGRaIjlVVfk1Ub77kxnyiP3LxzmMVV71H1eY1MYqImGUBIYs4JYkhpIFuVs15UN1LZSvLxt4rbFzaw3m3ZrDFdcOI6d0weiX2BMMPVB5EHTLUDTY0kW2EyS1MDu7RM4NLULYrmuDFseOe9n2CpQsjbiXLtjAvvm5odqgZEzaqKSQmtob2tkBGxMDfhMIaTTHLb2pHnbxF3+/dIbcXZOH/DSZyQknFETldRE5UHeDN42C85alPO1VTurXM40W0/eBHxvE3dZYAzpE5ALzqiJSqq78sA2g8+bxY6JrHsTSWaYvk72znrTUmSX3i2dWMX+w/PeTxW3fToI7ROQCwZqopLqrjywbb/OWmBrbRJAgJW1brjMCk4+TvY2vTkpuiV4Se9qADi+tLJuDL6CpGmBMZmh2/79QsTUB1FJdXd4s83gsxbYzjr9lJNBOmFKz5RJC5jenCY6bZx52sZ5YRUpoqzn71+UjLH2mjNqopJ8f3S3cZnBp2eoF0zdl3kv05bsQdMCebPZG/YecRqDD3kz9BhrrzmjJvIgKXN7evoqHJraVelH6EFm8C79oYHyC6N5s1nXMVQtxh7XnFETRWaQGbwtb5vwkRYwzWZdx1C1uj8B+cBATRShootvrsGpyrRASAHS5+JlHdiPmihQTfRZbrJH9ajL60fNGTVRgJqq9Q1p1kuvcQrUIvJDAD8DsAZg1RT1iciPJmt9Y0sLjIIiM+orVPUnlY2EiE6KsdaXqsPUB1EA0vnozhktHF9a2XBdyLW+VB3XOmoF8A0RmRORPVkXiMgeEZkVkdmFhQV/IyQaclmNjF76v1W0xta3Mwq91peq4zqjfpuqzovIzwN4QESOquqD/Reo6gyAGaBb9eF5nERDIz17XjqxuiEfvfKqotNu4czTTuGiHrkFalWd7/3vcyJyN4DLADyY/7eIKC2rmsPkheUVHLn51+saGgXMmvoQkTNF5OzkawC/DuCxqgdGNIxcmvonmI+mhMuM+o0A7haR5Pp/VdWvVzoqoiHlWrXBfDT1swZqVf0BgEtqGAvR0DNt0WY+mvKwPI+i1MT2ah9MjYk+cfVbjePvf9Zz2i2IAItLK1E9N5XDQE3RqXN7te0NoegbRtEt2uln7T8hJeu5Y30Do3xsykTR2Tl9wHhq9aGpXd5+jq1BUR0NjEzP2i95bjZUilteUyYeHEDRqWt7ta2Jfh2nj7s8U3JNE6ehUz0YqCk6dZ0UYntDqOMNw+WZkmvYH2R4MVBTdOo6Ssn2hlDHG0bWs/brf+5Qjroi/xioKTq2U6Z9sb0h1PGGkX7WTruF153RynzuKsZT5kRy8oeLiUQ5fFd9ND3eovfi4mR98hYTGaiJIlfVm0Vd1TXUxaO4iCJRNOhWWVPOxclwMEdNlNJUXjarL/XH7no09+dXWZLHxclwMFBTVKoOooMES9f72sY9SNCtctZbV3UN2TFQUzSqCqL9qpihuo57kKBb5ay3ruoasmOOmqJRx8ncVcxQXcdt6qyXF3RNTZ58zXp5InkYGKgpGnXtBCwaLG1M45tfXMbO6QMnFw6vuHAc++bmCwXdok2eKE4M1BSNKoJoWhUzVNO4Ba8dxTW/uIx9c/O4dscEDh5dKBR0OesdfgzUFI2qP+YD+TPUQeuVs8YtANI7GJZX1nDw6EItNcqhbdShfNzwQlFpKsBk7dIDgNed0cLN781u+p/X8N/UulQAPD19VRWPsG5c3HEYHm54oaFh+5hfVSA3HUp7fGklc4NJVsP/dmsMn/ngNuzePmHc9VdHjXIdi7LkF8vzyIsQmvdUWb6Xt2CZVb5nK/NrskaZOw7j4xyoRWRMRA6LyL1VDojiU0d9swtTcLzxzkdKj8U2000HOVswbLJGmTsO41Mk9fFRAE8A+LmKxkKRCuWjtCk4rqmW7n+RtSDYrz/I7T88j00iWMtY/+m/rqlqjToWZckvpxm1iLwJwFUAbq12OBSjUD5K580Iy+4uTGbAnXZrw/f6g1zy6SIrSIcSDLnjMD6uM+rPAvhzAGebLhCRPQD2AMDmzZvLj4yiUUd9swvbrLfsG0cyA85bsDQtOo6JBBUMWXsdF2ugFpH3AHhOVedE5B2m61R1BsAM0C3P8zZCCl4oH6WTwHPjnY9Y0w5lf44pyJneDF5VZWCkgbmkPnYCuFpEfgjgywB2ici/VDoqikpIH6V3b5/Apz9wSWMVFVyooyoU2vDSm1H/maq+J+86bnihpoW0MYabScgFN7zQyGkqB8smSVQFbiGnxjTZb4K9Lig0nFFTcHye9efrnMHZY88X7lxHVAfOqKkRpl4XQHcx0jVIFskJJwE9ryFS/38NzC1TnfJm1Oz1QY3Iq2kusgXd5eis/Yfnse2T38D1e48YgzSQ3XbUxyGxRGUxUFMjbOVqrkHStisymXEvLq8UH2TO/YnqxEBNznx2yMvqHpfmEiRtdcumnYJpUvD+RHVioCYnvjvk9W+SMXEJkrZ2oS7BfqLTxocv39zYJhkiG1Z9kJMqOuT1984YdAu6rW457zSV9GLh5Ftez5I9ChIDNTmpskNe2U0ieZtbTI2aso7QYqMiChUD9RCoY/NG1R3yqgqS3ClIw4CBOnI+N47kCaVD3iA4U6bYcTExci51xD6E1CGPaNRwRh25Ok9XqWNm2p/GOafdgkj3pO+x3tFWRXYtEg0LBurIhXK6io1LHj2dxunfpJIcBFBVaocoZEx9RM5WRxwC1xps180p3NpNo4aBOnIx5I5NefTr9x5Zt8OxSLqGW7tplDD1MQRCr2pwacAE5G9OSQsttUNUJQZq8i6dj+6c0cLxJXNTpCSVYTtFPBFaaoeoagzU5FVWXXdrk6A1JlhZM/c+f3ZxecPmFFZ9EHUxUJNXWfnolVcVnXYLZ552ijG1kaQy6k7j8EguigEXE8krUz76heUVHJrahc9+cFswVSq+OwISVcUaqEXkdBH5jog8IiKPi8gn6xgYvcZnH+iq2fpDh1SlUteuTqKyXFIfrwDYpaoviUgLwDdF5Guq+u2Kx0aor5eHLy49QUKpUqlzVydRGdZArd3Tb1/q/bbV++X/RFzK5LMPdB352Ji61cWyq5PI6RRyERkDMAfglwD8nar+RcY1ewDsAYDNmzfvOHbsmOehjqYLpu7LfFcUAE9PX+V8n6zm/Mmp21mVFKOwyFbkBHOiquWdQu5U9aGqawC2iUgHwN0icpGqPpa6ZgbADABMTk5yxu2Jr1lf1sw8eZHS6RRbuiWrcdLi0kp0AT2m2T+NtkLleaq6KCIHAbwbwGO264dVnbNNX32gbXnX/nSKbZHN1Dgp9Px5llDy5UR5XKo+xnszaYhIG8C7ABytemChqruky1eVhMsMPAnmeYtstsZJrJog8s9lRn0egNt7eepNAO5U1XurHVa4qjjk1cbHrM9le3YSzE3plk0iTr04WDVB5JdL1cd3AWyvYSxRiLGkK0nVLK+sndyKnSwkJvrTKaagnvX3siQBfxQWJInqwJ2JBdk2dISmP1UDdINtuzWGD1++2ZhOSdItYyIb7qfoVouYJAGfu/6I/GGvj4JCOuTVZcZqStUcPLqAQ1O7jPfevX0CN+w9kvm9pKQvr+pj5/SB2lNERMOKgbqgUEq6XHcslknVmHLVE512bpAv+3OJaD0G6gGEUNLluqhZpg67zKcH7voj8oc56opV1VDJdcZa5kzFMqWBMZzlSBQLzqgrVGVDJdcZa9lUzaCfHkJJERENA6deH0VNTk7q7Oys9/vGZuf0gcxgOiaCT3/gklJBi30qiIZL6V4fNBhTemJNtfTMmjNWotHBQF2hvFO1fZSq+V7U5AYVojBxMbFCWQtq/UIqVeMGFaJwMVBXKG+HHxBWqRqPpSIKF1MfFUtSB3XvZiyaxuAGFaJwMVDXwGXhz2d+eJCyQG5QIQoXA3VN8hb+fNdbD9KKNaQeJkS0HnPUAfCZH95/eN5YaZKXxvB1QAER+ccZdQB85YeTmbmJLY0RQg8TItqIM+oA+OpxnXdMFtMYRPHijLokH4uApvzwFReOY+f0gdKVGwCYxiCKGAN1Cb4WAbOqQq64cBz75ua9VG5MdNoM0kQRG9pAXXamW+b0FNet4Xk/Y5ATUli5QTScrIFaRN4M4IsA3ojuKUwzqvq5qgdWRtmZbh2np9h+xiD3HqRRE/t7EIXPZUa9CuBGVX1YRM4GMCciD6jq9yoe28DKznTrOD3F9jMGvXeRyo0q+2UTkT/Wqg9V/bGqPtz7+mcAngAQ9H/FZcvd6jg9xfYz6jghhf09iOJQqDxPRLYA2A7goYzv7RGRWRGZXVhY8DO6AZUtd3P9+2U2idh+Rh0bUNjfgygOzouJInIWgH0ArlfVF9PfV9UZADNA94QXbyN0kM6zpismgGKz0SKLcoNuEnH5GVVvQGF/D6I4OM2oRaSFbpC+Q1XvqnZIxWT1Ud43N49rd0wMPBsdZDZb9BDbELZs8wBaojhYz0wUEQFwO4DnVfV6l5vWeWai6VzCiU4bh6Z2lb6/S1VEzOcXsuqDKAx5Zya6BOq3AfgPAI8CeLX3xx9X1a+a/k6VgTodWEwNiATA09NXFb5ff6DKCsCCbo3iRN+1Vb9ZENHwK3W4rap+E9341LiscrIkcKa55Flt5WlZVRHJz+q/lotyRFSlqJoymQJn+l3ENc9qK0+zBdrkWl9NlYiIskQVqE2BM0lFCIBOu4XTW5tww94j1kU920zYJdA+u7jMRTkiqlRUgdoUOJNc8Gc+uA2vrL6K40srTidp22bCtlPEk2tDqOAgouEVVVMmW+1x0a3jtvv1987Iyoenr2VgJqIqRBWobU2Hii7quTQx6g/AIZSyhTAGIqqXtTxvEHXWUfcb9jK5mOu1iShfXnleVDlqm1gW9YruYkywiRLRaIoq9WEzSD/mupVpLcp6baLRFHWgNuVrBw3MvvK/efcp0yubTZSIRlO0gdp303tf9yt7cktekOdRW0SjKdocte98ra/72e6TV7ud1Qmwvw6c9dpEoym4GbVr+sF3vtbX/VxObjHNil3SIqzXJho9QQVqU9pg9tjzOHh0YV3w9p2v9XU/233yFjxv2Hsk855cLCQabUEFatOM8o5v//eGrnXX7pgodYpLmq/8b5mTW7hYSERZgspR5zVd6re8soaDRxe85mt95X/L3CeWOnAiqldQOxNNOwuzuB4MEBtuEScaTaUODqhTVtqgzMEAMeJiIRGlBZX6yEobfPjyzUwHENFIC2pGDWTPKCff8nqmA4hoZAUXqLMwHUBEo8waqEXkNgDvAfCcql5U/ZCqw4U6IoqRy4z6nwB8HsAXqx3KRj4Dq+/eIEREdbEuJqrqgwCer2Es69j6XhTFXs5EFKugqj76+Q6s7OVMRLHyFqhFZI+IzIrI7MLCQun7+Q6sthPHiYhC5S1Qq+qMqk6q6uT4+Hjp+/kOrNyeTUSxCjb14TOwJouSyytrGBMBAHTaLZze2oQb9h4pdG4hEVHdrIFaRL4E4FsAtorIMyLy+9UPy1+TpP5FSQBYU0Vrk+DlE6s4vrTiZaGSiKhKQTVlqkKRRk8TnTYOTe2qeERERBvlNWUKNvXhS5HFR1aAEFGIhj5QF1l8ZAUIEYVo6AN11qJka5OgNSbr/owVIEQUqiiaMpVhOqMw68+4lZyIQjT0i4lERDGI4oQXdrYjIsoWRKBmZzsiIrMgFhPZ2Y6IyCyIQM3OdkREZkEEana2IyIyCyJQs7MdEZFZEIuJplpnLiQSEQUSqAGeNE5EZBJE6oOIiMwYqImIAsdATUQUOAZqIqLAMVATEQWuku55IrIA4NiAf/1cAD/xOJwmDcuzDMtzAHyWEA3LcwDlnuUtqjqe9Y1KAnUZIjJravUXm2F5lmF5DoDPEqJheQ6gumdh6oOIKHAM1EREgQsxUM80PQCPhuVZhuU5AD5LiIblOYCKniW4HDUREa0X4oyaiIj6MFATEQWusUAtIu8WkSdF5CkRmcr4/mkisrf3/YdEZEv9o7RzeI7rRGRBRI70fv1BE+O0EZHbROQ5EXnM8H0Rkb/pPed3ReTSusfoyuFZ3iEiL/S9Jn9Z9xhdicibReSgiHxPRB4XkY9mXBP8a+P4HFG8LiJyuoh8R0Qe6T3LJzOu8Ru/VLX2XwDGAHwfwC8COBXAIwB+JXXNHwH4Qu/rDwHY28RYPTzHdQA+3/RYHZ7l7QAuBfCY4fu/CeBrAATA5QAeanrMJZ7lHQDubXqcjs9yHoBLe1+fDeC/Mv4/Fvxr4/gcUbwuvX/ns3pftwA8BODy1DVe41dTM+rLADylqj9Q1RMAvgzgfalr3gfg9t7XXwHwThGRGsfowuU5oqCqDwJ4PueS9wH4onZ9G0BHRM6rZ3TFODxLNFT1x6r6cO/rnwF4AkC6cXvwr43jc0Sh9+/8Uu+3rd6vdFWG1/jVVKCeAPCjvt8/g40v2slrVHUVwAsA3lDL6Ny5PAcAXNv7SPoVEXlzPUPzzvVZY/FrvY+uXxORtzY9GBe9j8/b0Z3B9Yvqtcl5DiCS10VExkTkCIDnADygqsbXxEf84mJi9f4dwBZV/VUAD+C1d1lqzsPo9lW4BMDfAtjf8HisROQsAPsAXK+qLzY9nkFZniOa10VV11R1G4A3AbhMRC6q8uc1FajnAfTPLN/U+7PMa0TkFADnAPhpLaNzZ30OVf2pqr7S++2tAHbUNDbfXF6zKKjqi8lHV1X9KoCWiJzb8LCMRKSFbnC7Q1XvyrgkitfG9hyxvS4AoKqLAA4CeHfqW17jV1OB+j8B/LKIXCAip6KbbL8ndc09AH639/X7ARzQXmY+INbnSOUKr0Y3NxejewD8Tq/C4HIAL6jqj5se1CBE5BeSfKGIXIbufwehTQIAdCs6APwDgCdU9a8NlwX/2rg8Ryyvi4iMi0in93UbwLsAHE1d5jV+NXK4raquisgfA7gf3cqJ21T1cRH5KwCzqnoPui/qP4vIU+guDH2oibHmcXyOPxWRqwGsovsc1zU24Bwi8iV0V93PFZFnANyM7iIJVPULAL6KbnXBUwCWAHykmZHaOTzL+wH8oYisAlgG8KEAJwGJnQB+G8CjvZwoAHwcwGYgqtfG5TlieV3OA3C7iIyh+2Zyp6reW2X84hZyIqLAcTGRiChwDNRERIFjoCYiChwDNRFR4BioiYgCx0BNRBQ4BmoiosD9P3fFq1WlCzEwAAAAAElFTkSuQmCC\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAbVklEQVR4nO3df4ylVXkH8O+zw0XuAjJaphVGx8XULKkQdtmJoZnGyGrFij82YNTGtKltsmmTNkIo7egfRf9iEmPVxqbNxNJqSnVpFzYGVDRZDHWj2Bl2EZClUXErs7aMZQfEmbqzw9M/7n2XO++85z3nve857z3n3u8n2bC795075903PPfMc57zHFFVEBFRvLYNegBERFSOgZqIKHIM1EREkWOgJiKKHAM1EVHkzgnxphdffLHu2LEjxFsTEQ2lxcXFn6nqRNFrQQL1jh07sLCwEOKtiYiGkoicML3G1AcRUeQYqImIIsdATUQUOQZqIqLIMVATEUUuSNUHEdEoOXR0CZ+4/0mcXFnDpeNt3HrdTuzbPent/RmoiYhqOHR0CR+5+1GsrW8AAJZW1vCRux8FAG/BmqkPIqIaPnH/k2eDdGZtfQOfuP9Jb9+DgZqIqIaTK2uV/r4fDNRERDVcOt6u9Pf9YKAmIkIn1zwzdxiXzd6HmbnDOHR0yenrbr1uJ9qtsU1/126N4dbrdnobGxcTiWjk1VkQzF5n1QcRUUBlC4IuAXff7kmvgTmPqQ8iGnlNLAjWYQ3UIrJTRI71/HpeRG5qYGxERI1oYkGwDmugVtUnVXWXqu4CsAfAKoB7Qg+MiKgpTSwI1lE1R/0WAD9UVWODayKi1FRZEOzdLn5RuwURYGV1PcgiYkZU1f1ikTsAPKyqny14bT+A/QAwNTW158QJxnIiGi756pC8dmsMt99wZV/BWkQWVXW66DXnxUQRORfAuwH8a9HrqjqvqtOqOj0xUXjsFxFR9MrqqYuqQ3r53jqeqZL6+B10ZtP/430UREQRsNVTu1SBhKgUqVKe97sAvuh9BEREkbA1WHKpAglRKeIUqEVkO4DfBnC39xEQEUXCVk9dVB3SK1SliFPqQ1VXAfyK9+9ORBSRS8fbWCoI1tksOV8d0lTVB7eQExF13Xrdzi1VHflZcujt4kUYqImIupposNQPBmoioh6DmDHbsCkTEVHkGKiJiCLHQE1EFDnmqIloJPQ2U4plkdAVAzURDYWyQNzvUVvZey6trGFMBBuqmBxAkGegJqJKYpyZlgViALjlrkewkesUajtqK/+e2ddXOU/Rl0ptTl1NT0/rwsKC9/closEqavNZp7WnLzNzhwt3FI63W/jlmRdLO95NjrcLP3RM79n7dUdm99YffJeXNqdERLamRYNi6tGxsrZeGqQFnRmy4qWZctbW1NYFr8nzFBmoichZrIfA9tOxTgDk8wlVOuU1eZ4iAzUROTMFp4vaLWOz/SaYzjx8xfZW4fVjIluCdMalU17T5ykyUBORs6Lg1dom+MXpM8YUQgj5U1gA4PYbrsTkeBuCTv749huuxG3vekNhAP/k+67CpOXk8X27J8++J9AJ7uh5b1Z9EFGUipoWrZ4+g1Or65uus1VU1GGq8Lj9hiuNi3umKpUYO+UVYdUHEdVy2ex9hWkEAfDU3PXev5+pGqOfKoyYSg3Lqj44oyaiWmzN9n3zuaAZy4zZhjlqIqrFtJAXarHN9AHQZBVG0xioiaiW3kW33oW8UDPVpj8YYsDUBxHV1mQKIdZTWEJyCtQiMg7gcwCuQKdG/A9V9dsBx0VEZJRKbtkX1xn1ZwB8TVXfKyLnAtgecExERIViqtJokjVQi8jLAbwJwB8AgKqeBnA67LCIiDbrt1XpMHBZTHwdgGUA/ygiR0XkcyJyfv4iEdkvIgsisrC8vOx9oEQ0WPndgE1vE4+1IVQTXAL1OQCuBvB3qrobwC8AzOYvUtV5VZ1W1emJiQnPwySiQcpms01uE+/93mUtRwfdEKoJLjnqpwE8raoPdf/8bygI1EQ0vEyz2Y99+fHSU1Xq5pOL+l/nDXP9dMYaqFX1v0XkJyKyU1WfBPAWAN8PPzQiikVZv+eVtU6fj/ypKj7yyUUfEL2GvX4641r18WcA7uxWfPwIwIfCDYmIYmPaJp7XmzM25ZOrBOqytMYgzi4cFKdArarHABQ2CyGi4XfrdTutKYhMWXCtmk82fUD4PgYrdtxCTkRWRdvETU35Lx1vO/XjcKkiGcXt4kW4hZyInOR3A5oOus2CaNlrLjXR2WLk2voGxkSwoTpS6Y5eDNREI6xOZYZLzw3Ta2U10ft2T24J5BuqZwP9qAVpgAcHEI0s04y4iWOmTIcNAJ20imnhsig3PSzbyssODmCOmmhEDXKnnymHLUBpdUl+MXKQG3GaxEBNNKJ8npRSVdEioQDGWXYmH+BHZVs5AzXRkDNVV5hmtQoE7+WRryIZb7esQbqo2mOQHzZN4mIi0RArq64oq4323ZnOlEfuXTgsY6r2aPq8xkFhoCYaQllgLApiWWogW5SzXVc3UNtK8cq2idsWN4s+bNqtMVx7+QRm5g4nv8CYYeqDyINBtwDNjyVbYDPJUgP7dk/iyOxeiOW6Omx55LLvYatAKdqIc+OeSRxcXBqqBUbOqIlqiq2hva2REbA1NeAzhZBPc9jak5ZtE3f598tvxJmZO+ylz0hMOKMmqmkQlQdlM3jbLLhoUc7XVu2icjnTbD37EPC9TdxlgTGmn4BccEZNVFPTlQe2GXzZLHZMZNOHSDbD9HWyd9GHlqK49G719BkcOrrk/VRx208Hsf0E5IKBmqimpisPbNuvixbYWtsEEGB9oxMui4KTj5O9TR9Oik4JXta7GgBOra5vGoOvIGlaYMxm6LZ/vxgx9UFUU9Md3mwz+KIFtgvOO+dskM6Y0jN10gKmD6fJ8TbOf9nWeWGIFFHR/fcuSqZYe80ZNVFNvn90t3GZwednqJfN3lf4XqYt2f2mBcpmszcfOOY0Bh/KZugp1l5zRk3kQVbm9tTc9Tgyuzfoj9D9zOBd+kMD9RdGy2azrmMILcUe15xREyWmnxm8LW+b8ZEWMM1mXccQWtM/AfnAQE2UoKqLb67BKWRaIKYA6XPxsgnsR00UqUH0WR5kj+pRV9aPmjNqoggNqtY3plkvvcQpUIvIjwH8HMAGgDOmqE9Efgyy1je1tMAoqDKjvlZVfxZsJER0Voq1vhQOUx9EEcjno8e3t3BqdX3LdTHX+lI4rnXUCuDrIrIoIvuLLhCR/SKyICILy8vL/kZINOSKGhm98H9n0Brb3M4o9lpfCsd1Rj2jqidF5FcBfENEjqvqg70XqOo8gHmgU/XheZxEQyM/e149fWZLPnr9RcV4u4XzX3YOF/XILVCr6snuf58RkXsAvBHAg+VfRUR5RdUcJs+trePYbW9ramgUMWvqQ0TOF5ELs98DeBuAx0IPjGgYuTT1zzAfTRmXGfWvAbhHRLLr/0VVvxZ0VERDyrVqg/lo6mUN1Kr6IwBXNTAWoqFn2qLNfDSVYXkeJWkQ26t9MDUm+ti732Acf++9XtRuQQRYWV1P6r6pHgZqSk6T26ttHwhVPzCqbtHO32vvCSlF953qBxiVY1MmSs7M3GHjqdVHZvd6+z62BkVNNDAy3Wuv7L7ZUCltZU2ZeHAAJaep7dW2JvpNnD7uck/ZNYM4DZ2awUBNyWnqpBDbB0ITHxgu95Rdw/4gw4uBmpLT1FFKtg+EJj4wiu61V+99x3LUFfnHQE3JsZ0y7YvtA6GJD4z8vY63W3jF9lbhfYcYT50TyckfLiYSlfBd9THo8VZ9Ly5ONqdsMZGBmihxoT4smqquoQ4exUWUiKpBN2RNORcn48EcNVHOoPKyRX2pP3L3o6XfP2RJHhcn48FATUkJHUT7CZau72sbdz9BN+Sst6nqGrJjoKZkhAqivULMUF3H3U/QDTnrbaq6huyYo6ZkNHEyd4gZquu4TZ31yoKuqcmTr1kvTySPAwM1JaOpnYBVg6WNaXxLK2uYmTt8duHw2ssncHBxqVLQrdrkidLEQE3JCBFE80LMUE3jFrx0FNfSyhoOLi7hxj2TeOD4cqWgy1nv8GOgpmSE/jEfKJ+h9luvXDRuAZDfwbC2voEHji83UqMc20YdKscNL5SUQQWYol16APCK7S3c9q7ipv9lDf9NrUsFwFNz14e4hU3j4o7D+HDDCw0N24/5oQK56VDaU6vrhRtMihr+t1tj+NT7d2Hf7knjrr8mapSbWJQlv1ieR17E0LwnZPle2YJlUfmercxvkDXK3HGYHudALSJjInJURO4NOSBKTxP1zS5MwfGWux6pPRbbTDcf5GzBcJA1ytxxmJ4qqY8PA3gCwMsDjYUSFcuP0qbguKFau/9F0YJgr94gd+joEraJYKNg/af3ukFVazSxKEt+Oc2oReTVAK4H8Lmww6EUxfKjdNmMsO7uwmwGPN5ubXmtN8hlP10UBelYgiF3HKbHdUb9aQB/AeBC0wUish/AfgCYmpqqPTBKRxP1zS5ss966HxzZDLhswdK06DgmElUwZO11WqyBWkTeCeAZVV0UkTebrlPVeQDzQKc8z9cAKX6x/CidBZ5b7nrEmnao+31MQc70YfCiKgMj9c0l9TED4N0i8mMAXwKwV0T+OeioKCkx/Si9b/ckPvm+qwZWUcGFOgqh0oaX7oz6z1X1nWXXccMLDVpMG2O4mYRccMMLjZxB5WDZJIlC4BZyGphB9ptgrwuKDWfUFB2fZ/35Omdw4cSzlTvXETWBM2oaCFOvC6CzGOkaJKvkhLOAXtYQqff/BuaWqUllM2r2+qCBKKtprrIF3eXorENHl7Dr41/HTQeOGYM0UNx21MchsUR1MVDTQNjK1VyDpG1XZDbjXllbrz7IkvcnahIDNTnz2SGvqHtcnkuQtNUtm3YK5knF9ydqEgM1OfHdIa93k4yJS5C0tQt1CfaT42188JqpgW2SIbJh1Qc5CdEhr7d3Rr9b0G11y2WnqeQXC6df+0qW7FGUGKjJScgOeXU3iZRtbjE1aio6QouNiihWDNRDoInNG6E75IUKktwpSMOAgTpxPjeOlImlQ14/OFOm1HExMXEudcQ+xNQhj2jUcEaduCZPV2liZtqbxrmo3YJI56Tvse7RVlV2LRINCwbqxMVyuoqNSx49n8bp3aSSHQQQKrVDFDOmPhJnqyOOgWsNtuvmFG7tplHDQJ24FHLHpjz6TQeObdrhWCVdw63dNEqY+hgCsVc1uDRgAso3p+TFltohComBmrzL56PHt7dwatXcFClLZdhOEc/EltohCo2BmrwqqutubRO0xgTrG+be5ydX1rZsTmHVB1EHAzV5VZSPXn9RMd5u4fyXnWNMbWSpjKbTODySi1LAxUTyypSPfm5tHUdm9+LT798VTZWK746ARKFYA7WInCci3xWRR0TkcRH5eBMDo5f47AMdmq0/dExVKk3t6iSqyyX18UsAe1X1BRFpAfiWiHxVVb8TeGyE5np5+OLSEySWKpUmd3US1WEN1No5/faF7h9b3V/+T8SlQj77QDeRj02pW10quzqJnE4hF5ExAIsAfh3A36rqXxZcsx/AfgCYmprac+LECc9DHU2Xzd5X+KkoAJ6au975fYqa82enbhdVUozCIluVE8yJQis7hdyp6kNVNwDsEpFxAPeIyBWq+ljumnkA8wAwPT3NGbcnvmZ9RTPz7CHl0ym2dEtR46SV1fXkAnpKs38abZXK81R1RUS+CeDtAB6zXD60mpxt+uoDbcu79qZTbItspsZJsefPi8SSLycq41L1MdGdSUNE2gDeCuB44HFFq+mSLl9VEi4z8CyYly2y2RonsWqCyD+XGfUlAD7fzVNvA3CXqt4bdljxCnHIq42PWZ/L9uwsmJvSLdtEnHpxsGqCyC+Xqo/vAdjdwFiSkGJJV5aqWVvfOLsVO1tIzPSmU0xBvejrimQBfxQWJImawJ2JFdk2dMSmN1UDdIJtuzWGD14zZUynZOmWMZEt76foVIuYZAGfu/6I/GGvj4piOuTVZcZqStU8cHwZR2b3Gt973+5J3HzgWOFrWUlfWdXHzNzhxlNERMOKgbqiWEq6XHcs1knVmHLVk+Pt0iBf9/sS0WYM1H2IoaTLdVGzTh12nZ8euOuPyB/mqAML1VDJdcZa50zFOqWBKZzlSJQKzqgDCtlQyXXGWjdV0+9PD7GkiIiGgVOvj6qmp6d1YWHB+/umZmbucGEwHRPBJ993Va2gxT4VRMOldq8P6o8pPbGhWntmzRkr0ehgoA6o7FRtH6Vqvhc1uUGFKE5cTAyoaEGtV0ylatygQhQvBuqAynb4AXGVqvFYKqJ4MfURWJY6aHo3Y9U0BjeoEMWLgboBLgt/PvPD/ZQFcoMKUbwYqBtStvDnu966n1asMfUwIaLNmKOOgM/88KGjS8ZKk7I0hq8DCojIP86oI+ArP5zNzE1saYwYepgQ0VacUUfAV4/rsmOymMYgShdn1DX5WAQ05YevvXwCM3OHa1duAGAagyhhDNQ1+FoELKoKufbyCRxcXPJSuTE53maQJkrY0AbqujPdOqenuG4NL/se/ZyQwsoNouFkDdQi8hoAXwDwKgAvAphX1c+EHlgddWe6TZyeYvse/bx3P42a2N+DKH4uM+ozAG5R1YdF5EIAiyLyDVX9fuCx9a3uTLeJ01Ns36Pf965SuRGyXzYR+WOt+lDVn6rqw93f/xzAEwCi/r+4brlbE6en2L5HEyeksL8HURoqleeJyA4AuwE8VPDafhFZEJGF5eVlT8PrT91yN9evr7NJxPY9mtiAwv4eRGlwXkwUkQsAHARwk6o+n39dVecBzAOdE168jdBBPs+ar5gAqs1GqyzK9btJxOV7hN6Awv4eRGlwmlGLSAudIH2nqt4ddkjVFPVRPri4hBv3TPY9G+1nNlv1ENsYtmzzAFqiNFjPTBQRAfB5AM+q6k0ub9rkmYmmcwknx9s4Mru39vu7VEWkfH4hqz6I4lB2ZqJLoP4tAP8O4FF0yvMA4KOq+hXT14QM1PnAYmpAJACemru+8vv1BqqiACwAFJ0Pguza0B8WRDT8ah1uq6rfQic+DVxROVkWOPNc8qy28rSiqojse/Vey0U5IgopqaZMpsCZ/xRxzbPaytNsgTa71ldTJSKiIkkFalPgzFIRAmC83cJ5rW24+cAx66KebSbsEmhPrqxxUY6IgkoqUJsCZ5YL/tT7d+GXZ17EqdV1p5O0bTNh2yni2bUxVHAQ0fBKqimTrfa46tZx2/v19s4oyofnr2VgJqIQkgrUtqZDVRf1XJoY9QbgGErZYhgDETXLWp7XjybrqHsNe5lcyvXaRFSurDwvqRy1TSqLelV3MWbYRIloNCWV+rDppx9z0+q0FmW9NtFoSjpQm/K1/QZmX/nfsvep0yubTZSIRlOygdp303tf71f35JayIM+jtohGU7I5at/5Wl/vZ3ufstrtok6AvXXgrNcmGk3Rzahd0w++87W+3s/l5BbTrNglLcJ6baLRE1WgNqUNFk48iweOL28K3r7ztb7ez/Y+ZQueNx84VvieXCwkGm1RBWrTjPLO7/zXlq51N+6ZrHWKS56v/G+dk1u4WEhERaLKUZc1Xeq1tr6BB44ve83X+sr/1nmfVOrAiahZUe1MNO0sLOJ6MEBquEWcaDTVOjigSUVpgzoHA6SIi4VElBdV6qMobfDBa6aYDiCikRbVjBoonlFOv/aVTAcQ0ciKLlAXYTqAiEaZNVCLyB0A3gngGVW9IvyQwuFCHRGlyGVG/U8APgvgC2GHspXPwOq7NwgRUVOsi4mq+iCAZxsYyya2vhdVsZczEaUqqqqPXr4DK3s5E1GqvAVqEdkvIgsisrC8vFz7/XwHVtuJ40REsfIWqFV1XlWnVXV6YmKi9vv5Dqzcnk1EqYo29eEzsGaLkmvrGxgTAQCMt1s4r7UNNx84VuncQiKiplkDtYh8EcC3AewUkadF5I/CD8tfk6TeRUkA2FBFa5vgF6fP4NTqupeFSiKikKJqyhRClUZPk+NtHJndG3hERERblTVlijb14UuVxUdWgBBRjIY+UFdZfGQFCBHFaOgDddGiZGuboDUmm/6OFSBEFKskmjLVYTqjsOjvuJWciGI09IuJREQpSOKEF3a2IyIqFkWgZmc7IiKzKBYT2dmOiMgsikDNznZERGZRBGp2tiMiMosiULOzHRGRWRSLiaZaZy4kEhFFEqgBnjRORGQSReqDiIjMGKiJiCLHQE1EFDkGaiKiyDFQExFFLkj3PBFZBnCizy+/GMDPPA5nkIblXoblPgDeS4yG5T6AevfyWlWdKHohSKCuQ0QWTK3+UjMs9zIs9wHwXmI0LPcBhLsXpj6IiCLHQE1EFLkYA/X8oAfg0bDcy7DcB8B7idGw3AcQ6F6iy1ETEdFmMc6oiYioBwM1EVHkBhKoReTtIvKkiPxARGYLXhcR+Zvu698TkasHMU4XDvfyZhF5TkSOdX/91SDGaSMid4jIMyLymOH1lJ6J7V5SeSavEZEHROQJEXlcRD5ccE0Sz8XxXlJ5LueJyHdF5JHuvXy84Bq/z0VVG/0FYAzADwG8DsC5AB4B8Bu5a94B4KsABMA1AB5qepwe7+XNAO4d9Fgd7uVNAK4G8Jjh9SSeieO9pPJMLgFwdff3FwL4z4T/X3G5l1SeiwC4oPv7FoCHAFwT8rkMYkb9RgA/UNUfqeppAF8C8J7cNe8B8AXt+A6AcRG5pOmBOnC5lySo6oMAni25JJVn4nIvSVDVn6rqw93f/xzAEwDyTduTeC6O95KE7r/1C90/trq/8lUZXp/LIAL1JICf9Pz5aWx9YC7XxMB1nL/Z/THpqyLyhmaG5l0qz8RVUs9ERHYA2I3O7K1Xcs+l5F6ARJ6LiIyJyDEAzwD4hqoGfS6DOOFFCv4u/2nkck0MXMb5MDp7+F8QkXcAOATg9aEHFkAqz8RFUs9ERC4AcBDATar6fP7lgi+J9rlY7iWZ56KqGwB2icg4gHtE5ApV7V0T8fpcBjGjfhrAa3r+/GoAJ/u4JgbWcarq89mPSar6FQAtEbm4uSF6k8ozsUrpmYhIC53Adqeq3l1wSTLPxXYvKT2XjKquAPgmgLfnXvL6XAYRqP8DwOtF5DIRORfABwB8OXfNlwH8fnfl9BoAz6nqT5seqAPrvYjIq0REur9/Izr/5v/b+EjrS+WZWKXyTLpj/AcAT6jqXxsuS+K5uNxLQs9lojuThoi0AbwVwPHcZV6fS+OpD1U9IyJ/CuB+dKom7lDVx0Xkj7uv/z2Ar6CzavoDAKsAPtT0OF043st7AfyJiJwBsAbgA9pdFo6JiHwRnVX3i0XkaQC3obNIktQzAZzuJYlnAmAGwO8BeLSbDwWAjwKYApJ7Li73kspzuQTA50VkDJ0Pk7tU9d6QMYxbyImIIsediUREkWOgJiKKHAM1EVHkGKiJiCLHQE1EFDkGaiKiyDFQExFF7v8B4SC4LI9GLoEAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": { - "tags": [], "needs_background": "light" - } + }, + "output_type": "display_data" } + ], + "source": [ + "np.random.seed(13) # pick the seed for reproducibility - change it to explore the effects of random variations\n", + "\n", + "train_x = np.linspace(0, 3, 120)\n", + "train_labels = 2 * train_x + 0.9 + np.random.randn(*train_x.shape) * 0.5\n", + "\n", + "plt.scatter(train_x,train_labels)" ] }, { @@ -682,37 +640,37 @@ "id": "Ng4rZmGc6oxk" }, "source": [ - "Линейная регрессия вычисляется как $f_{W,b}(x) = Wx+b$, где $W, b$ - параметры модели, которые необходимо найти. Функция ошибки на наборе данных $\\{x_i,y_u\\}_{i=1}^N$ может быть определена как среднеевадратичное отклонение\r\n", - "$$\r\n", - "\\mathcal{L}(W,b) = {1\\over N}\\sum_{i=1}^N (f_{W,b}(x_i)-y_i)^2\r\n", - "$$\r\n", - "\r\n", - "Опишем модель и функцию ошибки:" + "Linear regression is defined by a straight line $f_{W,b}(x) = Wx+b$, where $W, b$ are model parameters that we need to find. An error on our dataset $\\{x_i,y_u\\}_{i=1}^N$ (also called **loss function**) can be defined as mean square error:\n", + "$$\n", + "\\mathcal{L}(W,b) = {1\\over N}\\sum_{i=1}^N (f_{W,b}(x_i)-y_i)^2\n", + "$$\n", + "\n", + "Let's define our model and loss function:" ] }, { "cell_type": "code", + "execution_count": 15, "metadata": { "id": "QxhI4GlB6aiH" }, + "outputs": [], "source": [ - "input_dim = 1\r\n", - "output_dim = 1\r\n", - "learning_rate = 0.1\r\n", - "\r\n", - "# This is our weight matrix\r\n", - "w = torch.tensor([100.0],requires_grad=True,dtype=torch.float32)\r\n", - "# This is our bias vector\r\n", - "b = torch.zeros(size=(output_dim,),requires_grad=True)\r\n", - "\r\n", - "def f(x):\r\n", - " return torch.matmul(x,w) + b\r\n", - "\r\n", - "def compute_loss(labels, predictions):\r\n", + "input_dim = 1\n", + "output_dim = 1\n", + "learning_rate = 0.1\n", + "\n", + "# This is our weight matrix\n", + "w = torch.tensor([100.0],requires_grad=True,dtype=torch.float32)\n", + "# This is our bias vector\n", + "b = torch.zeros(size=(output_dim,),requires_grad=True)\n", + "\n", + "def f(x):\n", + " return torch.matmul(x,w) + b\n", + "\n", + "def compute_loss(labels, predictions):\n", " return torch.mean(torch.square(labels - predictions))" - ], - "execution_count": 189, - "outputs": [] + ] }, { "cell_type": "markdown", @@ -720,33 +678,33 @@ "id": "JUxwj3367gD2" }, "source": [ - "Обучать модель будем на сериях примеров - minibatches. Для обучения используем градиентный спуск, подстраивая парметры в соответствии с формулой:\r\n", - "$$\r\n", - "\\begin{array}{l}\r\n", - "W^{(n+1)}=W^{(n)}-\\eta\\frac{\\partial\\mathcal{L}}{\\partial W} \\\\\r\n", - "b^{(n+1)}=b^{(n)}-\\eta\\frac{\\partial\\mathcal{L}}{\\partial b} \\\\\r\n", - "\\end{array}\r\n", + "We will train the model on a series of minibatches. We will use gradient descent, adjusting model parameters using the following formulae:\n", + "$$\n", + "\\begin{array}{l}\n", + "W^{(n+1)}=W^{(n)}-\\eta\\frac{\\partial\\mathcal{L}}{\\partial W} \\\\\n", + "b^{(n+1)}=b^{(n)}-\\eta\\frac{\\partial\\mathcal{L}}{\\partial b} \\\\\n", + "\\end{array}\n", "$$" ] }, { "cell_type": "code", + "execution_count": 16, "metadata": { "id": "-991PErM7fJU" }, + "outputs": [], "source": [ - "def train_on_batch(x, y):\r\n", - " predictions = f(x)\r\n", - " loss = compute_loss(y, predictions)\r\n", - " loss.backward()\r\n", - " w.data.sub_(learning_rate * w.grad)\r\n", - " b.data.sub_(learning_rate * b.grad)\r\n", - " w.grad.zero_()\r\n", - " b.grad.zero_()\r\n", + "def train_on_batch(x, y):\n", + " predictions = f(x)\n", + " loss = compute_loss(y, predictions)\n", + " loss.backward()\n", + " w.data.sub_(learning_rate * w.grad)\n", + " b.data.sub_(learning_rate * b.grad)\n", + " w.grad.zero_()\n", + " b.grad.zero_()\n", " return loss" - ], - "execution_count": 190, - "outputs": [] + ] }, { "cell_type": "markdown", @@ -754,25 +712,26 @@ "id": "idr2VEWb9rr0" }, "source": [ - "Теперь приступаем к обучению: делаем несколько проходов по всему датасету (эпох), разбиваем его на minibatches, и вызываем функцию обучения:" + "Let's do the training. We will do several passes through the dataset (so-called **epochs**), divide it into minibatches and call the function defined above:" ] }, { "cell_type": "code", + "execution_count": 17, "metadata": { "id": "nOuu0qpx-wAp" }, + "outputs": [], "source": [ - "# Shuffle the data.\r\n", - "indices = np.random.permutation(len(train_x))\r\n", - "features = torch.tensor(train_x[indices],dtype=torch.float32)\r\n", + "# Shuffle the data.\n", + "indices = np.random.permutation(len(train_x))\n", + "features = torch.tensor(train_x[indices],dtype=torch.float32)\n", "labels = torch.tensor(train_labels[indices],dtype=torch.float32)" - ], - "execution_count": 191, - "outputs": [] + ] }, { "cell_type": "code", + "execution_count": 18, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -780,16 +739,9 @@ "id": "3zdIf6c_85Ht", "outputId": "6520288c-da59-4a9f-c37e-cd99779c3073" }, - "source": [ - "batch_size = 4\r\n", - "for epoch in range(10):\r\n", - " for i in range(0,len(features),batch_size):\r\n", - " loss = train_on_batch(features[i:i+batch_size].view(-1,1),labels[i:i+batch_size])\r\n", - " print('Epoch %d: last batch loss = %.4f' % (epoch, float(loss)))" - ], - "execution_count": 192, "outputs": [ { + "name": "stdout", "output_type": "stream", "text": [ "Epoch 0: last batch loss = 94.5247\n", @@ -802,13 +754,27 @@ "Epoch 7: last batch loss = 0.3390\n", "Epoch 8: last batch loss = 0.3384\n", "Epoch 9: last batch loss = 0.3382\n" - ], - "name": "stdout" + ] } + ], + "source": [ + "batch_size = 4\n", + "for epoch in range(10):\n", + " for i in range(0,len(features),batch_size):\n", + " loss = train_on_batch(features[i:i+batch_size].view(-1,1),labels[i:i+batch_size])\n", + " print('Epoch %d: last batch loss = %.4f' % (epoch, float(loss)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We now have obtained optimized parameters $W$ and $b$. Note that their values are similar to the original values used when generating the dataset ($W=2, b=1$)" ] }, { "cell_type": "code", + "execution_count": 19, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -816,27 +782,25 @@ "id": "US6q0nCBD-LL", "outputId": "c804b779-3231-4f6f-c854-032d211b2853" }, - "source": [ - "w,b" - ], - "execution_count": 193, "outputs": [ { - "output_type": "execute_result", "data": { "text/plain": [ "(tensor([1.8617], requires_grad=True), tensor([1.0711], requires_grad=True))" ] }, - "metadata": { - "tags": [] - }, - "execution_count": 193 + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" } + ], + "source": [ + "w,b" ] }, { "cell_type": "code", + "execution_count": 20, "metadata": { "colab": { "base_uri": "https://localhost:8080/", @@ -845,40 +809,36 @@ "id": "_e6xRMZFDnyI", "outputId": "79e6c360-265a-401d-ce39-8f211917a13d" }, - "source": [ - "plt.scatter(train_x,train_labels)\r\n", - "x = np.array([min(train_x),max(train_x)])\r\n", - "with torch.no_grad():\r\n", - " y = w.numpy()*x+b.numpy()\r\n", - "plt.plot(x,y)" - ], - "execution_count": 194, "outputs": [ { - "output_type": "execute_result", "data": { "text/plain": [ - "[]" + "[]" ] }, - "metadata": { - "tags": [] - }, - "execution_count": 194 + "execution_count": 20, + "metadata": {}, + "output_type": "execute_result" }, { - "output_type": "display_data", "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZhU1Z3/8ffp7uqFrRulQWhoQFZFZOsxGowKiBhN3KNJRo3JTJiJGbcYlGQWZ/L8ZgTcMyYTiZplYhKNInFiZFFQXFFWUeluEGRp9qWbpffq8/uju9qmum7Vra5b1beaz+t5eNJ03br3XMp876lzvud7jLUWERHxr4zOboCIiESnQC0i4nMK1CIiPqdALSLicwrUIiI+l5WMk/bp08cOGTIkGacWEemSVq9efcBaWxjptaQE6iFDhrBq1apknFpEpEsyxmxzek1DHyIiPqdALSLicwrUIiI+p0AtIuJzCtQiIj6XlKwPEZGTycK1FTywuIxdlTUMKMhj1oxRXDWhyLPzK1CLiCRg4doKfrRgAzUNQQAqKmv40YINAJ4Faw19iIgk4IHFZa1BOqSmIcgDi8s8u4YCtYhIAnZV1sT1+45QoBYRScCAgry4ft8RCtQiIjSPNU+es4yhs19m8pxlLFxb4ep9s2aMIi+QecLv8gKZzJoxyrO2aTJRRE56iUwIhl5X1oeISBJFmxB0E3CvmlDkaWAOp6EPETnppWJCMBExA7UxZpQxZl2bP0eMMXemonEiIqmQignBRMQM1NbaMmvteGvteGASUA28mPSWiYikSComBBMR7xj1NOBTa61jgWsRkXQTz4Rg2+Xi+XkBjIHK6oakTCKGGGut+4ONeRpYY619PMJrM4GZAMXFxZO2bVMsF5GuJTw7JFxeIJP7rxnboWBtjFltrS2J9JrryURjTDZwBfCnSK9ba+dba0ustSWFhRG3/RIR8b1o+dSRskPa8nrpeEg8Qx9fprk3vdfzVoiI+ECsfGo3WSDJyBSJJz3vG8AfPG+BiIhPxCqw5CYLJBmZIq4CtTGmOzAdWOB5C0REfCJWPnWk7JC2kpUp4mrow1p7HDjV86uLiPjIgII8KiIE61AvOTw7JFVZH1pCLiLSYtaMUe2yOsJ7ycleLh6JArWISItUFFjqCAVqEZE2OqPHHIuKMomIeOCjiiqeWZmchX7qUYuIJOCzA8d5cEkZf/lwN3175nDNhIHkZTtnhnSEArWISAfsO1rLf7+2mT+8v51AZga3TR3Ody843fMgDQrUInKSaFtMKZFJwqO1DcxfsYWn3tpKfWMTXz9nELdPG0HfnrlJaHUzBWoR6RKiBeKObrUVOmdFZQ0ZQBOQYaDJwlfO7s8PLxnFkD7dk31rCtQiEh+veqZet8kpEAPc/dx6gmGVQmNttRV+zqaW3zdZyMnK4OIz+qUkSEOcZU7dKikpsatWrfL8vCLSuSKV+UyktKdXJs9ZFnFFYUFegLrGpqgV74oK8iI+dL54/2vsqqqN+r63Z09NvPEtopU5VY9aRFxLdBPYZHGq0VFZ0xD1fQZaA3zbXnjxqd2iBulo10wGBWoRcc2vm8A61eiIxgDh4wk1DUFmL/iQ2oam1rHoaNdMFS14ERHXnIJTfl7Asdh+Kjjtedi7WyDi8ZnGtAvSIbUNTdw9fST3Xz3WsVJeqvdTVKAWEdciBcRAhuF4fSMVlTVYPh9CSGawDt+FBeD+a8ZSVJCHoXn8+P5rxnLfV8dEDOAPXT+OIoeHzmm9crlt2ghuOKe49ZzQHNxpc+5UDvVo6ENEXItUtKi6vpHD1SeOBSdz3Nopw+P+a8Y6Tu6FZ6nMGHMar27c2264JC+Qyewvj279u1/qfijrQ0QSMnT2yxGHEQywdc7lnl/PKcPDTRZGY7CJP63eyaOvlrP3SB1jBvRi/9E69h+t6/RUQ2V9iEjSxCq277WOTGhaa1n88R7mLS5jy/7jTCwu4L+/MZFzhp6SlDZ6TYFaRBLipti+l+J9MLz76UHmLipl3Y5KRvTtwfybJjH9zH6YljHndKBALSIJSXWxfbcPhk92HWHuolLeKN9P//xc5l13NtdOHEhmRvoE6BAFahFJWCon3WI9GHYcquahJWX8ef0ueuUG+PFlo7n5vCHkRtmU1u9cBWpjTAHwJHAWzTni37HWvpvMhomIOIn0YDhwrI7Hl23mmZXbyMww/OOFw/jHC4eRnxc5lzqduO1RPwYsstZeZ4zJBrolsU0iIhFFKgh18Zn9ePLNLfxyxRZqG5u4vmQQd148gn69kld2NNViBmpjTD5wAXALgLW2HqhPbrNERE4UKX961p/WkxPI5FhdI18+6zR+OGMUwwp7dHJLveemRz0U2A/8yhgzDlgN3GGtPd72IGPMTGAmQHFxsdftFJFO1tnlTSMVhGposphgEwu/P5nxgwpS1pZUc7OEPAuYCPyPtXYCcByYHX6QtXa+tbbEWltSWFjocTNFpDOFerOpXCbe9tpOi1wA6hubunSQBnc96p3ATmvtypa/P0+EQC0iXZdTedN/f+njqLuqJNoDj1T/OpxTzY6uJGagttbuMcbsMMaMstaWAdOAT5LfNBHxi2j1nkM1n8N3VenI1lfh/uuvG6MG6VRXsessbrM+bgOeacn42AJ8O3lNEhG/cVvvOVSMKfRzpNfcBOq9R2p59NVN7Dta53hMkU+2AUsFV4HaWrsOiFgsRES6vkirAZ1Eq7kRa4OBqpoGnnjjU55+eyvBJkv37EyO17e/ptfbYPmdViaKSExuy5vC5zU3YtXjaDuG3T8/l5Ihp/BG+X6O1DZw5bgB/GD6KNZsP5zSOiJ+pUAtIq6ErwZ02ug2FESjvRb+3l1Vtby0fhejT+vJ77/7BcYMyG8N5DUNQTKNIWjtSTXc0ZYCtchJLJHMDDfFmJxem7eoNOIwytHaxtYg3TaQB61tDfQnW5AGbRwgctJy6hEne5up97ce4vonnEsFFUWZuIw0Nt3ZC3G8oo0DRKQdp9zoZG2hVbrnCA8sKuO10n2OO3wbIo9th4RPRjptywXxpQH6nQK1yEmqIzuldMTOw9U8vLScF9dW0CMni3svHc2p3bO576WPT3hQGHDcGTwkfHOAVD9sOosCtUgX5zQ04JQbbWnelzDRIYRDx+v52fLN/O+728DAzC+dzvcuGkZBt2wAsrMyWtuVnxdoXTjjJFK2R6oeNp1NgVqkC4s2NBAtNzqRIYTq+kaeenMr81ds4Xh9I9dNGsgZ/XvxZMvv2j4srppQ1NrGaJyyPVK9X2NnUaAW6YJCvehIQSw0NBCalIt1nNtA3RBs4o8f7OCnr21i/9E6LjmzH7NmjOLjXUeijiNHGr4IiTW56bQt15TRhUyesyztJxhDFKhFPOCnzAM3hYxCQwOhXu3Q2S9HHB92M4TQ1GR5ecNuHlpSxmcHqzlnyCn84sZJTBrcG4BbfvVB1HHkaNeIlYESKUVwyuhCXlhd0aUmGBWoRRLkt8yDaD3UkPChgY4OIby16QBzF5WyoaKK0af15OlbSqiqbuD2P6xtDZxOWRyhAO10TFFBnqt/v/CFOJPnLOtyE4xu6lGLSBTRMg+SJVSjeejsl5k8Z9kJdaFj9YIjTcrNmjGKvLDNX6Mt1d6ws4obn1zJjU+t5NDxeh6+fhwv3/4ljtQ08uMXPzqhbrXTnt+hh0C8147FzQRjtH8/P1KPWiRBqc48iNWDj9aLzTTmhIdIqIfpZpUhwGcHjvPgkjL+8uFuencL8K9fOZMbzy0mJyuz9f3hDy1L5NS76vpGFq6tcH1tt2J9O/DbNyA3FKhFEpTqzINYucORJtgCGQYMNASbw2Wk4BRpZ++QfUdr+elrm/jj+zsIZGZw29ThfPeC0+mVe+IO304PJwsUhKXgHa5uOKENXgVJpwnGUA89HXOvFahFEhQrMHgtVg/ebaU7p+DUdmL0tPxcxhbl8+amAzQEm/jGOcXcNm04fXtG3uE72ngz0C5XOhkBMlYPPR1zrxWoRRLk9Vf3WNz04MN7qENnvxzxXLGWZO+uqmV3VS0TBhXwyA3jGdKne9S2RXto3fXsOldt8EK0Hno65l5rMlHEA1dNKOLt2VPZOudy3p49NalfoTsy+eYUhMJ/71TVbt/RuphBGpr/He6/ZixFBXkYmnvSoRQ7t21INq8nL1NBPWqRNNORHnys4RlrLcvL9rGrqjbi++Pp9Tr1ZlM9ROQk1d+AvKBALZKG4p18ixacVm87zNxXSnn/s0NkZhiCEcraedHr9VOA9HLyMhVUj1rEp5K92nHT3qPMW1zG0k/2UtgzhzumjSAvkMm/LPwo5TWqRfWoRdJOMnN9d1XW8Oir5Ty/eifds7P44SUj+c75Q+mW3RwOMjOML3q98jlXgdoY8xlwFAgCjU5RX0S8kYxc38rqen7++qf8+p3PwMJ3Jg/l1inDOaV79gnHpduwwMkgnh71FGvtgaS1RERaeZnrW1Mf5FfvbOV/Xv+UY3WNXDNhIHdNH8HA3t0SbaakiIY+RHwgfDy6oFug3QIViG9SrzHYxHOrdvLYa+XsPVLHtNF9mXXpKEaf1svLpksKuA3UFlhijLHAE9ba+eEHGGNmAjMBiouLvWuhSBcXaTw6kGEIZJrWJd/gPpXNWsuij/bwwJIytuw/zqTBvXn8mxP5myGnJO0eJLncBurzrbUVxpi+wFJjTKm1dkXbA1qC93xozvrwuJ0iXUZ477m6vrHdeHRDk6UgL0D3nKy4JvXe+fQAcxeVsX5HJSP69uCXN5dw8Rl9Mcaphp2kA1eB2lpb0fK/+4wxLwLnACuiv0tEwkXqPTupqmlg3X2XuDrvx7uqmLeojDfK9zMgP5d5153NtRMHkpmhAN0VxAzUxpjuQIa19mjLz5cAP0l6y0S6IDdF/UPcjEdvP1jNQ0vL+PO6XeTnBfjny87gpvMGkxu2RFrSm5sedT/gxZavTlnA7621i5LaKpEuym3WRqzx6APH6nh82WaeWbmNzAzDrRcN4x8uHEZ+XsDxPZK+YgZqa+0WYFwK2iLS5TlVbnM7Hn2srpFfrtjCk29uobaxietLBnHnxSPo1yty2VHpGpSeJ2nJT5vJxsOpMNG/XzHGsf0L11Ywb1Epu6pqT9gppW/PHL4w9BQF6ZOAArWknVRupRTrgRDvAyPewkQL1uzk3hc+bE3Ta5tOte9oXbv7TtcHmESnokySdibPWea4i8jbs6d6dp3wBwKcWKAo1uuJsNbyRvl+vvvbVSfkUkcSuu9ktkeSL1pRJm0cIGknVVspxdpdPFm7j6/bUck3fvket/zqg5hBGj6/787YDV1SQ4Fa0k6qdgqJ9UDw+oHx6f5jfO93q7nqZ2+zed8x/uOKMQzIjz3+HLrvdNwLUNxRoJa0k6qtlGI9ELx6YOypquVHCz7kkkdWsKJ8P3ddPJLXZ03hW18cwj2Xjm53r221vW+/bHUl3lOglrQTbV8+L8V6ICT6wKiqaWDuolIuenA5z6/eyU3nDuaNe6Zwx8Uj6JHTPM8ffq8FeQF6dwtEvO9kPMAWrq1g8pxlDJ39MpPnLGPh2ooOn0s6TpOJIlF4nfUBUNsQ5DfvfMbPX/+UI7UNXDW+iB9MH8mgUxIvO+pl1ocmJ1Mr2mSiArVIijQGm1iwpoJHXi1nd1UtF40q5J4ZozlzQGJlR5OVkpeq7Bpppq24RDqRtZYln+zlgcVlbN53jHGDCnj4+vGcN+zUdsfGG3STvWVXPL+X5FGgFgnjZQ/1/a2HmPPKRtZsr+T0wu784saJzBhzWsSyox0JusnYsivEabm7JidTT5OJklaSPbkVCpYVlTVYPg+W8V6ndM8RvvPrD7j+iXepqKzhhpJB1NYH+d7v1nD+3OURz9eRPOhk9npTlV0jsalHLWkjFUvHE+2h7jxczcNLy3lxbQU9c7K499LRnNo9m/te+jhmuzsSdJPZ6413ubskjwK1pI1kfs0P6WgP9dDxeh5ftpnfvbcNY2DmBadz64XDye8WYPKcZa7a3ZGg61Tkyater3Yk9wcFakkbqZjcijdYVtc38tSbW3lixRaq6xv52qRB3Dl9BP3zPz/eqX0VlTVMnrOstbc6ZXQhL6yuiCvoqtd7clCglrSRiskttz3UhmATf3x/O4+9tpkDx+q45Mx+3HPpKIb37em63YbPt+KqqKzhhdUVXDupiOWl++MKuur1dn0K1JI2kv01H6L3UNvWhc7MMASbLOcMPYUnbprEpMG942p327rSITUNQZaX7k9JjrLKoaYXBWpJG6n6mh+ph7pwbQX3PP8h9cEmAIJNzWG2fM8Rdhyqjhio2wbD/LwAuYEMKqsbHHvYkJoc5VTW8xZvKFBLWon1NT8ZPcUPd1Zy7wufB+m2KmsaIwa58GBYWdNAXiCTR24Yz1UTihxX/aUiRzkVk7LiLeVRiyf8ULzHqxzokK0HjvP936/hisffpq6xfZAOiZTrHCsnujNzlLXiMP247lEbYzKBVUCFtfYryWuSpBu/fJV2Co53P7c+rrbsO1rLT1/bxB/f30F2Vga3Tx3On1bvZHdVreN7woNcrGDYmdkaWnGYfuIZ+rgD2AgkVkFGuhy/fJV2Co5Ba109OI7UNjD/jS089dZWGoJNfPMLxdw2dQSFPXM4vbBHuwnBttoGuYVrK8gwhmCEgmdtj+usbI1UTMqKt1wFamPMQOBy4D+BHyS1RZJ2/PJVOtokXbQHR21DkN+9t42fLd/M4eoGvjpuAHdPH8mQPt1bjwm9799f+pjKmoYT3t82yIW+XUQK0n4Jhsq9Tj9ue9SPAvcA7ZNEWxhjZgIzAYqLixNvmaQNv3yVjtRTbCv8wRFssry4toJHlpZTUVnDl0b04d5LR3NWUX7E94d6wNEmLCN9uwDINMZXdZyVe51eYgZqY8xXgH3W2tXGmIucjrPWzgfmQ3M9as9aKL7nl6/SocBz93Prow47WGtZVrqPeYvKKNt7lLMH5jPvurOZPLyP6+s4BTmnbxFN1iowSoe56VFPBq4wxlwG5AK9jDG/s9bemNymSbrw01fp0DWdHhyrtx1iziulfPDZYYac2o2ffXMil42NXHa0I/zy7UK6lrh2eGnpUf8wVtaHdniRzhY+PHHjucWs3lbJqxv3UtgzhzumjeCGvxlEINPbDFVtXyUdpR1e5KQTGp7YVVnDI0vLeWBxGd2zs5g1YxTfnjyEbtnJ+U/fT98upOvQnonSaZJZb6Kyup6fv/4pv37nM7Bw83mD+f6U4fTunp30a4t0hHrU4jteLpJpG3RPy89lYnFvVmzaz7G6Rq6dOJC7po+kKCzPOdK1V207FHflOpFUUI9aOoVTrQto3uXabZCMNCYMMGZALx6+fjyjTut5wrEPLC5zvG54RTuNLUsqqUctvhNtMUw8vet5i0oj5i1XVje0BumFaysiLlQJF6nsqAoViR+oKJN0iljparE2dQV459MD7HKovxF6EIR63LGCtBMVKhI/UI9aXPNyAi7WKkJwDpIfVVQxb3EZK8r3kxmjpobTSsFwkQr5tz2PSGdSoBZXvK6Q1zaNzWnMODxIbj9YzYNLynhp/S4KugX458vOoKBbgH/788eOqyLd9IiLOrhfoUiqKFCLK8mokNe2dka0Jej7j9bx+LJN/P797WRmGL4/ZRgzLxhGfl4AgEBmhmNPP1qhpvDJwpLBpyhlT3xJgVpcSWaFPKdFIhef2Y+Hl5bz5JtbqGts4oa/GcQd00bQr1duu/c7BVSnIZbe3QLc99UxJ7xPhYrErxSou4BULN5Idg2LtkGyrjHI71du58J5yzl4vJ7Lx/bn7ktGcnphjw6dF7RSUNKbAnWaS9XuKqmokNfUZHlp/S4eXFLGzsM1fHHYqdx76WjGDSpI6LzqKUu6U6BOc6naXSWZPVNrLa+X72feojI27j7Cmf178dvvjOVLI/p4VtVOJJ0pUKe5VO6ukoye6drth5nzSikrtx6i+JRu3HTuYF7buJdvPf0++XkBjIHD1Q2taXjxrFoU6SoUqNNcutQ/Dh9Hv/m8wazdXsmij/fQp0c2P7lyDN0Cmfxrm1S7totUQrnSnbVxrkhn0srENDdrxijyApkn/M5v+b+hcfSKyhoszcH2/ldKWV62j7suHskbs6Zw83lDeOTVTa4Wp7hZtSjSlShQp7mrJhRx/zVjKSrIw9C8eMNvhYScVgfWNTbx3KodLP1kLxDfcI2WdsvJREMfXYCfsxpqG4KOC07gxKGMaItTwvltaEckmRSoxXML11Ywb1Epu6pqyXCRtBEaynBT/wP8N7QjkmwK1OKpF9fs5N4XNlAfbAKgyUKmgYwMQ0PQufb5rsqadimAyvoQaaZALZ5ZueXgCUE6JGihZ3YW3XOyYhZgSvUwjrbkknSgQC0J27j7CPMWlbK8bL/jMVU1Day775KYBZhSKVWrOkUSFTNQG2NygRVATsvxz1tr70t2w+Rzfu317ThUzSNLy3lxXQU9c7KY/eXR/PadzyIW82/bYwZ/1N5I1apOkUS56VHXAVOttceMMQHgLWPMK9ba95LcNsGfvb6Dx+p4fPlmnnlvO8bAzAtO59YLh5PfLcBpvXJj9pj9kqWSylWdIomIGaht8+63x1r+Gmj54/2OuBKRl72+RHvmx+saeeqtrcxfsYXq+kauLxnEHRePoH/+56lyfuoxx5IuqzpFXO1CbozJBFYDw4GfWWvvjXDMTGAmQHFx8aRt27Z53NST09DZL0d8Khpg65zLXZ8n0thwaPupSJkUbYN6//xczht2Km+UH+DAsTpmjOnHrBmjGN63Z/sLpRGn8XK/LRiSk0PCu5Bba4PAeGNMAfCiMeYsa+1HYcfMB+YDlJSUqMftEa96fZF65qEPKXw4JTyA7aqq5YU1FQwr7M78myex/WA133r6gxNS6CqrG3zde44knXr/cnKLK+vDWltpjFkOXAp8FOv4riqVk3te1YGONe7adjjFacl3TX2Q7QerT2hP28JJfhg/j5dfxstFoolZ68MYU9jSk8YYkwdMB0qT3TC/ilRg6EcLNrBwbUVSrudVLQ83PfBdlTV8uLPSMdd5d1VtzF29VTBJxHsxx6iNMWcDvwEyaQ7sz1lrfxLtPSUlJXbVqlWeNdJPJs9ZFjGQFRXk8fbsqZ3QIncijceGywtkUtMQJMM0rygMF1odGEu84+cikuAYtbX2Q2CC561KU+mY0hUaqqlpCLYG29BEYltBa7l92ggG5OfyH//3Sbug7vS+cKHeu1/zv0XSjVYmxindUrrCe9JBa8kLZPLVcf3564Y9HKtrBOD84X145IbxFPbMASA3kMndz61v14O2EDVYh8bP/Zj/LZKuVI86Tn4q1L9wbQWT5yxj6OyXmTxnWcRxcqc87D+t3smxukauGDeAN2ZdxO/+/gutQRqag2mTwzBHKKXPAAV5AXp3C7QbP4+W/y0i8VGPOk5+Sely22N1GpKxFv5y2/mcVZTveA2nbw9uxuPTcYhIxK8UqDvADyldblcs9s/PjVh7o6ggL2qQhsRSA9NtiEjEzzT0kWRuhic6wk2PddVnh8gJG6YB98E2kdRAPw0RiaQ79aiTKJkTatF6rOV7jzJvURmvbtxL3545fG3SQN7efIDdVbVxD9V09NuDX4aIRLoCV7U+4tWV86jj4ZRznWkMD10/LqGgFSkvOicrg3EDC1i17RDds7P4x4uG8e3JQ+iWreexiN8lXOtDOsZpeCJobcI967Y91orKGnrkZFHbEGTdzkr+7vyh3HrRcHp3z+5Yw0XEVxSokyjartpeFKi/ZEw/Kipr+MXrn3K8vpFrJg7krukjKerghJ0WqIj4kwJ1EsXaVbujqWoNwSaeW7WDx17dxL6jdVx8Rj/uuXQUI/t1vOyoFqiI+JcCdRKFAlykFX4Qf6qatZa/btjDg0vK2HrgOCWDe/Pzv51IyZBTEm6rtqUS8S8F6iQLBblES5W+s/kAcxeVsn5nFSP79eDJm0uYdkZfjDERj493GEMLVET8S4E6BdykqjkF1o8qqpi7qJQ3Nx1gQH4uD35tHFdPKCIzI3KADp0r3mEMLVAR8S+l5/mAU6rdmAG9WLO9koJuAf5pynBuPHcwuREWsITrSClWbUsl0rmUnudzkcaH6xqbWLu9ku9PGcY/XDiMXrkBV+dauLbCMdMk2jCGFqiI+JcCtQ84Fk4CZs0Y7fo8oV6xk1jDGH6oYSIi7anWRyerawzSKy9ybznefOho22SpzoZI+lKPOkEdXSTS1GT58/oKHlpSTlVNQ7vtr/ICmUwZXcjkOcsSztwANNYsksYUqBPQkewKay2vl+1n7qJSSvccZcyAXvzX1WM5eKyOB5eUtwblKaMLeWF1hSeZG0UFeQrSImmsywbqRJdDu3l/vItE1mw/zNxXSlm59RCDT+3GzecN5tVP9vKtp99vd43Jc5bFvQAlkfrRIuJfMQO1MWYQ8FugH83zW/OttY8lu2GJSHQ5dKK7p4T/fvO+YzywuJTFH++lT49sfnLlGLoFMvnXP3/seI2OLEDpSOaG6nuI+J+bHnUjcLe1do0xpiew2hiz1Fr7SZLb1mGJLod2+/5Yi0T2VNXy6KvlPLdqB92ys/jB9JH83flD6Z6TFbPH3NEFKPFkbqi+h0h6iJn1Ya3dba1d0/LzUWAj4Ov/Fye6HNrt+512Mfn+lGHc/8pGLnxgOS+s2cm3vjiEN2ZdxO3TRtA9J8vVNVKxQ4o2oBVJD3GNURtjhgATgJURXpsJzAQoLi72oGkdl+hyaLfvDx9q6J+fy6TBvZnzSilH6xq5enwRd00fyaBTusV9jVQsQFF9D5H04DpQG2N6AC8Ad1prj4S/bq2dD8yH5iXknrXQhfBx1vCMCYivNxrPpNxVE4r4ytn9eX71Th59dRP/9+Fupowq5J5LR3NG/14JXSPZC1BU30MkPbha8GKMCdAcpJ+x1i5IbpPiExpnraiswdI8zvrC6gqunVTUoU1Zwf2mrtZaFn20hxmPrmD2gg0cPF4HQPneY5TtOerJNZJJG9CKpIeYRZlMcx3N3wCHrLV3ujlpKosydaQAUTycsiLe23KQuYtKWbu9kr49c6isbjgHR4kAAAn7SURBVKA+2NT6vnQpaKSsDxF/iFaUyU2gPh94E9gAhCLRj621f3V6TzIDdXhgcSpAZICtcy6P+3xtA1WkinJt5ecF+PFlo3ns1U3sqqpt97pXDwsR6foSqp5nrX2L5rjX6SKlkxmak7vDuRlnjZWeFq12BkBdQ5CcrEx2RwjSoEk5EfFGWhVlihQ4Le2fIm7HWWOlp8UKtLWNTTywuMzxoaBJORHxQloF6mjlQEOTcgV5AXIDGdz17Domz1nGwrUVcZ9vV2UNx+sa6ZEbOylmV2WNJuVEJKnSKlA79VBDY8GP3DCeusYmDlc3tGaA/GjBBsdg7XS+XnkBLnxgOUdrG4my41XrOfyQwSEiXVdaBepYPdd4V9pFOp8BqmoaGFbYgwW3fpGHrx/fWhc62hDLVROKeHv2VLbOuZy3Z09VkBYRz6RV9bxYq/XiXWl31YQirLX8v5c3cvB4PQD983P5z2vGctHIQowxTCzufUIWSGensvmhDSKSWmkVqCH6ar14V9qt31HJn1bv5ODxegb2zuPuS0Zy5bgiMhzGOzp7qyoVURI5OaVdoI7G7dLvLfuP8dCScl7esJtTumdz31fP5JtfKCYnK/YO317oaK840aqAIpKeulSgjjU0su9ILY++tolnP9hBTlYGt08bwXe/NJSeLnf49kIivWIVURI5OaV1oHbqmYYHvCO1DTzxxqc89dZWgk2WG79QzD9NHUFhzxxX5/OqXZBYr1hFlEROTmkbqN30TGsbgvzvu9v42eubqaxu4IpxA7j7kpEMPrV7h87nRbti9YqjBXlttSVyckrbQB2tZ/rVcQNYsGYnjywtZ1dVLReMLOSeGaM4qyi/Q+eLJ1DHOk+0XnGsIJ+KGtUi4j++C9Ruhx+ceqYVlTV8+bEVlO89xriB+Tz4tXF8cXifmNf1avzXzc4tTr1iNw+Lzs48EZHU81WgdupRrtp2iOWl+08I3tEq5zUGLT//24l8+azTaK7SGptX47+J7Nxy17PrIp5Tk4UiJzdfBWqnHuUz721vrZAXCt7XTipqt4sLwPUlA/nPq8cSyIxv0aVX47+J7NyiyUIRicRXS8ijFV1qq6YhyNJP9jK2zZhzr9ws5l17NvOuGxd3kAbvdlxJ5Dwq7iQikcTcOKAjOrpxgNNuLU6yszK45YtD+N6Fw+jdPTvu6/mRloiLnJwS2jgglSINGzhtDNAtO5OlP7iwtWBSV6HJQhEJ56uhj0jDBt84ZxCBsNobOVkZ/NfVY7tckBYRicRXPWr4vEdpreXlDbt5aEk5DU2W7MwM6oNNFGk4QEROMr4L1ABvbz7A3EWlfLizipH9evDkzSVMO6Ov61Q7EZGuJGagNsY8DXwF2GetPSuZjfmoooq5i0p5c9MBigryePBr47h6QhGZsbZZcUkTdSKSjtz0qH8NPA78NpkNqapp4Gu/eJecQAb/cvkZ3HjuYBZ9tIcL5i33JLCqlrOIpKuYgdpau8IYMyTZDcnPC/DETZMYX1xAr9yA54FVtZxFJF35KuvjgpGF9GqpDR3v/oexqJaziKQrzwK1MWamMWaVMWbV/v37Ez6f14HVaRm2lmeLiN95FqittfOttSXW2pLCwsKEz+d1YNXybBFJV74a+mjLy8AayvaoaQiS2ZLiV5AXIDeQwV3PrmPynGUsXFvhSbtFRLwWM1AbY/4AvAuMMsbsNMb8XfKb5V2RpNCkZKiGSNBaAhmG4/WNHK5uwPL5RKWCtYj4ka+KMiVDPIWeigryeHv21CS3SESkvWhFmXw79OGVeCYflQEiIn7U5QN1PJOPygARET/q8oE60qRkIMMQyDxxWboyQETEr3xZlMlLTnsURvqdViiKiB91+clEEZF0kBY7vKiynYhIZL4I1KpsJyLizBeTiV4XYBIR6Up8EahV2U5ExJkvArUq24mIOPNFoFZlOxERZ76YTHTKddZEooiITwI1NAdrBWYRkfZ8MfQhIiLOFKhFRHxOgVpExOcUqEVEfE6BWkTE55JSPc8Ysx/Y1sG39wEOeNicztRV7qWr3AfoXvyoq9wHJHYvg621hZFeSEqgToQxZpVTqb9001XupavcB+he/Kir3Ack71409CEi4nMK1CIiPufHQD2/sxvgoa5yL13lPkD34kdd5T4gSffiuzFqERE5kR971CIi0oYCtYiIz3VaoDbGXGqMKTPGbDbGzI7weo4x5tmW11caY4akvpWxubiPW4wx+40x61r+/H1ntDMWY8zTxph9xpiPHF43xpifttznh8aYialuo1su7uUiY0xVm8/k31LdRreMMYOMMcuNMZ8YYz42xtwR4RjffzYu7yMtPhdjTK4x5n1jzPqWe/mPCMd4G7+stSn/A2QCnwKnA9nAeuDMsGNuBX7R8vPXgWc7o60e3MctwOOd3VYX93IBMBH4yOH1y4BXAAOcC6zs7DYncC8XAX/p7Ha6vJf+wMSWn3sC5RH+G/P9Z+PyPtLic2n5d+7R8nMAWAmcG3aMp/Grs3rU5wCbrbVbrLX1wB+BK8OOuRL4TcvPzwPTjDEmhW10w819pAVr7QrgUJRDrgR+a5u9BxQYY/qnpnXxcXEvacNau9tau6bl56PARiC8cLvvPxuX95EWWv6dj7X8NdDyJzwrw9P41VmBugjY0ebvO2n/obUeY61tBKqAU1PSOvfc3AfAtS1fSZ83xgxKTdM85/Ze08V5LV9dXzHGjOnsxrjR8vV5As09uLbS6rOJch+QJp+LMSbTGLMO2AcstdY6fiZexC9NJibf/wFDrLVnA0v5/CkrnWcNzXUVxgH/DSzs5PbEZIzpAbwA3GmtPdLZ7emoGPeRNp+LtTZorR0PDATOMcaclczrdVagrgDa9iwHtvwu4jHGmCwgHziYkta5F/M+rLUHrbV1LX99EpiUorZ5zc1nlhastUdCX12ttX8FAsaYPp3cLEfGmADNwe0Za+2CCIekxWcT6z7S7XMBsNZWAsuBS8Ne8jR+dVag/gAYYYwZaozJpnmw/aWwY14CvtXy83XAMtsyMu8jMe8jbKzwCprH5tLRS8DNLRkG5wJV1trdnd2ojjDGnBYaLzTGnEPz/w/81gkAmjM6gKeAjdbahx0O8/1n4+Y+0uVzMcYUGmMKWn7OA6YDpWGHeRq/OmVzW2ttozHmn4DFNGdOPG2t/dgY8xNglbX2JZo/1P81xmymeWLo653R1mhc3sftxpgrgEaa7+OWTmtwFMaYP9A8697HGLMTuI/mSRKstb8A/kpzdsFmoBr4due0NDYX93Id8D1jTCNQA3zdh52AkMnATcCGljFRgB8DxZBWn42b+0iXz6U/8BtjTCbND5PnrLV/SWb80hJyERGf02SiiIjPKVCLiPicArWIiM8pUIuI+JwCtYiIzylQi4j4nAK1iIjP/X8W5zmilIONHQAAAABJRU5ErkJggg==\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAArG0lEQVR4nO3deXhU1f3H8fdJiBBBCSKLBBGUxQUEBK02rmhdQePSuvxcaq1YxSouSLQu4BpFpXWpihbBrW5IQEGpNoqIoobFFVFBUYMKFMMaIAnn98fNxGEyy53Mncm94fN6njwNmTt3zvU+/d4z53zP9xhrLSIi4l9Zjd0AERGJT4FaRMTnFKhFRHxOgVpExOcUqEVEfK5ZOk668847265du6bj1CIiTdLcuXNXWmvbRXstLYG6a9eulJWVpePUIiJNkjFmaazXNPQhIuJzCtQiIj6nQC0i4nMK1CIiPqdALSLic2nJ+hAR2ZaUzC9nzIxFLKuopFNeLiOO6UVh/3zPzq9ALSKSgpL55Vz70idUVtUAUF5RybUvfQLgWbDW0IeISArGzFhUF6RDKqtqGDNjkWefoUAtIpKCZRWVSf29IRSoRURS0CkvN6m/N4QCtYgIzlhzQXEp3YqmUVBcSsn8clfvG3FML3Jzsrf6W25ONiOO6eVZ2zSZKCLbvFQmBEOvK+tDRCSN4k0Iugm4hf3zPQ3MkTT0ISLbvExMCKYiYaA2xvQyxiwI+1ljjBmegbaJiGREJiYEU5EwUFtrF1lr+1lr+wEDgA3A5HQ3TEQkUzIxIZiKZMeojwQWW2tjFrgWEQmaZCYEw5eLt87NwRio2FCVlknEEGOtdX+wMeOBedbaB6K8NhQYCtClS5cBS5cqlotI0xKZHRIpNyebO07p06BgbYyZa60dGO0115OJxpjtgBOBF6K9bq0dZ60daK0d2K5d1G2/RER8L14+dbTskHBeLx0PSWbo4zic3vTPnrdCRMQHEuVTu8kCSUemSDLpeWcC//a8BSIiPpGowJKbLJB0ZIq4CtTGmO2B3wEved4CERGfSJRPHS07JFy6MkVcDX1YazcAbT3/dBERH+mUl0t5lGAd6iVHZodkKutDS8hFRGqNOKZXvayOyF5yupeLR6NALSJSKxMFlhpCgVpEJExj9JgTUVEmEREvzJsHjzySllMrUIuIpOLrr+GMM2DAABg9GjZs8PwjFKhFRBrip59g2DDYay94+WW4/npYuBC2397zj9IYtYhsE8KLKaU0SbhmDYwZA2PHwqZNcOGFcOON0LGj942upUAtIk1CvEDc0K22Qucsr6gkt6aaM+dN46/vP0+b9avh9NPh1luhe/e0X5sCtYgkxbOeqcdtihWIAa56/iNqIiqFJtpqK3TOTZs2c/LnM7lq1lN0XrOcWbv14+9n/Ilzhp1CYffMXHdSZU7dGjhwoC0rK/P8vCLSuKKV+UyltKdXCopLo64ozMvNYVP1lrgV7/LzcqM+dAru+C89583impkT2WvFt3zcsTt3HvZHZnftV/e+2UWDPLuGeGVO1aMWEddS3QQ2XWLV6KiorIr7PgN1AX6r4ZBN33PvQ8P5zfef8k2bXbj0xGuYtufBWPNr/kUm91NUoBYR1/y6CWysGh3xGCByPKHTj9/S5uxb4fPZdG/VhuuPvoRn9z2a6uz6oTKT+ykqUIuIa7ECYuvcHAqKSxtt3DpWjY4WOVn8sqF+rzrbmK3GrDuuWcnw2c/w+0/eYENOc7jlFt494g9Mem0J1VGGTTK9n6LyqEXEtWhlPnOyDOs3V1NeUYnl1yGE8J1RvBa5CwvAHaf0IT8vF4MzfnzHKX24acg+UTetvecPfcnPy6V15VqK3hzPW48O5eTPSnl8wBBOv+ZpuP56hhT0rDsnOMGdsHNn8kGkHrWIuBataNGGzdX1eq3pHLeOleFxxyl9Yk7u1ctS6dWGHhPeoPO4+9lh43om9z6Cew8+m1U778Idp/TZ6nobO6MFlPUhIinqVjSt3lgvOGPA3xSf4PnnxcrwcJWFUV0Njz8Oo0bBsmX8dMhRjNjvdN5psUujpxoq60NE0iZRsX2vNWhC01qYPBmuuw4WLYKDDoJnn6XjIYfwZFpa6S2NUYtISqKNW6dzsi3WAyDmg+Gtt5zAfOqpkJ0NJSUwezYcckha2pcOCtQikpLC/vlRJ/LSNYTg+sHw0Udw3HFwxBFQXg7jx8PHH8NJJ0HtxGBQaOhDRFKWyUm3hLuwfPMN3HADPPMM5OU5BZSGDYPczOU9e81VoDbG5AGPAb1xcsT/ZK19L43tEhGJKeqDYflyp0jSww9Ds2YwcqTzk5fXKG30ktse9T+A16y1pxljtgO8L7gqIpJA1IJQ3XeEe++Fu++Gykq44AK46Sbo1Kmxm+uZhIHaGLMjcCjwRwBr7WZgc3qbJSKytcj86eX/W8On197KcR+8QPNf/udMFt52G/TK3IrBTHHTo94dWAE8bozpC8wFLrfWrg8/yBgzFBgK0KVLF6/bKSKNrLHLm4YKQhm7hSEL3+aqWU+xW8VPzN29HwNemw4HHJCxtmSam6yPZsB+wEPW2v7AeqAo8iBr7Thr7UBr7cB27dp53EwRaUyh3mwml4mHf3ZBcSnlv2zgsCVzeWXCcO57+W7Wb5fLub8fzWmn3dKkgzS461H/APxgrX2/9t8vEiVQi0jTFau86aipn8XdVSXVHnjoAdFz6eeMmTmR3373Md+17sBlQ67m5b0OxZqsulocTVnCQG2t/ckY870xppe1dhFwJPB5+psmIn4Rr95zqOZz5K4qDdn6KtKzT73B3dMe5YRFs1m5fWtuOuoinul3LFXZOUDmq9g1FrdZH38Fnq7N+FgCnJ++JomI37it9xwqxhT6PdprrgL1smUwejRPPfoYG3OaM7bgLB7bv5D1zX9NOMv3yTZgmeAqUFtrFwBRi4WISNMXrd5zLPFqbiTcYKCiAu66C/7+d6iuZvKBJ1E84FT+1zJvq8O83gbL77QyUUQSclveFH6tuZGoUFP4GHbXltncv2o2vSc+6ATrs86Cm28mZ3VzNrz0CURsCLAtDHeEU6AWEVciVwPG2ug2FETjvRa+w/dpn5ZyxTtP02ntSn7+7eF0eHAs9OtXF8grq2rqdmTZloY7wilQi2zDUsnMSFhzI85rY177goM/n82ImU/Q83/fsWCXnlw5+Eq+3/c3zK4N0uGBvsbaukC/rQVp0MYBItusWD3itG8zNWsWZWdexMDyhSzeqTN3HXouM3oeVFfRLj/OxGW0senGXojjFW0cICL1xMqNTtcWWnzyiVO4/5VX2G3HthQdcykv7Ps7arJ+LVlqiD62HRI5GRlrWy5ILg3Q71SPWmQb1aCdUhpi6VI47zzo2xdmzYLiYua8Oocp+59QL0gn+n4fuTlAvIdNU6JALdLERe7YHVr2HWtHFAtbHddgK1fClVdCz57w3HNw9dWwZAmMHMmQ33bfarOBvNychEE6WrZHxh42jUyBWqQJi1ejI9pOKSEp1fJYv96pC73HHvCPf8DZZzNj8iwKdjqWbne9V/cQKOyfz+yiQYw9vR+bqrfEPWWsXWOS3pYroDRGLdIEhSbYoo33hoYGQpNyiY5zPdZbVQWPPQY33ww//QSFhXDbbZRsah13HDna8EVIosnNaAtxcnOyOWLPdhQUlwZ+gjFEPWoRD8QaXmistoR60bGEhgZCvdpYOwi6GkLYssUZ2th7b7jkEujRA95919n1e++9E44jx/uMRBko0fZrPHVAPpPmljdKpb90UY9aJEV+yzyI10MNiRwaiFXLI+EQwhtvQFERzJ0LffrAK69QsktfxvznS5ZNmRa3RkgoQMc6Jj8v19V/v8iFOAXFpZnNZskA9ahFUtQYmQfxevCJesHRJuVc7+wdMncu/O53zs/KlfDEEzB/PiWd+nHt5E+36s3G6q2HHgJJf3YCbiYY/fQNyA31qEVSlOnMg0Q9+Hi92GxjtnqIhHqYblYZAvD113D99c5QR9u2MHYsXHwxNG9e9/7Ih5Yleurdhs3VdZOKrj7bpUTfDvz2DcgNBWqRFDV42KCBEi1UiTbBlpNlwEBVjRMuowWnqDt7h/z0kzNJ+OijsN12TrC++mpo3Xqrw2I9nCxOCl6odjXALxuqtmqDV0Ey1gRjqIee8YU+HtDQh0iKvP7qnkiiHny0CbZWLZrVBemQWMMz4cMCvxv1MosuHO6k2j36KAwdCosXwy231AvSEPvhlJ+XS8vm9fuF6Rgiinb94ZOSQcy9Vo9aJEVef3VPxE0PPrKH2q1oWtRzxVqSXVO5kfPnT+fS955jp8o1/HD0iXR+8B7o3j1u2+L1Zq94boGrNnghXg8909+AvKBALeIBL7+6J5Loq300boPTPa9+znHz/8OVs56m85rlvN21P3cddh6/7NmH2QmCNMR/aMXK1850gGzIf7/GpkAtEjAN6cEnDE7WwvTpPDp2GHuuXMrHHbsz8rjLmN21HwAmiV5vrIeWXwJkpr8BeUGBWiSAku3Bxw1O770HI0fCrFm0bJvPsBNHMn3PAqz5dQrLi16vnwJkJr8BeUGBWsSnvK6zXC84ff45FA6DKVOgY0d46CHm7XcspS9/gU1TrzdoAdIvFKhFfCitub7ffw+jRsGECdCqlVNAafhwaNmSkwCbk+OLXq/8ylWgNsZ8C6wFaoDqWLsQiIg30pLru2oV3HEH3H+/MyY9fDhcey3svPNWh6nX6z/J9KiPsNauTFtLRKSOp7m+GzbAffdBcTGsWQPnngujR8Nuu6XYSskUDX2I+EDkeHTe9jn8sqGq3nFJTepVV8P48U5QXrYMBg+G2293iidJoLhdmWiB/xhj5hpjhkY7wBgz1BhTZowpW7FihXctFGniohX3X7exmpzsrcsZuZ7UsxYmTYLeveGii6BrV2cLrJdfVpAOKLc96gJr7TJjTHvgdWPMF9bat8MPsNaOA8aBswu5x+0UaTIie88bNlfXG4+u2mLJy82hZfNmyU3qvfmmU3b0gw+c+tBTpsCQIXU7fEswuQrU1tpltf+73BgzGTgAeDv+u0QkUrRsjlhWV1ax4Kaj3Z14wQJnYvC112DXXZ0hj3PPhezoW21JsCQc+jDGtDTG7BD6HTga+DTdDRNpitwU9Q9xNR69ZAn83/9B//7w/vtw993w5Zdw/vkK0k2Imx51B2Cycb46NQOesda+ltZWiTRRbrM2Eo5HL1/u5D8//DA0a+b0pq+5BvLyvGmo+ErCQG2tXQL0zUBbRJq8WMWRXI9Hr10L99zj/FRWwgUXwE03QadOGWi9NBal50kgeb28OlNiFSYadeI+MdtfMr+csdM+5YiZk7ns3WfZacNqpvcq4MkTLuT0s4+iUEG6yVOglsDJ5FZKiR4IyT4wki1MVDL3e9659QGefHMiXVb/zLtd9uXOw87jo07OsMiCiOsO6gNM4jPWep9JN3DgQFtWVub5eUXA2WU61q7Vs4sGefY5kQ8EcHq/od1CEr2eEmthxgy+/NNf6fnj13zWfnfuPOw83u62X71Uu9B1p7U9knbGmLmxynNoKy4JnExtpZRod/G07T7+wQcwaBAcdxzNK9dx2ZARDP7j33l79wFR86FD190Yu6FLZmjoQwInU1spJXogeP7AWLQI/vY3Z1Vh+/Zw//2cW9GDpeuq474tdN1B3AtQ3FGPWgInU5vJxgr8ob8net218nJn09h99oEZM5zaHF9/DZdeyhUn9K53reHCr9uz9ojvKFBL4CTaZdoriR4IKT8wKiqc/OcePZza0MOGOTt833gj7LADUP9a83JzaLN9TtTrTscDLHxH8oLiUkrmlzf4XNJwmkwUicPrrA/AyX9+4AGnNnRFhbOy8OaboVu3tLc32XNpcjJz4k0mKlCLZEp1NTzxhLNA5Ycf4LjjnGDdN7X1ZOlKyctUdo044gVqTSaKpJu1ThW7666DhQvhgAPgySfh8MPrHZps0E1nTrkmJ/1DY9QiETwdl501CwoK4OSTYcsWJ6NjzpyYQTqyLvW1L30S9/PTmZKnyUn/UKCWQEn35FZDgmVUn3zi7Khy6KGwdCnzb7iLQ8+5n24fNKfgzjejnq8hQTedvd5MZddIYgrUEhieBdE4Uu6hLl0K553njDvPng3Fxbz8wkzOsn34bu3muO1uSNBNZ683U9k1kpjGqCUw0rIzd4QG91BXroTbboN//hOysmDECGenlTZtKC4uddXuhizkiVXkyater3Yk9wcFagmMTExuJR0s16+HsWPhrruc388/H0aNgs6dE7avvKKSguLSuonDI/Zsx6S55UkF3WSLPEkwKVBLYGRi6bjrHmpVFTz6qJP//PPPUFjo7PC9116u2234dSuu8opKJs0t59QB+bz5xYqkgq56vU2fArUERrq/5kP8HmrJ/HLufnUh/d5/naJ3nqLzqmXOZOHkyXDQQUm12wCRKxgqq2p484sVGclRVjnUYFGglsDI1Nf8aD3UkvnlvHz3RB7673j6/LyYhe268sfTbuKj3gdxU4suFEY5T3gwbJ2bQ4ucLCo2VMXsYUNmcpQzWc9bvKFALYGS6Gt+WnqKZWXkn3ER/1o8jx92bM8VJ1zJlL0PY0tWNlRWRw1ykcGworKK3Jxsxp7ej8L++TFX/WUiRzkTk7LiLaXniSf8ULzH8/S9r76C00+H/fdn92VfM/rICxl04SNM7j3ICdK1oqXvJUrza8wcZa04DB7XPWpjTDZQBpRbawenr0kSNH75Kh0rOF71/EfJteWnn5xJwkcfhebN4YYbOCNrf77aGLtfExnkEgXDxszWyFQ9b/FOMkMflwMLgR3T1BYJKL98lY4VHGusdffgWL0axoxx0u02b4aLLoIbboAOHRgWpZJcuPAgVzK/nCxjqIlS8Cz8uMbK1sjEpKx4y9XQhzGmM3AC8Fh6myNB5Jev0vF6hHFXF27c6ATnPfZwFq2ceKJTPOmBB6BDB+DXVXp5uTn13h4e5ELfLqIFab8EQ604DB63Peq/A9cAO8Q6wBgzFBgK0KVLl5QbJsHhl6/S0XqK4eo9OGpq4KmnnEL9330HRx/tlB3db7+o7w/1gONNWEb7dgGQbYyvgqFyr4MlYaA2xgwGlltr5xpjDo91nLV2HDAOnHrUXjVQ/M8vX6VDgeeq5z+KP+xgLUyb5uyu8umnMHAgjB8PRx7p+nNiBblY3yK2WKvAKA3mZuijADjRGPMt8CwwyBjzVFpbJYHip6/Shf3zuecPfWNnVLz7rrNIZcgQZ8jj+eedXb9dBulEVBpU0iGpHV5qe9RXJ8r60A4v0tgihydu7g5HPvkPmDoVOnZ0dlm54ALIqT/mnOrnavsqaQjt8CLbnLrhie+/d4Ly3yZCq1bOZOHll0PLlmn7XFCRJPGW9kyURpPWehOrVjkTg/ff74xJX3qpsxVW27bp/2yRBlCPWnzHy0Uy4UG32/aG+1fMYp8nH4I1a5wi/qNHQ1gmUqzPLlu6KunKdSKZoB61NIpYtS7AmYx0GyRDQXfzps384ePXGT77GTqsW8WPh/6OXR68F3r33urYMTMWxfzcyIp2GluWTFKPWnwn3mKYZHrXY177gsM/mcnVs55kj1XllOXvxbCTRvJj74HMrg3SJfPLGTX1Myoqq+KeK1rZURUqEj9QoJZGEa/UJ7gMkm++yT/vv5i+P37Fl2278OdTbuCN7geAMZjac0fLwkiGChWJHyhQi2teTsAlWkUIcYLk/PnOYpUZM+jQuj1XHz+cl/Y5YquKdqG85VgrBSNFK+Qffh6RxqQyp+KK1yVEwxfJxFIvSC5ZAmed5Szx/vBDuPtuPnjtXabtd8xWQTp8VaSbHnF+Xi7/d2CXRis7KpKIetTiSjoq5IXXzoi7BP3nn+HWW+GRR6BZMyfNbsQIyMvjRGBL8xYxe/rxhlgiJwsH7raTUvbElxSoxZV0VsiLuUik+47OYpV77nGWe//5z04BpU6d6r0/VkCNNcTSZvscbhqyz1bvU6Ei8SsF6iYgE4s30l0hb6sguWmT03s+5lZYsQJ+/3unR92zZ4POC1opKMGmQB1wmdpdJSMV8rZsgX//G66/Hr79FgYNguJi2H//lE6rnrIEnSYTAy7R3nxeSWuFPGvh1VedScKzz4a8PJgxA954I+UgLdIUqEcdcJncXSUtPdP334eRI2HmTNh9dz68/QGusL0oL91E6/dexxj4ZUMV2bVbWyWzalGkqVCgDji/7K6SSOQ4+uie2Rz19H3w0kvQvj088ABT9j+eopcXUVm1CWCrlYShjQAaa+NckcakoY+AG3FML9/n/4bnYLdfu5JL/30nh582iKrXZjgFkxYvhmHDuKv0G1eLU9IxtCPiZ+pRB1wQshrGzFhEztrVXDbnRc6fO5WsLVt4Yr/BPPDb08ndriMjvlpNYf9WSQ3XaGm3bEsUqJsAX2c1VFYyZMaT/GXOi+y4cT0l+xzOvQf/Hz/kdXReDxvKSFT/I5zfhnZE0kmBWjxXMr+ce6d/zkHvvMJV7z5D0ZqVlO4+kLsOO48v2nerd3xoKMNN/Q/w39COSLopUIunSub9wBvF4/hX6QR6/O975u/SiyuGXM0Hu/WhqiZ27fNlFZX1hnFa5+Yo60MEBWrx0ttvs8c5F1P43ed8vVNnLjr5Omb0OAiMIW+7ZrRs3izm0EZoKCPTwzjakkuCQIFaUvfxx07Z0enT2blVW0Ye+1de7HMUNWEV7VZXVrHgpqMTF2DKoEyt6hRJVcJAbYxpAbwNNK89/kVr7U3pbpj8yre9vm+/dYokPfUUtG4Nd97JWZv68M36LfUODe8xgz+yVNJREVAkHdz0qDcBg6y164wxOcA7xphXrbVz0tw2wae9vhUr4Lbb4KGHICvLKTlaVARt2nC5ix6zX7JUMrmqUyQVCQO1dXa/XVf7z5zaH+93xJWovOz1pdwzX7cOxo6FMWNg/Xr405+cMqSdO9cd4qcecyJBWdUp4moXcmNMNjAX6A48aK0dGeWYocBQgC5dugxYunSpx03dNnUrmhb1qWiAb4pPcH2eaGPDoe2nomVShAf1Lq2acd+6MvpOuN8p4n/yyU6Peq+9GnxdfhBrvFw7j0tjSHkXcmttDdDPGJMHTDbG9LbWfhpxzDhgHMDAgQPV4/aIV72+aD3z0E2KHE4JBbCNm6sYvHAWV816iq4VP7JywIHsXFJCSfNdGTNlEcsmLqlLoavYUOXr3nM0Qer9y7YtqawPa22FMeYt4Fjg0wSHN1mZnNzzqg50onHX8OGUMTMWMeDLMopmTqD3z4tZ2K4rfzxtFF/tdzAjmu+6VXvCCyf5Yvw8SX4ZLxeJx03WRzugqjZI5wJHAXemvWU+lenJPa96fW6WZy+rqISyMu58+EoOXvoRP+zYnuGDr2LK3odhTRZm9caEu3ora0LEe2561LsAE2vHqbOA5621r6S3Wf7VGCldXvT6Ei3P7rqqnBvmPAN3zmSflq0ZfeSFPN3veDY3y6k7JssYV7U4lDUh4i03WR8fA/0z0JZACGJKV2ioprKqpm4pdmgisd26VVw++9+c8dEMaNECbryR2UedybP/+ZbNEUE9/H3xhMbPfZv/LRIwWpmYpKCldEUO1dRYS25ONmftuSNdHnuQ38+exHY11Xz3+3PY/b47oUMHBgPVrXbgquc/qivYH2IhbrAOjZ/7Mv9bJKC0cUCS/FSov2R+OQXFpXQrmkZBcSkl88vrHRM5VNO8ejNnzX6Ryy48lvPeeobtTzuFZl8uYvfnJkCHDnXHFfbPZ0uM1M1QSp8B8nJzaLN9Tr19FDO1l6PItkA96iT5JaXLbY81NCSTtaWGUz57k+HvPE3nNSt4u2t/Dp30mLOhbAyxvj3k5+Uyu2hQ3PYFcYhIxK8UqBvADyldbic1O7VuwZ5zZ3LNzIn0WvkdH3XswYjjh/Nd3wOZHSdIQ2qpgUEbIhLxMwXqNEvXhJqrHuvs2Ux9/lraLviQJW06cclJRUzvVUDuds24w0WwTeXbg1f53yKiQJ1W6ZxQi9tj/ewzuO46mDqVtrvswoLrirls+/58v7Yq6cL7Df324JchIpGmwFWtj2QNHDjQlpWVeX7eoCkoLo0aTLON4Z4/9E0paEWrU7H7hv/x+LfT2G3ai9CqFYwcCZdfDi1bNvhzRCQzUq71IQ0Ta3iixtqUe9bhPdYNP/7MiPmTOf39qWQb4IornEL+bds26Nwi4i8K1GkUb9m2F6sZC3vmUfjqHJhwp1OC9NxzYfRo6NKlQefTAhURf1IedRpFy7kO1+BUtaoqeOQR6NED/vY3OPxwZzusxx9PKUhf+9InlFdUYvl1PD1abraIZJYCdRoV9s/njlP6kG1M1NeTTlWzFl54AfbZB/7yF9h9d3jnHZgyxflbCrRARcS/NPSRZqGhg5RT1UpLne2uPvzQCcpTp8LgwRDjIZDsMIYWqIj4lwJ1BrhJVYsZWOfPdwL0f/4Du+4KEybA2WdDduwhlYakBWqBioh/KT3PB6Kl2vVc+zP/WjKVXV+bAjvt5IxFX3KJU+EugVhpgfGWfmtbKpHGpfQ8nwsfH955/S/89d1nOWvBa9RkN3MWrlxzDbRu7epcJfPLY2aaxBvG0AIVEf9SoPaBZRWVtNq0gQs/mMyfP5xM8+rNPNv3GO4rOJMPbjvH9XlCveJYEg1j+KGGiYjUp0Dd2DZt4vLPpnPOf5+ibeUaXul1MPcceg7f7JRPvgcb2IaozoZIcClQp6jBi0S2bIFnnoEbbmD4t9/yXte+3H7oH/lklx6AE1iP2LMdBcWlKWduABprFgkw5VGnoEGLRKyF6dOhf3845xxo0wZmzODnSa+waq996wrwnzogn0lzy5M6d6yhjfy8XAVpkQBrsj3qVJdDu3l/0hvdzpnjpNrNnAl77MGHdzzIFVt6Ul66iU55X271GQXFpUlvoqvSoiJNU8JAbYzZFXgC6AhsAcZZa/+R7oalItXyosnunhKp3t+/+MLJ3pg8Gdq3hwceYMr+x1P08iIqqzZF/YyGLEBpSOaG6nuI+J+bHnU1cJW1dp4xZgdgrjHmdWvt52luW4Ml3dNt4PsTLhIpL4dRo2D8eKfU6M03O5XtWrXirgQ95oYuQEkmc0Mb0IoEQ8Ixamvtj9baebW/rwUWAr7+f3Gqy6Hdvj/WRrfXHdTBqQXdvTtMnAh//SssXgw33ODUiXbxGZnYRFf1PUSCIakxamNMV6A/8H6U14YCQwG6NLCCm1dSXQ7t9v2RQw1dW2Zx/8pZ9C78J6xe7Sz1vvlm6No16c/IxAIU1fcQCQbXgdoY0wqYBAy31q6JfN1aOw4YB84Scs9a6ELkOOsRe7Zj0tzyBk+qJTMpV9g/n8I+HZwaHKNGOcMdxx8Pd9wB++6b0mekewGK6nuIBIOr9DxjTA5OkH7aWvtSepuUnGgpcpPmlnPqAGfBSCjdLZk84lB50oTvt9aZIOzTBy68kE+zduT0s4opOOQqSmri767i+jPSKBPDKyKSuoRFmYwxBpgIrLLWDndz0kwWZWpIAaJkxMyKmDnTSbWbM4e1Xbtz7f5n8kq3A+rKjgaloJGyPkT8IV5RJjeB+mBgFvAJTnoewHXW2umx3pPOQB0ZWGIVIDLAN8UnJH2+8EAVraLcXsu/YcTMiQxaUkZl+47k3n4rh/7Uhe/Wbq53bq8eFiLS9KVUPc9a+w5O3Gt00dLJDBDtUeNmnDVRelp4VkTn1T9z5aynKPzsLdY2357bDz+f539zEqP2G8j3zy2Ien5NyomIFwK1MjFaOpmFesHa7ThronzpZRWV7LRhNZe++xxnz5/OlqwsHvnNqTx04GmsadGq7hyalBORdApUrY9YPVQLdZNyebk5tMjJ4ornFlBQXBq3Nkbc9LR16/jb3BeY+cifOW/eK0zqPYjDLxzHnYf/sS5Ih47VpJyIpFOgetSxeq6hseBkV9pFO19OTRUXLyqFPf7En5cv5z97FnDnwWezuO2uMdukovsikk6BCtSJco+TXToefj5jtzBk4SyunvUkXSp+gsMOgylT2NB8VzbOWARRxsPDP1tF90UkXQIVqBP1XJNdaVfYPx+sZeYDz3DB9HH0/nkxq3vsBc+Mh2OPBWMoDPtcP6Sy+aENIpJZgQrUEL/nmvSk3ocfUlhURGFpqbPM+8knaX3WWZAVfei+sXvNKqIksm0K1GRiIq4n9b78Ev7wBzjgAPj4Y/jHP5xSpGefHTNIe6lkfjkFxaV0K5qWcMIznIooiWybAtejjifhpN6PP8Lo0fDYY9CiBdx4I1x1Fey4Y8bamEqvWEWURLZNgQ7UscZr6wW81avhrrtg7FioroaLL4brr4cOHVydz6t2QWq1spWvLbJtCmygdtUz3bgRHnwQbr8dVq2CM8+EW26BPfZo2Pk8aFeiXnG8IK+ttkS2TYEdo447XltT45Qd7dkTrr4a9t8f5s1zdv2OEqQTns+rdhG799spLzfhZrl+qLgnIpnnux612+GHqD1Ta9n7w7eg72Xw2WdOgJ4wAQYlLozk1fivm51bYvWK3QyLNHbmiYhknq8Cdaxhg7Klq3jzixVbBe/I8dqBP3zGyLcmsn/5505P+oUX4NRT68qOJuLV+G8qO7dcoeJOIhKFrwJ1rB7l03O+q1sRGArepw7IZ9LccnZdtpgRbz/B777+gOWtdmL+34rpf9OVkJOT1Gd7Nf6bys4tmiwUkWh8FajjFV0KV1lVw+dzPuW1z0vY9ZUXWLfd9jx8zAXk31jEkN92b9Bne1WvI5XzaLJQRKJJuHFAQzR044BYu7WEy6tcw7D3nufcedNo3izL2eG7qAjaxt/6Kii0RFxk25TSxgGZFK1HGSqElLt5I38qm8JF70+iZdVGXt3vaAa/9Ag08o7nXtNkoYhE8lWgjjZscGT3PLLGP84lbz9N+/W/8Hr333DfkedzwUWDoYsCmog0fb4K1BDWo7TWydy47i/w1Vd81LU3lxRey4+9B2g4QES2Kb4L1AD897/OuHNZGeyzD0ydSt/Bg3nRZaqdiEhTkjBQG2PGA4OB5dba3mltzbx5ToB+/XVn7HnCBKeiXXZ2wre6oYk6EQkiN0vIJwDHprkdUFEBhxwCc+fCvffCokWU7HsUBWNmJl0ONJpEy7NFRPwqYY/aWvu2MaZr2luSlweTJ8NvfgOtW3teJD+VqnUiIo3JX0WZjj4aWrcGvC+Sr1rOIhJUngVqY8xQY0yZMaZsxYoVKZ/P68Aar2qdiIifeRaorbXjrLUDrbUD27Vrl/L5vA6srrfpEhHxGX8NfYTxMrCGsj0qq2rIrk3xy8vNoUVOFlc8tyDliUoRkXRKGKiNMf8G3gN6GWN+MMZckP5meVckPzzbA6DGWnKyDOs3V/PLhiplgIiI7/mqKFM6uCn0FJKfl8vsosSbDIiIeC1eUSbfDn14JZnJR2WAiIgfNflAnczkozJARMSPmnygjjYpmZNlyMneum6IMkBExK/8WZTJQ7F2XIn2N61QFBE/avKTiSIiQRCIHV5U2U5EJDpfBGqvCzCJiDQlvphM9LoAk4hIU+KLQK3KdiIisfkiUKuynYhIbL4I1KpsJyISmy8mE2PlOmsiUUTEJ4EanGCtwCwiUp8vhj5ERCQ2BWoREZ9ToBYR8TkFahERn1OgFhHxubRUzzPGrACWNvDtOwMrPWxOY2oq19JUrgN0LX7UVK4DUruW3ay17aK9kJZAnQpjTFmsUn9B01SupalcB+ha/KipXAek71o09CEi4nMK1CIiPufHQD2usRvgoaZyLU3lOkDX4kdN5TogTdfiuzFqERHZmh971CIiEkaBWkTE5xolUBtjjjXGLDLGfG2MKYryujHG3Ff7+sfGmP0ao51uuLiWw40xq40xC2p/bmyMdiZijBlvjFlujPk0xutBuieJriUo92RXY8ybxpiFxpjPjDGXRzkmEPfF5bUE5b60MMZ8YIz5qPZaRkc5xtv7Yq3N6A+QDSwGdge2Az4C9o445njgVcAABwLvZ7qdHl7L4cArjd1WF9dyKLAf8GmM1wNxT1xeS1DuyS7AfrW/7wB8GeD/r7i5lqDcFwO0qv09B3gfODCd96UxetQHAF9ba5dYazcDzwInRRxzEvCEdcwB8owxu2S6oS64uZZAsNa+DayKc0hQ7ombawkEa+2P1tp5tb+vBRYCkUXbA3FfXF5LINT+t15X+8+c2p/IrAxP70tjBOp84Puwf/9A/Rvm5hg/cNvOg2q/Jr1qjNknM03zXFDuiVuBuifGmK5Af5zeW7jA3Zc41wIBuS/GmGxjzAJgOfC6tTat96UxdngxUf4W+TRyc4wfuGnnPJw1/OuMMccDJUCPdDcsDYJyT9wI1D0xxrQCJgHDrbVrIl+O8hbf3pcE1xKY+2KtrQH6GWPygMnGmN7W2vA5EU/vS2P0qH8Adg37d2dgWQOO8YOE7bTWrgl9TbLWTgdyjDE7Z66JngnKPUkoSPfEGJODE9ietta+FOWQwNyXRNcSpPsSYq2tAN4Cjo14ydP70hiB+kOghzGmmzFmO+AMYGrEMVOBc2tnTg8EVltrf8x0Q11IeC3GmI7GGFP7+wE4/83/l/GWpi4o9yShoNyT2jb+C1horb03xmGBuC9uriVA96VdbU8aY0wucBTwRcRhnt6XjA99WGurjTGXAjNwsibGW2s/M8b8pfb1h4HpOLOmXwMbgPMz3U43XF7LacDFxphqoBI4w9ZOC/uJMebfOLPuOxtjfgBuwpkkCdQ9AVfXEoh7AhQA5wCf1I6HAlwHdIHA3Rc31xKU+7ILMNEYk43zMHneWvtKOmOYlpCLiPicViaKiPicArWIiM8pUIuI+JwCtYiIzylQi4j4nAK1iIjPKVCLiPjc/wPCCvjKMZGi9wAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": { - "tags": [], "needs_background": "light" - } + }, + "output_type": "display_data" } + ], + "source": [ + "plt.scatter(train_x,train_labels)\n", + "x = np.array([min(train_x),max(train_x)])\n", + "with torch.no_grad():\n", + " y = w.numpy()*x+b.numpy()\n", + "plt.plot(x,y,color='red')" ] }, { @@ -887,58 +847,27 @@ "id": "0giuwC9GHzi8" }, "source": [ - "## Вычислиения на GPU\r\n", - "\r\n", - "Для проведения вычислений на GPU PyTorch поддерживает перемещение тензоров на GPU и автоматическое построение вычислетельного графа там. Традиционный способ вычислений состоит в том, что вначале мы определяем доступное вычислительное устройство `device` (CPU или GPU), и затем перемещаем туда все необходимые тензоры по мере необходимости с помощью вызова `.to(device)`. Мы также можем заранее создавать тензоры на нужно устройстве, указывая параметр `device=...`. Такой код работает без изменений как на GPU, так и на CPU: " + "## Computations on GPU\n", + "\n", + "To use GPU for computations, PyTorch supports moving tensors to GPU and building computational graph for GPU. Traditionally, in the beginning of our code we define available computation device `device` (which is either `cpu` or `cuda`), and then move all tensors to this device using a call `.to(device)`. We can also create tensors on the specified device upfront, by passing the parameter `device=...` to tensor creation code. Such code works without changes both on CPU and GPU: " ] }, { "cell_type": "code", + "execution_count": 21, "metadata": { - "id": "HK7HPLz3Hyrl", "colab": { "base_uri": "https://localhost:8080/" }, + "id": "HK7HPLz3Hyrl", "outputId": "7e14cccb-d376-4e59-be66-4ab3f5c3f6f4" }, - "source": [ - "device = 'cuda' if torch.cuda.is_available() else 'cpu'\r\n", - "\r\n", - "print('Doing computations on '+device)\r\n", - "\r\n", - "### Changes here: indicate device\r\n", - "w = torch.tensor([100.0],requires_grad=True,dtype=torch.float32,device=device)\r\n", - "b = torch.zeros(size=(output_dim,),requires_grad=True,device=device)\r\n", - "\r\n", - "def f(x):\r\n", - " return torch.matmul(x,w) + b\r\n", - "\r\n", - "def compute_loss(labels, predictions):\r\n", - " return torch.mean(torch.square(labels - predictions))\r\n", - "\r\n", - "def train_on_batch(x, y):\r\n", - " predictions = f(x)\r\n", - " loss = compute_loss(y, predictions)\r\n", - " loss.backward()\r\n", - " w.data.sub_(learning_rate * w.grad)\r\n", - " b.data.sub_(learning_rate * b.grad)\r\n", - " w.grad.zero_()\r\n", - " b.grad.zero_()\r\n", - " return loss\r\n", - "\r\n", - "batch_size = 4\r\n", - "for epoch in range(10):\r\n", - " for i in range(0,len(features),batch_size):\r\n", - " ### Changes here: move data to required device\r\n", - " loss = train_on_batch(features[i:i+batch_size].view(-1,1).to(device),labels[i:i+batch_size].to(device))\r\n", - " print('Epoch %d: last batch loss = %.4f' % (epoch, float(loss)))" - ], - "execution_count": 195, "outputs": [ { + "name": "stdout", "output_type": "stream", "text": [ - "Doing computations on cuda\n", + "Doing computations on cpu\n", "Epoch 0: last batch loss = 94.5247\n", "Epoch 1: last batch loss = 9.3428\n", "Epoch 2: last batch loss = 1.4166\n", @@ -949,9 +878,40 @@ "Epoch 7: last batch loss = 0.3390\n", "Epoch 8: last batch loss = 0.3384\n", "Epoch 9: last batch loss = 0.3382\n" - ], - "name": "stdout" + ] } + ], + "source": [ + "device = 'cuda' if torch.cuda.is_available() else 'cpu'\n", + "\n", + "print('Doing computations on '+device)\n", + "\n", + "### Changes here: indicate device\n", + "w = torch.tensor([100.0],requires_grad=True,dtype=torch.float32,device=device)\n", + "b = torch.zeros(size=(output_dim,),requires_grad=True,device=device)\n", + "\n", + "def f(x):\n", + " return torch.matmul(x,w) + b\n", + "\n", + "def compute_loss(labels, predictions):\n", + " return torch.mean(torch.square(labels - predictions))\n", + "\n", + "def train_on_batch(x, y):\n", + " predictions = f(x)\n", + " loss = compute_loss(y, predictions)\n", + " loss.backward()\n", + " w.data.sub_(learning_rate * w.grad)\n", + " b.data.sub_(learning_rate * b.grad)\n", + " w.grad.zero_()\n", + " b.grad.zero_()\n", + " return loss\n", + "\n", + "batch_size = 4\n", + "for epoch in range(10):\n", + " for i in range(0,len(features),batch_size):\n", + " ### Changes here: move data to required device\n", + " loss = train_on_batch(features[i:i+batch_size].view(-1,1).to(device),labels[i:i+batch_size].to(device))\n", + " print('Epoch %d: last batch loss = %.4f' % (epoch, float(loss)))" ] }, { @@ -960,22 +920,24 @@ "id": "A10prCPowHl7" }, "source": [ - "## Пример 2: Задача классификации\n", + "## Example 2: Classification\n", "\n", - "Рассмотрим пример двухмерной задачи классификации на 2 класса. Примером такой задачи может быть классификация опухоли на 2 типа - доброкачественная и злокачественная, в зависимости от её размера и возраста.\n", + "Now we will consider binary classification problem. A good example of such a problem would be a tumour classification between malignant and benign based on it's size and age.\n", "\n", - "Сгенерируем тестовые данные случайным образом:\n" + "The core model is similar to regression, but we need to use different loss function. Let's start by generating sample data:\n" ] }, { "cell_type": "code", + "execution_count": 22, "metadata": { + "id": "j0OTPkGpwHl7", "scrolled": false, - "trusted": true, - "id": "j0OTPkGpwHl7" + "trusted": true }, + "outputs": [], "source": [ - "np.random.seed(0) # pick the seed for reproducability - change it to explore the effects of random variations\n", + "np.random.seed(0) # pick the seed for reproducibility - change it to explore the effects of random variations\n", "\n", "n = 100\n", "X, Y = make_classification(n_samples = n, n_features=2,\n", @@ -986,17 +948,17 @@ "split = [ 70*n//100, (15+70)*n//100 ]\n", "train_x, valid_x, test_x = np.split(X, split)\n", "train_labels, valid_labels, test_labels = np.split(Y, split)" - ], - "execution_count": 196, - "outputs": [] + ] }, { "cell_type": "code", + "execution_count": 23, "metadata": { + "id": "c-_BjSHPwHl8", "scrolled": false, - "trusted": true, - "id": "c-_BjSHPwHl8" + "trusted": true }, + "outputs": [], "source": [ "def plot_dataset(features, labels, W=None, b=None):\n", " # prepare the plot\n", @@ -1015,40 +977,45 @@ " ax.plot(cx,cy,'g')\n", " ax.set_ylim(min_y,max_y)\n", " fig.show()" - ], - "execution_count": 197, - "outputs": [] + ] }, { "cell_type": "code", + "execution_count": 24, "metadata": { - "scrolled": false, - "trusted": true, "colab": { "base_uri": "https://localhost:8080/", "height": 283 }, "id": "tq0vFchQwHl8", - "outputId": "919f1922-f789-4779-cbdc-4f9e742c358b" + "outputId": "919f1922-f789-4779-cbdc-4f9e742c358b", + "scrolled": false, + "trusted": true }, - "source": [ - "plot_dataset(train_x, train_labels)" - ], - "execution_count": 198, "outputs": [ { - "output_type": "display_data", + "name": "stderr", + "output_type": "stream", + "text": [ + "C:\\Users\\dmitryso\\AppData\\Local\\Temp/ipykernel_50828/2721537645.py:17: UserWarning: Matplotlib is currently using module://matplotlib_inline.backend_inline, which is a non-GUI backend, so cannot show the figure.\n", + " fig.show()\n" + ] + }, + { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEKCAYAAAASByJ7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO2dd5iU5dX/v2dmtrINdpfOsoBKV9AFsQRFNIgx1thiNJZojDH6mkSNKSbRn0l8TaKmGDUxRo2vFbEiYgsIEaWIIFUEgaXsLmzvOzP374/vPM6WqbvT93yua67dnXn2mTPzzJxz36eKMQaKoiiKYou3AIqiKEpioAZBURRFAaAGQVEURfGgBkFRFEUBoAZBURRF8aAGQVEURQEAOOItQF8oKioypaWl8RZDURQlqVizZs1BY0xx9/uT2iCUlpZi9erV8RZDURQlqRCRXb7uV5eRoiiKAkANgqIoqYgxQHs70NERb0mSiqR2GSmK0gecTmDjRuDNN4GdOwG7HZg+HZgzBxgzBhCJt4Th09YGrFoFLFoE7NvH+w47DDjjDOCoo/gaFb+oQVCiijFAYyN/5uQANt2TJgatrcCDDwKffALk5gKFhYDbDXz4IfD++8BZZwHnnptcRqGpCbj/fmDrVqCoCBg9mvdXVgL33QccfzxwzTWAQ9WeP/SdUaJCRwfwwQdcqFVUUK/k5wPz5wNf+QqQlRVvCfs5TzwBbNgAlJZ2VfrDh3PnsHAhlers2XETMWwefxz4/POeu5tBg4CBA4GVK4Fhw4BzzomfjAlOwqzXRGSUiLwnIptEZKOI3BRvmZTe0d4O/PnPwN//zt9LSnhzOIB//xv43e+AhoZ4S9mPqayktR41yvcOwOEAhg4FXnoJcLliL19vqKqiq2jkSN+vSYTGbvFi7o4UnySMQQDgBPAjY8wkALMAfF9EJsVZJqUXLFwIrF8PjB0L5OV5v58DBvC+8nLgscfiK2O/5uOP+TOQ/27AAKCmhrGFZGDDBvolA72mjAyuUD77LHZyJRkJ4zIyxuwHsN/ze4OIbAYwAsCmuAqmhEVTE/D228CIEf7dzyNHUidVVABDhsRWPgVAbS2Qlhb8OJuNFzRW1NRwlb9vH5CZCUydCkyYEFoguKkp9ICx7hD8kjAGoTMiUgpgOoAP4yuJEi7bt9MFHUjfiHAxt3EjMHgwvRJ2e3LFL5OavLzQ0jGNiU2wx+Wie+r11/mcmZn8EC1ezA/ID35A91YgCgpCc29Z2Q2KTxLOIIhIDoAFAP7HGFPv4/FrAVwLACUlJTGWTglGW1toit3lApYtA155hQtWhwM49ljg1FMZE1SiyLRpwLPPUjn6u1gtLcw+Gjs2+vIsWAC8+iqzgrpnAB06xKDTHXcE3k5OncodjbW68IX1mg4/PHKypxiJFEOAiKSBxuApY8yLvo4xxjxijCkzxpQVF/doxaHEmbw86plANDQAa9YAH30EpKcz0WXoUHoLfv1r4N13YyJq/2XYMODooxnM8XWx3G66bc48M/opmlVVwBtv8EPg67kKC7nKeP11733G0O3TeUdQUADMnQvs2uX7NTmdfE3nnqtppwFImHdGRATAowA2G2P+GG95lN5x2GFML21uBrKzez7ucgHLl/P3adNoEAB+R4cPZ8zvX//i7xMmxEzs/sfVVzNnf9s2pmValvzgQRaOfPWrVLDR5oMPuLIP5P8fOhRYsQI47TQGn5YsYcxABJg5k/ePGwdceCFXGx98wA9fQQFf06FDdJGdcw5w8snRf01JjJhgy7kYISInAngfwAYAbs/dPzXGLPL3P2VlZUab2yUe778PPPIIU027xxLKy4H33gPKyoCJE33/f1UVPRU/+lH0Ze3XtLcDq1dzhV5eTgU7dSqNwaRJsQnqPPAAA0+FhYGP27yZsQW7nXGFzEyuLioruYO47DIaMLebRu6tt1iTIAIceSSrr0eP1kCVBxFZY4wp635/wuwQjDHLAejVSgFOPBGorwdeeIGLv4ED+T2srfWmo44f7///CwsZcG5ooMtXiRLp6azePf54KlKR2CtMh4PPHQinE/j0U+4GOscN7Xa6v9rbWZQ2ciQ/WBMm6PaylySMQVBSBxHga1+jm3r5choBY2goLAUfKF3cZuM5rBigEgPi1VNk+nQGjwLFA3ftotHwl2mUns7MoUWLAq80lKCoQVCixrBhwAUX8Gbxl78AmzbRZe0Pt5sGxFcMQkkxpk/nhW5s9J0Oagw/MIcfHthoFRVx5eHvPEpIJFSWkZL6nHQSA86BqKwEjjlGv9f9gqws4PrrGfg9eLBrhlBbG/DFF/Q5jhsX+DzWtrKtLaripjpqEJSYMnEid/779/t+vKWFGYVnnBFbuZQ4MmUK8POf84OxaxewZw+wezeDTmefzQ9De3vgc7hcNAi6rewT6jJSYorDAdx8M/CHP3DxV1DAtjkdHVwg2mzADTdocVrCcegQW0s4HOxLEkrri3AYNw647Tb2M6mu5vOUlLD/0Cef0B0UKM5QWQnMmqVtdPuIGgQl5gwaBPziF8C6dUwpr6hgFuGZZzLwPHhwvCVUvmTXLuDFF6mQbTZvcOf004F58yJvGIYM6VmRPHmyd1s5bFjP/2lq4opi3rzIytIPSZg6hN6gdQiKEkW2bgXuvZdKv7jYG9RtaaFynjaNfYYibRR8UV3NbWV5eddt5aFD3E18//uciKaERMLXISiKkkC0tXGoRV5ez5SwrCz69NatA955h7uFaDNoEPsZffIJi84qK7mtPO881lEMGhT6uZqaaNDcbm5HCwqiJ3eSoQZBiQvxqoNSQmTdOirOoiLfj4uwpcQbb7AjYSz6A2VksDht5sze/X9DA/Dyy8DSpV37IM2cyR5H2otdDYISO+rr2ZLmrbfoAcjMZMxgzhzGKZUE4uOPgwdos7OZCXDgAKuEE5n6enZN3b+fjbIsN5fLxU6Ln34K/PSnfKwfo2mnSkzYu5eB5Oee42Jy9Gi2qFi6lPevXBlvCZUutLeHNnBGJDnGbD7zDLMXRo/uGvOw27kaMQZ46KHgrXpTHDUIStRpbWU80OXi9zE7m3okLY3fxcGDgYcfZhqqkiCUlASfluZyUYEOHBgbmXpLbS1XHIFW/0VFrH9IlpGhUUINghJ11q6li8ifOzozk+1oliyJrVxh4XJxbu8//gHcdx/w5JPsppmqK8rjjuNrC7T6r6hg7n+gPiSJgLXSCLTjsYJZ27ZFXZxERmMIStRZtowzEgIxeDDw4YfAt7/N2GFCUVHBNs3WrN/0dPbXeecd5sh/73up12djyBDOGVi8mNu67sq0poZK9Otfj4984RCqS8tmC220aAqjBkGJOrW1wZW83c4FaUtLghmE+nrgnnvo9yot7fqYMezT/6c/AbfemnqTuC66iEryzTep/LOyqFxbWxkAuv1234ViiUZhobdjYqC0Nrc7OV5PFEmxT7CSiAwcyKByoDYzTqdX5yQUy5ZxNTx6dM/HRFhBu3UrdwxHHhl7+aKJ3Q5cfDHTSleu5EXMyGCH0smTk8cAjh7N61Rb6z/e0drK1zZ1amxlSzCS5IoqycycOcCDD/qvHWpsZNq7zcaF9ogRHNo1ZUqcdY7bzcBGoF4aInQXvfVW6hkEi6Ii9hVJVkSAb30L+O1vmcnQ3b3X1kZj953vJNj2NPZoUFmJOkcdRZ1aWdnzsV276JHYvZuT1LKzmexx333A//4vjUXcaG1lMVOwbUtuLuMLSuIyfjzwwx/SJ7lrF2snKioYcD54ELjySmD27HhLGXd0h6BEnYwMfhfvvZffv8JCKv59+zh/ecAA7iKGDvUeP2gQR+0+9BBnK8elotnanrjdgYezuFwMNCuJzZFHAn/8I9tfbNrE6zp2LAd8p1pSQC9Rg6DEhKFDgTvv5LTEt97i4mzzZg7CmjKFRqEzlnv+009pROLSDjs9ncLt2BG49fKhQ+zb3wusrFVt4REj+tr+IsVRg6DEjAEDgJNP5q2qivGCkhL/ytAqXvvgg14ahMZGYONGZgoNGABMmhR+I7PTT2eW0cCBvgMaLS3cPZxwQsindDpZ0vDGG9wFiXBw0Lx5jNXGa7yxoqhBUOJCYyOTWIKtjDMz6eINC6eTPfyXLOHvFjYbZ3hefHHowcNJk7j6f+kl+rry8ym0MRSssZF1CP6q7rrR1gb87W8s1svP97YA2rmTLrXZs4ErrkieBB4ltdCPnRIXMjO9nQ8CGYX29jALYd1u4LHHGJwoKemqWV0u4L336OK58cbQtK4IWyyXlACvvMLot83G57GMxRFHhCzes88yo2rMmK6vu7iY9mbZMv4899wwXrOiRAg1CEpcGDqUrWXq6wNXMbe3A8ceG8aJt24Fli9nEVl334vdzpz0devYzXPGjNDOKcJjy8qYKtXWxiBkOD34wTT4//yHsRFfRtBm445h8WJg/nwaTUWJJeqtVOKCCHDWWYwldPbqdMbqqjx+fBgnfvttpjD5c8SLMB6waFHYMkOELR1KSsI2BoA3sSVQS530dBrBLVvCF09R+ooaBCVuzJxJb8zu3Vx4u928v6mJmUW5ucBNN4UZZN26NXj3zYICOu1j3La5oSH0bKKWlujKoii+SCiXkYj8E8CZACqNMVPiLY8SXUToK584kcVpn3zC+wsKgG9+kxk3tbVAXR3dSyG1tQhV48YhzzMvL/TmqIHafChKtEgogwDgXwD+AuCJOMuhxAgr5XLiRC7YnU4agZdf5kwTK6HH4QBOOYUdFALWEE2ezBiBVeXmi+pqFkCEMgAmgkyezKd0Ov3Hs9vamAAVlptMUSJEQrmMjDHLAFTHWw4lPtjtTAC66y4WsI0YwQBsSQmzOhcvZklAQ0OAk8ydS3+LP3eQMdxyzJ8fldcQiLw8GrU9e3zvFNxuoLwc+NrXNKCsxIeEMghK/8YY5ugbQ2PQeQGfns7Eob17geefD3CSww5jH/8vvuByuzMdHYwdzJoVt0Z0F17I2MnOnd6AutPJyu1du2jPzjgjLqIpSsK5jIIiItcCuBYASkpK4iyNEkk+/5wr5ECXdfhwYMUK4Bvf8FOfIMIAxMCBwKuvdjUKaWlMbTr77Ji7izqLcN11LEB7803gs88o8qRJtGMTJmgbCyV+JJ1BMMY8AuARACgrK0vR+YX9EyvVMpBCdDi4g/jiiwCLfJuNfpdTT+VJGxsZpR0/PiGitXY72+7389b7SgKSdAZBSV06OkJfuPurXehCRgZ7byuKEhIJFUMQkacBfABgvIiUi8jV8ZZJiR3DhwdX9MYw+FpYGBuZFKU/kVA7BGPMJfGWQYkM9fXeRqO5uUy5DNSiAqALKCPDm3oJAHZXO4bsX4fDPn8TefXlaOlw4OCYGShxnwygNMqvQlH6FwllEJTkp6ODWUDvvMPMT6uOwGYDvlpWjfOG/Rfp2zfz4IkTgeOP/7INRFYW57o/9hjTTXPQiGM/uB+F1Z+hNbMANWmD0druwpyMFZBfvceD58/XKKyiRAg1CErEcLmAv/8d+PBDKvQvi6+MQen2tzD8vmewbSAw6dhctqPYvJltqi+6iEOURTBnDt1Gzz1rcOT6h5HZuBPl2aVwtgnS0oCZJ9pRMGQELc/TT3M2Z1lZPF+2oqQMahCUiLFpE7ByZc/WziW738f0DU+idsgobKtPR2E7MGwYmBra3g78+9/cHsyeDRHahuNH7EbDDzdg/9DRyLYJiotZfPylkUlLY7XaSy8BxxyjuwRFiQAJFVRWkpslSxgv6Kybba4OTN74PBpyh8M40pGZySlhX5Kezmjyc89x1e8hZ/MqDBtpx9HHCKZPZ9fTHu0e8vI4mFkH3CtKRFCDoESMbdt6NhotOrgF6e0NcKaxM11WFlsJdWndkJXFWoHNm733VVcH798gwuBEU1PfhW9sZN+M1ta+nyuWVFez7HnfPm+7WEXpJeoyUiKGzdazR09WSzXS25swuGID0tsb4LKlQWwjANcQwNGt6KC21vt7fn7P1hPdMYaBi74Um23eDLz+Ov1dIrwdfzwHHI8Y0fvzRpvPPwcWLmQql/XGFxUBX/865zvrYGalF6hBUCLGpEnUq0OG8G+HsxVHbH0FQw98jLaMPLhtaXB2uDHJsRfydiZ7CnUeep+W5v195kxOoQ80Y7OhgU/WW8W9eDHwf/9HP5c1xszpZCDkv/8FfvhDvqhEY+1a4M9/piEcNcqr/BsagEceAXbsAC67TI2CEjb6iVHC4uBB4LXX2ITu0Uepm9rb+dhpp9F743YDMAbT1/wduQ170Z6eg/b0XHQ4stAiA5A9vICKfsWKTv8AtqS2KC1lYx9/8QGnk93hzjmndwHlrVuZpTRqFAcaW+dwOBjTyM8H/vQndkZNJOrqgIceoszFxV2Vfm4uI/rvvAOsWRM/GZWkRXcISki4XMALL3BRLcLFqcvFWfb5+cAPfsBWQaecArz7LjA1eweG71+D6sLxyGyrx4D6/ahHPvLzgQEDAEgWq9a2b2fZ8YwZdHlYiADf+x7wxz/SR15YSIXndtMQtLRw3FpYA5c78eabfBGddyUADdWhQ3yOigrm0V53XZAhDDHkww9pDP25yWw274jQsjLNvlLCQg1CClBVxWZvbjfT8ktLI68Hnn+eOmb06J79hurqgP/9X+AXv6CnorAQqP7jMtS1pKMWghr7UTjc2YiitFoMyM6B9bFz2jNg1m1Ey/zzkP3Ny3t+GPPzgdtvB1avpvto924+eVkZ+0QfdljvXmhbG/Dxx9wddH8hq1YxwGyzMevpySfZIO+882JfBFdfT3k++ojbsJISYMMGP21eO1FQwF7aDQ3Bj1WUTqhBSGIOHmQKvzV6UoRGYeRI4PLLgSOOiNzzLF7s2xgA1NstLSwJ+MEPGNfsWF2OA5tysP4LoL4lA58POxGNTTswuPxzHNrTBIcDaHc50Jxegservov2X+Vi/nzq+S6L9sxM4MQTebMi1n1Vyu3t3gwli8ZGYPly3m/FNTo6+JxDhtC95HZzZFssWLeOfrn2dr7BdjvwwQeMbwwezPiLw0GlX1/P/8nJoQGwguMhdQBUFC9qEJKUQ4eA3/yGeqxzXNEYJuv89rfALbdEJia6ciX1S6BOpIMHc9FdU0OPRYdkYOd2F9w2FpSJpKO1aAK2NR2OAztb4GwDikZmYnTaXhSNyYVUb8MX/+9DvP9wDWafPRCOE2b13AFEanWelcX6h/Z2/gS4C3C5uq6oOzqojNPSuDp/8UUaps6B8GiwfTvwwAN0oQ0Y4L0/N5dWeetW73tRVcWfVo+QgQN50dPSeLwSHk4nd2GLF3t3pEcfDcyZE52td4KhQeUk5bnnuDgcPrzrQleEOmHgQODhhyOzSNyzJ3hmp83GW7VnAOp/mmfC1ljfpVDN7QbK99vhzMyBLS8HTZWNaEgvQtlHf8Npq3+Dsub34dz8GfY9swy4+25avGgEdR0ObkUOHODfra0cxdY9TtDeDowdy9/T0qhwP/oo8vJ058UX+YZ3NgYWpaWUf+1aoLKSBquggD/z8xmkf/ttDlvoHh9RAtPcDPzhD8D99zOZobCQC4SVK4Ff/5oDl3zNPk0h1CAkITU1dC0HmiOfl0ddunFj358vPT20mie32+vFePXADKTnZSK9vfHLx5uavAPm7eJGfkcV9h1MQ379btQVlKIxbziksBAba0bAPWo0AyN/+EN0isXmzKE7qqaGiqC7C6m+ngp28GDvfdnZDHBHk4MHWRvROcDemUGDuJOxZm/62kFlZHAUW4orr4jzz39yp1haSmPgcHgr6UeOZCBt5cp4SxlVwjYIIjJAROIzf1ABwMUsEHyYjMPRrU1EL5k+nTGCQLS0cEE7fDiTc9ocA7Dq2BuQ2VKDnMYDEOP+Mlab7azHsNad2JAxA1vaxqAxz1tHkJbGhXlDo7C+YNcuroYjTVERcOut/H3vXhqd9nZardpaWtTjjuv6JlsWL5rU1fFN8ueaaGykIcvP57H19TRojY2UOy2Nk+JqayNz8fsL+/YxVbekxPd773BwcbBwYUpXhAc1CCJiE5FvisjrIlIJYAuA/SKySUTuFZHDoi+m0plAtVqdsYLMfWXqVK/+8SfP/v0s7rW8FCJA1eDJWHrSHagYPAV59eUY2FSOYR274LSl45Vh38VHOBbt6b7TOb+Ue+BA+nOjQWkp8LvfATfc4A3GFhXREMye3bN1RktL9OdepqcHXtnX11PO4mLGCsaO5Wp2+HDKPXeuNw6ya1d0ZU0l1q71BuP9kZvLmM3u3bGTK8aEstx5D8DbAG4H8Kkxxg0AIjIIwBwA94jIQmPMv6MnptKZoUOpMN3uwMWoHR3UeX0lLQ248UbgnnuoEwcP9j5vSwuNwZFHskupJZ+V5FJXMBqrZt6AjNY6lG+qx9YdaXAVDoaBIG/3Cgwb2rUPkaULv9TFVgO7aJGVRffRLbfQR+zvDWts5BYo2iM5hw9nTKCpyXcMwVoNuFwMund2afk6VgmN6mpvgkEgItU7K0EJxWV0qjHmLmPMessYAIAxptoYs8AYcz6AZ6MnotKd4mIq4IoK/8c0N9Pl7XcQfZiMGwfccQcnn5WX87Z7N5/nwgtpMKzvU04OF9id9XhbZj6yx4/CQcdQuIwNHU6Bw+7C+OzyLs/T1ESDkpXlucPp9BkcdbkivHM/4wyutnft6tJ1FcbQr3/oEHD99d5RbtHCbmfe7oEDvl9gTg6tcHa2/ziDRSL3Yko08vO7Xnd/GNPpw5l6BN0hGGM6RGQCgBEAPjTGfBklFJHTjTGLjTEhvJNKJLnoIuCuu7iDLSrqutNtaqKx+MEPIqu/Ro7kOWtquKCyujz4SmY56yxm7+3dy9kHNhsXvOPGMWbqcABnT6jFoOZ9aEwfDoD1Ym43O1Z8SVUVm7V5Hl+1ijVqe/fynFOncmcyaVIfMwKzsoAf/5g+4v/8hxYHoECHHQZcfDF/xoLZs9m8bulSXlzLndXeTpdRXh6LTPxtD+vr6UYaPz428qYCRx/Nax/IH9vUxN3b6NGxlS2GiAmyrRSRGwF8H8BmANMA3GSMednz2FpjzNFRl9IPZWVlZvXq1fF6+rhTXs7OCpar2EpFz89nxXC8B4nV1ABPPMEaKwur84TdDkzK2olzN/waNVkj0OZOQ0YGe9oVFnoO7uig5v/1r9EwaDTuu496ctAg6kRr8d7YyPjFJZdEqJ9bUxO3P04nhRk2LPb55243M1oWLfJaP4eD7q0jjmDRWnZ2z37j9fV843/8Y27nlNAwhm1SNm/myqc7Lhe/aFddBZx8cszFizQissYY00NDhGIQNgA4zhjTKCKlAF4A8KQx5gER+dgYMz0aAodCfzcIAD/HO3eywaXLRd01aVL0k2HCobLSK19hIXvYHTzIwltZ/AYmfPw08scMRNG4AjjSxFtdV10NXHopzLzTcd99TKHt3m0C4Hm/+AK48kr2UkoprPfC5WJQ09ry7djB1cCBA11XtYMGUWmpMQifhgb4XHVUVdE3evrp3CmmQHFaXwzCRmPM5E5/54BGYROAU4wx0yItbKioQUgBjOEW4qWXuCq32bg6Li0Fzj4bmDYN5eXAz3/Onbq/72JzMz0qv/99YhnDqOJ2M7V0927+PmIE/W3B8pEV/7S1eXtn7d3LD9y0aWzlO2FCShgDwL9BCOWrUyEi04wx6wDAs1M4E8A/AUQ5B09JeURY6DBtGrcSVjR88OAvv3zr1gVOzQf4LwcPcqcQK1d/3LHZ6D6KVNMqhTuwE07gze0OnoqaYoRiEC4H0KUBgjHGCeByEXk4KlIp/Q8R72SdbtTVhdaFQSR4AZ2ihEw/HDAUSpZReYDHVkRWHEXpycCBoWUEut2+U/eTjpoaNrBra6Mfe+LE4POlFSUC9Bdvq5LEHH00m/kFyghsbGTAOhKFeHGjuZkjPf/73641CJmZnAw3b16/cl8osUcNgpKQGMPEGrudhWqzZnFYmK9WM04n6y6uuy6Jd/ltbcxw2b6daY+dA8NtbcBTT9HqnX++GgUlaoRsEEREAFwKYKwx5k4RKQEw1BgTsX7AInI6gAcA2AH8wxjzu0idW0kO6uuZjvrGG8y2dDhYmzB7NvXhhg3MvszP5yL64EFmF11wAXD88fGWvg+sWEE30ZgxPRV+RgZTrF57jZbRV568okSAcHYIDwJwAzgFwJ0AGgAsADAjEoJ4Oqj+FcBpAMoBrBKRV4wxmyJxfiXxOXCAozhratieo7SUu4Q1a6gvL7mE2X9vvslsIrud/dxOOSVwSmrC43azAK1TZlUPHA7eli4FLr00tvIp/YZwDMKxxpijReRjADDG1IhICN2gQmYmgO3GmB0AICLPADgbrHdQUpyODhaKtrV17Qxgt7PYrr2d7vWf/MTbtTplaGxkEV5JSeDjCgpYSasoUSIcj2uHZxVvAEBEisEdQ6QYAWBPp7/LPfd1QUSuFZHVIrK6yhofqCQcLhdX+jU13rZAgfj0U5YhFBf7fjw9nQk3r70WWTkVRfESzg7hTwAWAhgsIncD+AaAn0dFqgAYYx4B8AjASuVYP78SmNZW4P336QGx5ifk5wPz5/seMWCxfHnwlNHCQg60qqvjOVOGnBy2Smhs7DnGszO1tZx3kAg0NbGid8MGWvxx4+i/+7IRVQRoaGDDLrebNSrBursqfSYkg+AJKC8DsAbAXAAC4BxjTCT3r3sBdO5UM9Jzn5IkNDXR7fPZZ/z+Wh6Qpibg3/9mltAPf+hb8dfXB+/MahWNtrSkmEGw2YCvfQ147DG+Ob7iCE4nFe9XvhJ7+bqzZg3wyCP07+XkUP5PPgEWLGB67Fln9S2gU1fHc63oVObkdnMWxYUXssWuEhVCchkZNjxaZIzZYoz5qzHmLxE2BgCwCsDhIjLGE5u4GMArEX4OJYo8/TQb7Y0d21XpDxjA+3buZPakLwYNCl5l7HYzHbUvxWetrRxt0NDQ+3NEheOPZ6+c3bt7+tja2thp88wz459htHEj8Oc/0yKXlnLVPmgQrf+IEcALLzBFrLfU1gK/+Q23jMOGsZuhdduyBbjzTmDPnuDnUXpFOC6jtSIywxizKhqCGGOcInIDgDfBtNN/GmMiMAuADNAAACAASURBVCJeiQU1NaynCjSTZcQIppRecEHPrs2zZtHN1NLCZJqiop7upaoqtjzKzQ1fvv37mZ20YgX1rTEcF/C1rwFTpiRAhlJGBnDzzbSqy5d7C9NEOKvhW9/yjqSLF8ZQvvx8No/qjsNBw7BwIXDSSb2z3M8+y1zi7gF2m40FKYcOAQ89xGEgSVt0kriElWUE4FIR2QWgCXQbGWNMhGZyAcaYRQAWRep8SuzYupU6LFCjTbudOmXrVhoAgP+zZAl1yJ49fCw7m3qwpITKOi2NhqKlhQo8XLZtYxdUY+jKcjj4+969wL33starr16OsLDmjhrDKLo1Azkri62rzzuPb0R7O63fxInRn9QWCrt3800LlA2VlsaUsY8/Bk48Mbzz19TQrxhoVVFYyN3S9u3a1C8KhGMQ5kVNiiTDGCqvigqv4go02rY/0N4e/rHG0FX8yiv0hMybxx1EfT31ys6d1BFjx9KYfO97jF2GQ1MT8MADdHVbehfgdSss5GJ3wQKed8qUEE/qcvHEDofvlbI/GhuZJvXee4wJWMyaRd+7lWJVUAAce2zo540VNTXB284CvHgHDoR/fmvSU7D23SK08moQIk7IBsEYsyuagiQLO3dyCtjOnd4gpzGcXfytb/lPm0x1OivbQIh4jy0vB15/na5oSwfMnk1d8vnn3krkuXPZlsJPM9SArFrFFkH+rovDQXkWLQrBINTVsTBsyRKe1BhO+znjDAY8AynKhgZW3e3Zw6CoNYDa5eKqeP164Kc/pd88UbG2VsFwuXrXjM9qNx0Mmy20XGYlbMJpXXGHr/uNMXdGTpzEZvt24He/486+c2Ws2816obvvBn72s/5pFCZO5GK5tdW/Lmht5Xs3cSL/XrqUi8nOC0KHg7sFK3ba0MDFeG/f0xUrgmckFRby+gXM+qys5MWvqfGmQBpD188f/kD//qWX+vdrP/cc3S3du+/Z7XyxlZX0jf/qVwkQ0PDDmDGUt6PDfz9yY/iFsC5yOBQXewM8gd4Dp5Nb8i1b+OHIzuYQjFB6pCsBCcdl1NTp90wAZ4JzlvsFLhfw8MNUGAUFXR+z2bjo27eP1bQ33RQfGeNJRgZd348/TmPZfWqZ00mPwDHHMLvS7WbsNFhqeW4uXdfNzYFT9P3R0hJcT4jwGvp1e7nd9Du1tHQto7b8TgMHctcwejS3ON2pqwsecS8u5htkpWklIgMGMFj89tv+28pWVfF96M1rGDmSRufgQf/1DM3NDCw/8QSzryyys5mF9dWvarC5D4TjMvpD579F5PdgRlC/YOtWftYDtVceOpTTvQ4e7J81NHPncpX98sv8TlqGs7aWC2mnkxXJVvLJxo3UtdOn++7pFgmGDgU2bQqc8NLRwYWv32O2bvW9urew2bhife01BlK7K6SdO7nqDeQbt/yPW7dGxiBYw7bffZd1A04nUzdPP52pWum97Dpz/vlsJPXZZ3RvWdvBjg76+nJyGOzpzcUUod/1N7+hFe/uh2xpoWEdMICPZWV5H2ttZU5zZSVw2WWJu8tKcPrS/jobLB7rF+zYEXzhYcXb9uwJbBA6Orjbra/n9/KII1Kj0EqEsdFjj+Xqf5OnC9XgwXytY8d2/Q6PG8dYwbp11JWdF98WDQ18L8OJ3XZmzhwW1AbyQlRUACefHCCR56OPgitQayuzb1/PWoHqan4oNm3iqjYjg2/GqFFd3xC7PbzovD+MoXFasIByFxfzw3noEPDXvzLu8T//07stV1YW8KMfAW+9xTzeqiqvMTvpJKaB9aVaedw44JZbgAcfpOFJT+e529poEHJygLKynlvQzEyuKt5+m49PmtR7Gfox4cQQNsDTxwisEygGcFc0hEpEgrk1Lawgs79zLF3K2p2mJu85RViAevHFXfVDsjJsGGsNAO4AbruNmVjdX9vo0TS0AwZw59B9DIAx1DdXXtl7L8CECdR/O3bw/N2vYV0dn/O00wKcpLExtBW1zdbVjQFwJf3oo1RuhYV8UYcOccuUnc1WrVaAxOmMTFD5ww8Zs+juuxs4kNu2HTtYaXzzzb1bSWdlMU/39NO5Ine7+doiNa5uwgTGZTZu5I7J5eLu7KOPuJLqbgwsbDYajLfeUoPQS8LZIZzZ6XcngArPbOV+wahRwRMbrKEuQ4f6fvy11/g9HT686w7C5QKWLaOO+NGPEiPlPFJ8/jldaL5W/3l5VNZbt/K9q6z06kOXi1lIEyb0bc6B3Q7ceCPwpz9RN2dlUW91dHDXkpPDBWnADKYhQ5hXHwjr4neumquupmIrLqby//xzCmSz8fjmZuD55+n7HjKEF/7IPpb1uN3ASy9xW+ZLcYrww7x+Pd/gUaN6HhMq6enRq5xOS6Nra9o0731PPBF891FYyP5Koa7glC6Es+663hizy3Pb66ksvidqkiUYkyfzu97U5P+YqioqMF+tVg4c4A7e10rZbuf9W7bQ1ZJKHDrk/3spwoXckUfSU7JjBxfSX3xBl/1XvsJFbF8NZG6ut2321Kk0CMOGsQbsnnuYoBKQY4/l6j1QymV1Na1b54KU5cv5whoaqPxtNiq69HS+KGtF/e67XA1ffHHfZyeXl9MHFqicW4QfutWr+/ZcscbtDr5VFOk6flQJi3B2CKcBuK3bffN93JeSpKVRgdx/Pxd83d2v1dVcIH7zm77/f/lyrz7whQh1yaJF9HunSqJEsNR1ESpky202YQL15YQJPbO5fFFdzZvDwSQef++v3U7j0ytPwsiR9EuvXu17Ek9LC7cb3/9+1/vffZd9fpYu5cXNy+M2sKOj6wVuaOAH6qSTeiFcNyzDE4yMDL5xycS4cTR4gXYJdXXRy1DoBwQ1CCLyPQDXAxgrIus7PZQL4L/REiwRmT6dK9bHHuMq1mbzLkaGD2cnT39V/Z9+2rN/T3dycph5WF8fmjJMBqyEGWs+si+sBJwzzgjdA7F7N9tdfPKJ9zpkZ9Ot/dWv9j6JxiciwNVXU5GvW8cnys31Dn2w2ZhZM2FC1/+rr6el6ujgbiAtjT+t4grAuyNIT+fFHzOmb7JmZ4e2Qm5vD/6BTDTmzWM73UGDfCt8Y3g9Lrss9rKlCKHsEP4PwBsAfgvgJ53ubzDGJNkSo+9Mm0a38KZNTCgRodIbNy4yq/pUW9gMGsTODB995N9dXVFBXRooTb8z27ax6DctjQbEet9bWxmj2bKFcYOIGoWsLBaYbNvGlf+uXVS+c+b4nwOQl0ffV+eLarMxpcxKK+vooGGx2Rhs6atBGDmSW9iGBv9uI2PoAisr69tzxZopU7gq++QTrrw6f+HcbmZyTZ3KqnGlVwQ1CMaYOgB1AC4RkYEADgcL0yAiMMYsi66IiYfDQb93OPG/CROYERfItdvURB3Sm26eicwll1B/7tpF372lqDs66EEpKAC+853QjGFbG7sv5+X1TFO3Mg/Xr+d7fcYZEX4hNhsvZPedgD9OOYVpnoF8Zs3NPF+wOoUQaGgAVq2yocl+DiYu/xuyJmRhyHBHz9jy3r1Urn0JKMcDh4M7saee8gbb7HZvtsesWcDll2vFch8IJ+30OwBuAmsP1gGYBeADAKdER7TUYvZsYPFiLsz8Zc1VVLAup496IeHIywNuv53xkXff9fZ1E2H+/5lncicRCuvXU/H5qxETodF54w2mksZVN5xwAvDMM1y5+sp6aWvjxR4xgtH3QFWPATCGmZbPPUfdmJV5HMpzKzB12UvYkZGJ8ScWY/BQO/3rVrfA7343ObejmZl0351zDt13dXX8gB11VP/sGRNhwgkq3wRgBoCVxpg5IjIBwG+iI1bqMXw48PWvMyNw5MiumTNuN2NlY8cmxkCsaJCbC1x0EdPXKyu9nZ/DTV3/+OPgRWpZWcz4OnAgzovgwkLgl79kj6OKCv7tcNAiNjfTGBx3HEu5Z8wI3Sp24513mJFZUmLtvgQVg89BxxFTMHzzO3D/Zw0ypriQP2UUM5mmT0/+3ObCwsQZJ5pChGMQWo0xrSICEckwxmwRkfFRkywFOfdcKrOXXmJMz/IkiFAfXH554hemNTQwHvDppzRkhx/OOoFQdZnVGLC3dE/Q8YfN1rXDdNwYP54Nnm65hZFwh4OrXCs/uaaGRSn+0tOC0NzMncGoUd1iJiKoLjwc1ScejkOHgKVFBr+4Q5JyU6DEjnAMQrmIFAB4CcBbIlIDQFtih4HNxmHzc+Yw7byujgu18eOTo/fRihXAv/5FRZuTQ0O2YQPw4otscXPGGdH3QowaxdY8gXC5aKx6ueCOPJMnMzVtwQL247YmBdXV0Zd47rm97l3yySc0koEW/IMGATu/EOzdy6detYq7tJwcNhuMVEKEkvyE09zuXM+vvxKR9wDkA1gcFalSnMxMfhGTibVr2e11+PCutVODBlEhPfMMldKpp0ZXjuOOY7ppoDTWigruuBKqP9SQIcD113NHYE1WGj68zxkEe/f6j0lZWLVaDz/McIbNxmvY3s64VmkpcMMNybEoUaJLyOsCId8SkTuMMUvBwPK0YP+nJD9uN0fpFhf7LqS10j+ff56pn5GkqYktJ7Zt44K6uJh1Bl984buVSG0tFeBZZ0VWjogxcCDdRePHRySdLCMjtJYqW7dyNzF6tHfC38iRNAYVFazYrq/vszhKkhOOy+hBAG4wq+hOAA0AFoCBZiWFCdSPyCIjg8bg008jk95eX8/RmkuXUuFZTQNnzfLOP37zTe8MepeLzz9wIJvphVrTkOyMHx+8Dq2ykklMJ5/s26U3ZAgN7LJlzPhS+i/hGIRjjTFHi8jHAGCMqRGRSJb+KAlKTU1ox9ntzO7pK3V1HE5WUcEUUit11OWi/3vDBk6mO/VUYOVKuk0yMlg0OHlylFJNnU5WI65fz3TRUaPol4pzte9hh9H4BZrBsW4ddwOB2iQNGUIDO39+6qU9K6ETjkHoEBE7PC2wRaQY3DEoKY7DEVqw2O2OTHXwM8/QsHRvA2Kl7FdW0h9+xx0xWtHu3An85S/s/ZOeTkGWL6egZ57JnPg4RWWtrhm/+Q0r54cM8Sr09nYW/tlswYt3rVTd5ubUK4xUQiccg/AnAAsBDBaRuwF8A8DPoyKVklCMG0eDEKwfkTF0YfSFmhqu+gP1NCoupo7etavXtVyhs28ftyuZmT19Zk4nI9wA54fGiZEjaRwXLuQOyuqubbez/U9xcXC3knX9dHfQvwmlud2TxpjLABQBuBXAXAAC4BxjTL+Zqdyfyc9nrcGKFf6b9x04wLnqffXd79zJn8GmTQIMNEfdILz0En/6cg05HDQSr71GB30c81yHDuVO4ZJLuNK35nxnZfH28suBB6RZBcwh1cE0NzPt7N13GcXPy+PrnzEjckNylLgQyj73GBEZDuAqABUAngYb3lWISKJkeitR5uKL6TbfubNrJlF7O1fqubnANdf0vQ4hWMaMRUwKz2pr2fI60PQcK+dz5cooCxMaBQUsFhw3zqvcTziB18VfBpjbTYMwf34I12/vXuDnPwf+8Q+vC62ujsV3P/0pPwxK0hKKQXgIwDsAJgBYA2C152b9rvQDBgxg9s5557Faec8ettuoqmIg97jjqDu3bOnbfBIrMBqoHxzA5/A3mS5iWNN9gsUHsrISWhEWFQFXXEHvV01N1/e2qYlGfs4c4Oijg5yosRH4/e+5CigtpfXJzOQWcvRonvjee2lIlaQklG6nfwLwJxH5mzHme9EQQkQuAPArABMBzDTGqKFJQLKzgbPP5kqyqor1AS++yFTTjRt5jNWj6NpruVINl9JSZhbV1fmfCdHSQlmmTOn1SwmNUB3qbnfCd9j8ylf4fr74oneWhzHU5VdcEeJQpo8+orL3l388aBDbc6xYAXztaxF+BUosCCWGIIb4NQbWMX2Q41MA5wF4uA/nUGJEejoXi48/TiXT3Y9fW8s47O23hzCeshsi7PhqzTvo7pJua2PmzDXXRHjegS+ssuzW1sA5my0tXWf/JihTp9KI7t/PXV5GBt2AIQeSlywJHicpLmbr1VAMwr59dLXt2cP3t6yMQkb9wir+CCXL6D0RWQDgZWPMbutOTw3CiQC+DeA9AP/qrRBWcFq081ZS4HYDjz5KY+CrPURBAY/517+Au+4KP64waRIn0z38MPPrMzK8PnCHg6NMZ85kRs3771O5DRzICZSTJwdv5RAy6elM01mwgFbP1wupr2dQNZzhGHHE6pjRK2pqus6M9kVmJgtIAs0/7ugAnnySF89mo9V3Omkc8vM53cgatafElFC+OqeDAeWnRWQMgFpwQI4dwBIA9xtjPo6eiF0RkWsBXAsAJf5SXpSosn07XUaBKpcHDqT3YOfO3n23jzoKuO8+FlVt3kz3xtixXETW1TGuWVXFzJmMDOqgtWuZgnnzzYHH7obFvHksSNu8mZrU6iLndtNatbcDt97aP1a1Awbw9QZKRerooFHwZwyMYa/upUtpZLsfV1vLPhq/+hV9h0pMCSWG0Aq2rXhQRNLA9NMWY0xYkSMReRuArzDgz4wxL4d6HmPMIwAeAYCysrK+uKmUXrJ/f/Cgr7WY3r+/94u9jAzg2GN5s6ivpzupo6Orq8oqptq3j7rkqqt4n3dGQC/JyKCFWbSIrpDWVioxl4v+l298o2/9vJOJk05i/mqghVhlZeA5BXv3cmfgyxgA3F42NzOV95pr+iyyEh5hba6NMR0A9vfmiYwxUe6DqcSKcFxAkfYCrljBHYIvHVxVxdqE8nLuTAYP5mJ2/nwu9Hsd983IYIvqM87gyZ1ObkH6W3vQE04AXn+dASRfRQ3NzVwpnHyy/3OsWEGfXqAI9pAhdB9dckng4gkl4kTK25o0VFdz1QrQAxDnVjRJiTWFzNdUSAur8jWSE8uMYb8dX5MS9+5lEkxmJnV1YyPdS62tHCCzYwdw/QVVcHz0X7b9dLmYBjV7NoUMxXJlZDDBv79SVAT84AfAAw/QKg8eTCvrdNJn53IB3/9+YFfPnj3Bi9esKHdtrRqEGJMQBkFEzgXwZwDFAF4XkXXGmHmRfI6KCrZnXrvW+903hsWVF1zQ/xZ7faG0lCv0Q4f8v29VVdS3gVpQhIvLRR3RPauprY1Dc3JyqJ9cLubXAzQQY0oNml99F1VvPEVdVVDAD8HSpcDbb7NL3iWXaN+GUJg6FbjzTs7tXLaMxsBu5+7h1FODrwBC6dcNMEYTsewAJVR69Y6LyFBjzIFICWGMWQj2SYoKBw4Ad99NxTFypHe36nLRQGzdyu6ZOqM7NEQ45/zuu+kyLi72Gtn2dhantbQAJ57IwHJJSWRcR3a7dyRxZ11RXt61FKB7WcCIfR/h6L3/wua8URg6M90rS24uPwRvvknf0vnn913IRMUYrs63bOG2afBgZkYFG1Dti+HDgcsu49jP1lYq+VCV94wZtN6BVmCNjdzmBctoUiJOb03wIgDB6hoTAmOYIul09ky3s7pn7t/PnPof/zg+MiYjJSXM9HnySfrtAb6PO3dSzx5xBNPWlyxhLcK11/b9+y3CiugPP+x6Lffv7zpCsrmZzw8AYtyYvOl5tBcMQUNL+pdFbV9it3O7s2gRJ++kYqvPgweBRx7hhbIqr10uRtvPO48Blt5YbLs9/N5F06ZxK2el63bHGG7nr7pK53rGgd6+40lTMFBezoraQMpo6FBW2x6I2J6nfzBqFIvP7r6b33Oru+bXv86up5Zrqbwc+O1vGb/pK3PnchfS3u69z+326rOODv5uJcIMrNmB7OZDaE+nL9pndpTDQQX5ySd9FzDRqK3lm797Ny/G6NG8cKWlXKU/9RTw6quxkyczk3UG9fX8wnXuc9LUxGDP8ccztqPEnN4ahL9HVIoosnNn4OAnwMdEvJ02ldAR4Yp7/Xpg+vSe+f8iNLh1dUxQCURzM/XWnj1dFX5nSkuBb3+bRqaykvpk4EC6AxsbqVPKyrwL14zWOhgRuFxccPodRu9wRGa6T6KxeDEt8dChPb8E6ek0EAsXchcRK444AvjlLxmPKC/nBd+9m0b5yiu5ndR4TlzorcvoxYhKEUWs8Yuh0JembP0Zq9FnoLTOYcMYgzz//J5u6+pqpp2//773GmRkAKedxh1H9+PnzKHL6PXXOT0tLY3G4PDDeeucOeaypwPGoLGRriu/rm6Xq3f+9ESmtRV4773AWT9WCugHH3BrFytGjWJGUl0ddzEOB42WGoK4kvIxhECdiztjjMawestnnwV3JaeleYt7O9c1VVZy2ldDA/WBZVRaWzmKYP16xna6n3/8eN4aG7mzWLiQKe7d3dLVgw5Dc3sasgZ0YOxYPxbLypGNere8GHPoEINnwSrzcnJ4EeNBfr7v/idKXEj5GML48VwxNjb6P6aujoajP6eY9wW7PXjlskXn3ZoxwEMPUfmPGtV1h5GZCYwZw67Szz/v/3w5OTTkV18NnH466xF276Z7urwc2L43C+UTT8VXxpQjK9OPkPv2sQlSr5v8JCjhbI21j5iCfhBDsNuByy/3zovtTmMje3ZdfrkmNfSWyZO9ef/+aGujG6jzLmzXLsYQA+3ihg/3NrALhMPBLMh77wUuugiYNYvupp/+FLjwqXOQNesoBok6n6ilhb2gi4oiM90n0SguZjqtv8k4Fo2NSdOcT4kuvXIZGWMejLQg0WT6dCY2PPooXRTWDrq9nSvMm2+mUlN6x4wZwP/9H/Wrv75n+/dzlkLnoO6WLd6Avj8cDu4kduwIPigeYFB7Xo+SxnRW2P73v0wv3e1p2jtgAHsRnXxyalbEpqVx2/T889xu+cJqITtzZmxlUxKSflMKeMwxdBGvX0/lAjAAOWVK/2hUGU0GDAC+8x3gr39lu/zOLmGnkx6Z0aN7Kuq2ttB3ZX0el5mWxuZsX/kKUx6NYc1BqlfDzp3L6sudO1l00/n11tczqHPddalZf6GETYp/G7qSkcHV7IwZ8ZYk9Zg5k37/p5+mK8ha9YuwYvmii3om8QwZEryLgRXvjdj8epvN/yi2VCQri1H5F15gmpeVdmcML8DNN3MLrSgIbWJaKF9Fd7jtsJXQsDoObN7MFXVxMV0niZgheeSRTC3fuZOppA4HW1/7Kki1jk9Pp+vO3y6tvp5Zk937FylhkJ3NINm553J73NHBTIsxYzRwpnQhlB3CPs8tUMTNDkCn1USYgwc5Neyzz7p2HEhLY8eB009PvDioCI1AKDMQsrPpwn/ySaaidq9jaGmhYfnxjxPvdSYlubmhBWJiSGOjN5anrtv4E4pB2GyMCbinFJGYTUzrL1gdB5qa6H/vrBDb2xnEbWsDzjknfjJGgtNO4+t50VPqOGAAd0WNjVQQN9zAXYeSOhjDWN7rr3OxY7MxG/Ckk/h50Hqg+CEmSAK5iGR6pqb16ZhoUFZWZlavXh3rp40JzzzDJpz+hnE5ncy5v/fe1GjdXV3Niuft26kcpkzp2oJCSQ2MofF/+WUmHwwaxMVORwd72jkcwG23+U+KUiKDiKwxxpR1vz/UEZr+TnqlMeaxeBiDVCbUjgMizKQ866zYyRYtBg3iQDIltfn4Y1agl5Z27VKRlsbW9LW1wP33c6xyZmbcxOy39DWi9OuISKF0IdSOA7m58es4oCjhYgx7Vg0c6L9lUUEBOwesWxdb2RQSSpbRen8PAQixU5ASDqEmfhijSSJK8lBbywy0kiDpJ7m53PnOmhUbuZKKxkbOtWhvp8/t8MMjWksTypmGAJgHoKbb/QLgvxGTRPmSoiJm4ASq/AXYhUE7DijJQns7FzDBMsbS0oK3Qul3tLay4nzp0q61JHl5TNU78cSIpOKFYhBeA5BjjOmxiROR//RZAqUHaWms6tWOA0oqYRVDu1yBu1w3NQETJ8ZGpqSgvR144AH2eulebd7cDPz971wdRiAIF9ThYIy52hiz3M9j3+yzBIpPTj2Vu8Fdu3q2baivZzuIK6/UjgNK8pCdTTdQRYX/Y4xhOrUOTOvEihXAxo30tXV3D2Vns1Xw888HfmNDRD3QCUpmJvDDH3IYzIED7Me2ezcNRHo6Ow4cf3y8pVSU8Jg/n922fXWvtaryJ07kYkgB36zXX2dxhj+XUFoafXErVvT56UIJKq81xgQchhPKMUr4WB0HzjsP+Pxz7hQKCrTjgJK8jBzJyvMHHmA2XUEBF71NTfR+TJzIQWr6+fZQX88inWCR+IICjg8877w+PV0oMYSJATKNAAaXdeRRFMnJSbiOA0osaWqiYkhLY3/vJO/jMWECCypXrWI2UWsrW53MmcNxyzpFsxOhTp4CIjIDOBSDMCGEY4L0rFQUJWz27wfeeINa0xh+4YcNA848k874JF5G5+TQAMyZE29JEpy8PAYKm5sDd7Ssq2Npfx8JpVJ5V+e/ReROz/+tA7DOGLOtz1IoitKVHTtYrut2c9i0NSmovh7429+YcXLFFUltFJQQsNvZxfKZZ/ynHLpc9CdHIBIf9qfJGHMHgAcA1AE4V0SSZpymoiQFbW3s35CV1TXNUITFSGPGsLfJBx/EV04lNpx0EmMI5eU93UIdHRwDO39+RGaCh2wQROQBETovjTEVxpg3jTH3GGOu6asQInKviGwRkfUislBE+tEEE0XpxiefcCfgb5CPzcbBGK++Gp6PWUlOsrOBW29lFeru3TQA1s+qKuDCC4ELLohZYZpFA4BXRORiY0yTiMwDcIcx5oQ+SwG8BeB2Y4xTRO4BcDuA2yJwXkVJPj78MPiM59xcKoWqKu0X3R/IzeVg+IoKTstqbWVHyClTIjotK2SDYIz5uYh8E8B/RKQdQCOAn0RCCGPMkk5/rgTwjUicV1GSEqsMPRDWxKT29tjIpCQGQ4bwFiXCcRnNBXANgCYARQBuNMa8HwWZrgLwRgA5rhWR1SKyuqqqKgpPryhxZsQINjELhDWMOl8zvpXIEU5Q+WcAfmGMORlcwT8rIqeE+s8i8raIfOrjdnanY34GWx6JlAAADjtJREFUwAngKX/nMcY8YowpM8aUFRcXhyG+oiQJJ5zArJFA8YGKCuDYY7V3iRJRwnEZndLp9w0iMh/AAgAhNVAwxpwa6HERuQLAmQDmmmBj3BQllSkpYdfCDz/kJJnuwcL6ehoLnSikRJheN9I2xuz3uJH6jIicDuBWACcZY5ojcU5FSVpEgKuvptJftYoVyjk53DU0NHCu6C23sA+EokSQPk1WMMa0REiOvwDIAPCWJ7N1pTHmugidW1GSj4wM4Prr2c3w/fc5QDsjgzuHo48OPChDUXpJ5Ebt9AFjzGHxlkFREg4RuoxKS+MtidJP0Lp3RVEUBYAaBEVRFMVDQriMFCVROXCALvyNGxnjnTCBPcSGD0/6LtSK0gM1CIriA2M4qGrBAir+gQN5/zvvAG++yYzPb3xDm40qqYUaBEXxwbJlwLPPsiQgLc17f24ui4RffZWZoFoKoKQSur5RlG50dAAvvEC3UGdjYGG3c675yy8DLZFKvFaUBEANgqJ0Y9s2thIKlOqfns6xBRs3xk4uRYk26jJSlG7U1YV2nAhQUxNdWRKGtjZg3ToGUaqr6S876SRgxozgrbqVpEENghJzqqqArVupYwoKgMmTgczMeEvlJT09tOOM6ScFw5WVwO9/z5/5+bxYdXXA448z6v6jH/kf76gkFWoQlJjR0AA88QSwejUnAYrwlp4OnHsuMG9eYqRyHnEEs4ecTv9jCSz5J0yIrWwxp6WFxqChoWvFdFYWjUNtLXDvvcBddwGFhXETU4kMGkNQYkJzM/XK2rUMyI4ZQ/0yejQHPz31FLBwYbylJHl5rDUoL/ffgXrvXqCsDCgqiq1sMWftWu4M/A1lKSjgQJ9ly2IrlxIV1CAoMWHpUo6AHTWqZ+5+RgYNwyuvsBAsEbjoIq7+d+7sOqumuRnYsYOv49vfjp98MeOdd/zPdrYYMgR46y2d75wCqEFQoo7TCSxeDAwd6v8Yh4PpnO9HYwZfL8jMBG6+GbjiCv69ezdv7e3At74F3HZbP4mlVlUFn9mbkUHXko7zTHo0hqBEnbo6uqCtal9/5OcDW7bERqZQyMgATjkFOPlkzqQBWJhmt8dVrNiSk0NFHyjS7nTyTfFVtKEkFbpDUKJOIgSK+4LNRq9JQUE/MwYAgymHDgU+prISOP547eORAugVVKJOfj5vTU2Bj6utZQqqkkAcdxy3Sg0Nvh9vbWVp96kBJ+QqSYIaBCXq2O3A/PmcC+8v7tjRwcdOPDG2silBKCgAbrqJPrPycl4ogA2d9u1jFsB3vsOmT0rSozEEJSbMns3xwNu3cxRwZ9dLSwuwfz8zewYPjp+Mih8mTgTuvBN4912mizmd9AMeeyxw2mlalJZCiEniVLGysjKzevXqeIuhhEhzM/DMM8CKFSzsssjJYSvp2bOTP96Q8jiddBNlZGgQOYkRkTXGmLLu9+sOQYkZ2dnAVVcB55/PBnLt7YwtjB+vuiVpcDj6Sb5t/0QNghJz8vPZE01RlMRCg8qKoigKADUIiqIoigc1CIqiKAoANQiKoiiKBzUIiqIoCoAEMQgicpeIrBeRdSKyRESGx1smRVGU/kZCGAQA9xpjjjTGTAPwGoA74i2QoihKfyMhDIIxpr7TnwMAJG/5tKIoSpKSMIVpInI3gMsB1AGYE+C4awFcCwAl2lBLURQlYsSsl5GIvA3A18ysnxljXu503O0AMo0xvwx2Tu1lpCiKEj5x72VkjAm1YfpTABYBCGoQFEVRlMiREDEEETm8059nA0igQYqKoij9g0SJIfxORMYDcAPYBeC6OMujKIrS70gIg2CMOT/eMiiKovR3EsJlpCiKosQfNQiKoigKADUIiqIoigc1CIqiKAoANQiKoiiKBzUIiqIoCgA1CIqiKIoHNQiKoigKADUIiqIoioeEqFRWoo/TCezYATQ1AVlZwLhxQFpavKVSFCWRUIOQ4hgDLF0KvPQSUFcHiPC+7GzgzDOBr34VsNvjLaWiKImAGoQUxhjgxRdpDIYNA0aP9j7W2go8/TRw4ADw7W8DNnUeKkq/R9VACrNzJ/DqqzQE2dldH8vMBEpLgffeAzZujIt4iqIkGGoQUpj33gPS0wGHn32gzQbk5gJvvhlbuRRFSUzUIKQw69YBhYWBjykq4g7B7Y6NTIqiJC5qEFIYtzu02IAxvCmK0r9Rg5DCjBsH1NYGPqa+Hhg1SjONFEVRg5DSnHYa0NDgf/VvDHDoEDB/fmzlUhQlMVGDkMJMmgSUlQG7dvWMERgD7NkDjB/PYxRFUbQOIYWx24Hvfhd46ing/fd5n8PBqmVjaAiuvJKZSIqiKGoQUpyMDOCqq4Czz2bWUU0NkJcHHHUUMGRIvKVTFCWRUIPQTygsBObOjbcUiqIkMhpDUBRFUQCoQVAURVE8qEFQFEVRAABikrhEVUSqAOyK4lMUATgYxfNHk2SVXeWOPckqe7LKDcRf9tHGmOLudya1QYg2IrLaGJOUWfrJKrvKHXuSVfZklRtIXNnVZaQoiqIAUIOgKIqieFCDEJhH4i1AH0hW2VXu2JOssier3ECCyq4xBEVRFAWA7hAURVEUD2oQFEVRFABqEIIiIneJyHoRWSciS0RkeLxlCgURuVdEtnhkXygiBfGWKVRE5AIR2SgibhFJuNS87ojI6SKyVUS2i8hP4i1PqIjIP0WkUkQ+jbcs4SAio0TkPRHZ5Pmc3BRvmUJBRDJF5CMR+cQj96/jLVN3NIYQBBHJM8bUe36/EcAkY8x1cRYrKCLyVQDvGmOcInIPABhjbouzWCEhIhMBuAE8DODHxpjVcRbJLyJiB7ANwGkAygGsAnCJMWZTXAULARGZDaARwBPGmCnxlidURGQYgGHGmLUikgtgDYBzEv09FxEBMMAY0ygiaQCWA7jJGLMyzqJ9ie4QgmAZAw8DACSFBTXGLDHGOD1/rgQwMp7yhIMxZrMxZmu85QiRmQC2G2N2GGPaATwD4Ow4yxQSxphlAKrjLUe4GGP2G2PWen5vALAZwIj4ShUcQxo9f6Z5bgmlT9QghICI3C0iewBcCuCOeMvTC64C8Ea8hUhRRgDY0+nvciSBckoVRKQUwHQAH8ZXktAQEbuIrANQCeAtY0xCya0GAYCIvC0in/q4nQ0AxpifGWNGAXgKwA3xldZLMLk9x/wMgBOUPWEIRXZFCYSI5ABYAOB/uu3kExZjjMsYMw3csc8UkYRy1emAHADGmFNDPPQpAIsA/DKK4oRMMLlF5AoAZwKYaxIsWBTGe57o7AUwqtPfIz33KVHE44NfAOApY8yL8ZYnXIwxtSLyHoDTASRMUF93CEEQkcM7/Xk2gC3xkiUcROR0ALcCOMsY0xxveVKYVQAOF5ExIpIO4GIAr8RZppTGE5x9FMBmY8wf4y1PqIhIsZXtJyJZYCJCQukTzTIKgogsADAezHrZBeA6Y0zCrwBFZDuADACHPHetTIbsKAAQkXMB/BlAMYBaAOuMMfPiK5V/ROQMAPcDsAP4pzHm7jiLFBIi8jSAk8FWzBUAfmmMeTSuQoWAiJwI4H0AG8DvJQD81BizKH5SBUdEjgTwOPg5sQF4zhhzZ3yl6ooaBEVRFAWAuowURVEUD2oQFEVRFABqEBRFURQPahAURVEUAGoQFEVRFA9qEBRFURQAahCUFEFESkWkxdMnxrqvR1tqEcnytDJvF5GiPj5nlogs9XQ8hYjcKCKbRSSsNiEiUiAi1/dFlhCeo0eraxFJF5FlIqIdCxQAahCU1OJzT58Yqy31XwHMBzAJwCUiMskY0+I5Zl8Enu8qAC8aY1yev68HcJox5tIwz1Pg+d+wEBLqd/hfYJuEL/F0Z30HwEXhPreSmqhBUJIGz1CU0zy//z8R+XOAw2PRlvpSAC975HkIwFgAb4jIzSLyLc8wlHUi8nCnXcRLIrLGMyDlWs95fgdgnOfYez27nc4r+R+LyK88v5d6dj1PgD1wRvl7rs4EaHX9kud1KIoaBCWp+CWAn4nIpWDL4/8JcGxU21J7+haNNcZ8AQCetiD7AMwBsBhcdZ/g2Y244FW6VxljjgFQBuBGESkE8BN4djfGmFtCePrDATxojJkMIDvAc4XCpwBmhHG8ksKo71BJGowxyzyNzX4I4GTLVSMid4HNzmJJEdhnyRdzARwDYBXFRRbY/x6gETjX8/soULkfCPO5d3WashXouYJijHF54im5nmEzSj9GDYKSNIjIVADDAByylJeIDAUnT3Un7LbUIvJ9ANd4/jwDwLmd/zbGdI47tADI9HcqAI8bY27vdv6TAZwK4DhjTLOI/MfPOZzounvvfkxTsOcKkwwArX34fyVFUJeRkhQI5+g+BcYBGj3tvQFgGoB1Pv4l7LbUxpi/etw204wx+7r/3e3YGgB2EfGl0N8B8A0RGeyRfZCIjAaQD6DGYwwmAJjlOb4BQG6n/68AMFhECkUkA5xp4Q9/zxUSHpfVQWNMR6j/o6QuahCUhEdEsgG8COBHxpjNAO6Cd0iRT4PgmSd9A4A3wZm7zxljNkZYtCUATvTx3JsA/BzAEhFZD+AtcGezGIBDRDaDgeSVnuMPAVjhmRh3r0c53wngI8//+u2ZH+C5uuBpdf0BgPEiUi4iV3semgPg9d68eCX10PbXSlIjIo+Cbp0SAK8ZY0IaSSgiXwAoM8Yc7MNzHw3gZmPMZb09R7wRkRcB/MQYsy3esijxR3cISlJjjLnaGOMGs2vyOxem+cIqTAPjDu5Ax4bw3GsBvOcrzTMZ8LjSXlJjoFjoDkFRFEUBoDsERVEUxYMaBEVRFAWAGgRFURTFgxoERVEUBYAaBEVRFMWDGgRFURQFgBoERVEUxYMaBEVRFAUA8P8BBYGhJSJXix0AAAAASUVORK5CYII=\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEKCAYAAAASByJ7AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAABOTElEQVR4nO2dd5iU5dX/v2dmtrINdpfOsoBKV9AFsQRFLGCMNbYYNWo0xhh9NTHGFJPoz8S8JrEkMWpeY9QYK4IdsQWEiFJEkKYIAkvZXdjed2bu3x/feZwtU3en7/lc11y7O/PsM2fmmTnnvk8VYwwURVEUxRZvARRFUZTEQA2CoiiKAkANgqIoiuJBDYKiKIoCQA2CoiiK4kENgqIoigIAcMRbgL5QVFRkSktL4y2GoihKUrFmzZoDxpji7vcntUEoLS3F6tWr4y2GoihKUiEiO33dry4jRVEUBYAaBEVRUhFjgPZ2oKMj3pIkFUntMlIUpQ84ncDGjcCbbwI7dgB2OzB9OjBnDjBmDCASbwnDp60NWLUKeP11YO9e3nfIIcDppwNHHMHXqPhFDYISVYwBGhv5MycHsOmeNDFobQUefBD45BMgNxcoLATcbuDDD4H33wfOPBM455zkMgpNTcB99wFbtwJFRcDo0by/shK4917g2GOBq68GHKr2/KHvjBIVOjqADz7gQq2ignolPx+YPx/42teArKx4S9jPeeIJYMMGoLS0q9IfPpw7h4ULqVRnz46biGHz+OPAF1/03N0MGgQMHAisXAkMGwacfXbcREx0Ema9JiKjROQ9EdksIhtF5MZ4y6T0jvZ24M9/Bv7+d/5eUsKbwwH861/A3XcDDQ3xlrIfU1lJaz1qlO8dgMMBDB0KLFoEuFwxF69XVFXRVTRypO/XJEJjt3gxd0eKTxLGIABwAviRMWYigFkAfiAik+Isk9ILFi4E1q8Hxo4F8vK8388BA3hfeTnw2GPxlbFf8/HH/BnIfzdgAFBTw9hCMrBhA/2SgV5TRgZXKJ9/Hju5koyEcRkZY/YB2Of5vUFENgMYAWBTXAVTwqKpCXj7bWDECP/u55EjqZMqKoAhQ2IrnwKgthZISwt+nM3GCxoramq4yt+7F8jMBKZOBSZMCC0Q3NQUesBYdwh+SRiD0BkRKQUwHcCHcRZFCZNt2+iCDqRvRLiY27gRGDyYXgm7Pbnil0lNXl5o6ZjGxCbY43LRPfXaa3zOzEx+iBYv5gfkhz+keysQBQWhubes7AbFJwlnEEQkB8ACAP9jjKn38fg1AK4BgJKSkhhLpwSjrS00xe5yAcuWAS+/zAWrwwEcfTRw8smMCSpRZNo04NlnqRz9XayWFmYfjR0bfXkWLABeeYVZQd0zgA4eZNDp9tsDbyenTuWOxlpd+MJ6TYceGjnZU4xEiiFARNJAY/CUMeZFX8cYYx4xxpQZY8qKi3u04lDiTF4e9UwgGhqANWuAjz4C0tOZ6DJ0KL0Fv/kN8O67MRG1/zJsGHDkkQzm+LpYbjfdNmecEf0Uzaoq4I03+CHw9VyFhVxlvPaa9z5j6PbpvCMoKADmzgV27vT9mpxOvqZzztG00wAkzDsjIgLgUQCbjTF/irc8Su845BCmlzY3A9nZPR93uYDly/n7tGk0CAC/o8OHM+b3z3/y9wkTYiV1P+Sqq5iz/9lnTMu0LPmBAywcOfVUKtho88EHXNkH8v8PHQqsWAGccgqDT0uWMGYgAsycyfvHjQMuuICrjQ8+4IevoICv6eBBusjOPhs48cTov6YkRkyw5VyMEJHjAbwPYAMAt+funxljXvf3P2VlZUab2yUe778PPPIIU027xxLKy4H33gPKyoCJE33/f1UVPRU/+lH0Ze3XtLcDq1dzhV5eTgU7dSqNwaRJsQnq3H8/A0+FhYGP27yZsQW7nXGFzEyuLioruYO49FIaMLebRu6tt1iTIAIcfjirr0eP1kCVBxFZY4wp635/wuwQjDHLAejVSgGOPx6orwdeeIGLv4ED+T2srfWmo44f7///CwsZcG5ooMtXiRLp6azePfZYKlKR2CtMh4PPHQinE/j0U+4GOscN7Xa6v9rbWZQ2ciQ/WBMm6PaylySMQVBSBxHg61+nm3r5choBY2goLAUfKF3cZuM5rBigEgPi1VNk+nQGjwLFA3fupNHwl2mUns7ModdfD7zSUIKiBkGJGsOGAeefz5vFX/4CbNpEl7U/3G4aEF8xCCXFmD6dF7qx0Xc6qDH8wBx6aGCjVVTElYe/8yghkVBZRkrqc8IJDDgHorISOOoo/V73C7KygOuuY+D3wIGuGUJtbcCXX9LnOG5c4PNY28q2tqiKm+qoQVBiysSJ3Pnv2+f78ZYWZhSefnps5VLiyJQpwC9+wQ/Gzp3A7t3Arl0MOp11Fj8M7e2Bz+Fy0SDotrJPqMtIiSkOB3DTTcAf/8jFX0EB2+Z0dHCBaLMB11+vxWkJx8GDbC3hcLAvSSitL8Jh3Djg1lvZz6S6ms9TUsL+Q598QndQoDhDZSUwa5a20e0jahCUmDNoEPDLXwLr1jGlvKKCWYRnnMHA8+DB8ZZQ+YqdO4EXX6RCttm8wZ1584DTTou8YRgypGdF8uTJ3m3lsGE9/6epiSuK006LrCz9kISpQ+gNWoegKFFk61bgnnuo9IuLvUHdlhYq52nT2Gco0kbBF9XV3FaWl3fdVh48yN3ED37AiWhKSCR8HYKiKAlEWxuHWuTl9UwJy8qiT2/dOuCdd7hbiDaDBrGf0SefsOisspLbynPPZR3FoEGhn6upiQbN7eZ2tKAgamInG2oQlLgQrzooJUTWraPiLCry/bgIW0q88QY7EsaiP1BGBovTZs7s3f83NAAvvQQsXdq1D9LMmexxpL3Y1SAosaO+ni1p3nqLHoDMTMYM5sxhnFJJID7+OHiANjubmQD797NKOJGpr2fX1H372CjLcnO5XOy0+OmnwM9+xsf6MZp2qsSEPXsYSH7uOS4mR49mi4qlS3n/ypXxllDpQnt7aANnRJJjzOYzzzB7YfTorjEPu52rEWOAhx4K3qo3xVGDoESd1lbGA10ufh+zs6lH0tL4XRw8GHj4YaahKglCSUnwaWkuFxXowIGxkam31NZyxRFo9V9UxPqHZBkZGiXUIChRZ+1auoj8uaMzM9mOZsmS2MoVFi4X5/b+3/8B994LPPkku2mm6orymGP42gKt/isqmPsfqA9JImCtNALteKxg1mefRV2cREZjCErUWbaMMxICMXgw8OGHwOWXM3aYUFRUsE2zNes3PZ39dd55hzny3/9+6vXZGDKEcwYWL+a2rrsyramhEv3GN+IjXziE6tKy2UIbLZrCqEFQok5tbXAlb7dzQdrSkmAGob4e+P3v6fcqLe36mDHs0//AA8BPfpJ6k7guvJBK8s03qfyzsqhcW1sZALrtNt+FYolGYaG3Y2KgtDa3OzleTxRJsU+wkogMHMigcqA2M06nV+ckFMuWcTU8enTPx0RYQbt1K3cMhx8ee/miid0OXHQR00pXruRFzMhgh9LJk5PHAI4ezetUW+s/3tHaytc2dWpMRUs0kuSKKsnMnDnAgw/6rx1qbGTau83GhfaIERzaNWVKnHWO283ARqBeGiJ0F731VuoZBIuiIvYVSVZEgG9/G/jd75jJ0N2919ZGY/fd7ybY9jT2aFBZiTpHHEGdWlnZ87GdO+mR2LWLk9Sys5nsce+9wP/+L41F3GhtZTFTsG1Lbi7jC0riMn48cPPN9Enu3MnaiYoKBpwPHACuuAKYPTveUsYd3SEoUScjg9/Fe+7h96+wkIp/717OXx4wgLuIoUO9xw8axFG7Dz3E2cpxqWi2tidud+DhLC4XA81KYnP44cCf/sT2F5s28bqOHcsB36mWFNBL1CAoMWHoUOCOOzgt8a23uDjbvJmDsKZMoVHojOWe//RTGpG4tMNOT6dw27cHbr188CD79vcCK2tVW3jEiL62v0hx1CAoMWPAAODEE3mrqmK8oKTEvzK0itc++KCXBqGxEdi4kZlCAwYAkyaF38hs3jxmGQ0c6Dug0dLC3cNxx4V8SqeTJQ1vvMFdkAgHB512GmO18RpvrChqEJS40NjIJJZgK+PMTLp4w8LpZA//JUv4u4XNxhmeF10UevBw0iSu/hctoq8rP59CG0PBGhtZh+Cv6q4bbW3A3/7GYr38fG8LoB076FKbPRv4zneSJ4FHSS30Y6fEhcxMb+eDQEahvT3MQli3G3jsMQYnSkq6alaXC3jvPbp4brghNK0rwhbLJSXAyy8z+m2z8XksY3HYYSGL9+yzzKgaM6br6y4upr1Ztow/zzkn9JesKJFCDYISF4YOZWuZ+vrAVczt7cDRR4dx4q1bgeXLWUTW3fditzMnfd06dvOcMSO0c4rw2LIypkq1tTEIGU4PfjAN/j//YWzElxG02bhjWLwYmD+fRlNRYol6K5W4IAKceSZjCZ29Op2xuiqPHx/Gid9+mylM/hzxIowHvP562DJDhC0dSkrCNgaAN7ElUEud9HQawS1bwhdPUfqKGgQlbsycSW/Mrl1ceLvdvL+piZlFubnAjTeGGWTdujV4982CAjrtY9y2uaEh9GyilpboyqIovkgol5GI/APAGQAqjTFT4i2PEl1E6CufOJHFaZ98wvsLCoBvfYsZN7W1QF0d3UshtbUIVePGIc8zLy/05qiB2nwoSrRIKIMA4J8A/gLgiTjLocQIK+Vy4kQu2J1OGoGXXuJMEyuhx+EATjqJHRQC1hBNnswYgVXl5ovqahZAhDIAJoJMnsyndDr9x7Pb2pgAFZabTFEiREK5jIwxywBUx1sOJT7Y7UwAuvNOFrCNGMEAbEkJszoXL2ZJQENDgJPMnUt/iz93kDHccsyfH5XXEIi8PBq13bt97xTcbqC8HPj61zWgrMSHhDIISv/GGOboG0Nj0HkBn57OxKE9e4Dnnw9wkkMOYR//L7/kcrszHR2MHcyaFbdGdBdcwNjJjh3egLrTycrtnTtpz04/PS6iKUrCuYyCIiLXALgGAEpKSuIsjRJJvviCK+RAl3X4cGDFCuCb3/RTnyDCAMTAgcArr3Q1CmlpTG0666yYu4s6i3DttSxAe/NN4PPPKfKkSbRjEyZoGwslfiSdQTDGPALgEQAoKytL0fmF/RMr1TKQQnQ4uIP48ssAi3ybjX6Xk0/mSRsbGaUdPz4horV2O9vu9/PW+0oCknQGQUldOjpCX7j7q13oQkYGe28rihISCRVDEJGnAXwAYLyIlIvIVfGWSYkdw4cHV/TGMPhaWBgbmRSlP5FQOwRjzMXxlkGJDPX13kajublMuQzUogKgCygjw5t6CQB2VzuG7FuHQ754E3n15WjpcODAmBkocZ8IoDTKr0JR+hcJZRCU5Kejg1lA77zDzE+rjsBmA04tq8a5w/6L9G2befDEicCxx37VBiIri3PdH3uM6aY5aMTRH9yHwurP0ZpZgJq0wWhtd2FOxgrIr9/jwfPnaxRWUSKEGgQlYrhcwN//Dnz4IRX6V8VXxqB021sYfu8z+GwgMOnoXLaj2LyZbaovvJBDlEUwZw7dRs89a3D4+oeR2bgD5dmlcLYJ0tKAmcfbUTBkBC3P009zNmdZWTxftqKkDGoQlIixaROwcmXP1s4lu97H9A1PonbIKHxWn47CdmDYMDA1tL0d+Ne/uD2YPRsitA3HjtiFhps3YN/Q0ci2CYqLWXz8lZFJS2O12qJFwFFH6S5BUSJAQgWVleRmyRLGCzrrZpurA5M3Po+G3OEwjnRkZnJK2FekpzOa/NxzXPV7yNm8CsNG2nHkUYLp09n1tEe7h7w8DmbWAfeKEhHUICgR47PPejYaLTqwBentDXCmsTNdVhZbCXVp3ZCVxVqBzZu991VXB+/fIMLgRFNT34VvbGTfjNbWvp8rllRXs+x5715vu1hF6SXqMlIihs3Ws0dPVks10tubMLhiA9LbG+CypUFsIwDXEMDRreigttb7e35+z9YT3TGGgYu+FJtt3gy89hr9XSK8HXssBxyPGNH780abL74AFi5kKpf1xhcVAd/4Buc762BmpReoQVAixqRJ1KtDhvBvh7MVh219GUP3f4y2jDy4bWlwdrgxybEH8nYmewp1Hnqflub9feZMTqEPNGOzoYFP1lvFvXgx8O9/089ljTFzOhkI+e9/gZtv5otKNNauBf78ZxrCUaO8yr+hAXjkEWD7duDSS9UoKGGjnxglLA4cAF59lU3oHn2Uuqm9nY+dcgq9N243AGMwfc3fkduwB+3pOWhPz0WHIwstMgDZwwuo6Fes6PQPYEtqi9JSNvbxFx9wOtkd7uyzexdQ3rqVWUqjRnGgsXUOh4Mxjfx84IEH2Bk1kairAx56iDIXF3dV+rm5jOi/8w6wZk38ZFSSFt0hKCHhcgEvvMBFtQgXpy4XZ9nn5wM//CFbBZ10EvDuu8DU7O0Yvm8NqgvHI7OtHgPq96Ee+cjPBwYMACBZrFrbto1lxzNm0OVhIQJ8//vAn/5EH3lhIRWe201D0NLCcWthDVzuxJtv8kV03pUANFQHD/I5KiqYR3vttUGGMMSQDz+kMfTnJrPZvCNCy8o0+0oJCzUIKUBVFZu9ud1Myy8tjbweeP556pjRo3v2G6qrA/73f4Ff/pKeisJCoPpPy1DXko5aCGrsR+BQZyOK0moxIDsH1sfOac+AWbcRLfPPRfa3Luv5YczPB267DVi9mu6jXbv45GVl7BN9yCG9e6FtbcDHH3N30P2FrFrFALPNxqynJ59kg7xzz419EVx9PeX56CNuw0pKgA0b/LR57URBAXtpNzQEP1ZROqEGIYk5cIAp/NboSREahZEjgcsuAw47LHLPs3ixb2MAUG+3tLAk4Ic/ZFyzY3U59m/KwfovgfqWDHwx7Hg0Nm3H4PIvcHB3ExwOoN3lQHN6CR6v+h7af52L+fOp57ss2jMzgeOP582KWPdVKbe3ezOULBobgeXLeb8V1+jo4HMOGUL3ktvNkW2xYN06+uXa2/kG2+3ABx8wvjF4MOMvDgeVfn09/ycnhwbACo6H1AFQUbyoQUhSDh4Efvtb6rHOcUVjmKzzu98Bt9wSmZjoypXUL4E6kQ4ezEV3TQ09Fh2SgR3bXHDbWFAmko7Wogn4rOlQ7N/RAmcbUDQyE6PT9qBoTC6k+jN8+f8+xPsP12D2WQPhOG5Wzx1ApFbnWVmsf2hv50+AuwCXq+uKuqODyjgtjavzF1+kYeocCI8G27YB999PF9qAAd77c3Nplbdu9b4XVVX8afUIGTiQFz0tjccr4eF0che2eLF3R3rkkcCcOdHZeicYGlROUp57jovD4cO7LnRFqBMGDgQefjgyi8Tdu4NndtpsvFV7BqD+p3kmbI31XQrV3G6gfJ8dzswc2PJy0FTZiIb0IpR99Decsvq3KGt+H87Nn2PvM8uAu+6ixYtGUNfh4FZk/37+3drKUWzd4wTt7cDYsfw9LY0K96OPIi9Pd158kW94Z2NgUVpK+deuBSorabAKCvgzP59B+rff5rCF7vERJTDNzcAf/wjcdx+TGQoLuUBYuRL4zW84cMnX7NMUQg1CElJTQ9dyoDnyeXnUpRs39v350tNDq3lyu71ejFf2z0B6XibS2xu/erypyTtg3i5u5HdUYe+BNOTX70JdQSka84ZDCguxsWYE3KNGMzDyxz9Gp1hszhy6o2pqqAi6u5Dq66lgBw/23pedzQB3NDlwgLURnQPsnRk0iDsZa/amrx1URgZHsaW48oo4//gHd4qlpTQGDoe3kn7kSAbSVq6Mt5RRJWyDICIDRCQ+8wcVAFzMAsGHyTgc3dpE9JLp0xkjCERLCxe0w4czOafNMQCrjr4emS01yGncDzHur2K12c56DGvdgQ0ZM7ClbQwa87x1BGlpXJg3NArrC3bu5Go40hQVAT/5CX/fs4dGp72dVqu2lhb1mGO6vsmWxYsmdXV8k/y5Jhobacjy83lsfT0NWmMj5U5L46S42trIXPz+wt69TNUtKfH93jscXBwsXJjSFeFBDYKI2ETkWyLymohUAtgCYJ+IbBSRe0Tk0GDnUCJLoFqtzlhB5r4ydapX//iTZ98+FvdaXgoRoGrwZCw94XZUDJ6CvPpyDGwqx7COnXDa0vHysO/hIxyN9nTf6ZxfyT1wIP250aC0FLj7buD6673B2KIiGoLZs3u2zmhpif7cy/T0wCv7+nrKWVzMWMHYsVzNDh9OuefO9cZBdu6MrqypxNq13mC8P3JzGbPZtSt2csWYUJY77wF4G8BtAD41xrgBQEQGAZgD4G4RWWiM+Vf0xFQ6M3QoFabbHbgYtaODOq+vpKUBN9wA/P731ImDB3uft6WFxuDww9ml1JLPSnKpKxiNVTOvR0ZrHco31WPr9jS4CgfDQJC3awWGDe3ah8jShV/pYquBXbTIyqL76JZb6CP294Y1NnILFO2RnMOHMybQ1OQ7hmCtBlwuBt07u7R8HauERnW1N8EgEJHqnZWghOIyOtkYc6cxZr1lDADAGFNtjFlgjDkPwLPRE1HpTnExFXBFhf9jmpvp8vY7iD5Mxo0Dbr+dk8/Ky3nbtYvPc8EFNBjW9yknhwvsznq8LTMf2eNH4YBjKFzGhg6nwGF3YXx2eZfnaWqiQcnK8tzhdPoMjrpcEd65n346V9s7d3bpugpj6Nc/eBC47jrvKLdoYbczb3f/ft8vMCeHVjg723+cwSKRezElGvn5Xa+7P4zp9OFMPYLuEIwxHSIyAcAIAB8aY76KEorIPGPMYmNMCO+kEkkuvBC4807uYIuKuu50m5poLH74w8jqr5Ejec6aGi6orC4PvpJZzjyT2Xt79nD2gc3GBe+4cYyZOhzAWRNqMah5LxrThwNgvZjbzY4VX1FVxWZtnsdXrWKN2p49POfUqdyZTJrUx4zArCzgxz+mj/g//6HFASjQIYcAF13En7Fg9mw2r1u6lBfXcme1t9NllJfHIhN/28P6erqRxo+PjbypwJFH8toH8sc2NXH3Nnp0TEWLJWKCbCtF5AYAPwCwGcA0ADcaY17yPLbWGHNktIX0R1lZmVm9enW8nj7ulJezs4LlKrZS0fPzWTEc70FiNTXAE0+wxsrC6jxhtwOTsnbgnA2/QU3WCLS505CRwZ52hYWegzs6qPl/8xs0DBqNe++lnhw0iDrRWrw3NjJ+cfHFEern1tTE7Y/TSWGGDYt9/rnbzYyW11/3Wj+Hg+6tww5j0Vp2ds9+4/X1fON//GNu55TQMIZtUjZv5sqnOy4Xv2hXXgmceGLMxYs0IrLGGNNDQ4RiEDYAOMYY0ygipQBeAPCkMeZ+EfnYGDM9KhKHQH83CAA/xzt2sMGly0XdNWlS9JNhwqGy0itfYSF72B04wMJbWfwGJnz8NPLHDETRuAI40sRbXVddDVxyCcxp83DvvUyh7d5tAuB5v/wSuOIK9lJKKaz3wuViUNPa8m3fztXA/v1dV7WDBlFpqTEIn4YG+Fx1VFXRNzpvHneKKVCc1heDsMkYM6nT3zmgUdgE4CRjzLQIyxoyahBSAGO4hVi0iKtym42r49JS4KyzgGnTUF4O/OIX3Kn7+y42N9Oj8oc/JJYxjCpuN1NLd+3i7yNG0N8WLB9Z8U9bm7d31p49/MBNm8ZWvhMmpIQxAPwbhFC+OvtFZJoxZh0AeHYKZwD4B4Ao5+ApKY8ICx2mTeNWwoqGDx781Zdv3brAqfkA/+XAAe4UYuXqjzs2G91HkWpapXAHdtxxvLndwVNRU4xQDMJlALo0QDDGOAFcJiIPR0Uqpf8h4p2s0426utC6MIgEL6BTlJDphwOGQskyKg/w2IrIiqMoPRk4MLSMQLfbd+p+0lFTwwZ2bW30Y0+cGHy+tKJEgP7ibVWSmCOPZDO/QBmBjY0MWEeiEC9uNDdzpOd//9u1BiEzk5PhTjutX7kvlNijBkFJSIxhYo3dzkK1WbM4LMxXqxmnk3UX116bxLv8tjZmuGzbxrTHzoHhtjbgqado9c47T42CEjVCNggiIgAuATDWGHOHiJQAGGqMiVg/YBGZB+B+AHYA/2eMuTtS51aSg/p6pqO+8QazLR0O1ibMnk19uGEDsy/z87mIPnCA2UXnnw8ce2y8pe8DK1bQTTRmTE+Fn5HBFKtXX6Vl9JUnrygRIJwdwoMA3ABOAnAHgAYACwDMiIQgng6qfwVwCoByAKtE5GVjzKZInF9JfPbv5yjOmhq25ygt5S5hzRrqy4svZvbfm28ym8huZz+3k04KnJKa8LjdLEDrlFnVA4eDt6VLgUsuia18Sr8hHINwtDHmSBH5GACMMTUiEkI3qJCZCWCbMWY7AIjIMwDOAusdlBSno4OFom1tXTsD2O0stmtvp3v9pz/1dq1OGRobWYRXUhL4uIICVtIqSpQIx+Pa4VnFGwAQkWJwxxApRgDY3envcs99XRCRa0RktYisrrLGByoJh8vFlX5NjbctUCA+/ZRlCMXFvh9PT2fCzauvRlZORVG8hLNDeADAQgCDReQuAN8E8IsIyuJrr9yjjNoY8wiARwBWKkfw+ZUI0NoKvP8+PSDW/IT8fGD+fN8jBiyWLw+eMlpYyIFWdXU8Z8qQk8NWCY2NPcd4dqa2lvMOEoGmJlb0bthAiz9uHP13XzWiigANDWzY5XazRiVYd1elz4RkEDwB5WUA1gCYCyrvs40xkdy/lgPo3KlmJIAoNsJXIk1TE90+n3/O76/lAWlqAv71L2YJ3Xyzb8VfXx+8M6tVNNrSkmIGwWYDvv514LHH+Ob4iiM4nVS8X/ta7OXrzpo1wCOP0L+Xk0P5P/kEWLCA6bFnntm3gE5dHc+1olOZk9vNWRQXXMAWu0pUCMllZNjwaJExZosx5q/GmL9E2BgAwCoAh4rIGE9s4iIAL0f4OZQo8vTTbLQ3dmxXpT9gAO/bsYPZk74YNCh4lbHbzXTUvhSftbZytEFDQ+/PERWOPZa9cnbt6ulja2tjp80zzoh/htHGjcCf/0yLXFrKVfugQbT+I0YAL7zAFLHeUlsL/Pa33DIOG8ZuhtZtyxbgjjuA3buDnkbpHeG4jFaKyAxjzKpoCGKMcYrI9QDeBNNO/2GMicCIeCUW1NSwnirQTJYRI5hSev75Pbs2z5pFN1NLC5Npiop6upeqqtjyKDc3fPn27WN20ooV1LfGcFzA178OTJmSABlKGRnATTfRqi5f7i1ME+Gshm9/2zuSLl4YQ/ny89k8qjsOBw3DwoXACSf0znI/+yxzibsH2G02FqQcPAg89BCHgSRt0UniEo5BmAPgeyKyE0AT6DYyxpgIzeQCjDGvA3g9UudTYsfWrdRhgRpt2u3UKVu30gAA/J8lS6hDdu/mY9nZ1IMlJVTWaWk0FC0tVODh8tln7IJqDF1ZDgd/37MHuOce1nr11csRFtbcUWMYRbdmIGdlsXX1uefyjWhvp/WbODH6k9pCYdcuvmmBsqHS0pgy9vHHwPHHh3f+mhr6FQOtKgoLuVvatk2b+kWBcAzC/KhJkWQYQ+VVUeFVXIFG2/YH2tvDP9YYuopffpmekNNO4w6ivp56ZccO6oixY2lMvv99xi7DoakJuP9+urotvQvwuhUWcrG7YAHPO2VKiCd1uXhih8P3StkfjY1Mk3rvPcYELGbNou/dSrEqKACOPjr088aKmprgbWcBXrz9+8M/vzXpKVj7bhFaeTUIESdkg2CM2RlNQZKFHTs4BWzHDm+Q0xjOLv72t/2nTaY6nZVtIES8x5aXA6+9Rle0pQNmz6Yu+eILbyXy3LlsS+GnGWpAVq1iiyB/18XhoDyvvx6CQairY2HYkiU8qTGc9nP66Qx4BlKUDQ2sutu9m0FRawC1y8VV8fr1wM9+Rr95omJtrYLhcvWuGZ/VbjoYNltoucxK2ITTuuJ2X/cbY+6InDiJzbZtwN13c2ffuTLW7Wa90F13AT//ef80ChMncrHc2upfF7S28r2bOJF/L13KxWTnBaHDwd2CFTttaOBivLfv6YoVwTOSCgt5/QJmfVZW8uLX1HhTII2h6+ePf6R//5JL/Pu1n3uO7pbu3ffsdr7Yykr6xn/96wQIaPhhzBjK29Hhvx+5MfxCWBc5HIqLvQGeQO+B08kt+ZYt/HBkZ3MIRig90pWAhOMyaur0eyaAM8A5y/0Clwt4+GEqjIKCro/ZbFz07d3Latobb4yLiHElI4Ou78cfp7HsPrXM6aRH4KijmF3pdjN2Giy1PDeXruvm5sAp+v5oaQmuJ0R4Df26vdxu+p1aWrqWUVt+p4EDuWsYPZpbnO7U1QWPuBcX8w2y0rQSkQEDGCx++23/bWWrqvg+9OY1jBxJo3PggP96huZmBpafeILZVxbZ2czCOvVUDTb3gXBcRn/s/LeI/AH9KC1061Z+1gO1Vx46lNO9DhzonzU0c+dylf3SS/xOWoaztpYLaaeTFclW8snGjdS106f77ukWCYYOBTZtCpzw0tHBha/fY7Zu9b26t7DZuGJ99VUGUrsrpB07uOoN5Bu3/I9bt0bGIFjDtt99l3UDTidTN+fNY6qW5bIKl/POYyOpzz+ne8vaDnZ00NeXk8NgT28upgj9rr/9La14dz9kSwsN64ABfCwry/tYaytzmisrgUsvTdxdVoLTl/bX2QASdCkTebZvD77wsOJtu3cHNggdHdzt1tfze3nYYalRaCXC2OjRR3P1v8nThWrwYL7WsWO7fofHjWOsYN066srOi2+Lhga+l+HEbjszZw4LagN5ISoqgBNPDJDI89FHwRWotZXZu7dnrUB1NT8UmzZxVZuRwTdj1Kiub4jdHl503h/G0DgtWEC5i4v54Tx4EPjrXxn3+J//6d2WKysL+NGPgLfeYh5vVZXXmJ1wAtPA+lKtPG4ccMstwIMP0vCkp/PcbW00CDk5QFlZzy1oZiZXFW+/zccnTfJ5eiUw4cQQNsDbSsIOoBjAndEQKhEJ5ta0sILM/s6xdClrd5qavOcUYQHqRRd11Q/JyrBhrDUAuAO49VZmYnV/baNH09AOGMCdQ/cxAMZQ31xxRe+9ABMmUP9t387zd7+GdXV8zlNOCXCSxsbQVtQ2W1c3BsCV9KOPUrkVFvJFHTzILVN2Nlu1WgESpzMyQeUPP2TMorvvbuBAbtu2b2el8U039W4lnZXFPN1587gid7v52iI1rm7CBMZlNm7kjsnl4u7so4+4kupuDCxsNhqMt95Sg9BLwtkhnNHpdyeACs9s5X7BqFHBExusoS5Dh/p+/NVX+T0dPrzrDsLlApYto4740Y8SI+U8UnzxBV1ovlb/eXlU1lu38r2rrPTqQ5eLWUgTJvRtzoHdDtxwA/DAA9TNWVnUWx0d3LXk5HBBGjCDacgQ5tUHwrr4navmqqup2IqLqfy/+IIC2Ww8vrkZeP55+r6HDOGFP7yPZT1uN7BoEbdlvhSnCD/M69fzDR41qucxoZKeHr3K6bQ0uramTfPe98QTwXcfhYXsrxTqCk7pQjjrruuMMTs9tz2eyuLfR02yBGPyZH7Xm5r8H1NVRQXmq9XK/v3cwftaKdvtvH/LFrpaUomDB/1/L0W4kDv8cHpKtm/nQvrLL+my/9rXuIjtq4HMzfW2zZ46lQZh2DDWgP3+90xQCcjRR3P1Hijlsrqa1q1zQcry5XxhDQ1U/jYbFV16Ol+UtaJ+912uhi+6qO+zk8vL6QMLVM4twg/d6tV9e65Y43YH3yqKdB0/qoRFODuEUwDc2u2++T7uS0nS0qhA7ruPC77u7tfqai4Qv/Ut3/+/fLlXH/hChLrk9dfp906VRIlgqesiVMiW22zCBOrLCRN6ZnP5orqaN4eDSTz+3l+7ncanV56EkSPpl1692vcknpYWbjd+8IOu97/7Lvv8LF3Ki5uXx21gR0fXC9zQwA/UCSf0QrhuWIYnGBkZfOOSiXHjaPAC7RLq6qKXodAPCGoQROT7AK4DMFZE1nd6KBfAf6MlWCIyfTpXrI89xlWszeZdjAwfzk6e/qr6P/20Z/+e7uTkMPOwvj40ZZgMWAkz1nxkX1gJOKefHroHYtcutrv45BPvdcjOplv71FN7n0TjExHgqquoyNet4xPl5nqHPthszKyZMKHr/9XX01J1dHA3kJbGn1ZxBeDdEaSn8+KPGdM3WbOzQ1sht7cH/0AmGqedxna6gwb5VvjG8HpcemnsZUsRQtkh/BvAGwB+B+Cnne5vMMYk2RKj70ybRrfwpk1MKBGh0hs3LjKr+lRb2AwaxM4MH33k311dUUFdGihNvzOffcai37Q0GhDrfW9tZYxmyxbGDSJqFLKyWGDy2Wdc+e/cSeU7Z47/OQB5efR9db6oNhtTyqy0so4OGhabjcGWvhqEkSO5hW1o8O82MoYusLKyvj1XrJkyhauyTz7hyqvzF87tZibX1KmsGld6RVCDYIypA1AH4GIRGQjgULAwDSICY8yy6IqYeDgc9HuHE/+bMIEZcYFcu01N1CG96eaZyFx8MfXnzp303VuKuqODHpSCAuC73w3NGLa1sftyXl7PNHUr83D9er7Xp58e4Rdis/FCdt8J+OOkk5jmGchn1tzM8wWrUwiBhgZg1SobmuxnY+LyvyFrQhaGDHf0jC3v2UPl2peAcjxwOLgTe+opb7DNbvdme8yaBVx2mVYs94Fw0k6/C+BGcHDNOgCzAHwA4KSoSJZizJ4NLF7MhZm/rLmKCtbl9FEvJBx5ecBttzE+8u673r5uIsz/P+MM7iRCYf16Kj5/NWIiNDpvvMFU0rjqhuOOA555hitXX1kvbW282CNGMPoeqOoxAMYw0/K556gbszKPQXluBaYuW4TtGZkYf3wxBg+1079udQv83veSczuamUn33dln031XV8cP2BFH9M+eMREmnKDyjQBmAFhpjJkjIhMA/CY6YqUew4cD3/gGMwJHjuyaOeN2M1Y2dmxiDMSKBrm5wIUXMn29stLb+Tnc1PWPPw5epJaVxYyv/fvjvAguLAR+9Sv2OKqo4N8OBy1iczONwTHHsJR7xozQrWI33nmHGZklJdbuS1Ax+Gx0HDYFwze/A/d/1iBjigv5U0Yxk2n69OTPbS4sTJxxoilEOAah1RjTKiIQkQxjzBYRGR81yVKQc86hMlu0iDE9y5MgQn1w2WWJX5jW0MB4wKef0pAdeijrBELVZVZjwN7SPUHHHzZb1w7TcWP8eDZ4uuUWRsIdDq5yrfzkmhoWpfhLTwtCczN3BqNGdYuZiKC68FBUH38oDh4ElhYZ/PJ2ScpNgRI7wjEI5SJSAGARgLdEpAY68zgsbDYOm58zh2nndXVcqI0fnxy9j1asAP75TyranBwasg0bgBdfZIub00+Pvhdi1Ci25gmEy0Vj1csFd+SZPJmpaQsWsB+3NSmoro6+xHPO6XXvkk8+oZEMtOAfNAjY8aVgzx4+9apV3KXl5LDZYKQSIpTkJ5zmdud4fv21iLwHIB/A4qhIleJkZvKLmEysXctur8OHd62dGjSICumZZ6iUTj45unIccwzTTQOlsVZUcMeVUP2hhgwBrruOOwJrstLw4X3OINizx39MysKq1Xr4YYYzbDZew/Z2xrVKS4Hrr0+ORYkSXUJeFwj5tojcboxZCgaWp0VLMCVxcLs5Sre42HchrZX++fzzTP2MJE1NbDnx2WdcUBcXs87gyy99txKpraUCPPPMyMoRMQYOpLto/PiIpJNlZITWUmXrVu4mRo/2TvgbOZLGoKKCFdv19X0WR0lywnEZPQjADWYV3QGgAcACMNCspDCB+hFZZGTQGHz6aWTS2+vrOVpz6VIqPKtp4KxZ3vnHb77pnUHvcvH5Bw5kM71QaxqSnfHjg9ehVVYyienEE3279IYMoYFdtowZX0r/JRyDcLQx5kgR+RgAjDE1IhLJ0h8lQampCe04u53ZPX2lro7DySoqmEJqpY66XPR/b9jAyXQnnwysXEm3SUYGiwYnT45SqqnTyWrE9euZLjpqFP1Sca72PeQQGr9AMzjWreNuIFCbpCFDaGDnz0+9tGcldMIxCB0iYoenBbaIFIM7BiXFcThCCxa73ZGpDn7mGRqW7m1ArJT9ykr6w2+/PUYr2h07gL/8hb1/0tMpyPLlFPSMM5gTH6eorNU147e/ZeX8kCFehd7ezsI/my148a6VqtvcnHqFkUrohGMQHgCwEMBgEbkLwDcB/CIqUikJxbhxNAjB+hEZQxdGX6ip4ao/UE+j4mLq6J07e13LFTp793K7kpnZ02fmdDLCDXB+aJwYOZLGceFC7qCs7tp2O9v/FBcHdytZ1093B/2bUJrbPWmMuRRAEYCfAJgLQACcbYzpNzOV+zP5+aw1WLHCf/O+/fs5V72vvvsdO/gz2LRJgIHmqBuERYv405dryOGgkXj1VTro45jnOnQodwoXX8yVvjXnOyuLt5deCjwgzSpgDqkOprmZaWfvvssofl4eX/+MGZEbkqPEhVD2uUeJyGgAVwKoAPA02PCuQkQSJdNbiTIXXUS3+Y4dXTOJ2tu5Us/NBa6+uu91CMEyZixiUnhWW8uW14Gm51g5nytXRlmY0CgoYLHguHFe5X7ccbwu/jLA3G4ahPnzQ7h+e/YAv/gF8H//53Wh1dWx+O5nP+OHQUlaQjEID4H1BhMArAGw2nOzflf6AQMGMHvn3HNZrbx7N9ttVFUxkHvMMdSdW7b0bT6JFRgN1A8O4HP4m0wXMazpPsHiA1lZCa0Ii4qA73yH3q+amq7vbVMTjfycOcCRRwY5UWMj8Ic/cBVQWkrrk5nJLeTo0TzxPffQkCpJSSjdTh8A8ICI/M0Y8/1oCCEi5wP4NYCJAGYaY9TQJCDZ2cBZZ3ElWVXF+oAXX2Sq6caNPMbqUXTNNVyphktpKTOL6ur8z4RoaaEsU6b09pWESKgOdbc74Ttsfu1rfD9ffNE7y8MY6vLvfCfEoUwffURl7y//eNAgtudYsQL4+tcjKr8SG0KJIYghfo2BdUwf5PgUwLkAHu7DOZQYkZ7OxeLjj1PJdPfj19YyDnvbbSGMp+yGCDu+WvMOuruk29qYOXP11RGed+ALqyy7tTVwzmZLS9fZvwnK1Kk0ovv2cZeXkUE3YMiB5CVLgsdJiovZejUUg7B3L11tu3fz/S0ro5BRv7CKP0LJMnpPRBYAeMkYs8u601ODcDyAywG8B+CfvRXCCk6Ldt5KCtxu4NFHaQx8tYcoKOAx//wncOed4ccVJk3iZLqHH2Z+fUaG1wfucHCU6cyZzKh5/30qt4EDOYFy8uTgrRxCJj2daToLFtDq+Xoh9fUMqoYzHCOOWB0zekVNTdeZ0b7IzGQBSaD5xx0dwJNP8uLZbLT6TieNQ34+pxtZo/aUmBLKV2ceGFB+WkTGAKgFB+TYASwBcK8xZl20BOyOiFwD4BoAKPGX8qJElW3b6DIKVLk8cCC9Bzt29O67fcQRwL33sqhq82a6N8aO5SKyro5xzaoqZs5kZFAHrV3LFMybbgo8djcsTjuNBWmbN1OTWl3k3G5aq/Z24Cc/6R+r2gED+HoDpSJ1dNAo+DMGxrBX99KlNLLdj6utZR+NX/+avkMlpoQSQ2gF21Y8KCJpYPppizGmNpwnEpG3AfgKA/7cGPNSqOcxxjwC4BEAKCsr64ubSukl+/YFD/pai+l9+3q/2MvIAI4+mjeL+nq6kzo6urqqrGKqvXupS668kvd5ZwT0kowMWpjXX6crpLWVSszlov/lm9/sWz/vZOKEE5i/GmghVlkZeE7Bnj3cGfgyBgC3l83NTOW9+uq+SqyESViba2NMB4B9vXkiY0yU+2AqsSIcF1CkvYArVnCH4EsHV1WxNqG8nDuTwYO5mJ0/nwv9Xsd9MzLYovr003lyp5NbkP7WHvS444DXXmMAyVdRQ3MzVwonnuj/HCtW0KcXKII9ZAjdRxdfHLh4Qok4kfK2Jg3V1Vy1AvQAxLkVTVJiTSHzNRXSwqp8jeTEMmPYb8fXpMQ9e5gEk5lJXd3YSPdSaysHyGzfDlx3fhUcH/2XbT9dLqZBzZ5NIUOxXBkZTPDvrxQVAT/8IXD//bTKgwfTyjqd9Nm5XMAPfhDY1bN7d/DiNSvKXVurBiHGJIRBEJFzAPwZQDGA10RknTHmtEg+R0UF2zOvXev97hvD4srzz+9/i72+UFrKFfrBg/7ft6oq6ttALSjCxeWijuie1dTWxqE5OTnUTy4X8+sBGogxpQbNr7yLqjeeoq4qKOCHYOlS4O232SXv4ou1b0MoTJ0K3HEH53YuW0ZjYLdz93DyycFXAKH06wYYo4lYdoASKr16x0VkqDFmf6SEMMYsBPskRYX9+4G77qLiGDnSu1t1uWggtm5l90yd0R0aIpxzftdddBkXF3uNbHs7i9NaWoDjj2dguaQkMq4ju907krizrigv71oK0L0sYMTej3Dknn9ic94oDJ2Z7pUlN5cfgjffpG/pvPP6LmSiYgxX51u2cNs0eDAzo4INqPbF8OHApZdy7GdrK5V8qMp7xgxa70ArsMZGbvOCZTQpEae3Jvh1AMHqGhMCY5gi6XT2TLezumfu28ec+h//OD4yJiMlJcz0efJJ+u0Bvo87dlDPHnYY09aXLGEtwjXX9P37LcKK6A8/7Hot9+3rOkKyuZnPDwBi3Ji86Xm0FwxBQ0v6V0VtX2G3c7vz+uucvJOKrT4PHAAeeYQXyqq8drkYbT/3XAZYemOx7fbwexdNm8atnJWu2x1juJ2/8kqd6xkHevuOJ03BQHk5K2oDKaOhQ1ltuz9ie57+wahRLD676y5+z63umt/4BrueWq6l8nLgd79j/KavzJ3LXUh7u/c+t9urzzo6+LuVCDOwZjuymw+iPZ2+aJ/ZUQ4HFeQnn/RdwESjtpZv/q5dvBijR/PClZZylf7UU8Arr8ROnsxM1hnU1/ML17nPSVMTgz3HHsvYjhJzemsQ/h5RKaLIjh2Bg58AHxPxdtpUQkeEK+7164Hp03vm/4vQ4NbVMUElEM3N1Fu7d3dV+J0pLQUuv5xGprKS+mTgQLoDGxupU8rKvAvXjNY6GBG4XFxw+h1G73BEZrpPorF4MS3x0KE9vwTp6TQQCxdyFxErDjsM+NWvGI8oL+cF37WLRvmKK7id1HhOXOity+jFiEoRRazxi6HQl6Zs/Rmr0WegtM5hwxiDPO+8nm7r6mqmnb//vvcaZGQAp5zCHUf34+fMocvotdc4PS0tjcbg0EN565w55rKnA8agsZGuK7+ubperd/70RKa1FXjvvcBZP1YK6AcfcGsXK0aNYkZSXR13MQ4HjZYagriS8jGEQJ2LO2OMxrB6y+efB3clp6V5i3s71zVVVnLaV0MD9YFlVFpbOYpg/XrGdrqff/x43hobubNYuJAp7t3d0tWDDkFzexqyBnRg7Fg/FsvKkY16t7wYc/Agg2fBKvNycngR40F+vu/+J0pcSPkYwvjxXDE2Nvo/pq6OhqM/p5j3Bbs9eOWyRefdmjHAQw9R+Y8a1XWHkZkJjBnDrtLPP+//fDk5NORXXQXMm8d6hF276J4uLwe27clC+cST8bUx5cjK9CPk3r1sgtTrJj8JSjhbY+0jpqAfxBDsduCyy7zzYrvT2MieXZddpkkNvWXyZG/evz/a2ugG6rwL27mTMcRAu7jhw70N7ALhcDAL8p57gAsvBGbNorvpZz8DLnjqbGTNOoJBos4namlhL+iioshM90k0iouZTutvMo5FY2PSNOdTokuvXEbGmAcjLUg0mT6diQ2PPkoXhbWDbm/nCvOmm6jUlN4xYwbw739Tv/rre7ZvH2cpdA7qbtniDej7w+HgTmL79uCD4gEGtU/rUdKYzgrb//6X6aW7PE17BwxgL6ITT0zNiti0NG6bnn+e2y1fWC1kZ86MrWxKQtJvSgGPOoou4vXrqVwABiCnTOkfjSqjyYABwHe/C/z1r2yX39kl7HTSIzN6dE9F3dYW+q6sz+My09LYnO1rX2PKozGsOUj1ati5c1l9uWMHi246v976egZ1rr02NesvlLBJ8W9DVzIyuJqdMSPekqQeM2fS7//003QFWat+EVYsX3hhzySeIUOCdzGw4r0Rm19vs/kfxZaKZGUxKv/CC0zzstLujOEFuOkmbqEVBaFNTAvlq+gOtx22EhpWx4HNm7miLi6m6yQRMyQPP5yp5Tt2MJXU4WDra18Fqdbx6el03fnbpdXXM2uye/8iJQyysxkkO+ccbo87OphpMWaMBs6ULoSyQ9jruQWKuNkB6LSaCHPgAKeGff55144DaWnsODBvXuLFQUVoBEKZgZCdTRf+k08yFbV7HUNLCw3Lj3+ceK8zKcnNDS0QE0MaG72xPHXdxp9QDMJmY0zAPaWIfBwheRQPVseBpib63zsrxPZ2BnHb2oCzz46XhJHhlFP4el70lDoOGMBdUWMjFcT113PXoaQOxjCW99prXOzYbMwGPOEEfh60Hih+iAmSQC4imZ6paX06JhqUlZWZ1atXx/ppY8Izz7AJp79hXE4nc+7vuSc1WndXV7Pieds2KocpU7q2oFBSA2No/F96ickHgwZxsdPRwZ52Dgdw663+k6KUyCAia4wxZd3vD3WEpr+TXmGMeSwexiCVCbXjgAgzKc88M3ayRYtBgziQTEltPv6YFeilpV27VKSlsTV9bS1w330cq5yZGR8Z+zN9jSj9JiJSKF0IteNAbm78Og4oSrgYw55VAwf6b1lUUMDOAevWxVIyxSKULKP1/h4CEGKnICUcQk38MEaTRJTkobaWGWglQdJPcnO58501KyZiJReNjZxr0d5On9uhh0a0liaUMw0BcBqAmm73C4D/RkwS5SuKipiBE6jyF2AXBu04oCQL7e1cwATLGEtLC94Kpd/R2sqK86VLu9aS5OUxVe/44yOSiheKQXgVQI4xZl33B0TkP32WQOlBWhqrerXjgJJKWMXQLlfgLtdNTcDEibGRKSlobwfuv5+9XrpXmzc3A3//O1eHEQjCBXU4GGOuMsYs9/PYt/osgeKTk0/mbnDnzp5tG+rr2Q7iiiu044CSPGRn0w1UUeH/GGOYTq0D0zqxYgWwcSN9bd3dQ9nZbBX8/POB39gQUQ90gpKZCdx8M4fB7N/Pfmy7dtFApKez48Cxx8ZbSkUJj/nz2W3bV/daqyp/4kQuhhTwzXrtNRZn+HMJpaXRF7diRZ+fLpSg8lpjTMBhOKEco4SP1XHg3HOBL77gTqGgQDsOKMnLyJGsPL//fmbTFRRw0dvURO/HxIkcpKafbw/19SzSCRaJLyjg+MBzz+3T04USQ5gYINMIYHBZRx5FkZychOs4oMSSpiYqhrQ09vdO8j4eEyawoHLVKmYTtbay1cmcORy3rFM0OxHq5CkgIjOAQzEIE0I4JkjPSkVRwmbfPuCNN6g1jeEXftgw4Iwz6IxP4mV0Tg4NwJw58ZYkwcnLY6CwuTlwR8u6Opb295FQKpV3dv5bRO4Am9mtA7DOGKOlUYoSabZvZ7mu281h09akoPp64G9/Y8bJd76T1EZBCQG7nV0sn3nGf8qhy0V/cgQi8WF/mowxtwN4AEADgPNEJGnGaSpKUtDWxv4NWVld0wxFWIw0Zgx7m3zwQVzFVGLECScwhlBe3tMt1NHBMbDz50dkJnjIBkFE7hOh89IYU2GMWWyMudsYc3VfhRCRe0Rki4isF5GFIlLQ13MqStLyySfcCfgb5GOzcTDGK6+E52NWkpPsbOAnP2EV6q5dNADWz6oq4IILgPPPj1lhmkUjgJdF5CJjTJOInArgV8aY4/osBfAWgNuMMU4R+T2A2wDcGoHzKkry8eGHwWc85+ZSKVRVab/o/kBuLgfDV1RwWlZrKztCTpkS0WlZIRsEY8wvRORbAP4jIm0AmgD8NBJCGGOWdPpzJYBvRuK8ipKUWGXogbAmJrW3x0YmJTEYMoS3KBGOy2gugKtBQ1AM4AZjzPtRkOlKAG8EkOMaEVktIqurqqqi8PSKEmdGjGATs0BYw6jzNeNbiRzhBJV/DuCXxpgTwRX8syJyUqj/LCJvi8inPm5ndTrm5wCcAJ7ydx5jzCPGmDJjTFlxcXEY4itKknDcccwaCRQfqKgAjj5ae5coESUcl9FJnX7fICLzASwAEFIDBWPMyYEeF5HLAZwBYK4JNsZNUVKZkhJ2LfzwQ06S6R4srK+nsdCJQkqE6XUjbWPMPo8bqc+IyDwwiHyCMaY5EudUlKRFBLjqKir9VatYoZyTw11DQwPnit5yC/tAKEoE6dNkBWNMS4Tk+AuADABveTJbVxpjro3QuRUl+cjIAK67jt0M33+fA7QzMrhzOPLIwIMyFKWXRG7UTh8wxhwSbxkUJeEQocuotDTekij9BK17VxRFUQCoQVAURVE8JITLSFESlf376cLfuJEx3gkT2ENs+PCk70KtKD1Qg6AoPjCGg6oWLKDiHziQ97/zDvDmm8z4/OY3tdmoklqoQVAUHyxbBjz7LEsC0tK89+fmskj4lVeYCaqlAEoqoesbRelGRwfwwgt0C3U2BhZ2O+eav/QS0BKpxGtFSQDUIChKNz77jK2EAqX6p6dzbMHGjbGTS1GijbqMFKUbdXWhHScC1NREV5aEoa0NWLeOQZTqavrLTjgBmDEjeKtuJWlQg6DEnKoqYOtW6piCAmDyZCAzM95SeUlPD+04Y/pJwXBlJfCHP/Bnfj4vVl0d8PjjjLr/6Ef+xzsqSYUaBCVmNDQATzwBrF7NSYAivKWnA+ecA5x2WmKkch52GLOHnE7/Ywks+SdMiK1sMaelhcagoaFrxXRWFo1DbS1wzz3AnXcChYXxklKJEBpDUGJCczP1ytq1DMiOGUP9Mno0Bz899RSwcGG8pSR5eaw1KC/334F6zx6grAwoKoqtbDFn7VruDPwNZSko4ECfZctiKpYSHdQgKDFh6VKOgB01qmfufkYGDcPLL7MQLBG48EKu/nfs6DqrprkZ2L6dr+Pyy+MnX8x45x3/s50thgwB3npL5zunAGoQlKjjdAKLFwNDh/o/xuFgOuf70ZjB1wsyM4GbbgK+8x3+vWsXb+3twLe/Ddx6az+JpVZVBZ/Zm5FB15KO80x6NIagRJ26OrqgrWpff+TnA1u2xEamUMjIAE46CTjxRM6kAViYZrfHVazYkpNDRR8o0u508k3xVbShJBW6Q1CiTiIEivuCzUavSUFBPzMGAIMpBw8GPqayEjj2WO3jkQLoFVSiTn4+b01NgY+rrWUKqpJAHHMMt0oNDb4fb21laffJASfkKkmCGgQl6tjtwPz5nAvvL+7Y0cHHjj8+trIpQSgoAG68kT6z8nJeKIANnfbuZRbAd7/Lpk9K0qMxBCUmzJ7N8cDbtnEUcGfXS0sLsG8fM3sGD46fjIofJk4E7rgDePddpos5nfQDHn00cMopWpSWQohJ4lSxsrIys3r16niLoYRIczPwzDPAihUs7LLIyWEr6dmzkz/ekPI4nXQTZWRoEDmJEZE1xpiy7vfrDkGJGdnZwJVXAuedxwZy7e2MLYwfr7olaXA4+km+bf9EDYISc/Lz2RNNUZTEQoPKiqIoCgA1CIqiKIoHNQiKoigKADUIiqIoigc1CIqiKAqABDEIInKniKwXkXUiskREhsdbJkVRlP5GQhgEAPcYYw43xkwD8CqA2+Msj6IoSr8jIQyCMaa+058DACRv+bSiKEqSkjCFaSJyF4DLANQBmBPguGsAXAMAJdpQS1EUJWLErJeRiLwNwNfMrJ8bY17qdNxtADKNMb8Kdk7tZaQoihI+ce9lZIwJtWH6vwG8BiCoQVAURVEiR0LEEETk0E5/ngkggQYpKoqi9A8SJYZwt4iMB+AGsBPAtXGWR1EUpd+REAbBGHNevGVQFEXp7ySEy0hRFEWJP2oQFEVRFABqEBRFURQPahAURVEUAGoQFEVRFA9qEBRFURQAahAURVEUD2oQFEVRFABqEBRFURQPCVGprEQfpxPYvh1oagKysoBx44C0tHhLpShKIqEGIcUxBli6FFi0CKirA0R4X3Y2cMYZwKmnAnZ7vKVUFCURUIOQwhgDvPgijcGwYcDo0d7HWluBp58G9u8HLr8csKnzUFH6PaoGUpgdO4BXXqEhyM7u+lhmJlBaCrz3HrBxY1zEUxQlwVCDkMK89x6Qng44/OwDbTYgNxd4883YyqUoSmKiBiGFWbcOKCwMfExREXcIbndMRFIUJYFRg5DCuN2hxQaM4U1RlP6NGoQUZtw4oLY28DH19cCoUZpppCiKGoSU5pRTgIYG/6t/Y4CDB4H582Mrl6IoiYkahBRm0iSgrAzYubNnjMAYYPduYPx4HqMoiqJ1CCmM3Q5873vAU08B77/P+xwOVi0bQ0NwxRXMRFIURVGDkOJkZABXXgmcdRazjmpqgLw84IgjgCFD4i2doiiJhBqEfkJhITB3brylUBQlkdEYgqIoigJADYKiKIriQQ2CoiiKAgAQk8QlqiJSBWBnFJ+iCMCBKJ4/miSr7Cp37ElW2ZNVbiD+so82xhR3vzOpDUK0EZHVxpikzNJPVtlV7tiTrLInq9xA4squLiNFURQFgBoERVEUxYMahMA8Em8B+kCyyq5yx55klT1Z5QYSVHaNISiKoigAdIegKIqieFCDoCiKogBQgxAUEblTRNaLyDoRWSIiw+MtUyiIyD0issUj+0IRKYi3TKEiIueLyEYRcYtIwqXmdUdE5onIVhHZJiI/jbc8oSIi/xCRShH5NN6yhIOIjBKR90Rks+dzcmO8ZQoFEckUkY9E5BOP3L+Jt0zd0RhCEEQkzxhT7/n9BgCTjDHXxlmsoIjIqQDeNcY4ReT3AGCMuTXOYoWEiEwE4AbwMIAfG2NWx1kkv4iIHcBnAE4BUA5gFYCLjTGb4ipYCIjIbACNAJ4wxkyJtzyhIiLDAAwzxqwVkVwAawCcnejvuYgIgAHGmEYRSQOwHMCNxpiVcRbtK3SHEATLGHgYACApLKgxZokxxun5cyWAkfGUJxyMMZuNMVvjLUeIzASwzRiz3RjTDuAZAGfFWaaQMMYsA1AdbznCxRizzxiz1vN7A4DNAEbEV6rgGNLo+TPNc0sofaIGIQRE5C4R2Q3gEgC3x1ueXnAlgDfiLUSKMgLA7k5/lyMJlFOqICKlAKYD+DDOooSEiNhFZB2ASgBvGWMSSm41CABE5G0R+dTH7SwAMMb83BgzCsBTAK6Pr7RegsntOebnAJyg7AlDKLInCeLjvoRa9aUqIpIDYAGA/+m2k09YjDEuY8w0cMc+U0QSylWnA3IAGGNODvHQfwN4DcCvoihOyASTW0QuB3AGgLkmwYJFYbzniU45gFGd/h4JYG+cZOk3eHzwCwA8ZYx5Md7yhIsxplZE/gNgHoCECerrDiEIInJopz/PBLAlXrKEg4jMA3ArgDONMc3xlieFWQXgUBEZIyLpAC4C8HKcZUppPMHZRwFsNsb8Kd7yhIqIFFvZfiKSBeBkJJg+0SyjIIjIAgDjwayXnQCuNcbsia9UwRGRbQAyABz03LUyGbKjAEBEzgHwZwDFAGoBrDPGnBZXoQIgIqcDuA+AHcA/jDF3xVei0BCRpwGcCLZirgDwK2PMo3EVKgRE5HgA7wPYAH4vAeBnxpjX4ydVcETkcACPg58TG4DnjDF3xFeqrqhBUBRFUQCoy0hRFEXxoAZBURRFAaAGQVEURfGgBkFRFEUBoAZBURRF8aAGQVEURQGgBkFJEUSkVERaPH1irPt6tKUWkSxPK/N2ESnq43NmichST8dTiMgNnpbMYbUJEZECEbmuL7KE8Bw9Wl2LSLqILBMR7VigAFCDoKQWX3j6xFhtqf8KYD6ASQAuFpFJxpgWzzGRaC9xJYAXjTEuz9/XATjdGHNJmOcp8PxvWAgJ9Tv8T7BNwld4urO+A+DCcJ9bSU3UIChJg2coyime3/+fiDwQ4PBYtKW+BMBLHnkeAjAWwMsicpOIfNszDGWdiDzcaRexSETWeAakXOM5z90AxnmOvcez2+m8kv+xiPza83upZxfyIIC1AEb5e67OBGh1vcjzOhRFDYKSVPwKwM9F5BKw5fFNAY6NaltqT9+iscaYLwHA0xZkL4A5ABaDq+7jPLsRF7xK90pjzFEAygDcICKFAH4Kz+7GGHNLCE8/HhxqMx1AdoDnCoVPAcwI43glhVHfoZI0GGOWeRqb3QzgRMtVIyJ3gs3OOhPtttRFYJ8lX8wFcBSAVRQXWWD/e4BG4BzP76MAHApgf5jPvbPTlK1AzxUUY4zLE0/J9QybUfoxahCUpEFEpgIYBuCApbxEZCh8f47DbkstIj8AcLXnz9MBnNP5b2NM5/9vAZDp71QAHjfG3Nbt/CeCHS6PMcY0e9of+zqHE113792PaQr2XGGSAaC1D/+vpAjqMlKSAuEc3afAOECTiFjdT6cDWOfjX8JuS22M+avHbTPNGLO3+9/djq0BYBcRXwr9HQDfFJHBHtkHichoAPkAajzGYAKAWZ7jGwDkdvr/CgCDRaRQRDLAmRb+8PdcIeFxWVUZYzpC/R8ldVGDoCQ8IpIN4EUAPzLGbAZwJ4Bfex6eBh8GwTNP+noAb4Izd58zxmyMsGhLABzv47k3AfgFgCUish7AW+DOZjEAh+e+O8FZ1zDGHASwwjMx7h6Pcr4DHAv5KgL0zA/wXF3wtLr+AMB4ESkXkas8D80BkNBto5XYoe2vlaRGRB4F3TolAF41xoQ0klBEvgRQZow50Ifnng7gZmPMpb09R7wRkRcB3GaM2RpvWZT4ozsEJakxxlxljHGD2TX5nQvTfGEVpgFIg3e4Sm+f+2MA7/lK80wGPK60RWoMFAvdISiKoigAdIegKIqieFCDoCiKogBQg6AoiqJ4UIOgKIqiAFCDoCiKonhQg6AoiqIAUIOgKIqieFCDoCiKogAA/j9ckqQf6WEScQAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": { - "tags": [], "needs_background": "light" - } + }, + "output_type": "display_data" } + ], + "source": [ + "plot_dataset(train_x, train_labels)" ] }, { @@ -1057,41 +1024,41 @@ "id": "SjPlpf2-wHl8" }, "source": [ - "## Обучение простейшего одноуровневого персептрона вручную\r\n", - "\r\n", - "Используем возможности tensorflow по вычислению градиента для обучения одноуровневого персептрона.\r\n", - "\r\n", - "Для начала, задаём архитектуру сети, в которой будет 2 входа и один выход. Соответственно, матрица весов $W$ будет иметь размерность $2\\times1$, а вектор сдвига $b$ -- $1$.\r\n", - "\r\n", - "Для удобства сгруппируем все параметры в отдельный класс:\r\n" + "## Training One-Layer Perceptron\n", + "\n", + "Let's use Tensorflow gradient computing machinery to train one-layer perceptron.\n", + "\n", + "Our neural network will have 2 inputs and 1 output. The weight matrix $W$ will have size $2\\times1$, and bias vector $b$ -- $1$.\n", + "\n", + "To make our code more structured, let's group all parameters into a single class:" ] }, { "cell_type": "code", + "execution_count": 28, "metadata": { "id": "J1KaixW-cMWJ" }, + "outputs": [], "source": [ - "class Network():\r\n", - " def __init__(self):\r\n", - " self.W = torch.randn(size=(2,1),requires_grad=True)\r\n", - " self.b = torch.zeros(size=(1,),requires_grad=True)\r\n", - "\r\n", - " def forward(self,x):\r\n", - " return torch.matmul(x,self.W)+self.b\r\n", - "\r\n", - " def zero_grad(self):\r\n", - " self.W.data.zero_()\r\n", - " self.b.data.zero_()\r\n", - "\r\n", - " def update(self,lr=0.1):\r\n", - " self.W.data.sub_(lr*self.W.grad)\r\n", - " self.b.data.sub_(lr*self.b)\r\n", - "\r\n", + "class Network():\n", + " def __init__(self):\n", + " self.W = torch.randn(size=(2,1),requires_grad=True)\n", + " self.b = torch.zeros(size=(1,),requires_grad=True)\n", + "\n", + " def forward(self,x):\n", + " return torch.matmul(x,self.W)+self.b\n", + "\n", + " def zero_grad(self):\n", + " self.W.data.zero_()\n", + " self.b.data.zero_()\n", + "\n", + " def update(self,lr=0.1):\n", + " self.W.data.sub_(lr*self.W.grad)\n", + " self.b.data.sub_(lr*self.b)\n", + "\n", "net = Network()" - ], - "execution_count": 199, - "outputs": [] + ] }, { "cell_type": "markdown", @@ -1099,22 +1066,25 @@ "id": "rQ7W6TOacIAI" }, "source": [ - "> Обратите внимание, что мы используем `W.data.zero_()` вместо `W.zero_()`. Это делается потому, что к тензору, состояние которого отслеживается через механизм *Autograd*, нельзя обращаться напрямую.\r\n", - "\r\n", - "Функция обучение будет такая же, как в прошлом примере, но функция ошибки будет представлять собой логистическую функцию ошибки. Для этого нам нужно получить на выходе сети значение **вероятности** класса 1, т.е. необходимо привести выход сети $z$ к диапазону [0,1] с помощью передаточной функции `sigmoid`: $p=\\sigma(z)$.\r\n", - "Далее, если для примера с номером класса $y_i\\in\\{0,1\\}$ был получен выход сети $p_i$, то ошибка вычисляется как $\\mathcal{L_i}=-(y_i\\log p_i + (1-y_i)log(1-p_i))$. \r\n", - "\r\n", - "В PyTorch оба эти этапа (применение сигмоиды и взятие логистической функции ошибки) делается одним вызовом `binary_cross_entropy_with_logits`. Поскольку мы делаем обучение по минибатчам, то необходимо усреднить ошибку по всем компонентам минибатча - это функция `binary_cross_entropy_with_logits` делает сама, возвращая одно число. \r\n", - "\r\n", - "> Следует отметить, что вызов `binary_crossentropy_with_logits` эквивалентен последовательному применению `sigmoid` и `binary_crossentropy`" + "> Note that we use `W.data.zero_()` instead of `W.zero_()`. We need to do this, because we cannot directly modify a tensor that is being tracked using *Autograd* mechanism.\n", + "\n", + "Core model will be the same as in previous example, but loss function will be a logistic loss. To apply logistic loss, we need to get the value of **probability** as the output of our network, i.e. we need to bring the output $z$ to the range [0,1] using `sigmoid` activation function: $p=\\sigma(z)$.\n", + "\n", + "If we get the probability $p_i$ for the i-th input value corresponding to the actual class $y_i\\in\\{0,1\\}$, we compute the loss as $\\mathcal{L_i}=-(y_i\\log p_i + (1-y_i)log(1-p_i))$. \n", + "\n", + "In PyTorch, both those steps (applying sigmoid and then logistic loss) can be done using one call to `binary_cross_entropy_with_logits` function. Since we are training our network in minibatches, we need to average out the loss across all elements of a minibatch - and that is also done automatically by `binary_cross_entropy_with_logits` function: \n", + "\n", + "> The call to `binary_crossentropy_with_logits` is equivalent to a call to `sigmoid`, followed by a call to `binary_crossentropy`" ] }, { "cell_type": "code", + "execution_count": 25, "metadata": { - "trusted": true, - "id": "kdDxWeCqwHl8" + "id": "kdDxWeCqwHl8", + "trusted": true }, + "outputs": [], "source": [ "def train_on_batch(net, x, y):\n", " z = net.forward(x).flatten()\n", @@ -1123,9 +1093,7 @@ " loss.backward()\n", " net.update()\n", " return loss" - ], - "execution_count": 200, - "outputs": [] + ] }, { "cell_type": "markdown", @@ -1133,34 +1101,26 @@ "id": "zAAgw0h6KzUd" }, "source": [ - "Для чтения данных воспользуемся встроенными функциями PyTorch по организации датасетов. Концепция датасетов основана на двух понятиях:\r\n", - "* **Dataset** - собественно источник данных, может быть **Iterable** и **Map-style**\r\n", - "* **Dataloader** отвечает за загрузку данных и разбиение на батчи.\r\n", - "\r\n", - "В нашем случае мы определяем датасет на основе тензора, и далее разбиваем входные данные на минибатчи по 16 элементов. Каждый минибатч включает в себя два тензора, входные данные (размером 16x2) и выходные (вектор длины 16 целого типа - номер класса)." + "To loop through our data, we will use built-in PyTorch mechanism for managing datasets. It is based on two concepts:\n", + "* **Dataset** is the main source of data, it can be either **Iterable** or **Map-style**\n", + "* **Dataloader** is responsible for loading the data from a dataset and splitting it into minibatches.\n", + "\n", + "In our case, we will define a dataset based on a tensor, and split it into minibatches of 16 elements. Each minibatch contains two tensors, input data (size=16x2) and labels (a vector of length 16 of integer type - class number)." ] }, { "cell_type": "code", + "execution_count": 26, "metadata": { - "trusted": true, "colab": { "base_uri": "https://localhost:8080/" }, "id": "PfyqjVb2wHl8", - "outputId": "b3a685a9-304c-4e7e-adf9-2858cc47c3a5" + "outputId": "b3a685a9-304c-4e7e-adf9-2858cc47c3a5", + "trusted": true }, - "source": [ - "# Create a tf.data.Dataset object for easy batched iteration\n", - "dataset = torch.utils.data.TensorDataset(torch.tensor(train_x),torch.tensor(train_labels,dtype=torch.float32))\n", - "dataloader = torch.utils.data.DataLoader(dataset,batch_size=16)\n", - "\n", - "list(dataloader)[0]" - ], - "execution_count": 201, "outputs": [ { - "output_type": "execute_result", "data": { "text/plain": [ "[tensor([[ 1.3383, -0.9861],\n", @@ -1182,11 +1142,17 @@ " tensor([1., 1., 0., 0., 0., 0., 1., 0., 0., 1., 0., 1., 0., 0., 0., 0.])]" ] }, - "metadata": { - "tags": [] - }, - "execution_count": 201 + "execution_count": 26, + "metadata": {}, + "output_type": "execute_result" } + ], + "source": [ + "# Create a tf.data.Dataset object for easy batched iteration\n", + "dataset = torch.utils.data.TensorDataset(torch.tensor(train_x),torch.tensor(train_labels,dtype=torch.float32))\n", + "dataloader = torch.utils.data.DataLoader(dataset,batch_size=16)\n", + "\n", + "list(dataloader)[0]" ] }, { @@ -1195,11 +1161,12 @@ "id": "xrwgkbQjhkEp" }, "source": [ - "Теперь мы можем пройтись по всему датасету и организовать процесс обучения:" + "Now we can loop through the whole dataset to train our network for 15 epochs:" ] }, { "cell_type": "code", + "execution_count": 29, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -1207,39 +1174,46 @@ "id": "QGchp9D6gVJa", "outputId": "b4c4751d-cb56-4104-d5b5-f1ae9d3d858d" }, - "source": [ - "for epoch in range(15):\r\n", - " for (x, y) in dataloader:\r\n", - " loss = train_on_batch(net,x,y)\r\n", - " print('Epoch %d: last batch loss = %.4f' % (epoch, float(loss)))" - ], - "execution_count": 202, "outputs": [ { + "name": "stdout", "output_type": "stream", "text": [ - "Epoch 0: last batch loss = 0.6447\n", - "Epoch 1: last batch loss = 0.6046\n", - "Epoch 2: last batch loss = 0.5817\n", + "Epoch 0: last batch loss = 0.6423\n", + "Epoch 1: last batch loss = 0.6035\n", + "Epoch 2: last batch loss = 0.5813\n", "Epoch 3: last batch loss = 0.5680\n", - "Epoch 4: last batch loss = 0.5596\n", - "Epoch 5: last batch loss = 0.5542\n", - "Epoch 6: last batch loss = 0.5507\n", - "Epoch 7: last batch loss = 0.5485\n", - "Epoch 8: last batch loss = 0.5470\n", - "Epoch 9: last batch loss = 0.5459\n", - "Epoch 10: last batch loss = 0.5453\n", - "Epoch 11: last batch loss = 0.5448\n", - "Epoch 12: last batch loss = 0.5445\n", - "Epoch 13: last batch loss = 0.5443\n", - "Epoch 14: last batch loss = 0.5442\n" - ], - "name": "stdout" + "Epoch 4: last batch loss = 0.5597\n", + "Epoch 5: last batch loss = 0.5544\n", + "Epoch 6: last batch loss = 0.5510\n", + "Epoch 7: last batch loss = 0.5487\n", + "Epoch 8: last batch loss = 0.5472\n", + "Epoch 9: last batch loss = 0.5462\n", + "Epoch 10: last batch loss = 0.5455\n", + "Epoch 11: last batch loss = 0.5450\n", + "Epoch 12: last batch loss = 0.5447\n", + "Epoch 13: last batch loss = 0.5445\n", + "Epoch 14: last batch loss = 0.5443\n" + ] } + ], + "source": [ + "for epoch in range(15):\n", + " for (x, y) in dataloader:\n", + " loss = train_on_batch(net,x,y)\n", + " print('Epoch %d: last batch loss = %.4f' % (epoch, float(loss)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Obtained parameters:" ] }, { "cell_type": "code", + "execution_count": 30, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -1247,19 +1221,18 @@ "id": "5QaDiCQUkFOT", "outputId": "45b4a66b-1222-40f4-c758-d58f1c7daf8c" }, - "source": [ - "print(net.W,net.b)" - ], - "execution_count": 203, "outputs": [ { + "name": "stdout", "output_type": "stream", "text": [ - "tensor([[1.0563],\n", - " [0.4450]], requires_grad=True) tensor([0.], requires_grad=True)\n" - ], - "name": "stdout" + "tensor([[1.0583],\n", + " [0.4454]], requires_grad=True) tensor([0.], requires_grad=True)\n" + ] } + ], + "source": [ + "print(net.W,net.b)" ] }, { @@ -1268,38 +1241,45 @@ "id": "s4_Atvn5K4K9" }, "source": [ - "Для демонстрации того, как сработало обучение, построим граничную прямую $W\\times x + b = 0.5$" + "To make sure our training worked, let's plot the line that separates two classes. Separation line is defined by the equation $W\\times x + b = 0.5$" ] }, { "cell_type": "code", + "execution_count": 31, "metadata": { - "trusted": true, "colab": { "base_uri": "https://localhost:8080/", "height": 283 }, "id": "PgRTHttLwHl9", - "outputId": "d9abf92f-cb70-4c56-ccd0-5e027239da58" + "outputId": "d9abf92f-cb70-4c56-ccd0-5e027239da58", + "trusted": true }, - "source": [ - "plot_dataset(train_x,train_labels,net.W.detach().numpy(),net.b.detach().numpy())" - ], - "execution_count": 204, "outputs": [ { - "output_type": "display_data", + "name": "stderr", + "output_type": "stream", + "text": [ + "C:\\Users\\dmitryso\\AppData\\Local\\Temp/ipykernel_50828/2721537645.py:17: UserWarning: Matplotlib is currently using module://matplotlib_inline.backend_inline, which is a non-GUI backend, so cannot show the figure.\n", + " fig.show()\n" + ] + }, + { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEKCAYAAAASByJ7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydd3iUVdrG7zOT3nsIISGh92YA6QgIglhZP+xlLWtvKEp1V0AF17oWLKyu3V0bKCKCUqVIly4ISYBUEtLLZDLn++NmnJSpybQk53ddcxFm3nnfZ9p5ztOFlBIKhUKhUGg8LYBCoVAovAOlEBQKhUIBQCkEhUKhUJxHKQSFQqFQAFAKQaFQKBTnUQpBoVAoFAAAH08L0BxiYmJkSkqKp8VQtCCOFx5Hma4MfeP7Qiu0nhZHofAIu3btOiuljG14f4tWCCkpKdi5c6enxVC0IHZm7cTgdwbj/8b9H2aNmuVpcRQKjyCEyDB3f4tWCAqFo6S1T8OUrlPwwtYX8MDQBxDiF+JpkTzL2bPA1q3AiROAVgv07QukpQHBwZ6WrGlICWRkABs2AEePAhoN0L8/MGoU0K6dp6XzekRLrlROS0uTykJQOMq209swbNkwLJmwBI+PeNzT4ngGgwFYvhxYsQIQAggJ4X3l5YCfH/C3vwGDBnlaSseorQU++ghYtw7w9QUiIqggzp3ja7vmGmDyZL7eNo4QYpeUMq3R/UohKFxFdTWwdy9w+DB/l506ec/mc+KHE7Evdx9OPnQSQb5BnhbH/Xz7LfDf/wIdOwI+DRwFFRVAXh7wxBNAz56eka8pfPkllVxKCi2DutTUAJmZVHQjR3pEPG/CkkJQWUYKl7BvH/DII8CbbwLbtgG//gq89x7w8MO05j29D5k/Zj7yyvPw9q63PSuIJygt5cKZnNxYGQBAUBAQHg58/rnnPyh7KS0Fvv+er6mhMgBoMbRrR6VRW+t++VoIXqMQhBBJQoh1QohDQoiDQoiHPC2TomkcOgS89BItgZQU/g7j4/l3TAywbBnwyy+elXFk8khclHIRFv+yGJU1lZ4Vxt3s2cNF0dfX8jEREUB6OnD6tNvEaha//cbXZE7BGQkOBoqKgOPH3SdXC8NrFAIAPYAZUspeAC4EcJ8QopeHZVI4iJTAxx9zPTHnGvL3BxISgE8+AXQ698tXl/lj5iOnLAfL9izzrCDuJjvbujIA6GfXaOh/bwkUFjIobgshgJIS18vTQvEahSClzJZS7j7/dymAwwASPSuVwlHS04GsLHocLBEYCFRWAgcOuE0ss4zpOAajkkfhuc3PoVpf7Vlh3ElAgP1uE2s7bmeTlQX87380L19/Hdi+nYEoewgKsu81ScldicIsXqMQ6iKESAEwEMB2M4/dJYTYKYTYmZ+f727RFDY4e5abMFuJHEIAOTnukcmyDALzRs/DmdIzeG/ve54Vxp307s3F01p8oKaGysAdhZ81NcC//w3Mng2sXg2cPEm/49KlwGOPAceO2T5Hr/POBFuvydcX6NLFOXK3QrxOIQghQgB8CeBhKWUj205K+baUMk1KmRYb26jQTuFhtFr74pB6PfDHH8AzzwAPPcSElhUrgIIC18tYlwmdJuDCDhfi2c3PQlfrYR+Wu+jcmcHXs2fNPy4lcOYMMG4cd96u5qOPgPXrKVOHDkBUFBAba8qAWrLEdiwjIQEYOJBym0NKnmPCBPe8phaKVykEIYQvqAw+llJ+5Wl5FI5j3FBas94rKpiFtH49rYSgIFNa/JNPArt3u0NSIoTA/NHzkVmciQ/3fei+C3sSIYB776X2PnWK2tlIZSV36N26AVde6XpZsrKAjRvNp4oC9D1qNNwt2OK226gYTp5kPQVARVBSwsK7AQPc85paMF5ThyCEEAD+A6BQSvmwPc9RdQjeyZtvAjt3AklJjR8zGIBVq6gwLrussWupooIb1/nz3eOtAAApJYa8OwQFFQU4ev9R+GptBFxbC2fPMlVz40YunFIyE2DSJO6kAwJcL8MXX/ALYe7LYqS2ljv/l16iwvr1V+4kgoNZPNe9u0mZVFQAmzfznMXFfE0JCcCUKcDQobaD6W0Ery9ME0KMBLAJwH4AhvN3z5ZSfm/pOUoheCfFxcCzz7K2KSHB9BusrWWR2sGD/H2Ghpp/fk4Ouw3cc4/7ZF5xdAWu+OwKvH/F+7hlwC3uu7A3UF7OLB2NBoiLc++i+fLL3L1HRVk/LiODRXKHDtGy8fenZVNdTTfTAw/Ub01hMABlZaYqbFWdXA+vVwhNQSkE76WkBPjmG2DTJv42heC/lZX8LVvb/dfW0pPw6qvuq2qWUmLgWwNRUVOBw/cdhlajOqG6haVLWUMQF2f5GCmBH38EEhMZPG7oWsrLY7uNp54CIiNdK28rQVUqK9xKWBhw883cAD75JDBzJvDii9zkRUdbf64xndzoBnYHQgjMHzMfxwqP4fODn7vvwm2dCy6gm8caWVncYfTsaT7OEBfHx3/+2TUytiGUQlC4lOBgoGtXxigjIugmspVaLiWtCXe4sOtyZY8r0SeuDxZuXIhag2pv4Bb69+fuobjY/ONS0seYmmq98Cw+Hli7lqmliiajFILCrYwcSdeuNQoK6BkIC3OPTEY0QoN5o+fh8NnD+PLwl+69eFvFz495x+XltATqpqeVlTG+EBHBHYU1/P2501BVyM1CKQSFW+nXjy4jSynwNTXsU3bppe6Vy8i0ntPQI6YHFmxcAIM02H6Covl07kz/f1oalcKpU+xMCjCV9MILbRe3GLOk3FlZ3QpR757Crfj7A48+CixezN98XJypk0JeHjd5N9xgKjx1N1qNFnNHzcWNX9+I5UeW46qeV3lGkLZGYiJw113AddexAZ2PD78cWi13CZ99Zr0fSkkJs43cbVa2MpSFoHA7iYnA008DV19Nr0BGBtPMBwwA5s4FJk70rHzT+0xH16iueHrj02jJWXgtktBQ1iQkJJhiBkOH8u9KC11ppaTJeemlKr20mSgLQeERIiKAqVP5G66uZuq7Pc0q3YGPxgdzRs3BrctvxXe/f4fLul/maZE8h7F4ZO1atn7w8+MCPWIEe5m7g4gIDrZ57TVaAFFRpoW/ooLdW0eNAoYMcY88rRhVh6BQmKGmtgY9Xu+BqMAo/HrHrxBtcedZXs5F+NAhFneFhFBBGIvY7riD/n13ceQIB9wcP87dg8FABXHppey75C07ihaApToEZSEoFGbw1fpi1shZuPPbO/HD8R8wuetkT4vkXqQE3n6bg+pTUuq7YkJC6L55800WgnXv7h6ZevRgR9S8PKap+vkxbqACyU5DxRAUCgvc3P9mJIcnt81YQkYGOxAmJZn3ywcG0t+/fLl75RKCNQfdulFROaoMpGTzuw0bgHXr6A4zqGwyI0q1KtyOlKZ2++YKT70FP60fZo2chXtW3oOfTv6ECZ0meFok97F1Kz8ga66y6GguqAUFtsvPvYFTpzi/NT2dX0Lja4uJYXpr794eFc8b8OKfo6K1ce4cuxg/8ABjhHfdxbkoGRmelswytw24DYmhifjHhn+0LSshL49WgDWMYzZLS90jU3M4cwZYtAjIz+echdRUWhgpKbQQ/vlPz4/w8wKUQlC4hcxMYN484Ouv6YLu2JHNKbdtY03Spk2eltA8/j7+eGLEE9icuRkbMjZ4Whz3ERpquw2EscdISxhJ+fHHVGCxsY2tHmPm0rvv1p8N0QZRCkHhciorgRde4GayY0dTjyIfH6B9e6acL1vGCWreyB2D7kC7kHZYsHGBp0VxH0OGAFVV1o8pKeGHV7fttDeSnc1MKWsdVUNDWRB3+LD75PJClEJQuJxdu7h2WGp57+9PJbF6tXvlspdA30DMHD4TP5/8GZu3fEbXQkaGfbNCWyo9e1Jb5+WZf7y2lsVg5qYceRvZ2dyN2JJTq/Vu/6UbUApB4XI2brTdUSA2llPWbG1KPYLBgL8VdUacIRALVszg5K6//50pkK21DkarZdO5gAAuksYqYWNVcEYG8//dWYfgalqzgrcTpRAULqekxLab2VhTZKs1ttsxGID330fQJ//DY/oh+DEwC9uSBAfCV1dzis+PP3paStcQH88Az7RprAjOyOCtY0fg8ceB6dO93zoA2CvFGO+whsHAYHMbRqWdKlxOdDSDykFBlo/R62nV20pscTv79gHr1wOpqbhHk4TF8lcswEasFNfT7AkIAD79FOjbl/701kZ4OHuMTJlCK8HHp2UEkesSHw/06QMcO2Y53lFczC9qjx7ulc3LUBaCwuWMHWs7MzE3l+1xvK7odNUqLooaDULgh0cxDN+LY9iFLD7u50dNtqGVZyBpNJx21NKUgZEbbuCXKze3vmtISuZDFxczD7qNt79QCkHhcvr358YsN9f846dPMwnkxx+Bv/4VeOIJbsptTVZ0OdXVwO+/14uG348hiJABWICNpuOio4E9ezwgoMJu2rUD5syhuyszk8VpRhdYSAgwa5b7WnB4Md62H1O0Qvz8gBkzWPuTns7mlcHBXG/37eNvsl8/oEsXbkTLyoD332fW0eOPW85OcjkGA33kdfzkYfDHI7gQT4n12CtzMADtKLQa3ej9JCRwwPeZM1QKUtKd1KlTy4iFuAFlISjcQmwsE3PuvJP90IqLmdGo19NF3acPrXUhmBKeksKOCP/6lwdbzQQEUNjy8np3P4ihCJP+WGi0EoqLKbCiZZCYCAwbBgwfzmltShn8iVIICrcRGMg4wVNPcaFPTTXNWDdH+/bsQ3bsmHvl/BMhgMmTG+XiRyAAD2IIvhSHcUDm0rc1oQ31OVK0WpRCUHiE3Fxa7damIgJ0N23f3owLSQnodE3PMR8xgi2Wz5ypd46HcSFCpB8WVfzAWcAO+p9rahh2+Ppr4JtvgP3723zXBIUXoGIICo9QWWlyEVnD358dBRwmP5+ZP2vXstrN15fpThddRNPDXoKCGMhYupQDWjQawM8P0Tod7g/vjsURB/DUtDHo4UDb1t272dSvrIyJL1Ky8DciArj77jaf+ajwIEohKDxCcDAXwbpdiM1RVdWEzsq//87mSXo9+9f4+3NLvm4db/fdBwwcaP/5IiKY+pSZyW19aSkQHY1Hez+GVz8ejEXbn8eHV31o16n27gVeeYViNZxAWVICLFnChJeuXR14vQqFk1AuI4VHiI1lcse5c5aPMc5NcKg7QlERW0sEB3O4izFv3teXrp+oKOD119nfxhGEYMrilVcCN90ETJmC2I69cE/aPfhk/yc4VmA70KHXM3sqNpbiNSQsjAH1//xHdVFQeAalEBQeQQjgiiuoEHQ688ecOUP3SadODpx4yxbms1qKVBvLpdevd0Rcizw2/DH4af3w7OZnbR575AgTkkJCLB8TEcG6jPR0p4inUDiEUggKj9GvH3Drrdysnz7NdVyvp5I4eZIb+nvvdTArcP162z6m+Hge54RteLuQdvjbBX/DB/s+wMlzJ60em5Nj+5LGmTOWivgUCleiFILCo4wfDyxYwHhvWRljwdHRdPPPnm27S2ojSkuZmmQNX19qn9rapopdj5kjZsJH42PTStBo7NNBUnr3aFFF68WrgspCiH8DmAogT0rZx9PyKNxDhw7AjTfy1hDjAmq3lRAVZcoqskR1NV1HTupb0z60Pe4YdAfe3vU25oyag44RHc0el5LC12EtkC4lb8nJThFNoXAIb9uHvA/gEk8LofAsNTUcM7BgAXD77axufuUV9juyucMePx4oLLR+TF4ecPHFTq1QfWLEEwCAxb8stnhMairj0mfPWj5PTg4bp3r7EDJF68SrFIKUciMAG79mRWumshJ4+WWOGTh7ltZDQgJw9CiweDHw4Yc2WlkMGcJqt4IC848XF9N6GD3aqXInhSfhtgG3YdmeZThTcsbsMUIAd9xB+XNy6r8Og4FB9IAAJjEpFJ7AqxSCQvHhh7QEUlPZRkijoWcnLo6767VrgTVrrJwgJAR47DE+KT2dyf01NQxQZGTQXfTYY00obrDNrFGzYJAGLPllicVjkpKAuXPZyC8z03Q7dYqWwdy51kf/KhSuREgvS3gWQqQA+M5SDEEIcReAuwAgOTn5gow2PgO1NZGfD8ycyUXTUlC1spKtg1580XqYAGVlwI4dwE8/MW0pLIxVykOH2u6X0QxuX347PjnwCU48eAIJodYH5uTm0lIQgv3WXKCjFAqzCCF2SSnTGt7vVUFle5BSvg3gbQBIS0vzLm2maBb79vFfaxk2gYEMARw/zjnwFgkJoQK46CKnymiLWaNm4f197+OfW/6JFya9YPXY+HjeFApvQbmMFF5DSYl9E9OE8ILhORboEtUFN/S9AUt3LUVeeZ7tJygUXoRXKQQhxKcAtgLoLoQ4LYS43dMyKdxHeLj9c2bMtX7wFuaMmoPKmkq8uPVFT4uiUDiEVykEKeV1UsoEKaWvlLKDlHKZp2VSuI/+/bn7t5ZFVFlJZdC5s/vkcpTuMd1xbZ9r8dqvr+FshZUcU4XCy/AqhaBoXZw7x5YUtsoCjMTEcPyAcbqhkaCKs+h29FtcsO11dNvwNm7svgO+hmrXCO0k5oyag4qaCry87WVPi6JQ2E2LCyorvJ8jR4AVK4DDh5n9WVvLds5XXAH0TigEsrJ4YPv2jQYm33gjYwl79wIRobUYnv0luv6xClXVAhUyCKOSDOiyYwtwJAR44AGvHYzeO643pvWahle3v4oZw2YgMjDS0yIpFDbxurRTR0hLS5M7d+70tBiKOmzYACxbxizP6GhTqwbdmXx0/+1/uDhyJxI7nK8QNhiACy4Apk9nT+jz6PXAgQPAiSX/Q8KOFSgMTUFcghadOlF/CAFqjZISYN48Fih4Ifty9mHAWwPw9zF/x1Njn/K0OArFn7SatFOF93LmDPv9JyaaxhAAQEh5LkbtXwSNrhI7czoguKcGERGgQti3jwNt5sz5MwfTxwcYkFyIAQGrgGtTzPccCgtjQOGrr4BHHnHHy3OY/u3648oeV+Ll7S/j4QsfRniA6+ofFApnoGIICqexfj3X7rrKAFJi4N73oK3VoSIiERofDU4au0RrNNQeOh3w3nv1AwfGQcrWGtDFxgK//Wa5TYUXMG/0PBRVFeG1X1/ztCgKhU2UQlA4ja1b63l+AAChpVmIPnsE5cHc/QcHM9Bcj/h4Bh6MsQWAvRwCAqxfUKPhzd6otQcYlDAIU7tNxYvbXkRpdamnxVEorKIUgsJp6HSNC8siitL5x/nOokKYZin/ibHraN0xYf7+9s0rkNK+ajZbVFQwLara+dlL80bPQ2FlId7Y8YbTz43CQr5vWVk2uv4pFLZRMQSF02jXjvNp6rYKEtIAH30Vos7+Dj9dGaqlL/xD20EgBkCD9tN1F7QBA4B166xfsKqKvSwSE5su9NGjwKpVjGUIQRfVqFFsj51gvReRvQxJHIJJnSfhha0v4P4h9yPYzwlVdX/8AXzzDaPvWi3fu9hYYOpU5u6qCTuKJqC+NQqnMWlSfe+NprYGSZmb0T5rJ2LPHkZoWTbCC9PRt/gX4OefqT2MCFHf39S7N1OKzp0zfzEpOXtz0iTbE9IssWYNsGgRg9pJSZxKEx8PbNwIzJ9PN5aTmD9mPvIr8vHWrreaf7K9e4GFC6kUkpLYIzw5melZb78NfPSRU8aDKtoeSiEoHEJKZhPt38821XV7Cl1wAdemrCwe2H/vfxCbfwgVQbHQ+wSgHMGoDQpFUPsI7u43b2amUEkJc1S7dTOdzMeHdQZVVTxhXfdRZSWHLvfpA1zSxHlKx45x4ezQgf2mjTtqHx/eFxbGqTwlJU07fwOGJw3H+NTxWPLLElTWVDb9RMXFwBtvsIqvrtwAEBrKvuFr1gC7djVfaEWbQykEhd0cP84N9dy5wEsvAS+8ADz0EPDxx1QMAQEcNZCYCJQezETs75uRE5iKE2EDYKiohL9Gh+Tk84lDwcEMOhw8yCyhW25p7OZITQWeegoYNIha6PRpBpsrKoD/+z9evKnWwQ8/cIymuedLyfuLixkpdxLzRs9Dbnku3tn9TtNPsn07LQFLzZw0GlpWK1cqK0HhMA4XpgkhggFUSSmdM6G8GajCNPdx4ABnEAQH1ykOA5vRZWVx7X78cbr0DQYg+/kPUfLdRhT4JyIwEEgNyEZE+h5oanSmhb+mhge/9x4nnVmjuJj+KK2Wvn2rwxBsUF0N3H1348ELRvPn+HGgqIgLb0AAMHs2MG4cWDzRPMa8PwbHC4/jjwf/QICPjSwqgJZRfj5liYykJi4osD7TQUr2/3jlFVo6CkUDmlyYJoTQALgWwA0ABgOoBuAvhDgLYCWAt6SUx50sr8JOioqAbdvoVtbrOYlr1Ch2hXDWyOCqKuDNN6kIQkLqP+bry0LhEycYm7366vPlBbp0JA4ORbmWa9OerAQYtHFI8M1FUug5BAVKFBgiUVlSg9+PpqBzFBvWWZQ5PNx5g210Ol6ooTLYu5euqMBAXkuvp9L67jvGFZ58stnDjuePno8JH07Ae3vewz2D77F8oF7Pa65cyTiKUdbcXLq0rL0Xxtem1zdLVkXbw54so3UA1gKYBeCAlNIAAEKIKAAXAVgshPhaSvmR68RUmOOXX7i5rq3l+qDRcMTk6tWcNX/dddbruuxl3z6gvLxxjUFdEhKAH38ELr30fGGary/OZNZi5x98PDAQEFot0qvbY/ex9tDpgKhIibjqU/hulRalP3HDfu+9TkvusUxgIF1COp3JZZSeTmUQGVnf/AkPZ8A2L4+DnhcsaNabOi51HIYnDcezm5/F7YNuh5/WjMtKr6cG/vVXBrmTk033Hz3KYPLEidTO2dm0GKSkxm7fnn/7+jbW3gr7qa3l1D2tlmaxs3ZXXo49CmGClLJRl3opZSGALwF8KYRohv2uaAq//Qa89RZ//3Xrt8LC+F1evZoL8zXXNP9a+/dzDbWGvz/X1+xsICUFOBmdhlN7PkZwQmS9MoGSEv7OhABQVgqZGIf41CD0zdmG2LUbsP/7IoROiUDI1LHsh22rOK0p+PhwktqPP1ILGQzMNAoJqf/D1+mATp34d1wcZzL//ruNUW3WEUJg/uj5uOTjS/Cfvf/BnRfc2fign36iMujUqb48Pj5Mx127ljdjrYbxDc7I4BcjPh647bamx1faMqWlbMi1ejVjVQYDFfKUKUBamnN2WF6MzaCylLJGCNFDCDFeCFFvyyGEuMR4jKsEVDRGSuDzz7khNLdearV046xaRdd7c9Hr7UtrN84ykBL47MQQaAN9ESBNaUh6Pd3hAQFAgL+Eb1E+jkYOw7h1czF411IkGM7AUFOL9C1ngKVL2bguO7v5L8Ac48ZxF11UxEWgqqr+AlpaSu1ad+K9ry/ghJjVxM4TMbj9YDyz+RnU1Db46ej1wPff00wytyuNiuItI4MWTEQEFVlICP+Wkv47FTtwnIICWoBffMGEA2MqcnEx8NprwLvv2lcs2YKx+TMXQjwIYDmABwAcEEJcUefhZ1wlmMIymZkM5FpzI/v4cHHevbv510tNtT2y0lh9HBPDwfFHs8Owf9jfEFyeh8AKujSMGZwBshLtq0/iQNBgFBzJh09NFYojUlAVGAW/iCD8URiF6nYdedHnn6dJ4WxiY4GZMyl4ZiatAZ2O1ywq4gI7bFj9HaGvL31nzUQIgflj5iO9KB0f7/+4/oNZWVRGQUGWTyAlU0zPnaPJVVnJ96ioiKbcRRcB335r//g5Bd/TN97g4p+SYjKJhaCi7dSJPtoff/SomK7GnrTTOwFcIKW8EsBYAPOEEA+df6xtONa8jOJifk9tuTX9/Oj6bi5Dh/Ja1taXnBzgwgu5MS0t5Tqak3gBNo+ajdLQBIQXZyL43Cm0r82Ev6zC2rjr8XPQVJRWalERbApOGC2Rigpw0T53ju4TV9CpE7B4MVNe/f158chIvpAxYxr7yaqq6lsMzeDSrpdiYLuBWLRpEfSGOsFfvd76B3vuHN+cxEQOmejUiTK3b08FNn48XUZlZUzpVdjHyZOMzVhKGhCCVtv337dqRWtPDEEjpSwDAClluhBiLIAvhBAdoRSCR7A341Kvt77RtJeICGYPff45LeiG1z97lsrnivO2o7+/KQW+ILobNo+ajeCyXBzfVYycAl+URSahVvgg9PA2+ISb93P/uSZGR9OfO25c81+IOYKCgMsvZ15tdjavZw4paU1ceKFTLmu0Eq76/Cp8duAz3NjvRj4QEUHTzmAw76czmmp6PZVTjx6WLuCc3UBbYe9e7mKsKePAQPo8MzKYztcKscdCyBVCDDD+57xymAogBkBfVwmmsExqKhdlnc7yMVJyTenTxznXvPRS4PrrmfVo7KV26hT/Dg0FZs36c5wBOnTgulbX01MeEg99p27I0KSiVvhQwJoadA6v37par+fv8s8EmaAg97S3njaNpk2lmSpiKbkIXHghd+JO4vLul6NvXF8s3LgQtYbzvumoKKBfPy485jAuWFLyjbaEMdNIYR9lZfa9X0K4pAGit2CPQrgZQE7dO6SUeinlzQBGu0QqhVUCAtjC58wZy8WoubnM609Jcc41hQAmTwZefhm4/XZu2C+9lIpg0aL6a5NWy013bm79fnXt2jG2odMBZeUC8QElSPar99VCWRnl/jMzSadzTaZRQ7p3B+67j8VvGRn0zZeX02rIyGBfjttuc+olNUKDeaPn4WjBUXxx6AvTA1ddxdddaqZddng47+/Y0XJaqfFL0bmzU+Vt1cTGWt9hAaZdVisO2Nt0GUkpG3avr/vYL84VR2EvU6dyd75nD3fmxrWhupr+/Kgo4J57nJ8+HRLCwjdbjB7NWO3atVzDoqK4AevThy2MwsOBYQOrEJKVh9LwDjAYuAZHRDSwxvPzqf3qUFREN5VWyw17vYE8zWHwYPrlt24FduzgApGWBowdS6FckIs+rdc09IrthQUbF+Ca3tdAIzTU4o8/zrqHggK+6VotFZRez1iBtRedl0d5k5KcLm+rZfBg4L//teyqAxi869DBumXWwlEzlVswNTXApk2McxUWcr3y9WXn5gkTnFfY21SkZFHb99+zlxzAzVXPnlRmVSezcfWeuSjzj4HeN2cw7loAACAASURBVBAdO/KxP7M/Kyu5IC5cCLRrh+xsdnzescO0Nhtf75QptmslvJVP93+K67+6Hl9c8wWm9ZpmeqCykhp/504qp44d2do6MBB49llqxfbtTeZUbS0tmuBgttsw+vAU9vH++2y5npLSWPlXV9MknzGD9TEtHEutK5RCaAUYDEw+MRioBLyxHqmqyhTk1mioLE6fBio2/IrYL95EaJQffBPjuBOureUuV6ejGyctDZmZXANra7nOGbNBq6sZz+jcmY31nBFEdze1hlr0eqMXAnwCsOdve2gl2KK4GFi+nO0tjL9hKWk9XHUV838VjqHTAf/+Ny3EgADuXow/LiGAm29m9lkroNkKQQghwH5GnaSUTwshkgG0k1K6KCfQNkohtBLS05nfvX07FzUhGMCdOBHo2BEGA9sIVVZaTgJKT6dn6brr3Cm48/hg3we45Ztb8M30b3BFjytsP8FIWRmtAimpKT1tFrZ0pGT66fr1TEX18WG33REjWpWSdYZCeBOAAcA4KWVPIUQkgB+llIOdK6r9KIXQyqiu5qofGFjPR374MMsFrAXIa2oYbnj55ZZpJegNenR/rTsiAyKx484dEG2kd47CM1hSCI7MQxgqpbwPQBUASCnPAfBC54SixeLvz6hyg4DpwYO2xyb7+tIllZHhQvlciI/GB3NGzcGu7F1YdXyVp8VRtFEcUQg1QggtAAkAQohY0GJQKFyKTmdfTzEhWnarmZv63YSO4R3x9Ian0ZJje4qWiyMK4VUAXwOIE0IsArAZqpeRwg0kJtqfIm4pxtAS8NX6Yvao2dh+ZjvWnFjjaXEUbRC7FML5gPJGADMBPAsgG8CVUsr/uVA2hQIAY3parfUWMgUFLCFw+SwFF3NLypXo4BeDpz+4HfLRR4DnnjPVRCgULsYuhSBpv34vpTwipXxdSvmalPKwi2VTKACwNcbVV7PQzZxSKC1li5/p090vm1M5dAj+c+bjyezO+EVzGusDcphT+/rrwD/+4Z4WHoo2jSMuo91CCJdmFAkhLhFCHBVCHBdCPOnKaym8G73e1NkZYNuM6dNNnSTy8019laqrWdjbojs1ZGVxXnJwMG6PmoAEGYKn/baxxDslhUVoL73UqjttKjyPPd1OjQwFcIMQIgNAOdjpVEop+zlDkPMB69cBXAzgNIAdQogVUspDzji/omVQUAD8/DOHhul0jA307k2FcOmlwMiR7IZtTBHv25eFo05rX+Ep1qz5c85BAIAnMAIPi9XYKDMwGh3pC0tPZ1fWgQM9La2ileKIQphk+5BmMQTAcSnlCQAQQnwG4AoASiG0ETIzgSVL6P5p144V11Jy8V+8mAW4V17JerVWhU7HHiR1evHfiQvwjNyMBdiINbiJd4aEsLWCUggKF2G3y0hKmWHu5kRZEgGcqvP/0+fvq4cQ4i4hxE4hxM58Sy2CFS2O6moWlWk0nLlgbL8hBBtRJicDX3/N3kitDuPs3jrFFkHwxeMYjrXiBLYafxZBQZbbYisUTsBuhSCEmG/u5krhzCGlfFtKmSalTIuNjbX9BEWLYN8+toyJijL/uI8Pa9a++85yy+8Wi3GikKF+Wc/dSEOMDMICbOQdOp3lltcKhRNwxGVUd5hsADgkx5mZRmcA1O3X2+H8fYoWRmkpMyWPHOH/e/Rgd+HQUMvP2bKFTTqtERnJNjNFRfy71RAYSDfQ4cP1OpSGwA8zMAyzxE/YIc9gcLGeg3y8BZ2OkX0pWQBi6wNUeD12KwQp5Qt1/y+E+CeA1U6UZQeArkKIVFARXAvgeieeX+EGNm4EPviAFcPGnkI7dgCffALcdJPlZpHl5ba7tApBl1KrHFh1ySXArl1cZOu8EfdhMJ6XW7Cg5iesCLmUg3o8TWUlg+CrV/Nv44DvUaM4qMMZ1YGFhdwl/PYbv0zduvH8TpxYp2iMIxZCQ4LAXbxTkFLqhRD3g0pGC+DfUko1JbwFsW0b8M47nB/SMOunuhp4913eb24scVwcR3JaG0ZlbEthzdJosXTrBtxyC7Wpvz/fEK0WodXAw1U9MT9iN/bcMgkDPd25r6ICePFFDrhISGCAB2Ce8MaNwO7dHKNnaVi9Paxfz/cBYPdWIaiAfviBLW3/7/8sD7FRNAtHYgj7hRC/nb8dBHAUwCvOFEZK+b2UspuUsrOUcpEzz61wLXo98OmnXAfMpYD6+/OxTz/lsQ0ZPZozE6zFB/Ly6HpqqmdCSs44OXSIrievS+kfNw6YNw8YMICCZmQARUV4YNhDCPcLw8IT73laQuCrr/jmpabWH23q48MJbTU1wJtvNj3Qs3MnZxLExzOTIDycuwTjpLKVK4EVK5zzWhSNcMRCmFrnbz2AXCmlmZ+2oi1y9CgLyTp2tHxMcDCTZI4c4SjNusTHU2ls3syNZ3x8fUugrIyKZMqUpsl34ADwxRdcY40DeoKDeb6JE+1rnucWOnfm7NM77uDi6u+PCK0WD637A09vfBr7c/ejb3xfz8hWXg5s2GDdbRMbyzf5xAnHKwUNBo6xjI01v6vQaqkkvvuOIwFVgN3pOGJ33Vsn3fTMeRfPYpdJpmhRFBTYvyksLDT9rddzDZgxg4t+cTFdTytXcu3Jz2d9QlkZ8PDDXA8cZcsW1jecO8fnJyXx36AgWixvv+3mLql6PQdfZ2XRLDKHry8FPK+pHrrwIYT6hWLhpoVuFLQBJ0/yjfL1tXyMMdBzqAnlQydO8AO35hM09jlvlfnHnscRC+FiAE80uG+ymfvaBBUVXASFoLvXG8dWuhNb8wrMHSsl8NFHrExOTub9nTuzS8OJE2xTsX8/R2OOGdO0YWCFhfRAtG9f38MBMLknNZUKY+BA87ENp1JVBaxdy2Bs+fmkPV9f4KKLGFSOiLD41KjAKNw/5H48t/k5/H3M39EztqeLhTWDXt941rA5tNqmRf6Li+07v48P/YcKp2PzZyyEuAfAvQA6CSF+q/NQKIAtrhLMWyks5ND4DRtMaeP+/ox1TZzYcge9Nxejd8BgsBzvM75fxmNPnmT8MCXF9ByNhgo2Ls50TEBA0ydDbtnC6zZUBkaEYFLMypXA0KE21qPaWvq7fvqJZoufH580fLgpuGqJykoGY48epU/MmImj01FB7NjBYKyVMY2PDnsUr25/FYs2LcJHV39k/XquIDKSb6ZxzKklamqaFlS2d1dVN4VN4VTscRl9AuAyACvO/2u8XSClvMGFsnkdeXnAggXc0cbF0fWQlMSY11dfmdoutEXi47nLzs62fEx2NvsOGVPt16/nGmAtYSQuDli1qlHNlt3s3m114w2Ayub0adZPWKSyks3lliyhUtBquZgvX86Bz1ts7I2++AI4fpwmSd1dg58fAy/l5fRdWfG7xQTF4N7B9+LTA5/i94LfrV/PFSQnM7BbVGT5mJoaWj1Naa/RubPtPudSmhpcKZyOTYUgpSyWUqZLKa8DUAIgHkBHAH2EEKNdLaC3ICXw1ltcF5KT67tRAwL4O09PB/7XhidE3HwzN5EN21TX1PC+yEjWIhQU0C109KjtnX9wMD0J5eXWj7OEXm87Q9Ho9rYYR5CS+bQHD9KciYvjoh4Swi9DbCywdCkLy8xRVkaTMjHR8s46Ph74/Xfm3lphxrAZ8Nf645lNHphNJQRw7bX8QMztfPR6ftBXXdW0VLCgIGD8eGZYWVKM2dn8sR06RIvqnnuAJ56glVVc7Pg1FfVwJO30DnBIzmoA/zj/799dI5b3kZHBbLs6haSNSExkKrbVnWYrJjISmDMHuPhixgZPneItP58xgGHDgEWL2Kp65kx6Xo4ftz77xbghbOrM+ZQUZj9Zo6qKSt1iLPP0aZoaycnmBTH6tL76yvxCduIEtY21QIuxuMuSUjlPfEg87k67Gx/99hH+KPzD6rEAuHAfOUJllptr+3hb9O4NPPAAF9+MDGr2ggIqgqws4Jpr2Jq2qVx1Fa9x4gQVqZGqKu64NBp+Hp9+yp1GdDTf288+A2bP5vMUTcaRoPJDAAYD2CalvEgI0QNtaITm0aOm36wlfHy4Hpw4QddIWyQ8HLjuOnYlNcb9QkO5gf7pJ1N6OcB15MAB/jtihPlMw5ISKtqm1h6MHWuK91iyFHJzKa/F9XrbNj5o7cOPimKxVn6+KQBipKaGz62sNLlUzAWbtFrLWUd1eHz443hjxxt4dvOzePfyd80fVFFBd9bPP5v8bQYD0LMnC7tSUmxexyIXXED32a5dpkriTp0YS7HUjMpe/P2Bhx5i99dVq6hohKDSnTiRfsagoPq9S0JCeCsqAv75T+46WlVvE/fhiEKoklJWCSEghPCXUh4RQnR3mWReRlWV/cWR5gqv6lJYyN9SXh6/2/370wpu6i7YGwkMNNUk/Oc/XCsbvsYuXbjJLCkB9uxpnOUjJTegf/lL09+b1FSuU7/8Uj94bTx/bi7XsLFjrZwkL89yVNqIEFQapaWNFUJ+PhdOo//KYGDwuHv3+sHo2trGzzVDQmgC7hx0J5buWop5o+ehY0SD4o+KCuD557mjbt/e5N+UkvctXEg3S9euNq9lkaAgtpIYNarp57CEnx9dRxddxFxhKbnTWLuWCtVSHUREBBXIL7+whYbCYRypQzgthIgA8A2ANUKI5QCc2f7aq4mPt52rbmxYaWlzotcDH3/MnPtPPqF76bvvgKef5m/07Fnny+1pSkr4Os25z0ND6R2oraUXoK6HQKfj2pWWxkSepiIE8Ne/sgj41CmuF7m5vF5GBpNhnnzSRiwjLMy+mcbm0pl+/pkfusHAhbm2lgGR48fp9z52jMfpdFQodpqWT4x8AhqhwXObn2v84Lff8s3r2LF+sMuYIx0aCrz2mheWajdAo6FLKCaGr+Onn6xmYQGggl271j3ytUIcaW531fk//y6EWAcgHMAPLpHKC+nXjxuXBr3H6lFSwozC1NTGj0nJnfKGDfyd1q2MNbZUWLyYnQus9fNpaRw71qjVfz26dKGXYPt2uo+Mmz9fX+Dyy3lzpMbBHL6+wK23cuLajh3c8AcHMxGmc2c7rI+0NC5G1igt5WKVkGC6LzMT+PBDZuYUFgJbt/LFGF9QRQX78+j1/FJdd53d6ZQdwjrgrwP+imV7lmH2qNlICj/fKLiykkqorhwNCQ9vmdPXioqsvy6ACjkvr3mBpzaMI0FlIYS4UQgxX0q5AcBeAANcJ5p3ERTEQe+nTpnfWFVW8jd/3XXmv4cZGdwpp6Q0bpMgBHeq+fl0kbYmbG1ChWBMYfBgxiPvuw945BEOy5k2zXpRrKPExrJVxa238lpduti5ZnTvzvxiS0HZ2louQpddVt8ntW4dF//sbH5xkpL44RsMvLCvLxeudevol3cwGPvkyCchIbHklyWmO0+f5ptuK6ff35+B5pZEaKhtS02no7ZXyqBJOOIyegPAMADXnf9/KTgDuc1gbLSYlcXNnzF9MiODrs777qMlYQ57cu7btaMXwVYMoiXhSEFZv35cF/v29bICP40GePBB7goyMqj9AVOQIyODi/nIkfWft20b/YcHDtDsi4ykSRIfz51sQAD/jolpUgZQx4iOuKX/LXhn9zvILj1fAGItet7wNXm7y6ghF11ke2JcXh79g4om4YgxPlRKOUgIsQcApJTnhBBtqmGDENwEjhhB6/+PP/i76tOHXgVrvbZOnrS9OBqt3dLS1pMk0bUrX3d5ueVMIZ2OG9aGDe+sodczFX3tWm6+jUXDI0faFZd1nNhY4KmnmP2yerUphapHDyqDvn0b70qrq7no6vUmN5GPDz9c4wcsJX2N2dlNagg3a+QsvL/3fTy/5Xm8OOlFymkw2FYMVVXWOxF6IyNGsKS8tNR8jnBZGS0wVwS62wiOKIQaIYQWgAQAIUQsgCbWj7ZsoqLoj3YEo6fAGsac+9bU6t3Hh1lCb71Fj0lDT4ZezwX9+uvttwrKyxkTPXSISjg0lF6blSt5++tfuXY4nbAwfvCTJ9NK0GqtZx+1a0chrbkvjGMxhaDP0UGF0DmqM27sdyOW7lyKJ0Y8gfioeAamDx+23D6ipoZfsrQ0h67lcaKj2eHwpZcYT4iL407COLlNo2HKqhqt22QcWXpeBfA1gDghxCIAm9GG6hCay6BB1iv+AVNQujUFlQEuzjfdxAafmZmsaSopoSI4fZo1AJdcYt+5pGSHh6NHGY+JjeWaHBxMhRMbS+Vjo76reWg0vKCtVNRJkxg4ttYGtqLCFMxoYg/u2aNmo7q2Gi9sPT/U8C9/4TXPnWt8sE7HD2HatJb5RevZkyl5U6bwB5WRQUU6aRLvd8TMVDTCnuZ2H0opbwIQA2AmgPEABIArpZSu/Nm1KoYNYyGrsSq2IUZ39N/+1vriYUKwpmjQILb82b+f1tKwYbTuHemDlpEB7N1LZWCtaPjrr7l2eJQhQ0zFFsbJX3UpLeWiHB9PF1SnTk26TLfobpje61q8tv0NBO+dibz0RMRWPYmr019HSkg6QiN9IbQaKgOtluaYvRrYG4mLo0K7+mqahlpt6/vReAh7XEYXCCHaA/grgA8AfGp8QAgRJaUstPhMxZ9ERwO33caWOFFR9deHqiq6kIcM4SLZWomJMaWSNpVt25icY2/RsEe9B4GBLDI5fJiZCKGh9KHp9byFh7MaLz+f/9rqwmeB6mogOWMOKvWfYkXJS7gkYhF04al43XcxYvIP47Kwgxg6sAYiqYPtYFdLwlgMqHAa9rybSwH8BKATgF117hdgPKFp25o2yKhRXBO++IJWu3FyV0AArfzJk71ocpeXkp9vO9ZgbFZXVuYF7uSYGFYhzpzJoLG/PxfkpCR+GXJyWKdw3XW2z2WBjz4Ccvb3Qq+Ua3Ag6F+YgBkIRBQSOmihb9cHb2T2QXUy+0kpFNawqRCklK8CeFUI8aaU8h43yNSqGTCAMb9Tp+hH9/Oj+6OlDNipqWEW5ZEjdPt07szXZMud7izsKRo2Voyb643kERISWFjx7becEQrQLDTOBJ0ypcm79rNnecrkZCAQc3FI/Bfb5Cu4CP8AwA10u3Z0oY0cqTYcCuvYE0MQklhUBsZjnCta68VYjNXSOHCAAduyMpPbZs0aLry33MKeQa5myBDWcVmjtJSLoK2iVrcSHc2KuL/8hWaOsRqxmZp0717+q9EA8eiLnvJqbMcrGIZHEAC6oIKCTFPoUlPZNaO0lJ9bly5q1ozChD0uo3VCiC8BLJdSZhrvPF+DMBLALQDWAXjfJRIqvIKjR4EXXqB/vmH6emUlu5lqNK4fQ2ksGs7JMR+Mrq3lenvvvV4aZzR25nQS587Vd6OPwlwcFl9hu/wXxmDen/dLyUr51183TaqUkor94ouBK65oOVaqwnXYk3Z6CYBaAJ8KIbKEEIeEECcAHAOrll+WUr7vQhkVHkZKtuQJCzNfDxQYyMX5o49cX/xqLBoOCWE7HuOcFoOBiiAjg8WDLp+P7CWEhdVvupiAgegmL8M2vIRqmAZznDrFbti+vlToycn8NyaGDRbfeKN1VcgrmoY9E9OqpJRvSClHgJPSxgMYJKXsKKW8U0q5x+VSKjxKejrrBawlwQQF0ZXkzPY4paXs0/bqq8Arr7DorLDQVDQ8fTozbDIyuOB16sTY7TXXeKl14AL69zeNOTYyGvNQJc5hx/nOMnl5bJ7Yq1fjanFjDGvXLjYYVLRtHMrZklLWALAyNVfRGjG6vG0tshoN02cHOKHl4ZYtwHvvcddqLOTdt48ZWn/5C+OwkyezHqm6msHStujyaNeOmaS7d9OVJgSQiMHoIidjK15AmuF+7N8fguRky+EKIWgpfP8940BtRZkqGqOSeBU2sTczxWBwThbLnj2MSbRvX38Ri4qiS+qzz7j4X3wxlZBbGuHl5XHwyq+/Ms0pKQmYMIHVbx5O3bntNsYFjh411biMkvPwnnY41pxbivYRj6G7jVFWYWFMha6oaPp0OkXLpxV1zVG4CuO0RWu9mIwuiy5dmnctg4HjcmNizO9ofX2Ztv/FF3ZNm3QOmzdzis6339Jk8fUFfv+dU8leesn8wHk3EhzMOdV3380YT0YGIE8NQ1fNBKQnPo9uvSts9scyWoAqV7Bt0ySFIIRwoNmAoqUTHc22E9lWnIVnzzJIaW44kCOcOEEXlbU2O/7+dBPt39+8a9nFwYNsnhQXx0issYdRXBw15cGDwLJlHl9J/fzo7lmwgOIsWwYsu3k+CqvzkJXwjs0+WuXljBGpFNS2TVMthO+dKoXC67nxRroiTp2qn9ViMJgUxV13Nd//XFho3zk0GlMHapchJSu6IiLMmyvGgpJduxh19xK0Wr4/ozqOwtiUsfi5ejEqdFVWLby8PDZytavTrsHAysT33uNQ+3ffZVdXWzNmFV5PU2MIKuzUxoiMBObMYYO+LVtM9xsMtB6uucaxJnWWMA4Rs4VbKpHz8jj0wloVobGfzrZtjCt4GfNHz8e4D8ZBXLAM6TvuQ3Jy/boF4/jWlBQ7CwuLipj2ZWzDERDASrfNm/n6H36YJqWiRdJUhfCOU6VQtAgiI4Hbb+fif+YMF5P4eAYyKyvpSg8MbJ6V0Lkzd6nGJpbmMCoMW4HSZlNaSmFsvaCAANuTvDzE2JSxGJE0AuuLnsOSqXdg7Wr/P99bY7rqoEEMTNsMzldXszoxJ8cUWKpLTg6wZAlzgpXvqUXSVIXwlTOFEEJcA+DvAHoCGCKl3OnM8yucS1iYqafQtm1MV8zJ4bqZkMCU0KFDmzYPOSyM8xOM/XnMkZtLZdChQ/Neh00CAuwzV3Q68xV7XoAQAvPHzMekjyahtPN/8PLLd+G33+iaCwri+ID4eDtPtns3U5EsBYratWPRyvbtHHepaHF4SwzhAICrAWx08nkVLqKyEnjxRbqPdTpuGJOTef9bb7GQrLq6aeeePp3nOnmyfiZRTQ3Xo+Bg4M473ZAv3749051KS60fp9OxyZKXcnGnizE0cSie2fQMAoJqMGIEq7nHj3dAGQAcHWprtmt0NPDDD44JqNfbHieocAtNVQhO/SlKKQ9LKY8685wK1/LJJ8x7T001ZQQJwcBzp05shPff/zbt3MHBrDi+6ip2hM3MZDA7P5+1B/PmcZ024rIEH42Gwxvy8iwHTHNyqL26dnWREM3HaCVkFGfgw98+bPqJsrNtW0IhITThbC3wZWXsjDhjBnDHHbwtXcp4hMJjtLgYghDiLgB3AUByS2wZ2go4d441Wh06mN+lC8HHNmzgeMymeFOCgvjcKVOoCKTk5tPo587P57z7n3/m2hIRwTqxESNsb2IdYsQIaqQffqDmi47mC6yooKKIjQUeeMDrB2FP7jIZFyRcgEWbFuHm/jfDR9OEn76/P3fz1grxamttTzDKz2eswTjBKCWFz9uzB9i6lSbi5MmqZNoDNOlbLKV8w9HnCCHWCiEOmLld4eC135ZSpkkp02I9Pv2kbWKchWBtXfDx4W/8yJHmXcvPD0hMpIIxKoMDB5jxtHIlN6QdO3Kt+uorYO5cupqchhAcXvPYY7QEMjNZ+VVVBfzf/wHz53vBFB7bCCEwb/Q8nDh3Ap/s/6RpJxk61HbwPC+Px1lazGtrORuitJSKwFgWrdUyANWhA0vR9+1rmoyKZuG21hVSygnuupbCtVRW2neclE2PI1giN5fxifDw+l2kAwO5XhcVAc89x/U7Koq3Zm80hQD69eOtupqLWkCA11sFDbm8++XoH98fizYtwg19b4BW42DLjbFjgbVr+R6Yy/mtqeFj48dbPsfhw6Y8V3P4+tLcW7GCnfuUleBWWtY3WuEVWKsirosQzk++Wb+e1om5kQKlpUyPX7eOXpzHH2cG5K+/OjHO4O9Pf1YLUwaAyUr4veB3/PdgEwI8iYmchHTmDEvTjW+qlExbysykNWWtXH3rVtv5rZGRNPMKChyXUdEs7JmYFmXHeQxSShvF8VavcRWAfwGIBbBSCLFXSjmpqedTuJbevenK0eksdxitrubvvkcP511XSi725jJjCgpMBXNRUYwrJCWx6du//gVccglw/XUS4vSp881+zhdRdO3aIhf3pnJVz6vQO7Y3FmxcgOl9pkMjHHztY8eybceKFcwq0GppMXXuzNSvvn2tP7+42HZbWuNQbHtNUYXTsMdllHX+Zs120wJocoRXSvk1gK+b+nx7MFbbr1nD7zHA/vATJjCnXVmm9hMYyLTF//6Xln/DWEJtLTs53HCDc6uJ9XoqmobrSU0N6yH8/Hi92lq6+IWg9yE0FNj2VRbGbFqGDlV/mJ4oBP3/t97KL0MbQCM0mDt6Lq778jp8eehLXNP7GsdP0qsXb4WFbIIUGGgKttsiOho4dsz6Mcah2E6cLKewD3sUwmEp5UBrBwghvHpIjl4P/PvfzIwJDjZloRw8COzYwRqam27yeBfjFsWllzIldM0aun2jovg7PneOC/TUqcDEic69po8PXfcNLZPsbF7TGJ+sra2viMIrsjHx2CJkZUgkTukIoamzcBUXs2vpjBms0moDXNPrGvx9/d+xcNNCTOs1zXErwYgxSOMIw4cz/cwaBQVsK+7UdDGFPdjzTRjmpGM8xldfsfK1Y0dau76+vMXH876ff2bGisJ+NBrg+uvpozf2wDEYmCQSF8f5vY89Bnz+ufUuqY4gBDBuHAPLdTl9ur4CqKhgLYSRvvs/RoBvLbL0caiqbrCLDQ/novbuu617hmRODvC//wEzZkD7wIOYe64vfsv9DSsOfuleObp25c1SM8CqKu40rnAo+VDhJOwaoWnpMSHEbbaO8TRlZcCPPzIDxZyrWKPhIrZypRv767cShGD88LbbmO4ZEcGgblUV11lfX773c+ZQQTiDMWNoydUtHtbrTZ9tZaUpVRUAgstyEZd3AOUh8RDCQn1ZaCgthUOHnCOkt7F5MzB7NmspfHyAiAhcmx+PLjWhePqLByFt9cZ2JhoNcP/9/IDSQR12+gAAGq5JREFU000mZWUlqw/z8tg21+WNqhTmaG407R9OkcKFGLvy+lhxjhkDpIcPu0+u1oSUHNJ+8iRjClFRfE8DAxnYjY83dUhuLnFxwCOPUNFnZnIdCQ6mK7u4mLIMH26yGEJLsyCFBgYpIISVmIZGw2Bza+PoUeCdd/jGJSXxQ/H1hU9UDOb4jMMekYOV/3rAva0jwsO5S7j3XsZwior4A5w0CVi0CBg50n2yKOphT5bRb5YeAuBIJxSPUFZmf8pheblrZWmtnDhBZdqxo/m4or8/14BvvrEeu9XpuDZoNHQfW4rp9OwJPPMMY0Lr1nGNq61lgktSkvnRBcaso6Y03GvRrFjB4KyZN+UG9MU/5AY8XbkKlx47BuHOXbmfH/s/eXEPqLaIPUHleACTAJxrcL8AsKXx4d6FI1143TKbtxWydSt/39aSTKKimFxy9mz9PkQA3T9r1zJAXV1NBR4Rwe4FY8aYz1KMjmabocsv5/Evv8zi1oYWQGlYIvTVBogAA7p0sWIQGwyWi6VaKufO0Syz0OLFF1rMxkjc5f8dfvzpbUzq/oKbBVR4G/a4jL4DECKlzGhwSwew3qXSOYFevbjTtBYvrKmhS6lnT/fJ1ZooKDC/K6+LEPwcysrq319YCCxcCCxfTisiKYnrl1YLfPghRxbbiu0IwXnCAwbQ65OVRfdRQQFwMD8Op6P6YXT3PMtFciUl1ECt7QtQVmZznsMtGICk2hD8o+BLSDVQuc1jT1D5dinlZguPXe98kZxLWBhraU6dMu86kpKPXXyxmunRVMLDbbeokJJunbpWmJR0b587x8153d19cDAD1ocPc4qlLQIDgYceYifU4cPpmk5NpaKY8tH1CI/2YcCy4ZegqIgC3HGH9UBTSyQwkK/XykLvBy1mlQ3EVkMGfj75sxuFU3gjrewXYJ7p07kT3bmTvmljenNhIXeSw4ax1bKiaQwdypYS1igu5s4/Ls503+nTptiDOYRgMsrPP9M1ZKwzsIQQLJjt3LnhI+0YxPz3v01FUUJwoWzXjlrDmSXV3kJ0NLXi2bNWc/r/ei4Fi9odxdMbn8b4Tlb6EClaPfYElXdLKQc19xhP4ucH3Hcf8NtvwKpVppbr3btzJGTfvqoorTl0787FPifH/FxlvZ7um5tuqu+9OHyY/7cWe/D15fP/+IO95ZpMQgJTL0+fpkloMDD9yTizszUiBEvKX3iBgWVzEfXcXPgndMDMUbPw0I+PYEP6BoxJGeN+WRVegT0WQk8rmUYAg8vhTpLHZfj4cHbsoEFcC2wtRAr70WiABx9kwW96Ot01wcF8n/PymBr6l7/wva9LZaV9ilgIxnmajRAMUiQlOeFkLYT+/YFrr2VRWkAAPxytlil1xgj/ww/jzshQPPPLc1iwcYFSCG0YexSCPba0hXFS3klr3RB6kpgY+u+3baMVZkzpT0tjv6hu3Ror4NhY+4qDpWScQtEEhGCfke7dmcplbP0aGcnOpMOHAyEhCAQwc8RMzPhxBn7J/AUjkkd4WnKFBxCOZhYIIZ4GFcleAHullL+7QjB7SEtLkzt37vTU5RUWkNI0WMua8i0rAx5+mJ4bS/UBpaU8z3PPKUXuFAwGU6VmAw1dritH6iupGJQwCD/c6OBcZEWLQgixS0qZ1vB+h39iUsr5AF4BUAzgKiGEx8ZpthXy89naecMGxkGc4j5xIUJwgbe1gIeEMFicmWm+pUR1NV1O//d/Shk4DY3G4ojLYL9gzBg2A6v/WI1fz/zqAeEUnsZuC0EI8QqAh6UXJSu3dguhuBj44ANg9+76mYMhIVwkR41q+XEQgwH44gvg++9pCYSF8bWWlPC13Xwzi9MU7qG0uhQpr6RgeNJwfHvdt54WR+EiLFkIjqSdlgJYIYS4VkpZLoSYBGC+lFI5G11AWRmweDF3yElJ9XfIlZXsDVRV5fwW0+5GozEpt82bmRWq0fB1DRumOiC7m1D/UDx64aOYu24udmfvxqAE1yQPFhWxwn3LFn6P27Xj5M0+fVpfOUhLwqEYghDiegCPANABKAOwUEq5yUWy2aQ1WwhffcU2NJa6Keh0TPN84QW1aCqcS3FVMVJeScHYlLH4errz51b99hvw+uv8DkdFUQGUlzNe1Lkz40r2jmlVNI1mxxCEEOMB3AmgHEAMgAc9qQxaMzod+/okJFg+xs+PrpVt29wnl6JtEB4QjoeHPoxvjnyDfTn7nHrujAz2nQoLY0FiaCgLqmNiWEOXmQm8+qp7m68qTDgSqpsDYJ6UciyAvwD4XAgxziVStXHOnqVSsDV+MjRUtexWuIYHhz6IUL9QLNy00KnnXbWKMW1LVeeJiSwcNY65VbgXuxWClHKcsaeRlHI/gMkAnPttUSgUXkFkYCQeHPogvjz0JQ7mHXTKOcvLWQZRt31JQ4Rg/ZyzBiopHKPJyXxSymwAqvGJC4iJoXVgq2FcaWnra9Cp8B4eufARBPkGYdGmRU45X1mZqeutNYKCmEyhMEN2NvDTT8B33zEi37B9cDNpVjxfSlnpLEEUJvz82H11+XLrQWWNhpk4CoUriA6Kxv1D7seSX5Zg/pj56BHTvAaA/v6MDUhpPV1ap2NqtaIOxcVszrjvfExHqzUVGF56KQt6nNCQTZX7eCkXX0x/6unTjQNsFRW8//rr2cZfoXAVjw57FAE+AXhm0zPNPldEBNC1K7sMW6OsDBihktlNlJezUdihQ4zEp6QwFz0lhWX+X30FfP65/aMhraAUgpcSEgI88QQwcCAzL9LTTbfKSs4hH6dC+goXExcch3vS7sEn+z/B8cLjzT7fZZdxs2uph1VhIVNR+/dv9qVaDxs2sENvYmJj08rXl4rhxx+BM2eafSmHexl5E625DqEuZ88y86KmhjUH3bu3wdnACo+RXZqNTq92wvV9rseyK5Y161xSAitXsvlqcDDjZRoN42W5uYwfPPFE22pIa5XaWuDRRxlptzbj9/Rp4KKLgBtusOu0zqhUVniImJjGc4gVCneREJqAuwbdhTd2voG5o+ciNTK1yecyNl/t3BlYvZoucSEYX5g6ldMNo6KcJ3uLp6SEPjRb1acREcDvze8zqhSCQqGwycwRM7F011I8t/k5vHXZW806lxDMjuvZk5ZBTQ03v2pIlRmMk/1sIaVTOkCqGIJC4c3U1HAbvWED0ww9lI+ZGJaI2wfejvf2vofM4kynndffn/EypQwsEBZG90BpqfXjioo4+rGZKAtBofBGpAQ2bWL2SHl5/VzNQYOAW25x+9SgJ0c+iXd3v4slvyzBa1Nec+u12ywaDTBlClNOQ0LM5+vqdPx+jBzZ/Ms1+wwKhcL5rFkDvPMOF4GUFDb6SUnh8Or9+9kK18lFSbZIDk/GrQNuxTu738GZkuZntCjsZPhw7v4zMhoPQykvZwbS9OnWS8DtRCkEhcLbOHcO+Owzpto0zCzRaJh+mJXFVEM3M2vkLNQaavH8lufdfu02i58fh5ZPmkSXYUaG6abXA3ff7bQ++F7hMhJCPA/gMrCt9h8AbpNSFnlWKoXCQ2zdyn/9/Cwfk5DAGclTp1o/zsmkRqbi5v43461db+HJkU+iXUg7t127TePvzxnYl1/OHHSdjvGFLl2cGoDxFgthDYA+Usp+AH4HMMvD8igUnuPIEbaytYax2dXZs+6RqQ6zR82GrlaHf275p9uv3eYJDmbV3uDBLEhycjTeKxSClPJHKaWxdnEbgA6elEeh8DheXDDaJaoLru97Pd7c+Sbyy/M9LY7CiXiFQmjAXwGssvSgEOIuIcROIcTO/Hz1ZVS0Qnr2tJ1mWF1NK8FDFYtzRs1BZU0lXtz6okeur3ANblMIQoi1QogDZm5X1DlmDgA9gI8tnUdK+baUMk1KmRYbG+sO0RUK9zJsGIPHOp3lY7Ky2AHRjfGDuvSI6YHpfabjtR2voaCiwCMyKJyP2xSClHKClLKPmdtyABBC3ApgKoAbZEtusKRQNJeICLayPXWKrW3rYjDw/qQkp2WWNJU5o+agTFeGl7e97FE5FM7DW7KMLgEwE8AYKWWFreMVilbPuHF0Cf3vf0B+fv3CtMGDgZtusjyH0k30ieuDaT2n4dVfX8WM4TMQEaB6sbd0vKLbqRDiOAB/AEbbc5uU8m5bz2sr3U4VbRi9nllHRUUchtK1KxAd7Wmp/mRvzl4MfGsg/jH2H5g/Zr6nxVHYiaVup16hEJqKUggKhee58rMrsSFjAzIezkCYf5inxVHYgSWF4I1ZRgqFogUxb/Q8FFUV4bVfVX+jlo5SCAqFollc0P4CTOk6BS9ufRFlOvf2V1I4F6UQFApFs5k3eh4KKgvw5o43PS2KohkohaBQ2KCqipmemZmcZ61ozIUdLsTEzhPx/JbnUVGjEgVbKl6RdqpQeCNlZcCqVewhZ+w67OPD0bWTJ7NcQGFi/uj5GPneSLy18y08MuwRT4ujaALKQlAozFBSAjz7LAfCR0VxDEFyMhAby67TixYBhYWeltK7GJE8AuNSx2HJliWorFGmVEtEKQSFwgwffwzk5HAmjb+/6X5fX6BjR6C4GFi2zGPieS3zRs9DTlkO3t39rqdFUTQBpRAUigYUFgI7dnAOjSUSEoBDh9hSSGFiTMcxGJU8Cot/WYxqfbWnxVE4iFIICkUD/viDnSI0Vn4dQvCY48fdJ1dLQAiB+WPm40zpGby39z1Pi6NwEKUQFIoGNBxbaw1rDUlbHTk5wHffAe+/D3z9NUc4mul0MD51PIZ1GIZnNz8LXW1beoNaPirLSOERqqu5mAYGMnPHm4iKsv9YD40jcC9VVVQC27bRbPLzo9ZcvpyzG+6+GwgP//NwIQTmjZ6HKZ9MwQf7PsAdg+7wnOwKh/Cyn6KitXPkCLB6NbB3L90ufn7A+PFM5fSWxbVrVyAykmmnISHmj6msBIKCgF693Cub26mtBd58E9i3j9F0Y8dVgNbBsWPAP/8JzJ5N7X6eS7pcgrT2aXhm0zO4pf8t8NX6ekB4haMol5HCbXz/PVM5jx5lO//kZO7Gf/gBeOopeiC8Aa2W4wjy8rg5bohOx2Dytdd6bD6N+zh8mNq7oTIA+P8OHVixt21bg4cE5o+ej5NFJ/HJ/k/cKLCiOSiFoHALBw8Cn33G9SMuzhSw9fOjcvDxAV56yfwC7AnS0oC//Q0oKADS0znLvqCASis3F7jlFmDUKE9L6QbWrKGZ1FAZ1CU2lhV8DeIJU7tNxYB2A7Bo0yLoDXoLT1Z4E8plpHALK1cCYWHM4zdHZCQX3r17gQsvdKtoFhkxAujXjymohw5xveveHRg6tJ7LvHWTns4PzhohITxOp6tXtGG0Eq7+79X4/MDnuKHfDS4VVdF8lEJQuJzSUnoekpOtHxcWBmza5D0KAQBCQzm8bNw4T0viIXx8OLbTGsZpblpto4eu6HEF+sT1wcJNC3Ftn2uh1TQ+RuE9KJeRwuVUVdFFZM3rANB6KC93j0wKOxk0iL4ya5w9y+i6mXQxjdBg3uh5OHL2CL48/KWLhFQ4C6UQFC7HmKlTW2v9uMpKuqMVXsSYMfzgLBVnGAw0AS+5xOIppvWchp4xPbFg4wIYpA1rQ+FRlEJQuJzAQLqBcnOtH1dRwfVH4UV06ABMn85MopKS+oHj8nLg5ElgwgSgTx+Lp9BqtJg7ei4O5B3AN0e+cYPQiqaiFILCLUyezI1mmYWBWmfOAJ06sc5J4WVMmgQ8+CADxpmZHA6RkUGr4bbbgBtvtOkPnN57OrpGdcWCjQvQkue4t3ZES/5w0tLS5M6dOz0thsJODh4E/vUvVilHRdHlXF5OJZGSAjzySBvK3mmJSEmFUF4OBASwNsFMINkS/9n7H9y6/FasuHYFLut+mQsFVdhCCLFLSpnW6H6lEBTupLgY2L4d2LyZMYN27Vip3Lu35ZRUReugprYGPV7vgajAKPx6x68QtrIMFC7DkkJQaacKtxIeDkycyJuibeGr9cXskbNxx7d34IfjP2By18meFknRABVDUCgUbuOm/v/f3v3HVlXecRx/fwaUFoqiCMqQ4VyMm3OGiTqZZNEozpltzrhfhpnMHyOIRFCZq2MZE9S4kCwmxmUu0UwNbiHC1Dh0oGMgy1DRoKL114hMZCoyyWRqHPDdH+epXmvbeyv2Pue2n1fSpOf23Pt82rTn2+ecc7/POUzYdwJXrr7S1xJKyAXBzOqmaVATbVPaeOjlh7h/0/2541gnLghmVlfnTjyXcSPGsWDNAs8SSsYFwczqaujgobRNaWPtP9eyevPq3HGsgguCmdXdBUdfwNjWsSxYvSB3FKvggmBmddc8uJnLT7icVS+u4sHND+aOY4kLgpllMX3SdMYMH8PCNQtzR7HEBcHMshg2ZBhzJ89l5aaVrNuyrvoTrM+VoiBIWijpCUkbJK2Q9Mncmcys71147IWMahnlWUJJlKIgAIsi4qiImAjcA/w8dyAz63utTa1cNvkylj+/nPVb3YYmt1IUhIj4T8XmcMA3J5sNEBcddxH7Ne/nWUIJlKIgAEi6WtJLwDR6mCFImi5pvaT127Ztq19AM+sT+wzdhznHz+HuZ+9mwysbcscZ0OrW7VTS/cBBXXxpXkTcVbHfFUBzRMyv9prudmrWP+x4ZwcTrpvA1EOncsd378gdp9/rrttp3WYIEXFKRBzZxcddnXZdDJxVr1xmlt/I5pHM/tJslrYvZeNrG3PHGbBKccpI0mEVm2cAz+TKYmZ5zDl+Dq1NrVy15qrcUQasUhQE4FpJGyU9AZwKzM4dyMzqa/+W/Zl17CyWPLWE9m3tueMMSKUoCBFxVjp9dFREfCMiXs6dyczq79LJl9IypIVr1l6TO8qAVIqCYGYGMHr4aGYeM5Pbn7yd57c/nzvOgOOCYGalMvfLc2ka1ORZQgYuCGZWKge2HsiMSTO47fHb2PTGptxxBpS6vQ+hL0jaBmzuwyEOAF7vw9fvS42a3bnrr1GzN2puyJ99QkSM7vxgQxeEviZpfVdv3mgEjZrdueuvUbM3am4ob3afMjIzM8AFwczMEheEnv02d4C90KjZnbv+GjV7o+aGkmb3NQQzMwM8QzAzs8QFwczMABeEqhp1vWdJiyQ9k7L/UdLI3JlqJek7kp6StEdS6W7N60zSaZKelfSCpLbceWol6WZJr0lqqH7TksZLWiXp6fR70hDNMCU1S3pY0uMp95W5M3XmawhVSNqnY4lPSRcDR0TEjMyxqpJ0KvCXiNgl6ZcAEfGTzLFqIulzwB7gRmBuRJR2FSRJg4DngKnAFuAR4OyIeDprsBpI+gqwE7g1Io7MnadWksYCYyPiMUkjgEeBb5X9Zy5JwPCI2ClpCLAWmB0R6zJHe49nCFU06nrPEbEiInalzXXAwTnz9EZEtEfEs7lz1Og44IWI2BQR7wJ/oFjTo/QiYg3w79w5eisi/hURj6XP3wTagXF5U1UXhZ1pc0j6KNXxxAWhBrWu91xi5wH35g7RT40DXqrY3kIDHJz6C0mHAF8EHsqbpDaSBknaALwGrIyIUuV2QaBY7zkt0NP54wyAiJgXEeMplveclTft+6rlTvvMA3ZRZC+NWrKb9URSK7AUmNNpJl9aEbE7IiZSzNiPk1SqU3WDcwcog4g4pcZdFwPLgfl9GKdm1XJL+iHwdeDkKNnFol78zMvuZWB8xfbB6THrQ+kc/FJgcUQsy52ntyJih6RVwGlAaS7qe4ZQRaOu9yzpNOBy4JsR8VbuPP3YI8Bhkj4tqQn4PnB35kz9Wro4exPQHhG/yp2nVpJGd9ztJ6mF4kaEUh1PfJdRFZKWAodT3PWyGZjRCEt8SnoBGApsTw+ta4S7owAknQlcD4wGdgAbIuKreVN1T9LpwHXAIODmiLg6c6SaSPo9cCJFK+ZXgfkRcVPWUDWQNAV4EHiS4u8S4KcRsTxfquokHQXcQvF78glgSUQsyJvqg1wQzMwM8CkjMzNLXBDMzAxwQTAzs8QFwczMABcEMzNLXBDMzAxwQbB+QtIhkt5OfWI6HvtQW2pJLamV+buSDtjLMVskrU4dT5F0saR2Sb1qEyJppKSZe5OlhjE+1OpaUpOkNZLcscAAFwTrX/6R+sR0tKW+AfgacARwtqQjIuLttM/Wj2G884BlEbE7bc8EpkbEtF6+zsj03F5Roda/4d9RtEl4T+rO+gDwvd6Obf2TC4I1jLQoytT0+VWSru9h93q0pZ4G3JXy/AY4FLhX0iWSfpAWQ9kg6caKWcSdkh5NC6RMT69zLfCZtO+iNNup/E9+rqRfpM8PSbOeWyl64IzvbqxKPbS6vjN9H2YuCNZQ5gPzJE2jaHk8p4d9+7QtdepbdGhEvAiQ2oJsBU4C7qP4r/uENBvZzfsH3fMiYhJwDHCxpFFAG2l2ExE/rmH4w4BfR8TngWE9jFWLjcCxvdjf+jGfO7SGERFrUmOzS4ETO07VSFpI0eysng6g6LPUlZOBScAjRVxaKPrfQ1EEzkyfj6c4uL/Sy7E3V6yy1dNYVUXE7nQ9ZURabMYGMBcEaxiSvgCMBbZ3HLwkHUSx8lRnvW5LLeki4Edp83TgzMrtiKi87vA20NzdSwG3RMQVnV7/ROAUYHJEvCXpr928xi4+OHvvvM9/q43VS0OBd/bi+dZP+JSRNQQV6+guprgOsDO19waYCGzo4im9bksdETek0zYTI2Jr5+1O+74BDJLU1QH9AeDbksak7PtLmgDsC7yRisFngePT/m8CIyqe/yowRtIoSUMp1rToTndj1SSdsno9Iv5X63Os/3JBsNKTNAxYBlwWEe3AQt5fpKjLgpDWk54F/Jlizd0lEfHUxxxtBTCli7GfBn4GrJD0BLCSYmZzHzBYUjvFheR1af/twN/SinGL0sF5AfBwem63PfN7GOsDUqvrvwOHS9oi6fz0pZOAP32Ub976H7e/toYm6SaK0zqfAu6JiJqWJJT0InBMRLy+F2MfDVwSEed81NfITdIyoC0insudxfLzDMEaWkScHxF7KO6u2bfyjWld6XhjGsV1hz097VvD2I8Bq7q6zbMRpFNpd7oYWAfPEMzMDPAMwczMEhcEMzMDXBDMzCxxQTAzM8AFwczMEhcEMzMDXBDMzCxxQTAzMwD+D0I+ZtV3h/8cAAAAAElFTkSuQmCC\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEKCAYAAAASByJ7AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAABbCklEQVR4nO2dd3yT5RbHfyfp3nSX0lL2nhaQDSIouCfinrjFiWzvZbi4XlAvigv3VhAVkCF7FBmylwhtKS2ddI80zXP/OInpyGyTvEn7fD+ffNomb973JE2e85xNQghIJBKJRKJSWgCJRCKRuAdSIUgkEokEgFQIEolEItEjFYJEIpFIAEiFIJFIJBI9UiFIJBKJBADgpbQATSEyMlIkJSUpLYbEgzhdcBqlmlL0iukFNamVFkciUYR9+/blCSGi6t/v0QohKSkJe/fuVVoMiQexN3MvBnwwALdediumD5+utDgSiSIQUZqp+z1aIUgk9pLcOhnjO47HG7vewJODnkSQT5DSIilLXh6waxdw5gygVgO9egHJyUBgoNKSNQ4hgLQ0YMsW4ORJQKUC+vQBhg8HYmOVls7tIU+uVE5OThbSQpDYy65zuzBk2RAsHLsQzw95XmlxlEGnA1auBH7+GSACgoL4vrIywMcHePhhoH9/paW0j5oa4IsvgE2bAG9vICyMFcTFi/zabrkFGD+eX28Lh4j2CSGSG9wvFYLEWVRVAQcOAMeP8/eyfXv32XyO+3wcDmYfxNkpZxHgHaC0OK7nl1+A774D2rYFvOo5CsrLgZwc4MUXgW7dlJGvMfz4Iyu5pCS2DGpTXQ2kp7OiGzZMEfHcCXMKQWYZSZzCwYPAM88A774LpKQAf/wBfPwx8PTTbM0rvQ+ZPWI2cspy8P6+95UVRAlKSnjhTExsqAwAICAACA0Fvv1W+X+UrZSUAKtX82uqrwwAthhiY1lp1NS4Xj4PwW0UAhElENEmIjpOREeJaIrSMkkax7FjwKJFbAkkJfH3MCaGf4+MBD76CNixQ1kZh7cdjlFJo/D6jtdRqa1UVhhX8+efvCh6e5s/JiwMSE0FMjJcJVXTOHSIX5MpBWcgMBAoLAROn3aZWJ6G2ygEAFoAzwkhugG4FMDjRNRdYZkkdiIE8OWXvJ6Ycg35+gJxccBXXwEajcvFq8OcEXOQVZqFj/Z/pKwgriYry7IyANjPrlKx/90TKCjgoLg1iIDiYufL46G4jUIQQmQJIfbrfy8BcBxAvLJSSewlNRXIzGSPgzn8/YGKCuDIEZeJZZJRSaMwNGEoXt3xKqq0VcoK40r8/Gx3m1jacTuazEzg++/ZvFyyBNi9mwNRthAQYNtrEoJ3JRKTuI1CqA0RJQHoB2C3iccmE9FeItqbm5vrctkklsnL402YtUQOIuDCBdfIZF4GwpyRc5BRnIFPD36qrDCupEcPXjwtxQeqq1kZuKLws7oaWLYMmDEDWLsWOHuW/Y5LlwLPPw/89Zf1c3TXOxOsvSZvb6BjR8fI3QxxO4VAREEAfgTwtBCigW0nhHhfCJEshEiOimpQaCdRGLXatjikVgv8/Tfw8svAlCmc0PLzz0B+vvNlrM3Y9mMxKH4QXt72Mqprql17caXo0IGDr3l5ph8XAjh/HrjsMt55O5svvgA2b2aZ2rQBwsOBqChjBtTrr1uPZcTFAf36sdymEILPcfnlrnlNHopbKQQi8gYrgy+FEMuVlkdiP4YNpSXrvbycs5A2b2YrISDAmBY/bRqwf78rJGWICLNHzEZaURo+P/S56y6sJETAY4+x9j53jrWzgYoK3qF37gxcf73zZcnMBLZuNZ0qCrDvUaXi3YI17ruPFcPZs1xPAbAiKC7mwru+fV3zmjwYt6lDICIC8CmAAiHE07Y8R9YhuCfvvgvs3QskJDR8TKcD1qxhhXHNNQ1dS+XlvHGdM8c13goAEEIg+YNkFFUW4cQTJ+ClaiEF/Hl5nKq5dSsvnEJwJsAVV/BO2s/P+TL88AN/IEx9WAzU1PDOf9EiVlh//ME7icBALp7r0sWoTMrLge3b+ZxFRfya4uKACROAQYOsB9NbCG5fmEZEwwBsA3AYgE5/9wwhxGpzz5EKwT0pKgJeeYVrm+LijN/BmhouUjt6lL+fwcGmn3/hAncbePRR18m88sRKXP/t9fjs+s9wV5+7XHdhd6CsjLN0VCogOtq1i+bixbx7Dw+3fFxaGhfJHTvGlo2vL1s2VVXsZnryybqtKXQ6oLTUWIUtq5Pr4PYKoTFIheC+FBcDP/0EbNvG300i/llRwd9lS7v/mhr2JLz1luuqmoUQ6PteX1RqK3HssWNQq2QnVJewdCnXEERHmz9GCGDdOiA+noPH9V1LOTncbuOll4BWrZwrbzNBVipLXEpICHD33bwBnDYNmDoV+O9/eZMXEWH5uYZ0coMb2BUYYgmn8k/h+2Pfu+7CLZ1LLmE3jyUyM3mH0a2b6ThDdDQ/vnGjc2RsQUiFIHEqgYFAp04cowwLYzeRtdRyIdiacIULuzY3drsR3aO6Y97WedAJnfUnSJpOnz68eygqMv24EOxjbNfOcuFZTAywYQOnlkoajVQIEpcybBi7di2Rn8+egZAQ18hkQEUqzBo+C8dyj2H5cZnk5hJ8fDjvuKyMLYHa6WmlpRxfCAvjHYUlfH15pyGrkJuEVAgSl9K7N7uMzKXAV1dzn7KrrnKtXAZu7XErOkd0llaCK+nQgf3/ycmsFM6d486kAKeSXnqp9eIWQ5aUKyurmyHy3ZO4FF9f4Nlngdde4+98dLSxk0JODm/y7rjDWHjqatQqNWYNn4W7f7obv5z8Bdd1vU4ZQVoa8fHA5MnApEncgM7Liz8cajXvEr75xnI/lOJizjZytVnZzJAWgsTlxMcDc+cCN97IXoG0NE4z79sXmDULGDdOWfkm9ZqEDq06YO7WufDkLDyPJDiYaxLi4owxg0GD+PeKCtPPEYJNzquukumlTURaCBJFCAsDrr6av8NVVZz6bkuzSlfgpfLCjOEz8MDPD2DN6TWY0GmC0iIph6F4ZMMGbv3g48ML9NCh3MvcFYSF8WCb//2PLYDwcOPCX17O3VuHDwcGDnSNPM0YWYcgkZiguqYand7uhNigWOx6YBeoJe48y8p4ET52jIu7goJYQRiK2B58kP37ruLECR5wc/o07x50OlYQV13FfZfcZUfhAZirQ5AWgkRiAm+1N6YPm45HVj2C9WfWY1wHhf1YrkYI4P33eVB9UlJdV0xQELtv3n2XC8G6dHGNTF27ckfUnBxOU/Xx4biBDCQ7DBlDkEjMcG/fe9EmpA3mbmmBsYS0NO5AmJBg2i/v78/+/pUrXSsXEdccdO7MispeZSAEN7/bsgXYtIndYTqZTWZAqlaJyxHC2G7fVOGpu+Dr5YtpQ6fhiTVPYHPqZoxuN1ppkVzHrl38D7LkKouI4AU1P996+bk7cO4cz29NTeUPoeG1RUZyemuPHoqK5w648ddR0ty4eJG7GD/5JMcIJ0/muShpaUpLZp4H+j+AuKA4zN06V2lRXEtODlsBljCM2SwpcY1MTeH8eWDBAiA3l+cstGvHFkZSElsI//mP8iP83ACpECQuIT0dmD0bWLGCXdBt23JzypQUrknatk1pCU3j5+WHqUOnYnPqZmxLc1MhnUFwsPU2EIYeI54wkvLLL1mBRUU1tHoMmUsfflh3NkQLRCoEidOpqADeeIM3k23bGnsUeXkBrVtzyvlHH/EENXdk8iWTER0YjXlb5yktiusYOBCorLR8THEx//Nqt512R7KyOFPKUkfV4GAuiDt+3GViuSNSIUiczr59vHaYa3nv68tKYu1a18plKwHeAXh+8PNYf2Y9UlK+Z9dCWppts0I9lW7dWFvn5Jh+vKaGi8FMTTlyN7KyeDdiTU612r39ly5AKgSJ09m61XpHgagonrJmbVOqCDodHi3ujAidH+ateJYnd/3rX5wC2VzrYNRqbjrn58eLpKFK2FAVnJbG+f+urENwNs1ZwduIVAgSp1NcbN3NbKgpstYa2+XodMAnnyDoqx/wnHYAVgdkYG+CmgfCV1XxFJ9165SW0jnExHCA56abuCI4LY1vbdsCL7wATJzo/tYBwL1SDPEOS+h0HGxuwci0U4nTiYjgoHJAgPljtFq26q0ltricgweBzZuBdu3wuKoNFoo9mIetWEm3sdnj5wd8/TXQqxf705sboaHcY2TCBLYSvLw8I4hcm5gYoGdP4K+/zMc7ior4g9q1q2tlczOkhSBxOqNGWc9MzM7m9jhuV3S6Zg0viioVQuCLp3EpfqaTOIAL/LiPD2uyLVuUldPZqFQ87cjTlIGBO+7gD1d2dl3XkBCcD11UxHnQLbz9hVQIEqfTpw9vzLKzTT+ekcFJIOvWAfffD7z4Im/KrU1WdDpVVcCpU3Wi4U9hEEKEL+Zjq/G4iAjgzz8VEFBiM7GxwMyZ7O5KT+fiNIMLLCgImD7ddS043Bh3249JmiE+PsBzz3HtT2oqN68MDOT19uBB/k727g107Mgb0dJS4JNPOOvohRfMZyc5HZ2OfeS1/ORh8MNTGIj5tA1HRA56IpqFlqMb3Z+4OB7wff48KwUh2J3Uvr1nxEJcgLQQJC4hKooTcx56iPuhFRVxRqNWyy7qnj3ZWifilPCkJO6I8PbbCraa8fNjYcvK6tz9NC5FkPDBAugL1YqKWGCJZxAfDwweDAwZwtPapDL4B6kQJC7D35/jBC+9xAt9u3bGGeumaN2a+5D99Zdr5fwHImD8+Aa5+BEIwBMYgG9xBCdELvu2Lr9cISElEschFYJEEbKz2Wq3NBURYHfT7t1NuJAQgEbT+BzzoUO5xfL583XO8SwGwx/eWFD+G88CttP/XF3NYYcVK4CffgIOH27xXRMkboCMIUgUoaLC6CKyhK8vdxSwm9xczvzZsIGr3by9Od1p9Gg2PWwlIIADGUuX8oAWlQrw8UGURoNHwzpjUegxvHTLGHS0o23r/v3c1K+0lBNfhODC37Aw4JFHWnzmo0RBpEKQKEJgIC+CtbsQm6KyshGdlU+d4uZJWi33r/H15S35pk18e/xxoF8/288XFsapT+npvK0vKQEiIvB892ex5MsBeHn3f7DsumU2nerAAeDNN1ms+hMoi4uB11/nhJdOnWwXTyJxFNJlJFGEqChO7rh40fwxhrkJdnVHKCzk1hKBgTzcxZA37+3Nrp/wcGDJEu5vYw9EnLJ4/fXAXXcBEyYgNqknJvefjM8OfoazF89aPYVWy9lTUVEsXn1CQjig/umnsouCRBmkQpAoAhFw3XWsEDQa08ecP8/uk/bt7Tjxzp2cz2ouUm0ol9682R5xzTJ16FSoVWq8uv1Vq8eeOMEJSUFB5o8JC+O6jNRUh4gnkdiFVAgSxejdG7j3Xt6sZ2TwOq7VspI4e5Y39I89ZmdW4ObN1n1MMTF8nAO24fEh8Xig3wP4+MDHSC9Kt3jshQvWL2mYOWOuiE8icSZSIUgUZcwYYN48jveWlnIsOCKC3fwzZljvktqAkhJOTbKEtzdrn5qaxopdh2nDpgEAXt/xusXjVCrbdJAQ7j1aVNJ8caugMhEtA3A1gBwhRE+l5ZG4hjZtgDvv5Ft9DAuozVZCeLgxq8gcVVXsOnJQ35rE0ETc2/defLD/A0wfNh3xIfEmj0tK4tdhKZAuBN8SEx0imkRiF+62D/kEwJVKCyFRlupqHjMwbx7wwANc3fzmm9zvyOoOe8wYoKDA8jE5OcDYsQ6tUJ02bBpqdDVYuHOh2WPateO4dF6e+fNcuMCNU919CJmkeeJWCkEIsRWAlW+zpDlTUQEsXsxjBvLy2HqIiwNOngReew34/HMrrSwGDuRqt/x8048XFbH1MGKEQ+Vu36o97upzF97b9x4ulF4weQwR8OCDLP+FC3Vfh07HQXQ/P05ikkiUwK0UgkTy+edsCbRrx22EVCr27ERH8+56wwZg/XoLJwgKAp5/np+UmsrJ/dXVHKBIS2N30fPPN6K4wTozhs2ApkaDN3a+YfaYhARg1ixu5JeebrydO8eWwaxZlkf/SiTOhISbJTwTURKAX83FEIhoMoDJAJCYmHhJWgufgdqcyM0Fpk7lRdNcULWiglsH/fe/lsMEKC0F9uwBfv+d05ZCQrhKedAg6/0ymsCdy+/EihMrkDolFVGBURaPzc5mS4GI+605QUdJJCYhon1CiOT697tVUNkWhBDvA3gfAJKTk91Lm0maxMGD/NNSho2/P4cATp/mOfBmCQpiBTB6tENltMbM4TPx1eGvsChlEV4e87LFY2Ni+CaRuAvSZSRxG4qLbZuYRuQGw3PM0C2qG27tcSve/uNtFFTIcJjEs3ArhUBEXwPYBaALEWUQ0QNKyyRxHaGhts+ZMdX6wV2YOXwmSjWlWJyyWGlRJBK7cCuFIISYJISIE0J4CyHaCCE+Ulomievo04d3/5ayiCoqWBl06OA6ueylV0wv3NjtRry1+y0UVhYqLY5EYjNupRAkzYuLF7klhbWyAAORkTx+wDDd0EBAeR46n/wFl6QsQect7+POLnvgratyjtAOYvaI2SiqKsLbu99WWhSJxGY8LqgscX9OnAB+/hk4fpyzP2tquJ3zddcBPeIKgMxMPrB16wYDk++8k2MJBw4AYcE1GJL1Izr9vQaVVYRyEYDhCTp03LMTOBEEPPmk2w5G7xvbF9d0vgaLUhZhyqVTEOJrbw8OicT1uF3aqT0kJyeLvXv3Ki2GpBZbtgAffcRZnhERxlYNmvO56HLoe4xttRfxbfQVwjodcMklwMSJ3BNaj1YLHDkCnHn9e8Tt+RkFwUmIjlOjfXvWH0RgrVFcDMyezQUKbsie83sw8MOBeGXMK//0O5JI3AFzaadSIUgcxvnzXFgVG2scQwAAgaXZGL5tAVRVFcgScRg5WoWwMLBCyMriXNKZM+vmYBYUcAFZfLz5nkPZ2UDnzsAzzzjzZTWJCV9OwJ7MPTg75SyCfCz0vZZIXIg5hSBjCBKHsXkzr921lQGEQL8DH0Ndo0F5WDxUXiqcNcySUal4wddogI8/rhs4MAxSttSALioKOHTIfJsKN2D2iNnIK8/D0r1LlRZFIrGKVAgSh7FrVx3PDwAguCQTEXknUBbIu//AQA401yEmhgMPhtgCwL0c/PwsX1Cl4putUWsFGJwwGJe3vxz/2fkflFe7afGERKJHKgSJw9BoGhaWhRWm8i/6zqJExlnK/2DoOlp7TJivr23zCoSwrZrNGuXlnBZV5fjspTkj5iC7LBsf7PvA4edGQQG/b5mZVrr+SSTWkVlGEocRG8vzaWq3CiKhg5e2EuF5p+CjKUWV8IZvcCwIkQDqtZ+uvaD17Qts2mT5gpWVHH+INz1/wCZOngTWrOG+GUTsoho+nNtjx8U1/ry1GN52OEa2HYnXdryGh5Mfhp+XFcvHFv7+G/jpJ46+q9X83kVFAVdfzbm7csKOpBHIT43EYVxxRV3vjaqmGgnp29E6cy+i8o4juDQLoQWp6FW0A9i4kbWHAaK6/qYePTil6OJF0xcTggPSV1xhfUKaOdavBxYsAE6d4o56iYnsvtq6FZgzh91YDmLOyDnIKs3Csj+XNf1kBw4A8+ezUkhI4B7hiYmcnvX++8AXXzhkPKik5SEVgsQuhOBsosOHuU117Z5Cl1zCa1NmJh/Y58CniMo9hvKAKGi9/FCGQNQEBCOgdRjv7rdv59Lj4mLOUe3c2XgyLy+uM6is5BPWdh9VVPDQ5Z49gSsbOU/pr7944WzThvtNG3bUXl58X0gIT+UpLm7c+esxOmk0hiYMxSvbX0GVtgluqaIi4J13uIqvttwAEBzMfcPXrwf27Wu60JIWh1QIEps5fZo31LNmAYsWAW+8AUyZAnz5JSsGPz9jpmjJ0XREndqOC/7tcCakL3TlFfBVaZCYqE8cCgzkoMPRo5wldM89Dd0c7doBL70E9O/PWigjg4PN5eXArbfyxRtrHfz2G4/RNPV8Ifj+oiKOlDsAIsLsEbORUZyBTw9+2vgT7d7NloC5Zk4qFVtWq1ZJK0FiN3bXIRBRIIBKIYRjJpQ3AVmH4DqOHOEZBIGBtYrDwM3oMjN57X7hBXbp63RA1sLPUfzrVuT7xsPfH2jnl4Ww1D+hqtYYF/7qaj7444950pkliorYH6VWs2/f4jAEK1RVAY880nDwgsH8OX0aKCzkhdfPD5gxA7jsMnDxROMRQuDSjy5FTlkOTj1xCt5qG15DTQ0PitBqeWLQokWsQC3NdBCC+3+8+SZbOhJJPRo9D4GIVABuA3AHgAEAqgD4ElEugNUA3hdC/OVgeSU2UlgIpKSwW1mr5Ulcw4dzVwhHjQyurATefZcVQVC92ipvby4UPnOGY7M33qgvL9CkIn5AMMrUvDb9mRkHnToacd7ZSAi+iAB/gXxdK1QUV+PUySR0COeGdWZlDg113GAbjYYvVF8ZHDjArih/f76WVstK69dfOa4wbVqThh0TEeaMmIOrv74aXxz6Avf1u8/8wVotX3PVKo6jGGTNzmaXlqX3wvDatNpGyyppmdiSZbQJwAYA0wEcEULoAICIwgGMBvAqEa0QQnzhPDElptixgzfXNTW8PqhUPGJy7VqeNT9pkuW6Lls5eBAoK2tYY1CbuDhg3Trgqqv0hWne3jifXoO9f/Pj/v4AqdVIrWqN/X+1hkYDhLcSiK46h1/XqFHyO2/YH3vMYck95vH3Z5eQRmN0GaWmsjJo1aqu+RMaygHbnBwe9DxvXpPe1AmdJqB/XH8s2LYAd/W5C14qE19BrZY18B9/cJA7MdF4/8mTHEweN461c1YWWwxCsMZu3Zp/9/ZuqL0ltlNTw1P31Go2ix21u3JzbFEIlwshGnSpF0IUAPgRwI9E1AT7XdIYDh0C3nuPv/+167dCQvizvHYtL8y33NL0ax0+zGuoJXx9eX3NygKSkoCzEck49+eXCIxrVadMoLiYv2dEAEpLIOKjEdMuAL0upCBqwxYcXl2I4AlhCLp6FPfDtlac1hi8vHiS2rp1rIV0Os40Cgqq+8XXaID27fn36GieyXzqlJVRbZYxxBJu+PYGfHPkG9zZ+86GB/3+OyuD9u3ryuPlxem4GzbwzVCrYXiD09L4gxETA9x3X+PjKy2ZkhJuyLV2LceqdDpWyBMmAMnJjtlhuTFWg8pCiGoi6kpEY4iozpaDiK40HOMsASUNEQL49lveEJpaL9VqduOsWcOu96ai1dqW1m6YZSAE8M2ZgVD7e8NPGNOQtFp2h/v5AX6+At6FuTjZajAu2zQLA/YtRZzuPHTVNUjdeR5YupQb12VlNf0FmOKyy3gXXVjIi0BlZd0FtKSEtWvtiffe3oADYlbXdrkWvWN6Y/7W+ajR1QvFabXA6tVsJpnalYaH8y0tjS2YsDBWZEFB/LsQ7L+TsQP7yc9nC/CHHzjhwJCKXFQE/O9/wIcf2lYs6cFY/ZoT0VMAVgJ4EsARIrqu1sOWh8ZKnEJ6OgdyLbmRvbx4cd6/v+nXa9fO+shKQ/VxZCQPjj+ZFYLDgx9GYFkO/MvZpWHI4PQTFWhddRZHAgYg/0QuvKorURSWhEr/cPiEBeDvgnBUxbbliy5cyCaFo4mKAqZOZcHT09ka0Gj4moWFvMAOHlx3R+jtzb6zJqIiFWYNn4WT+Sfx/bHv6z6YmcnKKCDA/AmE4BTTixfZ5Kqo4PeosJBNudGjgV9+sX38nITf03fe4cU/KcloEhOxom3fnn2069YpKaXTsSXt9CEAlwghrgcwCsBsIpqif6xlONbcjKIi/pxac2v6+LDru6kMGsTXsrS+XLgAXHopb0xLSngdvRB/CbYPn4GS4DiEFqUj8OI5tK5Jh6+oxIbo27Ex4GqUVKhRHmgMThgskfJy8KJ98SK7T5xB+/bAa69xyquvL1+8VSt+ISNHNvSTVVbWtRiawE3db0L3qO6Yv3U+dKJWhbZWa/kfe/Eivznx8Txkon17lrl1a1ZgY8awy6i0lFN6JbZx9izHZswlDRCx1bZ6dbNWtLbEENRCiFIAEEKkEtEoAD8QUVtIhaAItmZcarWWN5q2EhbG2UPffssWdP3r5+Wx8rlObzv6+hpT4PMjOmP78BkILM3G6X1FuJDvjdJWCaghLwQfT4FXqGk/9z9rYkQE+3Mvu6zpL8QUAQHAtddyXm1WFl/PFEKwNXHppQ65rMFKuH357VhxfAVu6n4TPxAWxqadTmfaT2cw1bRaVk5du5q+AJFjdgMthQMHeBdjSRn7+7PPMy2N0/maIbZYCBeIqK/hD71yuBpAJIBeTpJLYoF27XhR1mjMHyMEryk9ezrmmlddBdx+O2c9GnqpnTvHvwcHA9OnG8cZtGnD61ptT09ZUAy07TsjTdUONeTFAlZXo0No3dbVWi1/L/9JkAkIcE1765tuYtOmoqLhY0LwInDppbwTdxC39rgVnSM6Y97WefinHig8HOjdmxceUxgWLCH4jTaHIdNIYhulpba9X0ROaYDoLtiiEO4GcKH2HUIIrRDibgAjnCKVxCJ+ftzC5/x588Wo2dmc15+U5JhrEgHjxwOLFwMPPMAb9quuYkWwYEHdtUmt5k13dnbdfnWxsRzb0GiA0jJCjF8xEn3qfLRQWspy/5OZpNE4J9OoPl26AI8/zsVvaWnsmy8rY6shLY37ctxnoW6gEahVaswcPhMHsw/il1O/GB+44QZ+3bV7PRkIDeX727Y1n1Zq+FB06OBQeZs1UVGWd1iAcZfVjAP2cmKah6LRcOLDn3/yztywNlRVsT8/PJwX68hIZeTT6bhV0IYNvIYZqptTU7mFUWgo8Fjsj+ibuQYloW2g0/EaHBbGzTr/2aydO8fa79Zb/zl3YSG7qdRq3rDXGcjTVAoLuV3Fnj38JrdtC4waxS4CJ+Sia3VadPlfF7Tya4U9D+0BGa5x4gTXPZSX8z9XrWYFpdXyzdeXs2BMkZ3NMYaZM1tM/nyTycvjUvv61eu1KSxkc3juXI9/X+UIzWZIdTWwbRvHuQoK+DPq7c2dmy+/3HGFvY1FCC5qW72ae8kBvLnq1o0VQ+XZLNz45yyU+kZC6+2Ptm35sX+yPysq2F00fz4QG4usLO74vGeP8ftoeL0TJlivlXBXPtr/ER785UGsun0VJnSaYHygooI1/t69RuU0dCi/0Fde4UWsdWujOVVTwxZNYCC326g9klRinU8+4ZbrSUkNF/yqKjbJn3uO62M8HKkQmjE6HSef6HSsBNyxHqmy0hjkVqlYWWRkAOVb/kDUD+8iONwH3vHRvBOuqeGAqEbDbpzkZKSn8xpYU8PrnCEbtKqK4xkdOnBjPUcE0V2NpkaDzm93RmxQLHY9sMtoJViiqAhYuZLbWxi+w0JwptENNyhnGnoyGg2wbBlbiH5+vHsxfLmIgLvv5uyzZkCTFQLxp/QOAO2FEHOJKBFArBDCSTmB1pEKoZmQmsr53bt386JGxAHcceOAtm2h03EboYoK80lAqansWZo0yZWCO4739r6HR1Y9gnV3rsPYDmNtf2JpKVsFQrCmVNos9HSE4PTTzZs5FdXLi7vtDh3arJSsIxTCuwB0AC4TQnQjolYA1gkhBjhWVNuRCqGZUVXFq76/f53AwPHjXC5gKUBeXc2JOYsXe6aVUKWtQse3OyIpLAlb791qm5UgkTQScwrBnnkIg4QQjwOoBAAhxEUAbuickHgsvr4cVa4XJT561PrYZG9vdkmlpTlPPGfi6+WLF4e+iO3p27ElbYvS4khaKPYohGoiUgMQAEBEUWCLQSJxKhqNbT3FiDy71cyD/R9EXFAc5m6Zq7QokhaKPQrhLQArAEQT0QIA2yF7GUlcQHy87Sni5mIMnoCflx9eGPICNqVuwvb07UqLI2mB2KQQ9AHlrQCmAngFQBaA64UQ31t8okTiAPr3ZwvBUguZ/Hxu7eP0WQpO5uFOtyHaOwzzProHePZZ4NVXjTUREomTsUkhCI48/ySEOCGEWCKE+J8Q4riTZZNIAHAt0I03clNSU0qhpITrtyZOdL1sDuXYMQTM+jeez+6IdaozSPHN5ZzaJUuAf//bNS08JC0ae1xGKUTk1IwiIrqSiE4S0WkimubMa0ncG63W2NkZ4LYZEycaO0nk5hr7KlVVcZGpR3dqyMzkecmBgXi01VhECH/M893NJd5JSVyEtmhRs+60KVEeW7qdGhgN4GEiSgNQBu50KoQQvR0hiD5gvQTAWAAZAPYQ0c9CiGOOOL/EM8jPBzZu5KFhGg3HBnr0YIVw1VXAsGHcDduQIt6rFxeOOrR9hRKsX//PnIMgAM9iMGbSRuwTmbgErdkXlprKXVn79VNaWkkzxR6FMN5pUjADAZwWQpwBACL6BsB1AKRCaCGkpwOvv87un9hYrrgWghf/117jAtzrr+d6tWaFRsM9SGr14n8CA7FQ7MQ8bMVPuI3vDAri1gpSIUichM0uIyFEmqmbA2WJB3Cu1t8Z+vvqQESTiWgvEe3NNdciWOJxVFVxUZlKxTMXDO03iLgRZWIisGIF90Zqdhhm99YqtgiBL57GIKykkzhoaDYcEGC+LbZE4gBsVghENMfUzYGymCrNbFBGLYR4XwiRLIRIjoqKMvEUiSdy8CC3jAkPN/24lxfXrP36q/mW3x6LYaKQrm5Zz1MYhBDhi/nYxndoNOZbXkskDsAel1HtYbJ+4CE5jsw0ygBQu59vGwCZDjy/xEWUlHCm5IkT/HfXrsCAAZwtZI6dO7lJpyVateI2M4WF/Huzwd+f3UDHj9fpUNoK/ngSA7GAtuGoyEGPogoe5OMuaDQc2ReCC0Cs/QMlbo/NCkEI8Ubtv4noPwB+dqAsewB0IqJ2AM4DuA3A7Q48v8QFbN0KfPYZVwwbegrt2QN89RVw113mm0WWlVnv0krELqVmObDqyiuBfft4ka31RjyDS7FYpGBB9e/4KmgcD+pRmooKDoKvXcu/GwZ8Dx8OXH21Y6oDCwp4l3DoEH+YOnfm8ztwYp2kIfZYCPUJANDeUYIIIbRE9ASAtQDUAJYJIeSUcA8iJQX44AOenlY/66eqCvjwQ77f1Fji6GiehWNpGJWhLYUlS8Nj6dwZuOce1qa+vvyGqNWIqFLjiaqueD3kMOY8sARdle7cV14O/Pe/POAiLo4DPADnCW/dCuzfz5OZzA2rt4XNm/l9ALh7KxEroN9+Mw5LMjfERtIk7IkhHCaiQ/rbUQAnwe0sHIYQYrUQorMQooMQYoEjzy1xLlot8PXXvA6YSgH19eXHvv6aj63PiBE8M8FSfCAnh11PjfVMCMEzTo4dY9eT26X0X3YZMHs20LcvC5qWBhQW4tlBT8PP2x8vn/1MaQmB5cv5zWvXru5oUy8vnjZWXQ28+27jAz179/JMgpgYziQIDeVdQps2fFu1CvjZkY4JSW3ssRCurvW7FkC2EMLEV1vSEjl5kgvJ2rY1f0xgICfJnDgB9OxZ97GYGFYa27fzxjMmpq4lUFrKimTCBDSKI0eAH37gNdYwoCcwkM83bpxtzfNcQocOwKOPAg8+yIurry+i1Wo8uvYo3tz9JuaMnIOO4R2Vka2sDNiyxbLbJiqK3+QzZ+yvFNTpgO++43OY2lWo1awkfv2VRwLKALvDscfueqxWuul5vYvnNadJJvEo8vNt3xQWFBh/12p5DXjuOV70i4rY9bRqFa89ublcn1BaCjz9NK8H9rJzJ9c3XLzIz09I4J8BAWyxvP++i7ukarU8+Dozk80iU3h7s4B6TfX8kOfhrfbGK9tecaGg9Th7lt+ofwZem8AQ6DnWiPKhM2f4H27JJ2joc94s84+Vxx4LYSyAF+vdN97EfS2C8nJeBInY3euOYytdibV5BaaOFQL44guuTE5M5Ps7dOAuDWfOcJuKw4d5NObIkY0bBlZQwB6I1q3rejgATu5p144VRr9+pmMbDqWyEtiwgYOxZfqkPW9vYPRoDiqHhZl9alxwHB7q/xDe3fsuZo+cjaSwJCcLawKt1rbh8mp14yL/RUW2nd/Li/2HEodj9WtMRI8CeAxAeyI6VOuhYAA7nSWYu1JQwEPjt2wxpo37+nKsa9w4zx303lQM3gGdzny8z/B+GY49e5bjh0lJxueoVKxgo6ONx/j5NX4y5M6dfN36ysAAESfFrFoFDBpkZT2qqWF/1++/s9ni48NPGjLEGFw1R0UFB2NPnmSfmCETR6NhBbFnDwdjLYxpnDp0Kt7b9x5e3f4qll691PL1nEGrVvxmGsacmqO6unFBZVt3VbVT2CQOxRaX0VcArgGnmF5T63aJEOIOJ8rmduTkAPPm8Y42OppdDwkJHPNavtzYdqElEhPDu+ysLPPHZGVx3yFDqv3mzbwGWEoYiY4G1qxpULNlM/v3W9x4A2Blk5HB9RNmqajg5nKvv85KQa3mxXzlSh74vNPK3uiHH4DTp9kkqb1r8PHhwEtZGfuuLPjd2oS0wQP9HsCyP5fhXNE5s8c5jcREDuwWFpo/prqarZ7GtNfo0MF6n3MhjA2uJA7HqkIQQhQJIVKFEJMAFAOIAdAWQE8iGuFsAd0FIYD33uN1ITGxrhvVz4+/56mpwPcteELE3XfzJrJ+m+rqar6vVSuuRcjPZ7fQyZPWd/6BgexJKCuzfJw5tFrrGYoGt7fZOIIQnE979CibM9HRvKgHBfGHISoKWLqUC8tMUVrKJmV8vPmddUwMcOoU595a4MWhL0JA4LUdCoTviIDbbuN/iKmdj1bL/+gbbmhcKlhAADBmDGdYmVOMWVn8ZTt2jC2qRx8FXnyRrayiIvuvKamDPWmnD4KH5KwF8G/9z385Ryz3Iy2Ns+1qFZI2ID6eU7Et7jSbMa1aATNnAmPHcmzw3Dm+5eZyDGDwYGDBAm5VPXUqe15On7Y8+8WwIWzszPmkJM5+skRlJSt1s7HMjAw2NRITTQti8GktX256ITtzhrWNpUCLobjLnFLR0zasLe7tcy8+3P8hMktsKOQvL2eL5uhRripuKj16AE8+yYtvWhpr9vx8VgSZmcAtt3Br2sZyww18jTNnWJEaqKzkHZdKxf+Pr7/mnUZEBL+333wDzJjBz5M0GnuCylMADACQIoQYTURdwYqhRXDypPE7aw4vL14Pzpxh10hLJDQUmDSJu5Ia4n7BwbyB/v13Y3o5wOvIkSP8c+hQ05mGxcWsaBtbezBqlDHeY85SyM5mec2u1ykp/KClf354OBdr5eYaAyAGqqv5uRUVRpeKqWCTWm0+66gW04dPx8cHPsbCHQux6MpFpg8qL2d31saNRn+bTgd068aFXUlJVq9jlksuYffZvn3GSuL27TmWYq4Zla34+gJTpnD31zVrWNEQsdIdN479jAEBdXuXBAXxrbAQ+M9/eNfRrHqbuA57FEKlEKKSiEBEvkKIE0TUxWmSuRmVlbYXR5oqvKpNQQF/l3Jy+LPdpw9bwY3dBbsj/v7GmoRPP+W1sv5r7NiRN5nFxcCffzbM8hGCN6A339z496ZdO16nduyoG7w2nD87m9ewUaMsnCQnx3xU2gARK42SkoYKITeXF06D/0qn4+Bxly51g9E1NQ2fa4L2rdrjzt53Yum+pZg2bBpiguqZreXlwMKFvKNu3dro3xSC75s/n90snTpZvZZZAgK4lcTw4Y0/hzl8fNh1NHo05woLwTuNDRtYoZqrgwgLYwWyYwe30JDYjT11CBlEFAbgJwDriWglWlDzuZgY67nqhoaV5jYnWi3w5Zecc//VV+xe+vVXYO5c/o7m5TlebqUpLubXacp9HhzM3oGaGvYC1PYQaDS8diUncyJPYyEC7r+fi4DPneP1Ijubr5eWxskw06ZZiWWEhNg209hUOtPGjfxP1+l4Ya6p4YDI6dPs9/7rLz5Oo2GFYqNpOWP4DGhqNHhj1xsNH/zlF37z2ratG+wy5EgHBwP/+58blmrXQ6Vil1BkJL+O33+3mIUFgBXshg2uka8ZYk9zuxv0v/6LiDYBCAXwm1OkckN69+aNS73eY3UoLuaMwnbtGj4mBO+Ut2zh72ntylhDS4XXXuPOBZb6+Xgaf/3VoNV/HTp2ZC/B7t3sPjJs/ry9gWuv5Zs9NQ6m8PYG7r2XJ67t2cMb/sBAToTp0MEG6yM5mRcjS5SU8GIVF2e8Lz0d+PxzzswpKAB27eIXY3hB5eXcn0er5Q/VpEk2p1N2juiMST0nYcmeJXhhyAuICtRbGhUVrIRqy1Gf0FDPnL5WWGj5dQGskHNymhZ4asHYE1QmIrqTiOYIIbYAOACgr7MEczcCAnjQ+7lzpjdWFRX8nZ80yfTnMC2Nd8pJSQ3bJBDxTjU3l12kzQlrm1AijikMGMDxyMcfB555hofl3HST5aJYe4mK4lYV997L1+rY0cY1o0sXzi82F5StqeFF6Jpr6vqkNm3ixT8riz84CQn8z9fp+MLe3rxwbdrEfnk7g7Ezh89ERXUFFqXUiiNkZPCbbi2n39eXA82eRHCwdUtNo2FtL5VBo7DHZfQOgMEAJun/LgHPQG4xGBotZmby5s+QPpmWxq7Oxx9nS8IUtuTcx8ayF8FaDMKTsKegrHdvXhd79XKzAj+VCnjqKd4VpKWx9geMQY60NF7Mhw2r+7yUFPYfHjnCZl+rVmySxMTwTtbPj3+PjGxUBlC3qG64pcct+N8f/0NBhb4fiKXoef3X5O4uo/qMHm19YlxODvsHJY3CHmN8kBCiPxH9CQBCiItE1KIaNhDxJnDoULb+//6bv1c9e7JXwVKvrbNnrS+OBmu3pKT5JEl06sSvu6zMfKaQRsMb1voN7yyh1XIq+oYNvPk2FA0PG2ZTXNZ+oqKAl17i7Je1a40pVF27sjLo1avhrrSqihddrdboJvLy4n+u4R8sBPsas7Ia1RBu1vBZ+O7od3gz5U38e/S/WU6dzrpiqKy03InQHRk6lEvKS0pM5wiXlrIF5oxAdwvBHoVQTURq6MdaElEUgEbWj3o24eHsj7YHg6fAEoac++bU6t3Li7OE3nuPPSb1PRlaLS/ot99uu1VQVsYx0WPHWAkHB7PXZtUqvt1/P68dDickhP/x48ezlaBWW84+io1lIS25LwxjMYnY52inQugV0ws3dL0Bb+5+E88Ofhah4eEcmD5+3Hz7iOpq/pAlJ9t1LcWJiOAOh4sWcTwhOpp3EobJbSoVp6zK0bqNxp6l5y0AKwBEE9ECANsBvOwUqZoh/ftbrvgHjEHp5hRUBnhxvusubvCZns41TcXFrAgyMrgG4MorbTuXENzh4eRJjsdERfGaHBjICicqipWPlfqupqFS8QWtpaJecQUHji21gS0vNwYzGtmDe/aI2SiqKsLbf7zNd9x8M1/z4sWGB2s0/E+46SbP/KB168YpeRMm8BcqLY0V6RVX8P32mJmSBtjS3O5zIcRdACIBTAUwBgABuF4I4cyvXbNi8GAuZDVUxdbH4I5++OHmFw8j4pqi/v255c/hw2wtDR7M1r09fdDS0oADB1gZWCoaXrGC1w5FGTjQWGxhmPxVm5ISXpRjYtgF1b5xAwj7xfXD1Z2uwcLt/wV2T0FWajyiKqfhxtQlSApKRXArb5BaxcpArWZzzFYN7I5ER7NCu/FGNg3V6ub3pVEIW1xGlxBRWwD3A/gMwNeGB4goXAhRYPaZkn+IiADuu49b4oSH110fKivZhTxwIC+SzZXISGMqaWNJSeHkHFuLhhX1Hvj7c5HJ8eOciRAczD40rZZvoaFcjZebyz+tdeEzQ1UV0ClzNn6tHohVF5fgsrBp0IS2wxLv1xCZexzXhBzFoH7VoIQ21oNdnoShGFDiMGx5N5eC6w3aA9hX634CxxMcNle5uTN8OK8JP/zAVrthcpefH1v548e70eQuNyU313qswdCsrrTUDdzJkZFchTh1KgeNfX15QU5I4A/DhQtcpzBpkvVzmeGLL4CCIwPQIelKHAp8A6PxJHwQiLg2amhje+Kd9J6oSuR+UhKJJawqBCHEWwDeIqJ3hRCPukCmZk3fvhzzO3eO/eg+Puz+8JQBO9XVnEV54gS7fTp04NdkzZ3uKGwpGjZUjJvqjaQIcXFcWPHLLzwjFGCz0DATdMKERu/a8/L4lImJwEjMwTIagr1iKYbgOQC8gY6NZRfasGFywyGxjC0xBBKMWWVgOMaxojVfDMVYnsaRIxywLS01um3Wr+eF9557uGeQsxk4kOu4LFFSwougtaJWlxIRwRVxN9/MZo6hGrGJmvTAAf6pUgEJGIx2Ygx2YiEG4DF4g02pgADjFLp27bhrRkkJ/986dpSzZiRGbHEZbSKiHwGsFEKkG+7U1yAMA3APgE0APnGKhBK34ORJ4I032D9fP329ooK7mapUzh9DaSgavnDBdDC6pobX28cec9M4o6Ezp4O4eLGuG30k5uATGon94gMMwlP/3C8EV8ovWWKcVCkEK/axY4HrrvMcK1XiPGxJO70SQA2Ar4kok4iOEdEZAH+Bq5YXCSE+caKMEoURglvyhISYrgfy9+fF+YsvnF/8aigaDgridjyGOS06HSuCtDQuHnT6fGQ3ISSkbtPFthiBtmIkduA1aGFspX3uHHfD9vZmhZ6YyD8jI7nB4jvvNK8KeUnjsGViWqUQ4h0hxFDwpLQxAPoLIdoKIR4SQhxwtpASZUlN5XoBS0kwAQHsSnJke5ySEu7T9tZbwJtvctFZQYGxaHjiRM6wSUvjBa99e47d3nKLm1oHTqBPH+OYYwMjMBsllIk/sQwAZ7SePw90796wWtwQw9q3jxsMSlo2duVsCSGqAViYmitpjhhc3tYWWZWK02f79m36NXfuBD7+mHethkLegwc5Q+vmmzkOO3481yNVVXGwtCW6PGJjOZN0/352pREB7XAZEsQQbMer6Kt7EIcP+yAx0Xy4gogthdWrOQ7UUpSppCEyiVdiFVszU3Q6x2Sx/PknxyRat667iIWHs0vqm2948R87lpWQSxrh5eTw4JU//uA0p4QE4PLLufpN4dSd++7juMDJk4YaF8JwMQdfqa/ExvxPERn2ELpYGWUVEsKp0OXljZ9OJ/F8mlHXHImzMExbtNSLyeCy6NixadfS6XhcbmSk6R2ttzen7f/wg03TJh3D9u08ReeXX9hk8fYGTp3iqWSLFpkeOO9CAgN5TvUjj3CMJy0N8D43DgmqATiX9DK69qi22h/LYAHKXMGWTaMUAhHZ0WxA4ulERHDbiSwLzsK8PA5SmhoOZA9nzrCLylKbHV9fdhMdPty0a9nE0aPcPCk6miOxhh5G0dGsKY8eBT76SPGV1MeH3T3z5rE4yz4ivDNxDs6XpyI//kurfbTKyjhGJFNQWzaNtRBWO1QKidtz553caeHcubpZLTqdUVFMntx0/3NBgW3nUKmMHaidhhBc0RUWZtpcMRSU7NvHUXc3Qa3m9+eqTlehX2w/bKxegIoqrUULLyeHG7na1GlXp+PKxI8/5qH2H37IXV2tzZiVuD2NjSHIsFMLo1UrYOZMbtC3c6fxfp2OrYdbbrGvSZ05DEPErOGSSuScHB56YamK0NBPJyWF4wpuBBFh9ojZuPG7G3HFwG+QuvtOJCbWrVswjG9NSrKxsLCwkNO+DG04/Py40m37dn79Tz/NJqXEI2msQvjAoVJIPIJWrYAHHuDF//x5XkxiYjiQWVHBrnR//6ZZCR068C7V0MTSFAaFYS1Q2mRKSlgYay/Iz8/6JC+FuK7rdegV3QtbaxbgX9dMwrrf1P+8t4Z01f79OTBtNThfVcXViRcuGANLtblwAXj9dc4Jlr4nj6SxCmG5I4UgolsA/AtANwADhRB7HXl+iWMJCTH2FEpJ4XTFCxd43YyL45TQQYMaNw85JITnJxj685giO5uVQZs2TXsdVvHzs81c0WhMV+y5ASpSYfaI2bj1h1uBUT9g8ZUTcegQu+YCAnh8QEyMjSfbv59TkcwFimJjuWhl924edynxONwlhnAEwI0Atjr4vBInUVEB/Pe/7D7WaHjDmJjI97/3HheSVVU17twTJ/K5zp6tm0lUXc3rUWAg8NBDLsiXb92a051KSiwfp9FwkyU35abuN6FbZDfM2zoPgUE6DB3K1dxjxtihDAAeHWpttmtEBPDbb/YJqNVaHycocQmNVQgO/SoKIY4LIU468pwS5/LVV5z33q6dMSOIiAPP7dtzI7zvvmvcuQMDueL4hhu4I2x6Ogezc3O59mD2bF6nDTgtwUel4uENOTnmA6YXLrD26tTJSUI0HRWpMGvELBzNPYqfTvzU+BNlZVm3hIKC2ISztsCXlnJnxOeeAx58kG9Ll3I8QqIYHhdDIKLJACYDQKIntgxtBly8yDVabdqY3qUT8WNbtvB4zMZ4UwIC+LkTJrAiEII3nwY/d24uz7vfuJHXlrAwrhMbOtT6JtYuhg5ljfTbb6z5IiL4BZaXs6KIigKefNLtB2FP7DER/97yb8zdMhc3dL0B1BjzyteXd/OWCvFqaqxPMMrN5ViDYYJRUhI/788/gV272EQcP16WTCtAoz7FQoh37H0OEW0goiMmbtfZee33hRDJQojkKMWnn7RMDLMQLK0LXl78HT9xomnX8vEB4uNZwRiUwZEjnPG0ahVvSNu25bVq+XJg1ix2NTkMIh5e8/zzbAmkp3PlV2UlcOutwJw5bjCFxzpqlRozhs3AweyD+OXUL407yaBB1oPnOTl8nLnFvKaGZ0OUlLAiMJRFq9UcgGrThkvRDx5snIySJuGy1hVCiMtddS2Jc6mosO04IRofRzBHdjbHJ0JD63aR9vfn9bqwEHj1VV6/w8P51uSNJhHQuzffqqp4UfPzc3uroD6397odc7fOxbyt83BN52vstxJGjQI2bOD3wFTOb3U1PzZmjPlzHD9uzHM1hbc3m3s//8yd+6SV4FI86xMtcQssVRHXhsjxyTebN7N1YmqkQEkJp8dv2sRenBde4AzIP/5wYJzB15f9WR6mDADAW+2N6cOmY2/mXvx22s7AL8Cm2j338IKel2d8U4XgtKX0dLamLJWr79plPb+1VSs28/Lz7ZdR0iSsfqqJKNyGW1hThCCiG4goA8BgAKuIaG1TzidxLj16sCvH0ijLqir+3nft6rjrCsGLvanMmPx8Vhbnz7NVUFrKdVLFxcDbb3MQXOgEL1rbtvG0mJMnW1x2y9197kZiaCLmbp2LRg05HDWK+zq1acPvZUYG/4yO5kyAK6+0/PyiIuttaQ1DsW01RSUOwxaXUab+Zsl2UwNodIRXCLECwIrGPt8WDNX269fzOgBwf/jLL+ecdmmZ2o6/P6ctfvcdW/71Ywk1NbxO3HGHY6uJtVpWNPXXk+pqrofw8eHr1dSwi5+IvQ/BwUDK8kyM3PYR2lT+bXwiEfv/772XPwwtAB+1D6YPm45HVz2K38/+jsvbN8KT27073woKuAmSv78x2G6NiAjgr78sH2MYiu3AyXIS27BFIRwXQvSzdAAR/ekgeZyCVgssW8aZMYGBxiyUo0eBPXu4huauuxTvYuxRXHUV777Xr2e3b3g4f48vXuQF+uqrgXHjHHtNLy923Ws0dZVCVhZf0xCfrKmpq4hCy7Mw7q8FyEwTiJ/QFqSqtXAVFXHX0uee4yqtFsB9fe/D/K3zMXfL3MYpBAOGII09DBnC6WeWyM/ntuIOTReT2IItjtDBDjpGMZYv58rXtm3ZsvX25ltMDN+3cSNnrEhsR6UCbr+dffSGHjg6HXsSoqPZI/P888C331rukmoPRMBll3FguTYZGXUVQHk510IY6HX4S/h51yBTG43Kqnq72NBQXtQ+/LB5z5C8cAH4/nvguefgO+VZvFjWF9vSt2HL6Q2ulaNTJ76ZawZYWck7jevsSj6UOAibRmiae4yI7rN2jNKUlgLr1nEGiqk4oErFi9iqVS7sr99MIOL44X33cbpnWBgHdSsreZ319ub3fuZMVhCOYORItuRqFw9rtcb/bUWFMVUVAAJLsxGdcwRlQTEgMlNfFhzMlsKxY44R0t3Yvh2YMYNrKby8gLAwPJibiFitP+Z+MZlfu6tQqYAnnuB/UGqq0aSsqODqw5wcbpvr9EZVElM0NVXi3w6RwokYuvJ6WXCOGQKkx4+7Tq7mhBA8pP3sWY4phIfze+rvz4HdmBhjh+SmEh0NPPMMK/r0dF5HAgPZlV1UxLIMGWK0GIJLMiFIBZ0gEFmIaahUXF/Q3Dh5EvjgA37jEhL4n+LtDf/waLygHo6NdBY73n7BtcH10FDeJTz2GMdwCgv5C3jFFcCCBcCwYa6TRVIHqzEEIjpk7iEA9nRCUYTSUttTDsvKnCtLc+XMGVambduajiv6+vIa8NNPlmO3Gg2vDSoVu4/NxXS6dQNefpljQps28RpXUwP06sVrnqnRBYaso8Y03PNofv6Zg7Mm3pSHcQleFdsxr2QVfjt9Gujc2XVy+fhw/yc37gHVErElqBwD4AoAF+vdTwB2NjzcvbCnC69LZvM2Q3bt4u+3pSST8HBOLsnLq9uHCGD3z4YNHKCuqmIFHhbG3QtGjjSdpRgRwW2Grr2Wj1+8mItb61sAJSHx0FbpQH46dOxowSDW6cwXS3kqFy+yWWamxUsgfPA8huDFgA3YvfEzDOo838UCStwNW1xGvwIIEkKk1bulAtjsVOkcQPfuvNO0FC+srmaXUrdurpOrOZGfb3pXXhsi/j+Ulta9v6AAmD8fWLmSrYiEBF6/1Grg8895ZLG12A4RzxPu25e9PpmZ7D7KzweO5kYjI7w3RnTJMV8kV1zMGqi5fQBKS63Oc3gUyQjX+WFe7o8uFEzirtgSVH5ACLHdzGO3O14kxxISwrU0586Zdh0JwY+NHStnejSW0FDrLSqEYLdObStMCHZvX7zIm/Pau/vAQA5YHz/OUyyt4e8PTJnCnVCHDGHXdLt2rCgmfHE7QiO8OGBZ/0NQWMgCPPig5UCTJ+Lvz6/Xgs80GL54tqwXVulOYH/WfhcKJ3FHmtk3wDQTJ/JOdO9e9k0b0psLCngnOXgwt1qWNI5Bg7hK2BJFRbzzj4423peRYYw9mIKIk1E2bmTXkKHOwBxEPHGtQ4f6j8RyEHPZMmNRFBEvlLGxrDUcWVLtLkREsFbMy7OY0/9EQSf8J/wE5m2dhxUTnVofKnFzbAkq7xdC9G/qMUri4wM8/jhw6BCwZo2x5XqXLjwSslcvWZTWFLp04cX+wgXTc5W1Wnbf3HVXXe/F8eP8t6XYg7c3P//vv7m3XKOJi+PUy4wMNgl1Ok5/MszsbI4QcUn5G29wYNlURD07G6GxbTHl0qfx723zcCj7EHrHNOWNlngytlgI3SxkGgEcXA51kDxOw8uLZ8f2789rgbWFSGI7KhXw1FNc8Juayu6awEB+n3NyODX05pv5va9NRYVtipiI4zxNhoiDFAkJDjiZh9CnD3DbbVyU5ufH/xy1mlPqDBH+p5/GlGBv/Hf3YszfOh/f3dLIyUYSj8cWhWCLLW1mnJR70lw3hEoSGcn++5QUtsIMKf3JydwvqnPnhgo4Ksq24mAhOE4haQRE3GekSxdO5TK0fm3VijuTDhkCBAWhFYCnBj2Fl7e9jKM5R9EjuofSkksUgOzteEhEc8HN7A4AOCCEsNKpynkkJyeLvXv3KnV5iRmEMA7WsqR8S0uBp59mz425+oCSEj7Pq69KRe4QdDpjpWY9DZ1XnoekxUm4tsu1+OqmrxQSUOIKiGifECK5/v12f8WEEHMAvAWgBMBNRKTYOM2WQm4usHMn9wQ7dMhB7hMnQsQLvLUFPCiIg8Xp6aZbSlRVscvp1lulMnAYKpXZEZeRAZF4fMDj+PbotziZJ0ect0RsthCIaDGAZ0Sjmqg7h+ZuIRQVAZ99BuzfXzdzMCiIF8nhwz0/DqLTAT/8AKxezZZASAi/1uJifm13383FaRLXkFOWg6TFSbilxy349PpPlRZH4iTMWQj2pJ2WAviZiG4TQpQR0TgALwkhhjpMSsk/lJYCr73GO+SEhLo75IoK7g1UWen4FtOuRqUyKrft2zkrVKXi1zV4sOyA7GqiA6PxSPIjeGv3W5gzYg46hDfI4XUIhYVc4b5zJ3+OY2N58mbPns2vHMSTsCuGQES3A3gGQBWAMgDzhRDbnCSbVZqzhbB8ObehMddNQaPhNM833pCLpsSxZJVkod2b7XBn7zvx4bUfOvz8hw4BS5bwZzg8nBVAWRnHizp04LiSrWNaJY2jyTEEIhoD4CGwIogC8JSSyqA5o9FwX5+4OPPH+PiwayUlxXVySVoGccFxmHzJZHx68FOkFqY69Nxpadx3KiSECxKDg7mgOjKSa+jS04G33mpxk03dBntCdTMBzBZCjAJwM4Bviegyp0jVwsnLY6VgbfxkcLBs2S1xDlOHToWKVHh1+6sOPe+aNRzTNld1Hh/PhaMnZUxbEWxWCEKIyww9jYQQhwGMByDbI0okzZA2IW1wf9/7sezPZThXdM4h5ywr4zKI2u1L6kPE9XOOGqgksY9GJ/MJIbIAjHGgLBI9kZFsHVhrGFdS0vwadErch2nDpkFA4PUdrzvkfKWlxq63lggI4GQKiQmysoDffwd+/ZUj8vXbBzeRJsXzhRAVjhJEYsTHh7uvrlxpOaisUnEmjkTiDNqGtcU9fe7BB/s/wIzhMxAXbCGoZQO+vhwbEMJyurRGw6nVkloUFXFzxoMH+W+12lhgeNVVXNDjgIZsstzHTRk7lv2pGRkNA2zl5Xz/7bdzG3+JxFlMHzYdWp0WC3cubPK5wsKATp24y7AlSkuBoTKZ3UhZGTcKO3aMI/FJSZyLnpTEZf7LlwPffmv7aEgLSIXgpgQFAS++CPTrx5kXqanGW0UFzyG/TIb0JU6mQ3gH3Nn7TizduxTZpdlNPt811/Bm11wPq4ICTkXt06fJl2o+bNnCHXrj4xuaVt7erBjWrQPOn2/ypezuZeRONOc6hNrk5XHmRXU11xx06dICZwNLFONU/il0W9INzw1+Dq+PbVo8QQhg1SpuvhoYyPEylYrjZdnZHD948cWW1ZDWIjU1wLPPcqTd0ozfjAxg9GjgjjtsOq0jKpUlChEZ2XAOsUTiKjpHdMZtPW/DO3vewdShUxEZ0PgPo6H5aocOwNq17BIn4vjC1VfzdMPwcMfJ7vEUF7MPzVr1aVgYcOpUky8nFYJEIrHKzOEz8fXhr7Fo1yIsGLOgSeci4uy4bt3YMqiu5s2vHFJlAsNkP2sI4ZAOkDKGIJG4M9XVvI3esoXTDBXKx+we1R03d78Zb//xNgoqrESF7cDXl+NlUhmYISSE3QMlJZaPKyzk0Y9NRFoIEok7IgSwbRtnj5SV1c3V7N8fuOcel08NmjViFr4/9j3e2v0W/jXqXy69dotFpQImTOCU06Ag0/m6Gg1/PoYNa/rlmnwGiUTieNavBz74gBeBpCRu9JOUxMOrDx/mVrgOLkqyRu+Y3rih6w1YnLIYRZVFLr12i2bIEN79p6U1HIZSVsYZSBMnWi4BtxGpECQSd+PiReCbbzjVpn5miUrF6YeZmZxq6GJmjZiFoqoivP3H2y6/dovFx4eHll9xBbsM09KMN60WeOQRh/XBd4u0UyJaCOAaABoAfwO4TwhRaO15LSXtVNLCWL2apwYlJpo/pqqKE/oXL+YFw4Vc8/U12HluJ1KnpCLYN9il127xlJVxDrpGw/GFjh0bFYBx2AhNJ7EeQE8hRG8ApwBMV1geiUQ5TpzgVraWMDS7ystzjUy1mD1iNgoqCvDOnndcfu0WT2AgV+0NGMAFSQ6OxruFQhBCrBNCGGoXUwC0UVIeiURx3MByN8fA+IG4osMV+M+u/6BMU6a0OBIH4hYKoR73A1hj7kEimkxEe4lob25urgvFkkhcRLdu1tMMq6rYSlCoYnHOyDnIK8/De/veU+T6EufgMoVARBuI6IiJ23W1jpkJQAvgS3PnEUK8L4RIFkIkR0VFuUJ0icS1DB7MwWONxvwxmZncAdHF8QMDQxKGYEy7MXh9x+uoqJZNj5sLLlMIQojLhRA9TdxWAgAR3QPgagB3CHeIdEskShEWxq1sz53j1ra10en4/oQEh2WWNJbZI2YjuywbH+z/QFE5JI7DLQrTiOhKAC8CGCmEKLd2vETS7LnsMnYJff89kJtbtzBtwADgrrvMz6F0ESOTRmJE2xF4bcdrmHzJZPh5+Skqj6TpuEva6WkAvgDy9XelCCEesfY8mXYqafZotZx1VFjIw1A6dQIiIpSW6h9+P/M7Lv/8crwz4R08OuBRpcWR2Ii5tFO3UAiNRSoEiURZhBAYumwoMoozcPqp0/BRKxPTkNiHu9chSCQSD4SIMGfkHJwrPofPDn6mtDiSJiIVgkQiaRJXdLgCya2T8fK2l1FdU239CRK3RSoEiUTSJIgIc0bMwdnCs/jq8FdKiyNpAlIhSCRWqKzkTM/0dJ5nLWnI1Z2vRt/YvliwbQG0OjMDkyVuj1uknUok7khpKbBmDbBhg7HrsJcXj64dP57LBSQMEWH2iNm46bub8O2Rb3FHb9tm+0rcC5llJJGYoLiYRw6cPw/ExXFJAMCKITOTO0ZMny7n/9ZGJ3Tos7QPanQ1OPzoYahVcgyauyKzjCQSO/jyS+DCBZ5JY1AGAODtDbRty52nP/pIMfHcEhWpMHvEbBzPO44fj/+otDiSRiAVgkRSj4ICYM8enkNjjrg44NgxthYkRm7qdhO6RnbFvK3zoBM6pcWR2IlUCBJJPf7+mztFqCx8O4j4mNOnXSeXJ6BWqTFr+CwcyTmClSdWKi2OxE6kQpBI6lF/bK0lLDUkbXZcuAD8+ivwySfAihU8wtFEDHJiz4noFN4Jc7fOhSfHKFsiMstIoghVVbyY+vtz5o47YU+gWKFxBK6lspKVQEoKm00+Pqw1V67k2Q2PPAKEhv5zuJfKCzOGz8B9K+/Dr6d+xTVdrlFOdoldyCwjiUs5cQJYuxY4cIDdLj4+wJgxnMrpLotrTQ3wwgu89gUFmT6mooJn2Cgw0ti11NQAb70FHDzI0XRDx1WArQNDGtaMGazd9VTXVKPL/7ogMiASux/cDar9PIniyCwjieKsXg288gpw8iS3809M5N34b78BL73EHgh3QK3mcQQ5Obw5ro9Gw8Hk225r5soAAI4fZ+1dXxkA/HebNlyxl5JS5yFvtTdmDJ+BPZl7sPbvta6TV9IkpEKQuISjR4FvvuH1IzraGLD18WHl4OUFLFpkegFWguRk4OGHgfx8IDWVZ9nn57PSys4G7rkHGD5caSldwPr1bCZZ2uFHRXEFXz1vw9197kZiaCLmbpGxBE/Bzby3kubKqlVASAjn8ZuiVSteeA8cAC691JWSmWfoUKB3b05BPXaM17suXYBBg+q4zJs3qan8j7NEUBAfp9HUKdrwUftg2tBpeGz1Y9h4diPGtB/jVFElTUdaCBKnU1LCngdrc11CQoBt21wjk60EB/PwsieeAJ58kqdWthhlALDpprNST2CY5qZuWJl8f7/7ER8cj7lb5zpJQIkjkQpB4nQqK9lFZC2u6O0NlJW5RiaJjfTvz74yS+TlAd27m0wX8/XyxdShU7E1bSu2pG5xkpASRyEVgsTpGDJ1amosH1dRwe5oiRsxciT/48wVZ+h0bAJeeaXZUzzU/yHEBMZg3tZ5ThJS4iikQpA4HX9/jgtkZ1s+rryc1x+JG9GmDTBxImcSFRfXDRyXlQFnzwKXXw707Gn2FP7e/pg6dCp+P/s7dqTvcIHQksYiFYLEJYwfzxvN0lLTj58/D7Rvz3VOEjfjiiuAp57igHF6Og+HSEtjq+G++4A777TqD3z4kocRGRAprQQ3R2YZSVxCmzbAs88Cb7/NLunwcHY5l5WxkkhKAqZMMRmXlCgNETBgAOfipqfzP83Pj2sTbPyHBfoE4vnBz2Pa79Pwx/k/MDB+oJOFljQGWakscSlFRcDu3cD27RwziI3lSuUePcynpEqaByVVJUh6MwlDEobgl0m/KC1Oi8ZcpbK0ECQuJTSUUzfHjVNaEomrCfYNxjOXPoPZm2Zjf9Z+9I/rr7RIknrIGIJEInEZTw58EqG+oZi/db7SokhMIBWCRCJxGaF+oXj60qex4sQKHMo+pLQ4knpIhSCRSFzKlEFTEOwTLK0EN0QqBIlE4lJa+bfCkwOfxA/HfsCx3GNKiyOphVQIEonE5Twz+BkEeAdgwbYFSosiqYVUCBKJxOVEBkTisQGP4Zsj3+BU/imlxZHokQpBIpEownODn4Ov2hcvb3tZaVEkeqRCkEgkihATFINHkh/BF4e+wN8FfystjgRuohCIaB4RHSKiA0S0johaKy2TRCJxPi8MeQFeKi+8sv0VpUWRwE0UAoCFQojeQoi+AH4FMEdheSQSiQuIC47DQ/0fwqcHP0VaoZsM1W7BuIVCEEIU1/ozEIDnNliSSCR28eKwF6EiFV7d/qrSorR43EIhAAARLSCicwDugAULgYgmE9FeItqbm5vrOgElEolTaBPSBvf1vQ/LDixDRnGG0uK0aFymEIhoAxEdMXG7DgCEEDOFEAkAvgTwhLnzCCHeF0IkCyGSo+R4LYmkWTBt2DTohA6v73hdaVFaNC5TCEKIy4UQPU3cVtY79CsAN7lKLolEojxJYUm4p889eH/f+8gqyVJanBaLW7iMiKhTrT+vBXBCKVkkEokyTB82HVqdFgt3LlRalBaLWygEAK/q3UeHAIwDMEVpgSQSiWvpEN4Bd/S+A0v3LkVOWY7S4rRI3EIhCCFu0ruPegshrhFCnFdaJolE4npmDJuBqpoqvLHzDaVFaZG4hUKQSCQSAOgS2QUTe0zEkj1LkFeep7Q4LQ6pECQSiVsxc/hMlFeXY3HKYqVFaXFIhSCRSNyKHtE9cHP3m/HW7rdwseKi0uK0KEgIzy0KJqJcAM6sd48E4Kl2q6fKLuV2PZ4qu6fKDSgve1shRINCLo9WCM6GiPYKIZKVlqMxeKrsUm7X46mye6rcgPvKLl1GEolEIgEgFYJEIpFI9EiFYJn3lRagCXiq7FJu1+Opsnuq3ICbyi5jCBKJRCIBIC0EiUQikeiRCkEikUgkAKRCsIqnznsmooVEdEIv+woiClNaJlsholuI6CgR6YjI7VLz6kNEVxLRSSI6TUTTlJbHVohoGRHlENERpWWxByJKIKJNRHRc/znxiGaYRORHRH8Q0UG93P9WWqb6yBiCFYgoxDDik4ieAtBdCPGIwmJZhYjGAdgohNAS0WsAIIR4UWGxbIKIugHQAXgPwPNCiL0Ki2QWIlIDOAVgLIAMAHsATBJCHFNUMBsgohEASgF8JoToqbQ8tkJEcQDihBD7iSgYwD4A17v7e05EBCBQCFFKRN4AtgOYIoRIUVi0f5AWghU8dd6zEGKdEEKr/zMFQBsl5bEHIcRxIcRJpeWwkYEATgshzgghNAC+AXCdwjLZhBBiK4ACpeWwFyFElhBiv/73EgDHAcQrK5V1BFOq/9Nbf3Or9UQqBBuwdd6zG3M/gDVKC9FMiQdwrtbfGfCAxam5QERJAPoB2K2wKDZBRGoiOgAgB8B6IYRbyS0VAhw379nVWJNbf8xMAFqw7G6DLbJ7CGTiPrfa9TVXiCgIwI8Anq5nybstQogaIURfsMU+kIjcylXnpbQA7oAQ4nIbD/0KwCoALzlRHJuxJjcR3QPgagBjhJsFi+x4z92dDAAJtf5uAyBTIVlaDHof/I8AvhRCLFdaHnsRQhQS0WYAVwJwm6C+tBCs4KnznonoSgAvArhWCFGutDzNmD0AOhFROyLyAXAbgJ8VlqlZow/OfgTguBDiv0rLYytEFGXI9iMifwCXw83WE5llZAUi+hFAF3DWSxqARzxhxCcRnQbgCyBff1eKJ2RHAQAR3QDgbQBRAAoBHBBCXKGoUBYgogkAFgNQA1gmhFigrES2QURfAxgFbsWcDeAlIcRHigplA0Q0DMA2AIfB30sAmCGEWK2cVNYhot4APgV/TlQAvhNCzFVWqrpIhSCRSCQSANJlJJFIJBI9UiFIJBKJBIBUCBKJRCLRIxWCRCKRSABIhSCRSCQSPVIhSCQSiQSAVAiSZgIRJRFRhb5PjOG+Bm2pichf38pcQ0SRTbymPxFt0Xc8BRE9pW/JbFebECIKI6LHmiKLDddo0OqaiHyIaCsRyY4FEgBSIUiaF3/r+8QY2lIvATAeQHcAk4iouxCiQn+MI9pL3A9guRCiRv/3YwAmCCHusPM8Yfrn2gUxtn6HPwG3SfgHfXfW3wFMtPfakuaJVAgSj0E/FGWs/vf5RPSWhcNd0Zb6DgAr9fIsBdAewM9E9AwR3akfhnKAiN6rZUX8RET79ANSJuvP8yqADvpjF+qtndo7+eeJ6F/635P0Vsg7APYDSDB3rdpYaHX9k/51SCRSIUg8ipcAzCSiO8Atj5+xcKxT21Lr+xa1F0KkAoC+LUgmgNEAfgPvuofqrZEaGBfd+4UQlwBIBvAUEUUAmAa9dSOEeMGGy3cBD7XpByDAwrVs4QiAAXYcL2nGSN+hxGMQQmzVNzZ7FsAog6uGiOaBm53VxtltqSPBfZZMMQbAJQD2sLjwB/e/B1gJ3KD/PQFAJwAX7Lx2Wq0pW5auZRUhRI0+nhKsHzYjacFIhSDxGIioF4A4AHmGxYuIYmH6c2x3W2oiehzAQ/o/JwC4ofbfQojaz68A4GfuVAA+FUJMr3f+UeAOl4OFEOX69semzqFFXeu9/jFl1q5lJ74AKpvwfEkzQbqMJB4B8RzdL8FxgDIiMnQ/7QfggImn2N2WWgixRO+26SuEyKz/d71jLwJQE5GpBf13ADcTUbRe9nAiagsgFMBFvTLoCuBS/fElAIJrPT8bQDQRRRCRL3imhTnMXcsm9C6rXCFEta3PkTRfpEKQuD1EFABgOYDnhBDHAcwD8C/9w31hQiHo50k/AWAteObud0KIow4WbR2AYSaufQzALADriOgQgPVgy+Y3AF76++aBZ11DCJEPYId+YtxC/eI8FzwW8ldY6Jlv4Vp10Le63gWgCxFlENED+odGA3DrttES1yHbX0s8GiL6COzWSQTwqxDCppGERJQKIFkIkdeEa/cD8KwQ4q7GnkNpiGg5gOlCiJNKyyJRHmkhSDwaIcQDQggdOLsmtHZhmikMhWkAvGEcrtLYa/8JYJOpNE9PQO9K+0kqA4kBaSFIJBKJBIC0ECQSiUSiRyoEiUQikQCQCkEikUgkeqRCkEgkEgkAqRAkEolEokcqBIlEIpEAkApBIpFIJHqkQpBIJBIJAOD/VoGtZSrXN5MAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": { - "tags": [], "needs_background": "light" - } + }, + "output_type": "display_data" } + ], + "source": [ + "plot_dataset(train_x,train_labels,net.W.detach().numpy(),net.b.detach().numpy())" ] }, { @@ -1308,11 +1288,12 @@ "id": "1W4TZfXOmIlS" }, "source": [ - "Посчитаем точность на тестовом датасете:" + "Not let's compute the accuracy on the validation dataset:" ] }, { "cell_type": "code", + "execution_count": 32, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -1320,24 +1301,34 @@ "id": "HUjdeIefsIsg", "outputId": "a1a363d4-a307-4769-9ccf-fe8a857b62af" }, - "source": [ - "pred = torch.sigmoid(net.forward(torch.tensor(valid_x)))\r\n", - "torch.mean(((pred.view(-1)>0.5)==(torch.tensor(valid_labels)>0.5)).type(torch.float32))" - ], - "execution_count": 205, "outputs": [ { - "output_type": "execute_result", "data": { "text/plain": [ "tensor(0.8667)" ] }, - "metadata": { - "tags": [] - }, - "execution_count": 205 + "execution_count": 32, + "metadata": {}, + "output_type": "execute_result" } + ], + "source": [ + "pred = torch.sigmoid(net.forward(torch.tensor(valid_x)))\n", + "torch.mean(((pred.view(-1)>0.5)==(torch.tensor(valid_labels)>0.5)).type(torch.float32))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's explain what is going on here:\n", + "* `pred` is the vector of predicted probabilities for the whole validation dataset. We compute it by running original validation data `valid_x` through our network, and applying `sigmoid` to get probabilities.\n", + "* `pred.view(-1)` creates a flattened view of the original tensor. `view` is similar to `reshape` function in numpy.\n", + "* `pred.view(-1)>0.5` returns a boolean tensor or truth value showing the predicted class (False = class 0, True = class 1)\n", + "* Similarly, `torch.tensor(valid_labels)>0.5)` creates the boolean tensor of truth values for validation labels\n", + "* We compare those two tensors element-wise, and get another boolean tensor, where `True` corresponds to correct prediction, and `False` - to incorrect.\n", + "* We convert that tensor to floating point, and take it's mean value using `torch.mean` - that is the desired accuracy " ] }, { @@ -1346,25 +1337,27 @@ "id": "_95qF9lY2kHp" }, "source": [ - "## Нейросети и оптимизаторы\r\n", - "\r\n", - "В PyTorch реализованы специальные модули `torch.nn.Module` для реализации нейросетей в виде набора слоёв. При этом есть два способа описания нейросетей:\r\n", - "* **Sequential**, в виде перечисления набора слоёв\r\n", - "* В виде **класса**, унаследованного от `Module`\r\n", - "\r\n", - "Первый способ проще позволяет описывать стандартные сети, второй - более гибкий, позволяет описывать произвольные сложные конфигурации вычислений.\r\n", - "\r\n", - "Внутри модулей можно использовать стандартные слои, среди которых:\r\n", - "* `Linear` - линейный слой, аналогичный нашей нейросети, приведенной выше\r\n", - "* `Softmax`, `Sigmoid`, `ReLU` - слои для передаточных функций\r\n", - "\r\n", - "> Обратите внимание, что большинство передаточных функций и функций потерь в PyTorch доступны в двух вариантах: **функциональный** (внутри `torch.nn.functional`) и **в виде класса** (внутри `torch.nn`). Для передаточных функций удобно использовать функции из `torch.nn.functional`, не создавая отдельного слоя.\r\n", - "\r\n", - "Для однослойной сети мы можем просто взять экземпляр слоя `Linear`:" + "## Neural Networks and Optimizers\n", + "\n", + "In PyTorch, a special module `torch.nn.Module` is defined to represent a neural network. There are two methods to define your own neural network:\n", + "* **Sequential**, where you just specify a list of layers that comprise your network\n", + "* As a **class** inherited from `torch.nn.Module`\n", + "\n", + "First method allows you to specify standard networks with sequential composition of layers, while the second one is more flexible, and gives an opportunity to express networks of arbitrary complex architectures. \n", + "\n", + "Inside modules, you can use standard **layers**, such as:\n", + "* `Linear` - dense linear layer, equivalent to one-layered perceptron. It has the same architecture as we have defined above for our network\n", + "* `Softmax`, `Sigmoid`, `ReLU` - layers that correspond to activation functions \n", + "* There are also other layers for special network types - convolution, recurrent, etc. We will revisit many of them later in the course.\n", + "\n", + "> Most of the activation function and loss functions in PyTorch are available in two form: as a **function** (inside `torch.nn.functional` namespace) and **as a layer** (inside `torch.nn` namespace). For activation functions, it is often easier to use functional elements from `torch.nn.functional`, without creating separate layer object.\n", + "\n", + "If we want to train one-layer perceptron, we can just use one built-in `Linear` layer:" ] }, { "cell_type": "code", + "execution_count": 33, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -1372,22 +1365,21 @@ "id": "D77pXPR6oFRs", "outputId": "efa49e5c-72d4-4781-89d4-4ab6597d2b0e" }, - "source": [ - "net = torch.nn.Linear(2,1) # 2 inputs, 1 output\r\n", - "\r\n", - "print(list(net.parameters()))" - ], - "execution_count": 206, "outputs": [ { + "name": "stdout", "output_type": "stream", "text": [ "[Parameter containing:\n", - "tensor([[-0.1892, -0.5979]], requires_grad=True), Parameter containing:\n", - "tensor([0.1538], requires_grad=True)]\n" - ], - "name": "stdout" + "tensor([[-0.6290, 0.2565]], requires_grad=True), Parameter containing:\n", + "tensor([-0.6059], requires_grad=True)]\n" + ] } + ], + "source": [ + "net = torch.nn.Linear(2,1) # 2 inputs, 1 output\n", + "\n", + "print(list(net.parameters()))" ] }, { @@ -1396,21 +1388,21 @@ "id": "0tbe0Et_oiNo" }, "source": [ - "Мы видим, что у сети есть метод `parameters()`, возвращающий все параметры, которые необходимо подстраивать.\r\n", - "\r\n", - "Методы обучения, такие, как градиентный спуск, реализованы в виде отдельных объектов, которым передается список параметров:" + "As you can see, `parameters()` method returns all the parameters that need to be adjusted during training. They correspond to weight matrix $W$ and bias $b$. You may note that they have `requires_grad` set to `True`, because we need to compute gradients with respect to parameters.\n", + "\n", + "PyTorch also contains built-in **optimizers**, which implement optimization methods such as **gradient descent**. Here is how we can define a **stochastic gradient descent optimizer**:" ] }, { "cell_type": "code", + "execution_count": 34, "metadata": { "id": "B4AxyrFMozh0" }, + "outputs": [], "source": [ "optim = torch.optim.SGD(net.parameters(),lr=0.05)" - ], - "execution_count": 207, - "outputs": [] + ] }, { "cell_type": "markdown", @@ -1418,11 +1410,12 @@ "id": "6eB8v58eo9pp" }, "source": [ - "С использованием оптимизатора, наш процесс обучения будет выглядеть так:" + "Using the optimizer, our training loop will look like this:" ] }, { "cell_type": "code", + "execution_count": 35, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -1430,38 +1423,37 @@ "id": "ups7nlV22ofp", "outputId": "503d8ae9-35f3-4ecb-e2ff-4da2ec2914eb" }, - "source": [ - "val_x = torch.tensor(valid_x)\r\n", - "val_lab = torch.tensor(valid_labels)\r\n", - "\r\n", - "for ep in range(10):\r\n", - " for (x,y) in dataloader:\r\n", - " z = net(x).flatten()\r\n", - " loss = torch.nn.functional.binary_cross_entropy_with_logits(z,y)\r\n", - " optim.zero_grad()\r\n", - " loss.backward()\r\n", - " optim.step()\r\n", - " acc = ((torch.sigmoid(net(val_x).flatten())>0.5).float()==val_lab).float().mean()\r\n", - " print(f\"Epoch {ep}: last batch loss = {loss}, val acc = {acc}\")" - ], - "execution_count": 208, "outputs": [ { + "name": "stdout", "output_type": "stream", "text": [ - "Epoch 0: last batch loss = 1.1270943880081177, val acc = 0.6666666865348816\n", - "Epoch 1: last batch loss = 1.0163036584854126, val acc = 0.7333333492279053\n", - "Epoch 2: last batch loss = 0.9239333271980286, val acc = 0.800000011920929\n", - "Epoch 3: last batch loss = 0.8493780493736267, val acc = 0.9333333373069763\n", - "Epoch 4: last batch loss = 0.7907424569129944, val acc = 0.8666666746139526\n", - "Epoch 5: last batch loss = 0.7453570365905762, val acc = 0.8666666746139526\n", - "Epoch 6: last batch loss = 0.7104158401489258, val acc = 0.8666666746139526\n", - "Epoch 7: last batch loss = 0.6834256649017334, val acc = 0.8666666746139526\n", - "Epoch 8: last batch loss = 0.6623864769935608, val acc = 0.8666666746139526\n", - "Epoch 9: last batch loss = 0.645784318447113, val acc = 0.8666666746139526\n" - ], - "name": "stdout" + "Epoch 0: last batch loss = 0.65546053647995, val acc = 0.20000000298023224\n", + "Epoch 1: last batch loss = 0.6187976002693176, val acc = 0.3333333432674408\n", + "Epoch 2: last batch loss = 0.5907494425773621, val acc = 0.3333333432674408\n", + "Epoch 3: last batch loss = 0.5694093108177185, val acc = 0.4000000059604645\n", + "Epoch 4: last batch loss = 0.5532841086387634, val acc = 0.46666666865348816\n", + "Epoch 5: last batch loss = 0.5412127375602722, val acc = 0.46666666865348816\n", + "Epoch 6: last batch loss = 0.5322896838188171, val acc = 0.46666666865348816\n", + "Epoch 7: last batch loss = 0.5258066058158875, val acc = 0.46666666865348816\n", + "Epoch 8: last batch loss = 0.5212085843086243, val acc = 0.6000000238418579\n", + "Epoch 9: last batch loss = 0.5180616974830627, val acc = 0.6000000238418579\n" + ] } + ], + "source": [ + "val_x = torch.tensor(valid_x)\n", + "val_lab = torch.tensor(valid_labels)\n", + "\n", + "for ep in range(10):\n", + " for (x,y) in dataloader:\n", + " z = net(x).flatten()\n", + " loss = torch.nn.functional.binary_cross_entropy_with_logits(z,y)\n", + " optim.zero_grad()\n", + " loss.backward()\n", + " optim.step()\n", + " acc = ((torch.sigmoid(net(val_x).flatten())>0.5).float()==val_lab).float().mean()\n", + " print(f\"Epoch {ep}: last batch loss = {loss}, val acc = {acc}\")" ] }, { @@ -1470,13 +1462,14 @@ "id": "vRLXEQ4Qrcvx" }, "source": [ - "> Обратите внимание, что для вызова (применения) нейросети к входному аргументу мы используем синтаксис `net(x)`, вместо `net.forward(x)`, поскольку `nn.Module` реализует метод `__call__()`\r\n", - "\r\n", - "С учётом описанного, можем описать универсальную обучающую функцию:" + "> You may notice that to apply our network to input data we can use `net(x)` instead of `net.forward(x)`, because `nn.Module` implements Python `__call__()` function\n", + "\n", + "Taking this into account, we can define generic `train` function:" ] }, { "cell_type": "code", + "execution_count": 36, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -1484,41 +1477,40 @@ "id": "5c6WsBhlrlIs", "outputId": "54de8404-4170-4a15-abba-039d06d5e946" }, - "source": [ - "def train(net, dataloader, val_x, val_lab, epochs=10, lr=0.05):\r\n", - " optim = torch.optim.Adam(net.parameters(),lr=lr)\r\n", - " for ep in range(epochs):\r\n", - " for (x,y) in dataloader:\r\n", - " z = net(x).flatten()\r\n", - " loss = torch.nn.functional.binary_cross_entropy_with_logits(z,y)\r\n", - " optim.zero_grad()\r\n", - " loss.backward()\r\n", - " optim.step()\r\n", - " acc = ((torch.sigmoid(net(val_x).flatten())>0.5).float()==val_lab).float().mean()\r\n", - " print(f\"Epoch {ep}: last batch loss = {loss}, val acc = {acc}\")\r\n", - "\r\n", - "net = torch.nn.Linear(2,1)\r\n", - "\r\n", - "train(net,dataloader,val_x,val_lab,lr=0.03)" - ], - "execution_count": 214, "outputs": [ { + "name": "stdout", "output_type": "stream", "text": [ - "Epoch 0: last batch loss = 0.6797666549682617, val acc = 0.06666667014360428\n", - "Epoch 1: last batch loss = 0.6262387037277222, val acc = 0.06666667014360428\n", - "Epoch 2: last batch loss = 0.5835054516792297, val acc = 0.20000000298023224\n", - "Epoch 3: last batch loss = 0.5517064332962036, val acc = 0.2666666805744171\n", - "Epoch 4: last batch loss = 0.5291333794593811, val acc = 0.2666666805744171\n", - "Epoch 5: last batch loss = 0.5140607357025146, val acc = 0.3333333432674408\n", - "Epoch 6: last batch loss = 0.5050028562545776, val acc = 0.4000000059604645\n", - "Epoch 7: last batch loss = 0.5006228089332581, val acc = 0.6000000238418579\n", - "Epoch 8: last batch loss = 0.4996645152568817, val acc = 0.6000000238418579\n", - "Epoch 9: last batch loss = 0.5009827017784119, val acc = 0.7333333492279053\n" - ], - "name": "stdout" + "Epoch 0: last batch loss = 0.717000424861908, val acc = 0.800000011920929\n", + "Epoch 1: last batch loss = 0.6802961826324463, val acc = 0.800000011920929\n", + "Epoch 2: last batch loss = 0.6523239016532898, val acc = 0.800000011920929\n", + "Epoch 3: last batch loss = 0.6305356621742249, val acc = 0.8666666746139526\n", + "Epoch 4: last batch loss = 0.6132137775421143, val acc = 0.8666666746139526\n", + "Epoch 5: last batch loss = 0.5993289947509766, val acc = 0.8666666746139526\n", + "Epoch 6: last batch loss = 0.5882464051246643, val acc = 0.8666666746139526\n", + "Epoch 7: last batch loss = 0.5795158743858337, val acc = 0.8666666746139526\n", + "Epoch 8: last batch loss = 0.5727649331092834, val acc = 0.8666666746139526\n", + "Epoch 9: last batch loss = 0.5676581263542175, val acc = 0.8666666746139526\n" + ] } + ], + "source": [ + "def train(net, dataloader, val_x, val_lab, epochs=10, lr=0.05):\n", + " optim = torch.optim.Adam(net.parameters(),lr=lr)\n", + " for ep in range(epochs):\n", + " for (x,y) in dataloader:\n", + " z = net(x).flatten()\n", + " loss = torch.nn.functional.binary_cross_entropy_with_logits(z,y)\n", + " optim.zero_grad()\n", + " loss.backward()\n", + " optim.step()\n", + " acc = ((torch.sigmoid(net(val_x).flatten())>0.5).float()==val_lab).float().mean()\n", + " print(f\"Epoch {ep}: last batch loss = {loss}, val acc = {acc}\")\n", + "\n", + "net = torch.nn.Linear(2,1)\n", + "\n", + "train(net,dataloader,val_x,val_lab,lr=0.03)" ] }, { @@ -1527,13 +1519,14 @@ "id": "KzuIDqJ8sFYm" }, "source": [ - "## Описание сети в виде набора слоёв\r\n", - "\r\n", - "Простейшую многослойную нейросеть можно описать в виде набора последовательно-применяемых слоёв. При этом такая сеть будет обладать всеми характеристиками вышеописанных - она автоматически соберёт в методе `parameters` параметры всех промежуточных слоёв" + "## Defining Network as a Sequence of Layers\n", + "\n", + "Now let's train multi-layered perceptron. It can be defined just by specifying a sequence of layers. The resulting object will automatically inherit from `Module`, e.g. it will also have `parameters` method that will return all parameters of the whole network." ] }, { "cell_type": "code", + "execution_count": 37, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -1541,13 +1534,9 @@ "id": "tBtytmEAsq-O", "outputId": "06ad840b-c2b7-409e-e01e-a9170548151d" }, - "source": [ - "net = torch.nn.Sequential(torch.nn.Linear(2,5),torch.nn.Sigmoid(),torch.nn.Linear(5,1))\r\n", - "print(net)" - ], - "execution_count": 215, "outputs": [ { + "name": "stdout", "output_type": "stream", "text": [ "Sequential(\n", @@ -1555,9 +1544,12 @@ " (1): Sigmoid()\n", " (2): Linear(in_features=5, out_features=1, bias=True)\n", ")\n" - ], - "name": "stdout" + ] } + ], + "source": [ + "net = torch.nn.Sequential(torch.nn.Linear(2,5),torch.nn.Sigmoid(),torch.nn.Linear(5,1))\n", + "print(net)" ] }, { @@ -1566,11 +1558,12 @@ "id": "5r5RbLB1s6YB" }, "source": [ - "Обучать такую сеть мы можем с помощью описанного ранее метода `train`:" + "We can train this multi-layered network using the function `train` that we have defined above:" ] }, { "cell_type": "code", + "execution_count": 38, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -1578,27 +1571,26 @@ "id": "ogXKdcfIs_ND", "outputId": "957ccd8d-0076-4e9b-89f1-edc1de75f18e" }, - "source": [ - "train(net,dataloader,val_x,val_lab)" - ], - "execution_count": 216, "outputs": [ { + "name": "stdout", "output_type": "stream", "text": [ - "Epoch 0: last batch loss = 0.6622872352600098, val acc = 0.6666666865348816\n", - "Epoch 1: last batch loss = 0.6727332472801208, val acc = 0.800000011920929\n", - "Epoch 2: last batch loss = 0.6326467990875244, val acc = 0.800000011920929\n", - "Epoch 3: last batch loss = 0.5876082181930542, val acc = 0.7333333492279053\n", - "Epoch 4: last batch loss = 0.5610458254814148, val acc = 0.7333333492279053\n", - "Epoch 5: last batch loss = 0.5546905398368835, val acc = 0.7333333492279053\n", - "Epoch 6: last batch loss = 0.5626044273376465, val acc = 0.7333333492279053\n", - "Epoch 7: last batch loss = 0.5754281878471375, val acc = 0.800000011920929\n", - "Epoch 8: last batch loss = 0.5850474238395691, val acc = 0.800000011920929\n", - "Epoch 9: last batch loss = 0.5874522924423218, val acc = 0.800000011920929\n" - ], - "name": "stdout" + "Epoch 0: last batch loss = 0.6754228472709656, val acc = 0.6000000238418579\n", + "Epoch 1: last batch loss = 0.639519453048706, val acc = 0.7333333492279053\n", + "Epoch 2: last batch loss = 0.606279194355011, val acc = 0.800000011920929\n", + "Epoch 3: last batch loss = 0.5834296345710754, val acc = 0.7333333492279053\n", + "Epoch 4: last batch loss = 0.5749474763870239, val acc = 0.7333333492279053\n", + "Epoch 5: last batch loss = 0.5772255063056946, val acc = 0.7333333492279053\n", + "Epoch 6: last batch loss = 0.5822412967681885, val acc = 0.7333333492279053\n", + "Epoch 7: last batch loss = 0.582870364189148, val acc = 0.7333333492279053\n", + "Epoch 8: last batch loss = 0.5753149390220642, val acc = 0.800000011920929\n", + "Epoch 9: last batch loss = 0.5589768886566162, val acc = 0.800000011920929\n" + ] } + ], + "source": [ + "train(net,dataloader,val_x,val_lab)" ] }, { @@ -1607,13 +1599,14 @@ "id": "jY4R1XEGtEzJ" }, "source": [ - "## Описание нейросети в виде класса\r\n", - "\r\n", - "Это более гибкий способ описания нейросетей, поскольку он позволяет выполнять произвольные вычисления." + "## Defining a Network as a Class\n", + "\n", + "Using a class inherited from `torch.nn.Module` is a more flexible method, because we can define any computations inside it. `Module` automates a lot of things, eg. it automatically understands all internal variables that are PyTorch layers, and gathers their parameters for optimization. You just need to define all layers of the network as members of the class:" ] }, { "cell_type": "code", + "execution_count": 39, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -1621,26 +1614,9 @@ "id": "SlsJmGu0tMsZ", "outputId": "240d5c89-096c-4392-99cd-1ade5ff3e3e1" }, - "source": [ - "class MyNet(torch.nn.Module):\r\n", - " def __init__(self,hidden_size=10,func=torch.nn.Sigmoid()):\r\n", - " super().__init__()\r\n", - " self.fc1 = torch.nn.Linear(2,hidden_size)\r\n", - " self.func = func\r\n", - " self.fc2 = torch.nn.Linear(hidden_size,1)\r\n", - "\r\n", - " def forward(self,x):\r\n", - " x = self.fc1(x)\r\n", - " x = self.func(x)\r\n", - " x = self.fc2(x)\r\n", - " return x\r\n", - " \r\n", - "net = MyNet(func=torch.nn.ReLU())\r\n", - "print(net)" - ], - "execution_count": 230, "outputs": [ { + "name": "stdout", "output_type": "stream", "text": [ "MyNet(\n", @@ -1648,13 +1624,30 @@ " (func): ReLU()\n", " (fc2): Linear(in_features=10, out_features=1, bias=True)\n", ")\n" - ], - "name": "stdout" + ] } + ], + "source": [ + "class MyNet(torch.nn.Module):\n", + " def __init__(self,hidden_size=10,func=torch.nn.Sigmoid()):\n", + " super().__init__()\n", + " self.fc1 = torch.nn.Linear(2,hidden_size)\n", + " self.func = func\n", + " self.fc2 = torch.nn.Linear(hidden_size,1)\n", + "\n", + " def forward(self,x):\n", + " x = self.fc1(x)\n", + " x = self.func(x)\n", + " x = self.fc2(x)\n", + " return x\n", + " \n", + "net = MyNet(func=torch.nn.ReLU())\n", + "print(net)" ] }, { "cell_type": "code", + "execution_count": 40, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -1662,27 +1655,26 @@ "id": "HwdapRxft-7M", "outputId": "6eb900cf-4902-4a04-c62b-497b68455406" }, - "source": [ - "train(net,dataloader,val_x,val_lab,lr=0.005)" - ], - "execution_count": 231, "outputs": [ { + "name": "stdout", "output_type": "stream", "text": [ - "Epoch 0: last batch loss = 0.6930208206176758, val acc = 0.8666666746139526\n", - "Epoch 1: last batch loss = 0.6779811382293701, val acc = 0.9333333373069763\n", - "Epoch 2: last batch loss = 0.6627538800239563, val acc = 0.9333333373069763\n", - "Epoch 3: last batch loss = 0.6472628712654114, val acc = 0.9333333373069763\n", - "Epoch 4: last batch loss = 0.6321150660514832, val acc = 0.9333333373069763\n", - "Epoch 5: last batch loss = 0.618022620677948, val acc = 0.9333333373069763\n", - "Epoch 6: last batch loss = 0.6052160859107971, val acc = 0.9333333373069763\n", - "Epoch 7: last batch loss = 0.5925578474998474, val acc = 0.9333333373069763\n", - "Epoch 8: last batch loss = 0.5803073048591614, val acc = 0.9333333373069763\n", - "Epoch 9: last batch loss = 0.569009006023407, val acc = 0.9333333373069763\n" - ], - "name": "stdout" + "Epoch 0: last batch loss = 0.7940112948417664, val acc = 0.20000000298023224\n", + "Epoch 1: last batch loss = 0.7393167614936829, val acc = 0.20000000298023224\n", + "Epoch 2: last batch loss = 0.693408727645874, val acc = 0.13333334028720856\n", + "Epoch 3: last batch loss = 0.6558710336685181, val acc = 0.13333334028720856\n", + "Epoch 4: last batch loss = 0.6256343722343445, val acc = 0.2666666805744171\n", + "Epoch 5: last batch loss = 0.6012560129165649, val acc = 0.3333333432674408\n", + "Epoch 6: last batch loss = 0.5820637345314026, val acc = 0.3333333432674408\n", + "Epoch 7: last batch loss = 0.5663362145423889, val acc = 0.3333333432674408\n", + "Epoch 8: last batch loss = 0.5532793998718262, val acc = 0.3333333432674408\n", + "Epoch 9: last batch loss = 0.541694700717926, val acc = 0.4000000059604645\n" + ] } + ], + "source": [ + "train(net,dataloader,val_x,val_lab,lr=0.005)" ] }, { @@ -1691,36 +1683,60 @@ "id": "dvAiaj_JndyP" }, "source": [ - "**Задание 1**: Постройте графики ошибок на обучающей и тестовой выборке в процессе обучения\r\n", - "\r\n", - "**Задание 2**: Попробуйте решить задачу классификации на датасете MNIST с помощью этого кода. Подсказка: используйте `crossentropy_with_logits` в качестве функции ошибки. При этом в первом случае на выход сети необходимо подавать целевые значения в формате *one hot encoding*, а во втором - в виде целочисленного номера класса." + "**Task 1**: Plot the graphs of loss function and accuracy on training and validation data during training\n", + "\n", + "**Task 2**: Try to solve MNIST classificiation problem using this code. Hint: use `crossentropy_with_logits` as a loss function." ] }, { "cell_type": "markdown", - "metadata": { - "id": "gZ-kWx84bMDH" - }, + "metadata": {}, "source": [ - "**Задание 3**: \r\n", - "Используйте PyTorch для обучения классификатора на сети MNIST. При этом:\r\n", - "* Обратите внимание, что в keras заложены типовые датасеты, включая MNIST. Для обращения к нему достаточно пары строчек кода с использованием библиотеки `torchvision` (см, например, [тут](https://pytorch.org/vision/0.8/datasets.html))\r\n", - "* Попробуйте несколько конфигураций сети с несколькими полносвязными слоями, передаточными функциями, и разным количеством нейронов\r\n", - "\r\n", - "Какой точности вам удалось достичь?\r\n" + "## Takeaways\n", + "\n", + "* PyTorch allows you to operate on tensors at low level, you have most flexibility.\n", + "* There are convenient tools to work with data, such as Datasets and Dataloaders.\n", + "* You can define neural network architectures using `Sequential` syntax, or inheriting a class from `torch.nn.Module`\n", + "* For even simpler approach to defining and training a network - look into PyTorch Lightning" ] }, { - "cell_type": "code", - "metadata": { - "trusted": true, - "id": "MG64NKzawHl-" - }, - "source": [ - "" - ], - "execution_count": null, - "outputs": [] + "cell_type": "markdown", + "metadata": {}, + "source": [] } - ] -} \ No newline at end of file + ], + "metadata": { + "accelerator": "GPU", + "celltoolbar": "Slideshow", + "colab": { + "collapsed_sections": [], + "name": "IntroPyTorch.ipynb", + "provenance": [] + }, + "interpreter": { + "hash": "0cb620c6d4b9f7a635928804c26cf22403d89d98d79684e4529119355ee6d5a5" + }, + "kernelspec": { + "display_name": "Python 3.8.12 64-bit (conda)", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.12" + }, + "livereveal": { + "start_slideshow_at": "selected" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +}