1433 lines
159 KiB
Plaintext
1433 lines
159 KiB
Plaintext
{
|
||
"nbformat": 4,
|
||
"nbformat_minor": 0,
|
||
"metadata": {
|
||
"celltoolbar": "Slideshow",
|
||
"kernelspec": {
|
||
"name": "python3",
|
||
"display_name": "Python 3",
|
||
"language": "python"
|
||
},
|
||
"language_info": {
|
||
"mimetype": "text/x-python",
|
||
"nbconvert_exporter": "python",
|
||
"name": "python",
|
||
"file_extension": ".py",
|
||
"version": "3.7.4-final",
|
||
"pygments_lexer": "ipython3",
|
||
"codemirror_mode": {
|
||
"version": 3,
|
||
"name": "ipython"
|
||
}
|
||
},
|
||
"livereveal": {
|
||
"start_slideshow_at": "selected"
|
||
},
|
||
"colab": {
|
||
"name": "IntroKerasTF.ipynb",
|
||
"provenance": [],
|
||
"collapsed_sections": []
|
||
}
|
||
},
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "En2vX4FuwHlu"
|
||
},
|
||
"source": [
|
||
"# Введение в нейронные сети\n",
|
||
"\n",
|
||
"## Эпизод 2а: Многослойный персептрон на TensorFlow и Keras\n",
|
||
"\n",
|
||
"Дмитрий Сошников | dmitri@soshnikov.com\n",
|
||
"\n",
|
||
"http://github.com/shwars/NeuroWorkshop\n",
|
||
"-> Notebooks -> IntroKerasTF.ipynb"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "fVjlmWBwwHl2"
|
||
},
|
||
"source": [
|
||
"## Нейросетевые фреймворки\n",
|
||
"\n",
|
||
"Мы видели, что для обучения нейросетей нужно:\n",
|
||
"* Быстро умножать матрицы (тензоры)\n",
|
||
"* Считать производные для вычисления градиента для метода обратного распространения ошибки\n",
|
||
"\n",
|
||
"Что позволяют делать нейросетевые фреймворки:\n",
|
||
"* Оперировать с тензорами, как на CPU, так и на GPU\n",
|
||
"* Автоматически вычислять производные (они вручную прописаны для всех элементарных функций)\n",
|
||
"\n",
|
||
"Опционально:\n",
|
||
"* Конструктор для нейросетей (описание сети как набора слоёв)\n",
|
||
"* Простые функции для обучения (`fit`, как в Scikit Learn)\n",
|
||
"* Набор алгоритмов оптимизации\n",
|
||
"* Набор абстракций для работы с данными"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "8cACQoFMwHl3"
|
||
},
|
||
"source": [
|
||
"## Основные фреймворки\n",
|
||
"\n",
|
||
"* Tensorflow 1.0 - первый, получивший широкое распространение (Google). Позволял определять статический computation graph, и затем в явном виде выполнять вычисления\n",
|
||
"* PyTorch - Facebook\n",
|
||
"* Keras - надстройка над Tensorflow/PyTorch для унификации (Francois Chollet)\n",
|
||
"* Tensorflow 2.0 + Keras - динамический вычислительный граф, код получается похожим на обычные вычисления в numpy\n",
|
||
"\n",
|
||
"Мы рассмотрим Tensorflow 2.0 и Keras. Вам необходимо убедиться, что у вас установлена версия 2.x.x Tensorflow:\n",
|
||
"```\n",
|
||
"pip install tensorflow\n",
|
||
"```\n",
|
||
"или\n",
|
||
"```\n",
|
||
"conda install tensorflow\n",
|
||
"```\n",
|
||
"или выполняйте код в [Google Colab](https://colab.research.google.com/)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"tags": [],
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/"
|
||
},
|
||
"id": "xwqVx9-bwHl3",
|
||
"outputId": "2aa591b4-b647-441f-9c8e-4e0da2d517a0"
|
||
},
|
||
"source": [
|
||
"import tensorflow as tf\n",
|
||
"import numpy as np\n",
|
||
"print(tf.__version__)"
|
||
],
|
||
"execution_count": 1,
|
||
"outputs": [
|
||
{
|
||
"output_type": "stream",
|
||
"text": [
|
||
"2.4.1\n"
|
||
],
|
||
"name": "stdout"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "6tp2xGV7wHl4"
|
||
},
|
||
"source": [
|
||
"## Основные понятия в TensorFlow\n",
|
||
"\n",
|
||
"**Тензор** - это многомерный массив произвольной размерности. Удобно использовать при обучении нейросетей, например:\n",
|
||
"* 400x400 - чёрно-белая картинка\n",
|
||
"* 400x400x3 - цветная картинка\n",
|
||
"* 16x400x400x3 - minibatch из 16 картинок, используемый для одного шага обучения\n",
|
||
"* 25x400x400x3 - секунда видео\n",
|
||
"* 8x25x400x400x3 - minibatch из 8 1-секундных видео"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "qG2bsaR7wHl4"
|
||
},
|
||
"source": [
|
||
"### Простые тензоры"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"trusted": true,
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/"
|
||
},
|
||
"id": "ybpnk08HwHl4",
|
||
"outputId": "fad9ed4a-df82-44a0-84ea-324bc71ea46f"
|
||
},
|
||
"source": [
|
||
"a = tf.constant([[1,2],[3,4]])\n",
|
||
"print(a)\n",
|
||
"a = tf.random.normal(shape=(10,3))\n",
|
||
"print(a)"
|
||
],
|
||
"execution_count": 2,
|
||
"outputs": [
|
||
{
|
||
"output_type": "stream",
|
||
"text": [
|
||
"tf.Tensor(\n",
|
||
"[[1 2]\n",
|
||
" [3 4]], shape=(2, 2), dtype=int32)\n",
|
||
"tf.Tensor(\n",
|
||
"[[ 1.1846514 0.7508411 -1.2235912 ]\n",
|
||
" [ 0.9710432 -0.15184928 -0.10070106]\n",
|
||
" [ 0.01476252 -0.19518584 0.43259642]\n",
|
||
" [ 1.8912938 0.6803918 -0.41853315]\n",
|
||
" [-1.0234195 -0.46338254 -1.6521171 ]\n",
|
||
" [-2.6472843 0.72314113 -0.8514363 ]\n",
|
||
" [-0.2650735 -0.22901714 -0.4702833 ]\n",
|
||
" [ 0.4691955 -0.6513927 -0.22865944]\n",
|
||
" [-1.0068002 2.3035717 -0.8701199 ]\n",
|
||
" [ 0.35074237 0.192767 -1.013126 ]], shape=(10, 3), dtype=float32)\n"
|
||
],
|
||
"name": "stdout"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "AXFMsV3r09Ux"
|
||
},
|
||
"source": [
|
||
"С тензорами можно производить обычные вычисления, которые производятся поэлементно (как в numpy). При этом тензоры автоматически дополняются до нужной размерности. Можно извлечь numpy-массив из тензора при помощи `.numpy()`:"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/"
|
||
},
|
||
"id": "e5Nu5Xgj1DnQ",
|
||
"outputId": "0dfc8758-4ffd-4968-c7bf-6ba8d435df2e"
|
||
},
|
||
"source": [
|
||
"print(a-a[0])\r\n",
|
||
"print(tf.exp(a)[0].numpy())"
|
||
],
|
||
"execution_count": 3,
|
||
"outputs": [
|
||
{
|
||
"output_type": "stream",
|
||
"text": [
|
||
"tf.Tensor(\n",
|
||
"[[ 0. 0. 0. ]\n",
|
||
" [-0.21360815 -0.90269035 1.1228901 ]\n",
|
||
" [-1.1698889 -0.9460269 1.6561877 ]\n",
|
||
" [ 0.7066424 -0.07044929 0.80505806]\n",
|
||
" [-2.2080708 -1.2142236 -0.42852592]\n",
|
||
" [-3.8319356 -0.02769995 0.3721549 ]\n",
|
||
" [-1.4497249 -0.9798582 0.75330794]\n",
|
||
" [-0.7154559 -1.4022338 0.99493176]\n",
|
||
" [-2.1914515 1.5527306 0.35347128]\n",
|
||
" [-0.83390903 -0.5580741 0.2104652 ]], shape=(10, 3), dtype=float32)\n",
|
||
"[3.2695467 2.1187813 0.29417184]\n"
|
||
],
|
||
"name": "stdout"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "uQ5zN6cVyrG7"
|
||
},
|
||
"source": [
|
||
"## Переменные\r\n",
|
||
"\r\n",
|
||
"Переменные могут содержать какие-то значения, которые мы затем можем модифицировать с помощью методов `assign` и `assign_add`. \r\n",
|
||
"\r\n",
|
||
"Например, вот глупый способ посчитать сумму всех строк тензора `a`"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/"
|
||
},
|
||
"id": "7pu0UZ-_yqfB",
|
||
"outputId": "6708c83e-02e6-4442-8757-45918eb1fbc2"
|
||
},
|
||
"source": [
|
||
"s = tf.Variable(tf.zeros_like(a[0]))\r\n",
|
||
"for i in a:\r\n",
|
||
" s.assign_add(i)\r\n",
|
||
"\r\n",
|
||
"print(s)"
|
||
],
|
||
"execution_count": 4,
|
||
"outputs": [
|
||
{
|
||
"output_type": "stream",
|
||
"text": [
|
||
"<tf.Variable 'Variable:0' shape=(3,) dtype=float32, numpy=array([-0.0608891, 2.9598851, -6.3959713], dtype=float32)>\n"
|
||
],
|
||
"name": "stdout"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "rIh1EHcezlNo"
|
||
},
|
||
"source": [
|
||
"Умный способ:"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/"
|
||
},
|
||
"id": "aQIdWZ1kzn6P",
|
||
"outputId": "1c123d9a-ecd2-4f2e-828e-5ade85ac8f63"
|
||
},
|
||
"source": [
|
||
"tf.reduce_sum(a,axis=0)"
|
||
],
|
||
"execution_count": 8,
|
||
"outputs": [
|
||
{
|
||
"output_type": "execute_result",
|
||
"data": {
|
||
"text/plain": [
|
||
"<tf.Tensor: shape=(3,), dtype=float32, numpy=array([1.8912938 , 2.3035717 , 0.43259642], dtype=float32)>"
|
||
]
|
||
},
|
||
"metadata": {
|
||
"tags": []
|
||
},
|
||
"execution_count": 8
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "U-auwezDwHl6"
|
||
},
|
||
"source": [
|
||
"## Вычисляем производные\r\n",
|
||
"\r\n",
|
||
"Для обратного распространения ошибки, нам нужно уметь вычислять градиенты. Это делается с помощью `tf.GradientTape()`:\r\n",
|
||
" * Оборачиваем интересующие нас вычисления в `with tf.GradientTape`\r\n",
|
||
" * Помечаем интересующие нас тензоры вызовом `tape.watch` (переменные отслеживаются автоматически)\r\n",
|
||
" * Проводим вычисления\r\n",
|
||
" * Получаем градиенты через `tape.gradient` "
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"trusted": true,
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/"
|
||
},
|
||
"id": "m8vFOXr7wHl6",
|
||
"outputId": "860ac72e-50c7-4ff2-f258-747f27194f90"
|
||
},
|
||
"source": [
|
||
"a = tf.random.normal(shape=(2, 2))\n",
|
||
"b = tf.random.normal(shape=(2, 2))\n",
|
||
"\n",
|
||
"with tf.GradientTape() as tape:\n",
|
||
" tape.watch(a) # Start recording the history of operations applied to `a`\n",
|
||
" c = tf.sqrt(tf.square(a) + tf.square(b)) # Do some math using `a`\n",
|
||
" # What's the gradient of `c` with respect to `a`?\n",
|
||
" dc_da = tape.gradient(c, a)\n",
|
||
" print(dc_da)"
|
||
],
|
||
"execution_count": 9,
|
||
"outputs": [
|
||
{
|
||
"output_type": "stream",
|
||
"text": [
|
||
"tf.Tensor(\n",
|
||
"[[0.9086961 0.954213 ]\n",
|
||
" [0.85484374 0.9623889 ]], shape=(2, 2), dtype=float32)\n"
|
||
],
|
||
"name": "stdout"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "8sfjBMBu59B5"
|
||
},
|
||
"source": [
|
||
"## Пример 1: Линейная регрессия\r\n",
|
||
"\r\n",
|
||
"Попробуем с помощью полученных знаний решить классическую задачу линейной регрессии. Для этого сгенерируем небольшой синтетический датасет:"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"trusted": true,
|
||
"id": "j723455WwHl7"
|
||
},
|
||
"source": [
|
||
"import matplotlib.pyplot as plt\n",
|
||
"from sklearn.datasets import make_classification, make_regression\n",
|
||
"from sklearn.model_selection import train_test_split\n",
|
||
"import random"
|
||
],
|
||
"execution_count": 10,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/",
|
||
"height": 282
|
||
},
|
||
"id": "WJNK_J6v6I-Z",
|
||
"outputId": "eb4a66a6-6b9a-4c8a-bc24-d81eeb2d3f27"
|
||
},
|
||
"source": [
|
||
"np.random.seed(13) # pick the seed for reproducability - change it to explore the effects of random variations\r\n",
|
||
"\r\n",
|
||
"train_x = np.linspace(0, 3, 120)\r\n",
|
||
"train_labels = 2 * train_x + 0.9 + np.random.randn(*train_x.shape) * 0.5\r\n",
|
||
"\r\n",
|
||
"plt.scatter(train_x,train_labels)"
|
||
],
|
||
"execution_count": 11,
|
||
"outputs": [
|
||
{
|
||
"output_type": "execute_result",
|
||
"data": {
|
||
"text/plain": [
|
||
"<matplotlib.collections.PathCollection at 0x7f4cc4d54e90>"
|
||
]
|
||
},
|
||
"metadata": {
|
||
"tags": []
|
||
},
|
||
"execution_count": 11
|
||
},
|
||
{
|
||
"output_type": "display_data",
|
||
"data": {
|
||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAb2ElEQVR4nO3df4xlZXkH8O+zwwUuP8pVmVoYXZemzZIKZZedEJo1RtZYbFHcgFEb0xbbZpM2bYVQ2tE/ivYfJjFWbW1qJpQWW6pLXdhQUJFkN6FuFDvDLgKyNChuZbBllB0QZsrODE//uPcsd86c97zvuec957zvvd9PsnF25+yZ9+yV5773eZ/3eUVVQURE4drU9ACIiCgfAzURUeAYqImIAsdATUQUOAZqIqLAnVLFTc8991zdsmVLFbcmIhpKc3NzP1HV8azvVRKot2zZgtnZ2SpuTUQ0lETkmOl7TH0QEQWOgZqIKHAM1EREgWOgJiIKHAM1EVHgKqn6ICIaJfsPz+NT9z+JZxeXcX6njZuu3Ird2ye83Z+BmoiohP2H5/Gxux7F8soaAGB+cRkfu+tRAPAWrJn6ICIq4VP3P3kySCeWV9bwqfuf9PYzGKiJiEp4dnG50J8PgoGaiKiE8zvtQn8+CAZqIiJ0c807pw/ggqn7sHP6APYfnnf6ezdduRXt1ti6P2u3xnDTlVu9jY2LiUQ08sosCCbfZ9UHEVGF8hYEXQLu7u0TXgNzGlMfRDTy6lgQLMMaqEVkq4gc6fv1oohcX8fgiIjqUMeCYBnWQK2qT6rqNlXdBmAHgCUAd1c+MiKimtSxIFhG0Rz1OwF8X1WNDa6JiGJTZEGwf7v4Oe0WRIDFpZVKFhEToqruF4vcBuBhVf18xvf2ANgDAJs3b95x7BhjORENl3R1SFq7NYZbrrl4oGAtInOqOpn1PefFRBE5FcDVAP4t6/uqOqOqk6o6OT6eeewXEVHw8uqps6pD+vneOp4okvr4DXRn0//rfRRERAGw1VO7VIFUUSlSpDzvtwB8yfsIiIgCYWuw5FIFUkWliFOgFpEzAbwLwF3eR0BEFAhbPXVWdUi/qipFnFIfqvoygDd4/+lERAE5v9PGfEawTmbJ6eqQuqo+uIWciKjnpiu3bqjqSM+Sq94unoWBmoiop44GS4NgoCYi6tPEjNmGTZmIiALHQE1EFDgGaiKiwDFHTUQjob+ZUiiLhK4YqIloKOQF4kGP2kruOb+4jDERrKliooEgz0BNRIWEODPNC8QAcOOdj2At1SnUdtRW+p7J3y9ynqIvhdqcupqcnNTZ2Vnv9yWiZmW1+SzT2tOXndMHMncUdtotvLL6am7Hu4lOO/NNx3TP/r93aGpX+cH3eGlzSkRka1rUFFOPjsXlldwgLejOkBWvzZSTtqa2Lnh1nqfIQE1EzkI9BHaQjnUCIJ1PKNIpr87zFBmoiciZKTid024Zm+3XwXTm4evOaGVePyayIUgnXDrl1X2eIgM1ETnLCl6tTYKXT6waUwhVSJ/CAgC3XHMxJjptCLr541uuuRg3v/etmQH80x+4BBOWk8d3b584eU+gG9zRd29WfRBRkLKaFi2dWMXxpZV119kqKsowVXjccs3FxsU9U5VKiJ3ysrDqg4hKuWDqvsw0ggB4evoq7z/PVI0xSBVGSKWGeVUfnFETUSm2Zvu++VzQDGXGbMMcNRGVYlrIq2qxzfQGUGcVRt0YqImolP5Ft/6FvKpmqnW/MYSAqQ8iKq3OFEKop7BUySlQi0gHwK0ALkK3Rvz3VPVbVQ6MiMgkltyyL64z6s8B+Lqqvl9ETgVwRoVjIiLKFFKVRp2sgVpEzgHwdgDXAYCqngBwotphERGtN2ir0mHgsph4AYAFAP8oIodF5FYROTN9kYjsEZFZEZldWFjwPlAialZ6N2Dd28RDbQhVB5dAfQqASwH8vapuB/AygKn0Rao6o6qTqjo5Pj7ueZhE1KRkNlvnNvH+n53XcrTphlB1cMlRPwPgGVV9qPf7ryAjUBPR8DLNZj9xz+O5p6qUzSdn9b9OG+b66YQ1UKvq/4jIj0Rkq6o+CeCdAL5X/dCIKBR5/Z4Xl7t9PtKnqvjIJ2e9QfQb9vrphGvVx58AuKNX8fEDAB+pbkhEFBrTNvG0/pyxKZ9cJFDnpTWaOLuwKU6BWlWPAMhsFkJEw++mK7daUxCJvOBaNJ9seoPwfQxW6LiFnIissraJm5ryn99pO/XjcKkiGcXt4lm4hZyInKR3A5oOuk2CaN73XGqik8XI5ZU1jIlgTXWk0h39GKiJRliZygyXnhum7+XVRO/ePrEhkK+pngz0oxakAR4cQDSyTDPiOo6ZMh02AHTTKqaFy6zc9LBsK887OIA5aqIR1eROP1MOW4Dc6pL0YmSTG3HqxEBNNKJ8npRSVNYioQDGWXYiHeBHZVs5AzXRkDNVV5hmtQpU3ssjXUXSabesQTqr2qPJN5s6cTGRaIjlVVfk1Ub77kxnyiP3LxzmMVV71H1eY1MYqImGUBIYs4JYkhpIFuVs15UN1LZSvLxt4rbFzaw3m3ZrDFdcOI6d0weiX2BMMPVB5EHTLUDTY0kW2EyS1MDu7RM4NLULYrmuDFseOe9n2CpQsjbiXLtjAvvm5odqgZEzaqKSQmtob2tkBGxMDfhMIaTTHLb2pHnbxF3+/dIbcXZOH/DSZyQknFETldRE5UHeDN42C85alPO1VTurXM40W0/eBHxvE3dZYAzpE5ALzqiJSqq78sA2g8+bxY6JrHsTSWaYvk72znrTUmSX3i2dWMX+w/PeTxW3fToI7ROQCwZqopLqrjywbb/OWmBrbRJAgJW1brjMCk4+TvY2vTkpuiV4Se9qADi+tLJuDL6CpGmBMZmh2/79QsTUB1FJdXd4s83gsxbYzjr9lJNBOmFKz5RJC5jenCY6bZx52sZ5YRUpoqzn71+UjLH2mjNqopJ8f3S3cZnBp2eoF0zdl3kv05bsQdMCebPZG/YecRqDD3kz9BhrrzmjJvIgKXN7evoqHJraVelH6EFm8C79oYHyC6N5s1nXMVQtxh7XnFETRWaQGbwtb5vwkRYwzWZdx1C1uj8B+cBATRShootvrsGpyrRASAHS5+JlHdiPmihQTfRZbrJH9ajL60fNGTVRgJqq9Q1p1kuvcQrUIvJDAD8DsAZg1RT1iciPJmt9Y0sLjIIiM+orVPUnlY2EiE6KsdaXqsPUB1EA0vnozhktHF9a2XBdyLW+VB3XOmoF8A0RmRORPVkXiMgeEZkVkdmFhQV/IyQaclmNjF76v1W0xta3Mwq91peq4zqjfpuqzovIzwN4QESOquqD/Reo6gyAGaBb9eF5nERDIz17XjqxuiEfvfKqotNu4czTTuGiHrkFalWd7/3vcyJyN4DLADyY/7eIKC2rmsPkheUVHLn51+saGgXMmvoQkTNF5OzkawC/DuCxqgdGNIxcmvonmI+mhMuM+o0A7haR5Pp/VdWvVzoqoiHlWrXBfDT1swZqVf0BgEtqGAvR0DNt0WY+mvKwPI+i1MT2ah9MjYk+cfVbjePvf9Zz2i2IAItLK1E9N5XDQE3RqXN7te0NoegbRtEt2uln7T8hJeu5Y30Do3xsykTR2Tl9wHhq9aGpXd5+jq1BUR0NjEzP2i95bjZUilteUyYeHEDRqWt7ta2Jfh2nj7s8U3JNE6ehUz0YqCk6dZ0UYntDqOMNw+WZkmvYH2R4MVBTdOo6Ssn2hlDHG0bWs/brf+5Qjroi/xioKTq2U6Z9sb0h1PGGkX7WTruF153RynzuKsZT5kRy8oeLiUQ5fFd9ND3eovfi4mR98hYTGaiJIlfVm0Vd1TXUxaO4iCJRNOhWWVPOxclwMEdNlNJUXjarL/XH7no09+dXWZLHxclwMFBTVKoOooMES9f72sY9SNCtctZbV3UN2TFQUzSqCqL9qpihuo57kKBb5ay3ruoasmOOmqJRx8ncVcxQXcdt6qyXF3RNTZ58zXp5InkYGKgpGnXtBCwaLG1M45tfXMbO6QMnFw6vuHAc++bmCwXdok2eKE4M1BSNKoJoWhUzVNO4Ba8dxTW/uIx9c/O4dscEDh5dKBR0OesdfgzUFI2qP+YD+TPUQeuVs8YtANI7GJZX1nDw6EItNcqhbdShfNzwQlFpKsBk7dIDgNed0cLN781u+p/X8N/UulQAPD19VRWPsG5c3HEYHm54oaFh+5hfVSA3HUp7fGklc4NJVsP/dmsMn/ngNuzePmHc9VdHjXIdi7LkF8vzyIsQmvdUWb6Xt2CZVb5nK/NrskaZOw7j4xyoRWRMRA6LyL1VDojiU0d9swtTcLzxzkdKj8U2000HOVswbLJGmTsO41Mk9fFRAE8A+LmKxkKRCuWjtCk4rqmW7n+RtSDYrz/I7T88j00iWMtY/+m/rqlqjToWZckvpxm1iLwJwFUAbq12OBSjUD5K580Iy+4uTGbAnXZrw/f6g1zy6SIrSIcSDLnjMD6uM+rPAvhzAGebLhCRPQD2AMDmzZvLj4yiUUd9swvbrLfsG0cyA85bsDQtOo6JBBUMWXsdF2ugFpH3AHhOVedE5B2m61R1BsAM0C3P8zZCCl4oH6WTwHPjnY9Y0w5lf44pyJneDF5VZWCkgbmkPnYCuFpEfgjgywB2ici/VDoqikpIH6V3b5/Apz9wSWMVFVyooyoU2vDSm1H/maq+J+86bnihpoW0MYabScgFN7zQyGkqB8smSVQFbiGnxjTZb4K9Lig0nFFTcHye9efrnMHZY88X7lxHVAfOqKkRpl4XQHcx0jVIFskJJwE9ryFS/38NzC1TnfJm1Oz1QY3Iq2kusgXd5eis/Yfnse2T38D1e48YgzSQ3XbUxyGxRGUxUFMjbOVqrkHStisymXEvLq8UH2TO/YnqxEBNznx2yMvqHpfmEiRtdcumnYJpUvD+RHVioCYnvjvk9W+SMXEJkrZ2oS7BfqLTxocv39zYJhkiG1Z9kJMqOuT1984YdAu6rW457zSV9GLh5Ftez5I9ChIDNTmpskNe2U0ieZtbTI2aso7QYqMiChUD9RCoY/NG1R3yqgqS3ClIw4CBOnI+N47kCaVD3iA4U6bYcTExci51xD6E1CGPaNRwRh25Ok9XqWNm2p/GOafdgkj3pO+x3tFWRXYtEg0LBurIhXK6io1LHj2dxunfpJIcBFBVaocoZEx9RM5WRxwC1xps180p3NpNo4aBOnIx5I5NefTr9x5Zt8OxSLqGW7tplDD1MQRCr2pwacAE5G9OSQsttUNUJQZq8i6dj+6c0cLxJXNTpCSVYTtFPBFaaoeoagzU5FVWXXdrk6A1JlhZM/c+f3ZxecPmFFZ9EHUxUJNXWfnolVcVnXYLZ552ijG1kaQy6k7j8EguigEXE8krUz76heUVHJrahc9+cFswVSq+OwISVcUaqEXkdBH5jog8IiKPi8gn6xgYvcZnH+iq2fpDh1SlUteuTqKyXFIfrwDYpaoviUgLwDdF5Guq+u2Kx0aor5eHLy49QUKpUqlzVydRGdZArd3Tb1/q/bbV++X/RFzK5LMPdB352Ji61cWyq5PI6RRyERkDMAfglwD8nar+RcY1ewDsAYDNmzfvOHbsmOehjqYLpu7LfFcUAE9PX+V8n6zm/Mmp21mVFKOwyFbkBHOiquWdQu5U9aGqawC2iUgHwN0icpGqPpa6ZgbADABMTk5yxu2Jr1lf1sw8eZHS6RRbuiWrcdLi0kp0AT2m2T+NtkLleaq6KCIHAbwbwGO264dVnbNNX32gbXnX/nSKbZHN1Dgp9Px5llDy5UR5XKo+xnszaYhIG8C7ABytemChqruky1eVhMsMPAnmeYtstsZJrJog8s9lRn0egNt7eepNAO5U1XurHVa4qjjk1cbHrM9le3YSzE3plk0iTr04WDVB5JdL1cd3AWyvYSxRiLGkK0nVLK+sndyKnSwkJvrTKaagnvX3siQBfxQWJInqwJ2JBdk2dISmP1UDdINtuzWGD1++2ZhOSdItYyIb7qfoVouYJAGfu/6I/GGvj4JCOuTVZcZqStUcPLqAQ1O7jPfevX0CN+w9kvm9pKQvr+pj5/SB2lNERMOKgbqgUEq6XHcslknVmHLVE512bpAv+3OJaD0G6gGEUNLluqhZpg67zKcH7voj8oc56opV1VDJdcZa5kzFMqWBMZzlSBQLzqgrVGVDJdcZa9lUzaCfHkJJERENA6deH0VNTk7q7Oys9/vGZuf0gcxgOiaCT3/gklJBi30qiIZL6V4fNBhTemJNtfTMmjNWotHBQF2hvFO1fZSq+V7U5AYVojBxMbFCWQtq/UIqVeMGFaJwMVBXKG+HHxBWqRqPpSIKF1MfFUtSB3XvZiyaxuAGFaJwMVDXwGXhz2d+eJCyQG5QIQoXA3VN8hb+fNdbD9KKNaQeJkS0HnPUAfCZH95/eN5YaZKXxvB1QAER+ccZdQB85YeTmbmJLY0RQg8TItqIM+oA+OpxnXdMFtMYRPHijLokH4uApvzwFReOY+f0gdKVGwCYxiCKGAN1Cb4WAbOqQq64cBz75ua9VG5MdNoM0kQRG9pAXXamW+b0FNet4Xk/Y5ATUli5QTScrIFaRN4M4IsA3ojuKUwzqvq5qgdWRtmZbh2np9h+xiD3HqRRE/t7EIXPZUa9CuBGVX1YRM4GMCciD6jq9yoe28DKznTrOD3F9jMGvXeRyo0q+2UTkT/Wqg9V/bGqPtz7+mcAngAQ9H/FZcvd6jg9xfYz6jghhf09iOJQqDxPRLYA2A7goYzv7RGRWRGZXVhY8DO6AZUtd3P9+2U2idh+Rh0bUNjfgygOzouJInIWgH0ArlfVF9PfV9UZADNA94QXbyN0kM6zpismgGKz0SKLcoNuEnH5GVVvQGF/D6I4OM2oRaSFbpC+Q1XvqnZIxWT1Ud43N49rd0wMPBsdZDZb9BDbELZs8wBaojhYz0wUEQFwO4DnVfV6l5vWeWai6VzCiU4bh6Z2lb6/S1VEzOcXsuqDKAx5Zya6BOq3AfgPAI8CeLX3xx9X1a+a/k6VgTodWEwNiATA09NXFb5ff6DKCsCCbo3iRN+1Vb9ZENHwK3W4rap+E9341LiscrIkcKa55Flt5WlZVRHJz+q/lotyRFSlqJoymQJn+l3ENc9qK0+zBdrkWl9NlYiIskQVqE2BM0lFCIBOu4XTW5tww94j1kU920zYJdA+u7jMRTkiqlRUgdoUOJNc8Gc+uA2vrL6K40srTidp22bCtlPEk2tDqOAgouEVVVMmW+1x0a3jtvv1987Iyoenr2VgJqIqRBWobU2Hii7quTQx6g/AIZSyhTAGIqqXtTxvEHXWUfcb9jK5mOu1iShfXnleVDlqm1gW9YruYkywiRLRaIoq9WEzSD/mupVpLcp6baLRFHWgNuVrBw3MvvK/efcp0yubTZSIRlO0gdp303tf9yt7cktekOdRW0SjKdocte98ra/72e6TV7ud1Qmwvw6c9dpEoym4GbVr+sF3vtbX/VxObjHNil3SIqzXJho9QQVqU9pg9tjzOHh0YV3w9p2v9XU/233yFjxv2Hsk855cLCQabUEFatOM8o5v//eGrnXX7pgodYpLmq/8b5mTW7hYSERZgspR5zVd6re8soaDRxe85mt95X/L3CeWOnAiqldQOxNNOwuzuB4MEBtuEScaTaUODqhTVtqgzMEAMeJiIRGlBZX6yEobfPjyzUwHENFIC2pGDWTPKCff8nqmA4hoZAUXqLMwHUBEo8waqEXkNgDvAfCcql5U/ZCqw4U6IoqRy4z6nwB8HsAXqx3KRj4Dq+/eIEREdbEuJqrqgwCer2Es69j6XhTFXs5EFKugqj76+Q6s7OVMRLHyFqhFZI+IzIrI7MLCQun7+Q6sthPHiYhC5S1Qq+qMqk6q6uT4+Hjp+/kOrNyeTUSxCjb14TOwJouSyytrGBMBAHTaLZze2oQb9h4pdG4hEVHdrIFaRL4E4FsAtorIMyLy+9UPy1+TpP5FSQBYU0Vrk+DlE6s4vrTiZaGSiKhKQTVlqkKRRk8TnTYOTe2qeERERBvlNWUKNvXhS5HFR1aAEFGIhj5QF1l8ZAUIEYVo6AN11qJka5OgNSbr/owVIEQUqiiaMpVhOqMw68+4lZyIQjT0i4lERDGI4oQXdrYjIsoWRKBmZzsiIrMgFhPZ2Y6IyCyIQM3OdkREZkEEana2IyIyCyJQs7MdEZFZEIuJplpnLiQSEQUSqAGeNE5EZBJE6oOIiMwYqImIAsdATUQUOAZqIqLAMVATEQWuku55IrIA4NiAf/1cAD/xOJwmDcuzDMtzAHyWEA3LcwDlnuUtqjqe9Y1KAnUZIjJravUXm2F5lmF5DoDPEqJheQ6gumdh6oOIKHAM1EREgQsxUM80PQCPhuVZhuU5AD5LiIblOYCKniW4HDUREa0X4oyaiIj6MFATEQWusUAtIu8WkSdF5CkRmcr4/mkisrf3/YdEZEv9o7RzeI7rRGRBRI70fv1BE+O0EZHbROQ5EXnM8H0Rkb/pPed3ReTSusfoyuFZ3iEiL/S9Jn9Z9xhdicibReSgiHxPRB4XkY9mXBP8a+P4HFG8LiJyuoh8R0Qe6T3LJzOu8Ru/VLX2XwDGAHwfwC8COBXAIwB+JXXNHwH4Qu/rDwHY28RYPTzHdQA+3/RYHZ7l7QAuBfCY4fu/CeBrAATA5QAeanrMJZ7lHQDubXqcjs9yHoBLe1+fDeC/Mv4/Fvxr4/gcUbwuvX/ns3pftwA8BODy1DVe41dTM+rLADylqj9Q1RMAvgzgfalr3gfg9t7XXwHwThGRGsfowuU5oqCqDwJ4PueS9wH4onZ9G0BHRM6rZ3TFODxLNFT1x6r6cO/rnwF4AkC6cXvwr43jc0Sh9+/8Uu+3rd6vdFWG1/jVVKCeAPCjvt8/g40v2slrVHUVwAsA3lDL6Ny5PAcAXNv7SPoVEXlzPUPzzvVZY/FrvY+uXxORtzY9GBe9j8/b0Z3B9Yvqtcl5DiCS10VExkTkCIDnADygqsbXxEf84mJi9f4dwBZV/VUAD+C1d1lqzsPo9lW4BMDfAtjf8HisROQsAPsAXK+qLzY9nkFZniOa10VV11R1G4A3AbhMRC6q8uc1FajnAfTPLN/U+7PMa0TkFADnAPhpLaNzZ30OVf2pqr7S++2tAHbUNDbfXF6zKKjqi8lHV1X9KoCWiJzb8LCMRKSFbnC7Q1XvyrgkitfG9hyxvS4AoKqLAA4CeHfqW17jV1OB+j8B/LKIXCAip6KbbL8ndc09AH639/X7ARzQXmY+INbnSOUKr0Y3NxejewD8Tq/C4HIAL6jqj5se1CBE5BeSfKGIXIbufwehTQIAdCs6APwDgCdU9a8NlwX/2rg8Ryyvi4iMi0in93UbwLsAHE1d5jV+NXK4raquisgfA7gf3cqJ21T1cRH5KwCzqnoPui/qP4vIU+guDH2oibHmcXyOPxWRqwGsovsc1zU24Bwi8iV0V93PFZFnANyM7iIJVPULAL6KbnXBUwCWAHykmZHaOTzL+wH8oYisAlgG8KEAJwGJnQB+G8CjvZwoAHwcwGYgqtfG5TlieV3OA3C7iIyh+2Zyp6reW2X84hZyIqLAcTGRiChwDNRERIFjoCYiChwDNRFR4BioiYgCx0BNRBQ4BmoiosD9P3fFq1WlCzEwAAAAAElFTkSuQmCC\n",
|
||
"text/plain": [
|
||
"<Figure size 432x288 with 1 Axes>"
|
||
]
|
||
},
|
||
"metadata": {
|
||
"tags": [],
|
||
"needs_background": "light"
|
||
}
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "Ng4rZmGc6oxk"
|
||
},
|
||
"source": [
|
||
"Линейная регрессия вычисляется как $f_{W,b}(x) = Wx+b$, где $W, b$ - параметры модели, которые необходимо найти. Функция ошибки на наборе данных $\\{x_i,y_u\\}_{i=1}^N$ может быть определена как среднеевадратичное отклонение\r\n",
|
||
"$$\r\n",
|
||
"\\mathcal{L}(W,b) = {1\\over N}\\sum_{i=1}^N (f_{W,b}(x_i)-y_i)^2\r\n",
|
||
"$$\r\n",
|
||
"\r\n",
|
||
"Опишем модель и функцию ошибки:"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"id": "QxhI4GlB6aiH"
|
||
},
|
||
"source": [
|
||
"input_dim = 1\r\n",
|
||
"output_dim = 1\r\n",
|
||
"learning_rate = 0.1\r\n",
|
||
"\r\n",
|
||
"# This is our weight matrix\r\n",
|
||
"w = tf.Variable([[100.0]])\r\n",
|
||
"# This is our bias vector\r\n",
|
||
"b = tf.Variable(tf.zeros(shape=(output_dim,)))\r\n",
|
||
"\r\n",
|
||
"def f(x):\r\n",
|
||
" return tf.matmul(x,w) + b\r\n",
|
||
"\r\n",
|
||
"def compute_loss(labels, predictions):\r\n",
|
||
" return tf.reduce_mean(tf.square(labels - predictions))"
|
||
],
|
||
"execution_count": 14,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "JUxwj3367gD2"
|
||
},
|
||
"source": [
|
||
"Обучать модель будем на сериях примеров - minibatches. Для обучения используем градиентный спуск, подстраивая парметры в соответствии с формулой:\r\n",
|
||
"$$\r\n",
|
||
"\\begin{array}{l}\r\n",
|
||
"W^{(n+1)}=W^{(n)}-\\eta\\frac{\\partial\\mathcal{L}}{\\partial W} \\\\\r\n",
|
||
"b^{(n+1)}=b^{(n)}-\\eta\\frac{\\partial\\mathcal{L}}{\\partial b} \\\\\r\n",
|
||
"\\end{array}\r\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"id": "-991PErM7fJU"
|
||
},
|
||
"source": [
|
||
"def train_on_batch(x, y):\r\n",
|
||
" with tf.GradientTape() as tape:\r\n",
|
||
" predictions = f(x)\r\n",
|
||
" loss = compute_loss(y, predictions)\r\n",
|
||
" # Note that `tape.gradient` works with a list as well (w, b).\r\n",
|
||
" dloss_dw, dloss_db = tape.gradient(loss, [w, b])\r\n",
|
||
" w.assign_sub(learning_rate * dloss_dw)\r\n",
|
||
" b.assign_sub(learning_rate * dloss_db)\r\n",
|
||
" return loss"
|
||
],
|
||
"execution_count": 13,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "idr2VEWb9rr0"
|
||
},
|
||
"source": [
|
||
"Теперь приступаем к обучению: делаем несколько проходов по всему датасету (эпох), разбиваем его на minibatches, и вызываем функцию обучения:"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"id": "nOuu0qpx-wAp"
|
||
},
|
||
"source": [
|
||
"# Shuffle the data.\r\n",
|
||
"indices = np.random.permutation(len(train_x))\r\n",
|
||
"features = tf.constant(train_x[indices],dtype=tf.float32)\r\n",
|
||
"labels = tf.constant(train_labels[indices],dtype=tf.float32)"
|
||
],
|
||
"execution_count": 12,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/"
|
||
},
|
||
"id": "3zdIf6c_85Ht",
|
||
"outputId": "43b04684-8b90-4c65-d5ff-20ebac61c73c"
|
||
},
|
||
"source": [
|
||
"batch_size = 4\r\n",
|
||
"for epoch in range(10):\r\n",
|
||
" for i in range(0,len(features),batch_size):\r\n",
|
||
" loss = train_on_batch(tf.reshape(features[i:i+batch_size],(-1,1)),tf.reshape(labels[i:i+batch_size],(-1,1)))\r\n",
|
||
" print('Epoch %d: last batch loss = %.4f' % (epoch, float(loss)))"
|
||
],
|
||
"execution_count": 15,
|
||
"outputs": [
|
||
{
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Epoch 0: last batch loss = 94.5247\n",
|
||
"Epoch 1: last batch loss = 9.3428\n",
|
||
"Epoch 2: last batch loss = 1.4166\n",
|
||
"Epoch 3: last batch loss = 0.5224\n",
|
||
"Epoch 4: last batch loss = 0.3807\n",
|
||
"Epoch 5: last batch loss = 0.3495\n",
|
||
"Epoch 6: last batch loss = 0.3413\n",
|
||
"Epoch 7: last batch loss = 0.3390\n",
|
||
"Epoch 8: last batch loss = 0.3384\n",
|
||
"Epoch 9: last batch loss = 0.3382\n"
|
||
],
|
||
"name": "stdout"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/"
|
||
},
|
||
"id": "US6q0nCBD-LL",
|
||
"outputId": "65a79620-a3eb-445b-aafb-60a60575ab0e"
|
||
},
|
||
"source": [
|
||
"w,b"
|
||
],
|
||
"execution_count": 16,
|
||
"outputs": [
|
||
{
|
||
"output_type": "execute_result",
|
||
"data": {
|
||
"text/plain": [
|
||
"(<tf.Variable 'Variable:0' shape=(1, 1) dtype=float32, numpy=array([[1.8616778]], dtype=float32)>,\n",
|
||
" <tf.Variable 'Variable:0' shape=(1,) dtype=float32, numpy=array([1.0710956], dtype=float32)>)"
|
||
]
|
||
},
|
||
"metadata": {
|
||
"tags": []
|
||
},
|
||
"execution_count": 16
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/",
|
||
"height": 282
|
||
},
|
||
"id": "_e6xRMZFDnyI",
|
||
"outputId": "d202b7fe-4383-4d82-b98e-a20f3180093e"
|
||
},
|
||
"source": [
|
||
"plt.scatter(train_x,train_labels)\r\n",
|
||
"x = np.array([min(train_x),max(train_x)])\r\n",
|
||
"y = w.numpy()[0,0]*x+b.numpy()[0]\r\n",
|
||
"plt.plot(x,y)"
|
||
],
|
||
"execution_count": 17,
|
||
"outputs": [
|
||
{
|
||
"output_type": "execute_result",
|
||
"data": {
|
||
"text/plain": [
|
||
"[<matplotlib.lines.Line2D at 0x7f4cc487efd0>]"
|
||
]
|
||
},
|
||
"metadata": {
|
||
"tags": []
|
||
},
|
||
"execution_count": 17
|
||
},
|
||
{
|
||
"output_type": "display_data",
|
||
"data": {
|
||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZhU1Z3/8ffp7uqFrRulQWhoQFZFZOsxGowKiBhN3KNJRo3JTJiJGbcYlGQWZ/L8ZgTcMyYTiZplYhKNInFiZFFQXFFWUeluEGRp9qWbpffq8/uju9qmum7Vra5b1beaz+t5eNJ03br3XMp876lzvud7jLUWERHxr4zOboCIiESnQC0i4nMK1CIiPqdALSLicwrUIiI+l5WMk/bp08cOGTIkGacWEemSVq9efcBaWxjptaQE6iFDhrBq1apknFpEpEsyxmxzek1DHyIiPqdALSLicwrUIiI+p0AtIuJzCtQiIj6XlKwPEZGTycK1FTywuIxdlTUMKMhj1oxRXDWhyLPzK1CLiCRg4doKfrRgAzUNQQAqKmv40YINAJ4Faw19iIgk4IHFZa1BOqSmIcgDi8s8u4YCtYhIAnZV1sT1+45QoBYRScCAgry4ft8RCtQiIjSPNU+es4yhs19m8pxlLFxb4ep9s2aMIi+QecLv8gKZzJoxyrO2aTJRRE56iUwIhl5X1oeISBJFmxB0E3CvmlDkaWAOp6EPETnppWJCMBExA7UxZpQxZl2bP0eMMXemonEiIqmQignBRMQM1NbaMmvteGvteGASUA28mPSWiYikSComBBMR7xj1NOBTa61jgWsRkXQTz4Rg2+Xi+XkBjIHK6oakTCKGGGut+4ONeRpYY619PMJrM4GZAMXFxZO2bVMsF5GuJTw7JFxeIJP7rxnboWBtjFltrS2J9JrryURjTDZwBfCnSK9ba+dba0ustSWFhRG3/RIR8b1o+dSRskPa8nrpeEg8Qx9fprk3vdfzVoiI+ECsfGo3WSDJyBSJJz3vG8AfPG+BiIhPxCqw5CYLJBmZIq4CtTGmOzAdWOB5C0REfCJWPnWk7JC2kpUp4mrow1p7HDjV86uLiPjIgII8KiIE61AvOTw7JFVZH1pCLiLSYtaMUe2yOsJ7ycleLh6JArWISItUFFjqCAVqEZE2OqPHHIuKMomIeOCjiiqeWZmchX7qUYuIJOCzA8d5cEkZf/lwN3175nDNhIHkZTtnhnSEArWISAfsO1rLf7+2mT+8v51AZga3TR3Ody843fMgDQrUInKSaFtMKZFJwqO1DcxfsYWn3tpKfWMTXz9nELdPG0HfnrlJaHUzBWoR6RKiBeKObrUVOmdFZQ0ZQBOQYaDJwlfO7s8PLxnFkD7dk31rCtQiEh+veqZet8kpEAPc/dx6gmGVQmNttRV+zqaW3zdZyMnK4OIz+qUkSEOcZU7dKikpsatWrfL8vCLSuSKV+UyktKdXJs9ZFnFFYUFegLrGpqgV74oK8iI+dL54/2vsqqqN+r63Z09NvPEtopU5VY9aRFxLdBPYZHGq0VFZ0xD1fQZaA3zbXnjxqd2iBulo10wGBWoRcc2vm8A61eiIxgDh4wk1DUFmL/iQ2oam1rHoaNdMFS14ERHXnIJTfl7Asdh+Kjjtedi7WyDi8ZnGtAvSIbUNTdw9fST3Xz3WsVJeqvdTVKAWEdciBcRAhuF4fSMVlTVYPh9CSGawDt+FBeD+a8ZSVJCHoXn8+P5rxnLfV8dEDOAPXT+OIoeHzmm9crlt2ghuOKe49ZzQHNxpc+5UDvVo6ENEXItUtKi6vpHD1SeOBSdz3Nopw+P+a8Y6Tu6FZ6nMGHMar27c2264JC+Qyewvj279u1/qfijrQ0QSMnT2yxGHEQywdc7lnl/PKcPDTRZGY7CJP63eyaOvlrP3SB1jBvRi/9E69h+t6/RUQ2V9iEjSxCq277WOTGhaa1n88R7mLS5jy/7jTCwu4L+/MZFzhp6SlDZ6TYFaRBLipti+l+J9MLz76UHmLipl3Y5KRvTtwfybJjH9zH6YljHndKBALSIJSXWxfbcPhk92HWHuolLeKN9P//xc5l13NtdOHEhmRvoE6BAFahFJWCon3WI9GHYcquahJWX8ef0ueuUG+PFlo7n5vCHkRtmU1u9cBWpjTAHwJHAWzTni37HWvpvMhomIOIn0YDhwrI7Hl23mmZXbyMww/OOFw/jHC4eRnxc5lzqduO1RPwYsstZeZ4zJBrolsU0iIhFFKgh18Zn9ePLNLfxyxRZqG5u4vmQQd148gn69kld2NNViBmpjTD5wAXALgLW2HqhPbrNERE4UKX961p/WkxPI5FhdI18+6zR+OGMUwwp7dHJLveemRz0U2A/8yhgzDlgN3GGtPd72IGPMTGAmQHFxsdftFJFO1tnlTSMVhGposphgEwu/P5nxgwpS1pZUc7OEPAuYCPyPtXYCcByYHX6QtXa+tbbEWltSWFjocTNFpDOFerOpXCbe9tpOi1wA6hubunSQBnc96p3ATmvtypa/P0+EQC0iXZdTedN/f+njqLuqJNoDj1T/OpxTzY6uJGagttbuMcbsMMaMstaWAdOAT5LfNBHxi2j1nkM1n8N3VenI1lfh/uuvG6MG6VRXsessbrM+bgOeacn42AJ8O3lNEhG/cVvvOVSMKfRzpNfcBOq9R2p59NVN7Dta53hMkU+2AUsFV4HaWrsOiFgsRES6vkirAZ1Eq7kRa4OBqpoGnnjjU55+eyvBJkv37EyO17e/ptfbYPmdViaKSExuy5vC5zU3YtXjaDuG3T8/l5Ihp/BG+X6O1DZw5bgB/GD6KNZsP5zSOiJ+pUAtIq6ErwZ02ug2FESjvRb+3l1Vtby0fhejT+vJ77/7BcYMyG8N5DUNQTKNIWjtSTXc0ZYCtchJLJHMDDfFmJxem7eoNOIwytHaxtYg3TaQB61tDfQnW5AGbRwgctJy6hEne5up97ce4vonnEsFFUWZuIw0Nt3ZC3G8oo0DRKQdp9zoZG2hVbrnCA8sKuO10n2OO3wbIo9th4RPRjptywXxpQH6nQK1yEmqIzuldMTOw9U8vLScF9dW0CMni3svHc2p3bO576WPT3hQGHDcGTwkfHOAVD9sOosCtUgX5zQ04JQbbWnelzDRIYRDx+v52fLN/O+728DAzC+dzvcuGkZBt2wAsrMyWtuVnxdoXTjjJFK2R6oeNp1NgVqkC4s2NBAtNzqRIYTq+kaeenMr81ds4Xh9I9dNGsgZ/XvxZMvv2j4srppQ1NrGaJyyPVK9X2NnUaAW6YJCvehIQSw0NBCalIt1nNtA3RBs4o8f7OCnr21i/9E6LjmzH7NmjOLjXUeijiNHGr4IiTW56bQt15TRhUyesyztJxhDFKhFPOCnzAM3hYxCQwOhXu3Q2S9HHB92M4TQ1GR5ecNuHlpSxmcHqzlnyCn84sZJTBrcG4BbfvVB1HHkaNeIlYESKUVwyuhCXlhd0aUmGBWoRRLkt8yDaD3UkPChgY4OIby16QBzF5WyoaKK0af15OlbSqiqbuD2P6xtDZxOWRyhAO10TFFBnqt/v/CFOJPnLOtyE4xu6lGLSBTRMg+SJVSjeejsl5k8Z9kJdaFj9YIjTcrNmjGKvLDNX6Mt1d6ws4obn1zJjU+t5NDxeh6+fhwv3/4ljtQ08uMXPzqhbrXTnt+hh0C8147FzQRjtH8/P1KPWiRBqc48iNWDj9aLzTTmhIdIqIfpZpUhwGcHjvPgkjL+8uFuencL8K9fOZMbzy0mJyuz9f3hDy1L5NS76vpGFq6tcH1tt2J9O/DbNyA3FKhFEpTqzINYucORJtgCGQYMNASbw2Wk4BRpZ++QfUdr+elrm/jj+zsIZGZw29ThfPeC0+mVe+IO304PJwsUhKXgHa5uOKENXgVJpwnGUA89HXOvFahFEhQrMHgtVg/ebaU7p+DUdmL0tPxcxhbl8+amAzQEm/jGOcXcNm04fXtG3uE72ngz0C5XOhkBMlYPPR1zrxWoRRLk9Vf3WNz04MN7qENnvxzxXLGWZO+uqmV3VS0TBhXwyA3jGdKne9S2RXto3fXsOldt8EK0Hno65l5rMlHEA1dNKOLt2VPZOudy3p49NalfoTsy+eYUhMJ/71TVbt/RuphBGpr/He6/ZixFBXkYmnvSoRQ7t21INq8nL1NBPWqRNNORHnys4RlrLcvL9rGrqjbi++Pp9Tr1ZlM9ROQk1d+AvKBALZKG4p18ixacVm87zNxXSnn/s0NkZhiCEcraedHr9VOA9HLyMhVUj1rEp5K92nHT3qPMW1zG0k/2UtgzhzumjSAvkMm/LPwo5TWqRfWoRdJOMnN9d1XW8Oir5Ty/eifds7P44SUj+c75Q+mW3RwOMjOML3q98jlXgdoY8xlwFAgCjU5RX0S8kYxc38rqen7++qf8+p3PwMJ3Jg/l1inDOaV79gnHpduwwMkgnh71FGvtgaS1RERaeZnrW1Mf5FfvbOV/Xv+UY3WNXDNhIHdNH8HA3t0SbaakiIY+RHwgfDy6oFug3QIViG9SrzHYxHOrdvLYa+XsPVLHtNF9mXXpKEaf1svLpksKuA3UFlhijLHAE9ba+eEHGGNmAjMBiouLvWuhSBcXaTw6kGEIZJrWJd/gPpXNWsuij/bwwJIytuw/zqTBvXn8mxP5myGnJO0eJLncBurzrbUVxpi+wFJjTKm1dkXbA1qC93xozvrwuJ0iXUZ477m6vrHdeHRDk6UgL0D3nKy4JvXe+fQAcxeVsX5HJSP69uCXN5dw8Rl9Mcaphp2kA1eB2lpb0fK/+4wxLwLnACuiv0tEwkXqPTupqmlg3X2XuDrvx7uqmLeojDfK9zMgP5d5153NtRMHkpmhAN0VxAzUxpjuQIa19mjLz5cAP0l6y0S6IDdF/UPcjEdvP1jNQ0vL+PO6XeTnBfjny87gpvMGkxu2RFrSm5sedT/gxZavTlnA7621i5LaKpEuym3WRqzx6APH6nh82WaeWbmNzAzDrRcN4x8uHEZ+XsDxPZK+YgZqa+0WYFwK2iLS5TlVbnM7Hn2srpFfrtjCk29uobaxietLBnHnxSPo1yty2VHpGpSeJ2nJT5vJxsOpMNG/XzHGsf0L11Ywb1Epu6pqT9gppW/PHL4w9BQF6ZOAArWknVRupRTrgRDvAyPewkQL1uzk3hc+bE3Ta5tOte9oXbv7TtcHmESnokySdibPWea4i8jbs6d6dp3wBwKcWKAo1uuJsNbyRvl+vvvbVSfkUkcSuu9ktkeSL1pRJm0cIGknVVspxdpdPFm7j6/bUck3fvket/zqg5hBGj6/787YDV1SQ4Fa0k6qdgqJ9UDw+oHx6f5jfO93q7nqZ2+zed8x/uOKMQzIjz3+HLrvdNwLUNxRoJa0k6qtlGI9ELx6YOypquVHCz7kkkdWsKJ8P3ddPJLXZ03hW18cwj2Xjm53r221vW+/bHUl3lOglrQTbV8+L8V6ICT6wKiqaWDuolIuenA5z6/eyU3nDuaNe6Zwx8Uj6JHTPM8ffq8FeQF6dwtEvO9kPMAWrq1g8pxlDJ39MpPnLGPh2ooOn0s6TpOJIlF4nfUBUNsQ5DfvfMbPX/+UI7UNXDW+iB9MH8mgUxIvO+pl1ocmJ1Mr2mSiArVIijQGm1iwpoJHXi1nd1UtF40q5J4ZozlzQGJlR5OVkpeq7Bpppq24RDqRtZYln+zlgcVlbN53jHGDCnj4+vGcN+zUdsfGG3STvWVXPL+X5FGgFgnjZQ/1/a2HmPPKRtZsr+T0wu784saJzBhzWsSyox0JusnYsivEabm7JidTT5OJklaSPbkVCpYVlTVYPg+W8V6ndM8RvvPrD7j+iXepqKzhhpJB1NYH+d7v1nD+3OURz9eRPOhk9npTlV0jsalHLWkjFUvHE+2h7jxczcNLy3lxbQU9c7K499LRnNo9m/te+jhmuzsSdJPZ6413ubskjwK1pI1kfs0P6WgP9dDxeh5ftpnfvbcNY2DmBadz64XDye8WYPKcZa7a3ZGg61Tkyater3Yk9wcFakkbqZjcijdYVtc38tSbW3lixRaq6xv52qRB3Dl9BP3zPz/eqX0VlTVMnrOstbc6ZXQhL6yuiCvoqtd7clCglrSRiskttz3UhmATf3x/O4+9tpkDx+q45Mx+3HPpKIb37em63YbPt+KqqKzhhdUVXDupiOWl++MKuur1dn0K1JI2kv01H6L3UNvWhc7MMASbLOcMPYUnbprEpMG942p327rSITUNQZaX7k9JjrLKoaYXBWpJG6n6mh+ph7pwbQX3PP8h9cEmAIJNzWG2fM8Rdhyqjhio2wbD/LwAuYEMKqsbHHvYkJoc5VTW8xZvKFBLWon1NT8ZPcUPd1Zy7wufB+m2KmsaIwa58GBYWdNAXiCTR24Yz1UTihxX/aUiRzkVk7LiLeVRiyf8ULzHqxzokK0HjvP936/hisffpq6xfZAOiZTrHCsnujNzlLXiMP247lEbYzKBVUCFtfYryWuSpBu/fJV2Co53P7c+rrbsO1rLT1/bxB/f30F2Vga3Tx3On1bvZHdVreN7woNcrGDYmdkaWnGYfuIZ+rgD2AgkVkFGuhy/fJV2Co5Ba109OI7UNjD/jS089dZWGoJNfPMLxdw2dQSFPXM4vbBHuwnBttoGuYVrK8gwhmCEgmdtj+usbI1UTMqKt1wFamPMQOBy4D+BHyS1RZJ2/PJVOtokXbQHR21DkN+9t42fLd/M4eoGvjpuAHdPH8mQPt1bjwm9799f+pjKmoYT3t82yIW+XUQK0n4Jhsq9Tj9ue9SPAvcA7ZNEWxhjZgIzAYqLixNvmaQNv3yVjtRTbCv8wRFssry4toJHlpZTUVnDl0b04d5LR3NWUX7E94d6wNEmLCN9uwDINMZXdZyVe51eYgZqY8xXgH3W2tXGmIucjrPWzgfmQ3M9as9aKL7nl6/SocBz93Prow47WGtZVrqPeYvKKNt7lLMH5jPvurOZPLyP6+s4BTmnbxFN1iowSoe56VFPBq4wxlwG5AK9jDG/s9bemNymSbrw01fp0DWdHhyrtx1iziulfPDZYYac2o2ffXMil42NXHa0I/zy7UK6lrh2eGnpUf8wVtaHdniRzhY+PHHjucWs3lbJqxv3UtgzhzumjeCGvxlEINPbDFVtXyUdpR1e5KQTGp7YVVnDI0vLeWBxGd2zs5g1YxTfnjyEbtnJ+U/fT98upOvQnonSaZJZb6Kyup6fv/4pv37nM7Bw83mD+f6U4fTunp30a4t0hHrU4jteLpJpG3RPy89lYnFvVmzaz7G6Rq6dOJC7po+kKCzPOdK1V207FHflOpFUUI9aOoVTrQto3uXabZCMNCYMMGZALx6+fjyjTut5wrEPLC5zvG54RTuNLUsqqUctvhNtMUw8vet5i0oj5i1XVje0BumFaysiLlQJF6nsqAoViR+oKJN0iljparE2dQV459MD7HKovxF6EIR63LGCtBMVKhI/UI9aXPNyAi7WKkJwDpIfVVQxb3EZK8r3kxmjpobTSsFwkQr5tz2PSGdSoBZXvK6Q1zaNzWnMODxIbj9YzYNLynhp/S4KugX458vOoKBbgH/788eOqyLd9IiLOrhfoUiqKFCLK8mokNe2dka0Jej7j9bx+LJN/P797WRmGL4/ZRgzLxhGfl4AgEBmhmNPP1qhpvDJwpLBpyhlT3xJgVpcSWaFPKdFIhef2Y+Hl5bz5JtbqGts4oa/GcQd00bQr1duu/c7BVSnIZbe3QLc99UxJ7xPhYrErxSou4BULN5Idg2LtkGyrjHI71du58J5yzl4vJ7Lx/bn7ktGcnphjw6dF7RSUNKbAnWaS9XuKqmokNfUZHlp/S4eXFLGzsM1fHHYqdx76WjGDSpI6LzqKUu6U6BOc6naXSWZPVNrLa+X72feojI27j7Cmf178dvvjOVLI/p4VtVOJJ0pUKe5VO6ukoye6drth5nzSikrtx6i+JRu3HTuYF7buJdvPf0++XkBjIHD1Q2taXjxrFoU6SoUqNNcutQ/Dh9Hv/m8wazdXsmij/fQp0c2P7lyDN0Cmfxrm1S7totUQrnSnbVxrkhn0srENDdrxijyApkn/M5v+b+hcfSKyhoszcH2/ldKWV62j7suHskbs6Zw83lDeOTVTa4Wp7hZtSjSlShQp7mrJhRx/zVjKSrIw9C8eMNvhYScVgfWNTbx3KodLP1kLxDfcI2WdsvJREMfXYCfsxpqG4KOC07gxKGMaItTwvltaEckmRSoxXML11Ywb1Epu6pqyXCRtBEaynBT/wP8N7QjkmwK1OKpF9fs5N4XNlAfbAKgyUKmgYwMQ0PQufb5rsqadimAyvoQaaZALZ5ZueXgCUE6JGihZ3YW3XOyYhZgSvUwjrbkknSgQC0J27j7CPMWlbK8bL/jMVU1Day775KYBZhSKVWrOkUSFTNQG2NygRVATsvxz1tr70t2w+Rzfu317ThUzSNLy3lxXQU9c7KY/eXR/PadzyIW82/bYwZ/1N5I1apOkUS56VHXAVOttceMMQHgLWPMK9ba95LcNsGfvb6Dx+p4fPlmnnlvO8bAzAtO59YLh5PfLcBpvXJj9pj9kqWSylWdIomIGaht8+63x1r+Gmj54/2OuBKRl72+RHvmx+saeeqtrcxfsYXq+kauLxnEHRePoH/+56lyfuoxx5IuqzpFXO1CbozJBFYDw4GfWWvvjXDMTGAmQHFx8aRt27Z53NST09DZL0d8Khpg65zLXZ8n0thwaPupSJkUbYN6//xczht2Km+UH+DAsTpmjOnHrBmjGN63Z/sLpRGn8XK/LRiSk0PCu5Bba4PAeGNMAfCiMeYsa+1HYcfMB+YDlJSUqMftEa96fZF65qEPKXw4JTyA7aqq5YU1FQwr7M78myex/WA133r6gxNS6CqrG3zde44knXr/cnKLK+vDWltpjFkOXAp8FOv4riqVk3te1YGONe7adjjFacl3TX2Q7QerT2hP28JJfhg/j5dfxstFoolZ68MYU9jSk8YYkwdMB0qT3TC/ilRg6EcLNrBwbUVSrudVLQ83PfBdlTV8uLPSMdd5d1VtzF29VTBJxHsxx6iNMWcDvwEyaQ7sz1lrfxLtPSUlJXbVqlWeNdJPJs9ZFjGQFRXk8fbsqZ3QIncijceGywtkUtMQJMM0rygMF1odGEu84+cikuAYtbX2Q2CC561KU+mY0hUaqqlpCLYG29BEYltBa7l92ggG5OfyH//3Sbug7vS+cKHeu1/zv0XSjVYmxindUrrCe9JBa8kLZPLVcf3564Y9HKtrBOD84X145IbxFPbMASA3kMndz61v14O2EDVYh8bP/Zj/LZKuVI86Tn4q1L9wbQWT5yxj6OyXmTxnWcRxcqc87D+t3smxukauGDeAN2ZdxO/+/gutQRqag2mTwzBHKKXPAAV5AXp3C7QbP4+W/y0i8VGPOk5+Sely22N1GpKxFv5y2/mcVZTveA2nbw9uxuPTcYhIxK8UqDvADyldblcs9s/PjVh7o6ggL2qQhsRSA9NtiEjEzzT0kWRuhic6wk2PddVnh8gJG6YB98E2kdRAPw0RiaQ79aiTKJkTatF6rOV7jzJvURmvbtxL3545fG3SQN7efIDdVbVxD9V09NuDX4aIRLoCV7U+4tWV86jj4ZRznWkMD10/LqGgFSkvOicrg3EDC1i17RDds7P4x4uG8e3JQ+iWreexiN8lXOtDOsZpeCJobcI967Y91orKGnrkZFHbEGTdzkr+7vyh3HrRcHp3z+5Yw0XEVxSokyjartpeFKi/ZEw/Kipr+MXrn3K8vpFrJg7krukjKerghJ0WqIj4kwJ1EsXaVbujqWoNwSaeW7WDx17dxL6jdVx8Rj/uuXQUI/t1vOyoFqiI+JcCdRKFAlykFX4Qf6qatZa/btjDg0vK2HrgOCWDe/Pzv51IyZBTEm6rtqUS8S8F6iQLBblES5W+s/kAcxeVsn5nFSP79eDJm0uYdkZfjDERj493GEMLVET8S4E6BdykqjkF1o8qqpi7qJQ3Nx1gQH4uD35tHFdPKCIzI3KADp0r3mEMLVAR8S+l5/mAU6rdmAG9WLO9koJuAf5pynBuPHcwuREWsITrSClWbUsl0rmUnudzkcaH6xqbWLu9ku9PGcY/XDiMXrkBV+dauLbCMdMk2jCGFqiI+JcCtQ84Fk4CZs0Y7fo8oV6xk1jDGH6oYSIi7anWRyerawzSKy9ybznefOho22SpzoZI+lKPOkEdXSTS1GT58/oKHlpSTlVNQ7vtr/ICmUwZXcjkOcsSztwANNYsksYUqBPQkewKay2vl+1n7qJSSvccZcyAXvzX1WM5eKyOB5eUtwblKaMLeWF1hSeZG0UFeQrSImmsywbqRJdDu3l/vItE1mw/zNxXSlm59RCDT+3GzecN5tVP9vKtp99vd43Jc5bFvQAlkfrRIuJfMQO1MWYQ8FugH83zW/OttY8lu2GJSHQ5dKK7p4T/fvO+YzywuJTFH++lT49sfnLlGLoFMvnXP3/seI2OLEDpSOaG6nuI+J+bHnUjcLe1do0xpiew2hiz1Fr7SZLb1mGJLod2+/5Yi0T2VNXy6KvlPLdqB92ys/jB9JH83flD6Z6TFbPH3NEFKPFkbqi+h0h6iJn1Ya3dba1d0/LzUWAj4Ov/Fye6HNrt+512Mfn+lGHc/8pGLnxgOS+s2cm3vjiEN2ZdxO3TRtA9J8vVNVKxQ4o2oBVJD3GNURtjhgATgJURXpsJzAQoLi72oGkdl+hyaLfvDx9q6J+fy6TBvZnzSilH6xq5enwRd00fyaBTusV9jVQsQFF9D5H04DpQG2N6AC8Ad1prj4S/bq2dD8yH5iXknrXQhfBx1vCMCYivNxrPpNxVE4r4ytn9eX71Th59dRP/9+Fupowq5J5LR3NG/14JXSPZC1BU30MkPbha8GKMCdAcpJ+x1i5IbpPiExpnraiswdI8zvrC6gqunVTUoU1Zwf2mrtZaFn20hxmPrmD2gg0cPF4HQPneY5TtOerJNZJJG9CKpIeYRZlMcx3N3wCHrLV3ujlpKosydaQAUTycsiLe23KQuYtKWbu9kr49c6isbjgHR4kAAAn7SURBVKA+2NT6vnQpaKSsDxF/iFaUyU2gPh94E9gAhCLRj621f3V6TzIDdXhgcSpAZICtcy6P+3xtA1WkinJt5ecF+PFlo3ns1U3sqqpt97pXDwsR6foSqp5nrX2L5rjX6SKlkxmak7vDuRlnjZWeFq12BkBdQ5CcrEx2RwjSoEk5EfFGWhVlihQ4Le2fIm7HWWOlp8UKtLWNTTywuMzxoaBJORHxQloF6mjlQEOTcgV5AXIDGdz17Domz1nGwrUVcZ9vV2UNx+sa6ZEbOylmV2WNJuVEJKnSKlA79VBDY8GP3DCeusYmDlc3tGaA/GjBBsdg7XS+XnkBLnxgOUdrG4my41XrOfyQwSEiXVdaBepYPdd4V9pFOp8BqmoaGFbYgwW3fpGHrx/fWhc62hDLVROKeHv2VLbOuZy3Z09VkBYRz6RV9bxYq/XiXWl31YQirLX8v5c3cvB4PQD983P5z2vGctHIQowxTCzufUIWSGensvmhDSKSWmkVqCH6ar14V9qt31HJn1bv5ODxegb2zuPuS0Zy5bgiMhzGOzp7qyoVURI5OaVdoI7G7dLvLfuP8dCScl7esJtTumdz31fP5JtfKCYnK/YO317oaK840aqAIpKeulSgjjU0su9ILY++tolnP9hBTlYGt08bwXe/NJSeLnf49kIivWIVURI5OaV1oHbqmYYHvCO1DTzxxqc89dZWgk2WG79QzD9NHUFhzxxX5/OqXZBYr1hFlEROTmkbqN30TGsbgvzvu9v42eubqaxu4IpxA7j7kpEMPrV7h87nRbti9YqjBXlttSVyckrbQB2tZ/rVcQNYsGYnjywtZ1dVLReMLOSeGaM4qyi/Q+eLJ1DHOk+0XnGsIJ+KGtUi4j++C9Ruhx+ceqYVlTV8+bEVlO89xriB+Tz4tXF8cXifmNf1avzXzc4tTr1iNw+Lzs48EZHU81WgdupRrtp2iOWl+08I3tEq5zUGLT//24l8+azTaK7SGptX47+J7Nxy17PrIp5Tk4UiJzdfBWqnHuUz721vrZAXCt7XTipqt4sLwPUlA/nPq8cSyIxv0aVX47+J7NyiyUIRicRXS8ijFV1qq6YhyNJP9jK2zZhzr9ws5l17NvOuGxd3kAbvdlxJ5Dwq7iQikcTcOKAjOrpxgNNuLU6yszK45YtD+N6Fw+jdPTvu6/mRloiLnJwS2jgglSINGzhtDNAtO5OlP7iwtWBSV6HJQhEJ56uhj0jDBt84ZxCBsNobOVkZ/NfVY7tckBYRicRXPWr4vEdpreXlDbt5aEk5DU2W7MwM6oNNFGk4QEROMr4L1ABvbz7A3EWlfLizipH9evDkzSVMO6Ov61Q7EZGuJGagNsY8DXwF2GetPSuZjfmoooq5i0p5c9MBigryePBr47h6QhGZsbZZcUkTdSKSjtz0qH8NPA78NpkNqapp4Gu/eJecQAb/cvkZ3HjuYBZ9tIcL5i33JLCqlrOIpKuYgdpau8IYMyTZDcnPC/DETZMYX1xAr9yA54FVtZxFJF35KuvjgpGF9GqpDR3v/oexqJaziKQrzwK1MWamMWaVMWbV/v37Ez6f14HVaRm2lmeLiN95FqittfOttSXW2pLCwsKEz+d1YNXybBFJV74a+mjLy8AayvaoaQiS2ZLiV5AXIDeQwV3PrmPynGUsXFvhSbtFRLwWM1AbY/4AvAuMMsbsNMb8XfKb5V2RpNCkZKiGSNBaAhmG4/WNHK5uwPL5RKWCtYj4ka+KMiVDPIWeigryeHv21CS3SESkvWhFmXw79OGVeCYflQEiIn7U5QN1PJOPygARET/q8oE60qRkIMMQyDxxWboyQETEr3xZlMlLTnsURvqdViiKiB91+clEEZF0kBY7vKiynYhIZL4I1KpsJyLizBeTiV4XYBIR6Up8EahV2U5ExJkvArUq24mIOPNFoFZlOxERZ76YTHTKddZEooiITwI1NAdrBWYRkfZ8MfQhIiLOFKhFRHxOgVpExOcUqEVEfE6BWkTE55JSPc8Ysx/Y1sG39wEOeNicztRV7qWr3AfoXvyoq9wHJHYvg621hZFeSEqgToQxZpVTqb9001XupavcB+he/Kir3Ack71409CEi4nMK1CIiPufHQD2/sxvgoa5yL13lPkD34kdd5T4gSffiuzFqERE5kR971CIi0oYCtYiIz3VaoDbGXGqMKTPGbDbGzI7weo4x5tmW11caY4akvpWxubiPW4wx+40x61r+/H1ntDMWY8zTxph9xpiPHF43xpifttznh8aYialuo1su7uUiY0xVm8/k31LdRreMMYOMMcuNMZ8YYz42xtwR4RjffzYu7yMtPhdjTK4x5n1jzPqWe/mPCMd4G7+stSn/A2QCnwKnA9nAeuDMsGNuBX7R8vPXgWc7o60e3MctwOOd3VYX93IBMBH4yOH1y4BXAAOcC6zs7DYncC8XAX/p7Ha6vJf+wMSWn3sC5RH+G/P9Z+PyPtLic2n5d+7R8nMAWAmcG3aMp/Grs3rU5wCbrbVbrLX1wB+BK8OOuRL4TcvPzwPTjDEmhW10w819pAVr7QrgUJRDrgR+a5u9BxQYY/qnpnXxcXEvacNau9tau6bl56PARiC8cLvvPxuX95EWWv6dj7X8NdDyJzwrw9P41VmBugjY0ebvO2n/obUeY61tBKqAU1PSOvfc3AfAtS1fSZ83xgxKTdM85/Ze08V5LV9dXzHGjOnsxrjR8vV5As09uLbS6rOJch+QJp+LMSbTGLMO2AcstdY6fiZexC9NJibf/wFDrLVnA0v5/CkrnWcNzXUVxgH/DSzs5PbEZIzpAbwA3GmtPdLZ7emoGPeRNp+LtTZorR0PDATOMcaclczrdVagrgDa9iwHtvwu4jHGmCwgHziYkta5F/M+rLUHrbV1LX99EpiUorZ5zc1nlhastUdCX12ttX8FAsaYPp3cLEfGmADNwe0Za+2CCIekxWcT6z7S7XMBsNZWAsuBS8Ne8jR+dVag/gAYYYwZaozJpnmw/aWwY14CvtXy83XAMtsyMu8jMe8jbKzwCprH5tLRS8DNLRkG5wJV1trdnd2ojjDGnBYaLzTGnEPz/w/81gkAmjM6gKeAjdbahx0O8/1n4+Y+0uVzMcYUGmMKWn7OA6YDpWGHeRq/OmVzW2ttozHmn4DFNGdOPG2t/dgY8xNglbX2JZo/1P81xmymeWLo653R1mhc3sftxpgrgEaa7+OWTmtwFMaYP9A8697HGLMTuI/mSRKstb8A/kpzdsFmoBr4due0NDYX93Id8D1jTCNQA3zdh52AkMnATcCGljFRgB8DxZBWn42b+0iXz6U/8BtjTCbND5PnrLV/SWb80hJyERGf02SiiIjPKVCLiPicArWIiM8pUIuI+JwCtYiIzylQi4j4nAK1iIjP/X8W5zmilIONHQAAAABJRU5ErkJggg==\n",
|
||
"text/plain": [
|
||
"<Figure size 432x288 with 1 Axes>"
|
||
]
|
||
},
|
||
"metadata": {
|
||
"tags": [],
|
||
"needs_background": "light"
|
||
}
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "0giuwC9GHzi8"
|
||
},
|
||
"source": [
|
||
"## Вычислительный граф\r\n",
|
||
"\r\n",
|
||
"Для проведения вычислений Tensorflow строит внутри себя вычислительный граф, который, в т.ч., может вычисляться на GPU. Однако в нашем случае, поскольку мы использовали пользовательские Python-функции, они не включались в вычислительный граф, и при вычислениях на GPU производилась бы передача данных между GPU и CPU и обратно.\r\n",
|
||
"\r\n",
|
||
"Для ускорения высчислений и построения единого статического графа, необходимо отметить все функции соответствующим декоратором:"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"id": "HK7HPLz3Hyrl"
|
||
},
|
||
"source": [
|
||
"@tf.function\r\n",
|
||
"def train_on_batch(x, y):\r\n",
|
||
" with tf.GradientTape() as tape:\r\n",
|
||
" predictions = f(x)\r\n",
|
||
" loss = compute_loss(y, predictions)\r\n",
|
||
" # Note that `tape.gradient` works with a list as well (w, b).\r\n",
|
||
" dloss_dw, dloss_db = tape.gradient(loss, [w, b])\r\n",
|
||
" w.assign_sub(learning_rate * dloss_dw)\r\n",
|
||
" b.assign_sub(learning_rate * dloss_db)\r\n",
|
||
" return loss"
|
||
],
|
||
"execution_count": null,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "J7HusxWkGjLX"
|
||
},
|
||
"source": [
|
||
"## Dataset API\r\n",
|
||
"\r\n",
|
||
"Для работы с данными в Tensorflow присутствует удобное API, которым мы в данном случае можем воспользоваться:"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/"
|
||
},
|
||
"id": "oYro9Lbr8q0M",
|
||
"outputId": "78c0a6de-71bd-4eef-8819-439495b28672"
|
||
},
|
||
"source": [
|
||
"w.assign([[10.0]])\r\n",
|
||
"b.assign([0.0])\r\n",
|
||
"\r\n",
|
||
"# Create a tf.data.Dataset object for easy batched iteration\r\n",
|
||
"dataset = tf.data.Dataset.from_tensor_slices((train_x.astype(np.float32), train_labels.astype(np.float32)))\r\n",
|
||
"dataset = dataset.shuffle(buffer_size=1024).batch(256)\r\n",
|
||
"\r\n",
|
||
"for epoch in range(10):\r\n",
|
||
" for step, (x, y) in enumerate(dataset):\r\n",
|
||
" loss = train_on_batch(tf.reshape(x,(-1,1)), tf.reshape(y,(-1,1)))\r\n",
|
||
" print('Epoch %d: last batch loss = %.4f' % (epoch, float(loss)))"
|
||
],
|
||
"execution_count": 18,
|
||
"outputs": [
|
||
{
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Epoch 0: last batch loss = 173.4585\n",
|
||
"Epoch 1: last batch loss = 13.8459\n",
|
||
"Epoch 2: last batch loss = 4.5407\n",
|
||
"Epoch 3: last batch loss = 3.7364\n",
|
||
"Epoch 4: last batch loss = 3.4334\n",
|
||
"Epoch 5: last batch loss = 3.1790\n",
|
||
"Epoch 6: last batch loss = 2.9458\n",
|
||
"Epoch 7: last batch loss = 2.7311\n",
|
||
"Epoch 8: last batch loss = 2.5332\n",
|
||
"Epoch 9: last batch loss = 2.3508\n"
|
||
],
|
||
"name": "stdout"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "A10prCPowHl7"
|
||
},
|
||
"source": [
|
||
"## Пример\n",
|
||
"Рассмотрим пример двухмерной задачи классификации на 2 класса. Примером такой задачи может быть классификация опухоли на 2 типа - доброкачественная и злокачественная, в зависимости от её размера и возраста.\n",
|
||
"\n",
|
||
"Сгенерируем тестовые данные случайным образом:\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"scrolled": false,
|
||
"trusted": true,
|
||
"id": "j0OTPkGpwHl7"
|
||
},
|
||
"source": [
|
||
"np.random.seed(0) # pick the seed for reproducability - change it to explore the effects of random variations\n",
|
||
"\n",
|
||
"n = 100\n",
|
||
"X, Y = make_classification(n_samples = n, n_features=2,\n",
|
||
" n_redundant=0, n_informative=2, flip_y=0.2,class_sep=1)\n",
|
||
"X = X.astype(np.float32)\n",
|
||
"Y = Y.astype(np.int32)\n",
|
||
"\n",
|
||
"split = [ 70*n//100, (15+70)*n//100 ]\n",
|
||
"train_x, valid_x, test_x = np.split(X, split)\n",
|
||
"train_labels, valid_labels, test_labels = np.split(Y, split)"
|
||
],
|
||
"execution_count": 19,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"scrolled": false,
|
||
"trusted": true,
|
||
"id": "c-_BjSHPwHl8"
|
||
},
|
||
"source": [
|
||
"def plot_dataset(features, labels, W=None, b=None):\n",
|
||
" # prepare the plot\n",
|
||
" fig, ax = plt.subplots(1, 1)\n",
|
||
" ax.set_xlabel('$x_i[0]$ -- (feature 1)')\n",
|
||
" ax.set_ylabel('$x_i[1]$ -- (feature 2)')\n",
|
||
" colors = ['r' if l else 'b' for l in labels]\n",
|
||
" ax.scatter(features[:, 0], features[:, 1], marker='o', c=colors, s=100, alpha = 0.5)\n",
|
||
" if W is not None:\n",
|
||
" min_x = min(features[:,0])\n",
|
||
" max_x = max(features[:,1])\n",
|
||
" min_y = min(features[:,1])*(1-.1)\n",
|
||
" max_y = max(features[:,1])*(1+.1)\n",
|
||
" cx = np.array([min_x,max_x],dtype=np.float32)\n",
|
||
" cy = (0.5-W[0]*cx-b)/W[1]\n",
|
||
" ax.plot(cx,cy,'g')\n",
|
||
" ax.set_ylim(min_y,max_y)\n",
|
||
" fig.show()"
|
||
],
|
||
"execution_count": 20,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"scrolled": false,
|
||
"trusted": true,
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/",
|
||
"height": 283
|
||
},
|
||
"id": "tq0vFchQwHl8",
|
||
"outputId": "9a5aa6a0-c92f-4d72-9e78-c0f615804bff"
|
||
},
|
||
"source": [
|
||
"plot_dataset(train_x, train_labels)"
|
||
],
|
||
"execution_count": 21,
|
||
"outputs": [
|
||
{
|
||
"output_type": "display_data",
|
||
"data": {
|
||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEKCAYAAAASByJ7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO2dd5iU5dX/v2dmtrINdpfOsoBKV9AFsQRFNIgx1thiNJZojDH6mkSNKSbRn0l8TaKmGDUxRo2vFbEiYgsIEaWIIFUEgaXsLmzvOzP374/vPM6WqbvT93yua67dnXn2mTPzzJxz36eKMQaKoiiKYou3AIqiKEpioAZBURRFAaAGQVEURfGgBkFRFEUBoAZBURRF8aAGQVEURQEAOOItQF8oKioypaWl8RZDURQlqVizZs1BY0xx9/uT2iCUlpZi9erV8RZDURQlqRCRXb7uV5eRoiiKAkANgqIoqYgxQHs70NERb0mSiqR2GSmK0gecTmDjRuDNN4GdOwG7HZg+HZgzBxgzBhCJt4Th09YGrFoFLFoE7NvH+w47DDjjDOCoo/gaFb+oQVCiijFAYyN/5uQANt2TJgatrcCDDwKffALk5gKFhYDbDXz4IfD++8BZZwHnnptcRqGpCbj/fmDrVqCoCBg9mvdXVgL33QccfzxwzTWAQ9WeP/SdUaJCRwfwwQdcqFVUUK/k5wPz5wNf+QqQlRVvCfs5TzwBbNgAlJZ2VfrDh3PnsHAhlers2XETMWwefxz4/POeu5tBg4CBA4GVK4Fhw4BzzomfjAlOwqzXRGSUiLwnIptEZKOI3BRvmZTe0d4O/PnPwN//zt9LSnhzOIB//xv43e+AhoZ4S9mPqayktR41yvcOwOEAhg4FXnoJcLliL19vqKqiq2jkSN+vSYTGbvFi7o4UnySMQQDgBPAjY8wkALMAfF9EJsVZJqUXLFwIrF8PjB0L5OV5v58DBvC+8nLgscfiK2O/5uOP+TOQ/27AAKCmhrGFZGDDBvolA72mjAyuUD77LHZyJRkJ4zIyxuwHsN/ze4OIbAYwAsCmuAqmhEVTE/D228CIEf7dzyNHUidVVABDhsRWPgVAbS2Qlhb8OJuNFzRW1NRwlb9vH5CZCUydCkyYEFoguKkp9ICx7hD8kjAGoTMiUgpgOoAP4yuJEi7bt9MFHUjfiHAxt3EjMHgwvRJ2e3LFL5OavLzQ0jGNiU2wx+Wie+r11/mcmZn8EC1ezA/ID35A91YgCgpCc29Z2Q2KTxLOIIhIDoAFAP7HGFPv4/FrAVwLACUlJTGWTglGW1toit3lApYtA155hQtWhwM49ljg1FMZE1SiyLRpwLPPUjn6u1gtLcw+Gjs2+vIsWAC8+iqzgrpnAB06xKDTHXcE3k5OncodjbW68IX1mg4/PHKypxiJFEOAiKSBxuApY8yLvo4xxjxijCkzxpQVF/doxaHEmbw86plANDQAa9YAH30EpKcz0WXoUHoLfv1r4N13YyJq/2XYMODooxnM8XWx3G66bc48M/opmlVVwBtv8EPg67kKC7nKeP11733G0O3TeUdQUADMnQvs2uX7NTmdfE3nnqtppwFImHdGRATAowA2G2P+GG95lN5x2GFML21uBrKzez7ucgHLl/P3adNoEAB+R4cPZ8zvX//i7xMmxEzs/sfVVzNnf9s2pmValvzgQRaOfPWrVLDR5oMPuLIP5P8fOhRYsQI47TQGn5YsYcxABJg5k/ePGwdceCFXGx98wA9fQQFf06FDdJGdcw5w8snRf01JjJhgy7kYISInAngfwAYAbs/dPzXGLPL3P2VlZUab2yUe778PPPIIU027xxLKy4H33gPKyoCJE33/f1UVPRU/+lH0Ze3XtLcDq1dzhV5eTgU7dSqNwaRJsQnqPPAAA0+FhYGP27yZsQW7nXGFzEyuLioruYO47DIaMLebRu6tt1iTIAIceSSrr0eP1kCVBxFZY4wp635/wuwQjDHLAejVSgFOPBGorwdeeIGLv4ED+T2srfWmo44f7///CwsZcG5ooMtXiRLp6azePf54KlKR2CtMh4PPHQinE/j0U+4GOscN7Xa6v9rbWZQ2ciQ/WBMm6PaylySMQVBSBxHga1+jm3r5choBY2goLAUfKF3cZuM5rBigEgPi1VNk+nQGjwLFA3ftotHwl2mUns7MoUWLAq80lKCoQVCixrBhwAUX8Gbxl78AmzbRZe0Pt5sGxFcMQkkxpk/nhW5s9J0Oagw/MIcfHthoFRVx5eHvPEpIJFSWkZL6nHQSA86BqKwEjjlGv9f9gqws4PrrGfg9eLBrhlBbG/DFF/Q5jhsX+DzWtrKtLaripjpqEJSYMnEid/779/t+vKWFGYVnnBFbuZQ4MmUK8POf84OxaxewZw+wezeDTmefzQ9De3vgc7hcNAi6rewT6jJSYorDAdx8M/CHP3DxV1DAtjkdHVwg2mzADTdocVrCcegQW0s4HOxLEkrri3AYNw647Tb2M6mu5vOUlLD/0Cef0B0UKM5QWQnMmqVtdPuIGgQl5gwaBPziF8C6dUwpr6hgFuGZZzLwPHhwvCVUvmTXLuDFF6mQbTZvcOf004F58yJvGIYM6VmRPHmyd1s5bFjP/2lq4opi3rzIytIPSZg6hN6gdQiKEkW2bgXuvZdKv7jYG9RtaaFynjaNfYYibRR8UV3NbWV5eddt5aFD3E18//uciKaERMLXISiKkkC0tXGoRV5ez5SwrCz69NatA955h7uFaDNoEPsZffIJi84qK7mtPO881lEMGhT6uZqaaNDcbm5HCwqiJ3eSoQZBiQvxqoNSQmTdOirOoiLfj4uwpcQbb7AjYSz6A2VksDht5sze/X9DA/Dyy8DSpV37IM2cyR5H2otdDYISO+rr2ZLmrbfoAcjMZMxgzhzGKZUE4uOPgwdos7OZCXDgAKuEE5n6enZN3b+fjbIsN5fLxU6Ln34K/PSnfKwfo2mnSkzYu5eB5Oee42Jy9Gi2qFi6lPevXBlvCZUutLeHNnBGJDnGbD7zDLMXRo/uGvOw27kaMQZ46KHgrXpTHDUIStRpbWU80OXi9zE7m3okLY3fxcGDgYcfZhqqkiCUlASfluZyUYEOHBgbmXpLbS1XHIFW/0VFrH9IlpGhUUINghJ11q6li8ifOzozk+1oliyJrVxh4XJxbu8//gHcdx/w5JPsppmqK8rjjuNrC7T6r6hg7n+gPiSJgLXSCLTjsYJZ27ZFXZxERmMIStRZtowzEgIxeDDw4YfAt7/N2GFCUVHBNs3WrN/0dPbXeecd5sh/73up12djyBDOGVi8mNu67sq0poZK9Otfj4984RCqS8tmC220aAqjBkGJOrW1wZW83c4FaUtLghmE+nrgnnvo9yot7fqYMezT/6c/AbfemnqTuC66iEryzTep/LOyqFxbWxkAuv1234ViiUZhobdjYqC0Nrc7OV5PFEmxT7CSiAwcyKByoDYzTqdX5yQUy5ZxNTx6dM/HRFhBu3UrdwxHHhl7+aKJ3Q5cfDHTSleu5EXMyGCH0smTk8cAjh7N61Rb6z/e0drK1zZ1amxlSzCS5IoqycycOcCDD/qvHWpsZNq7zcaF9ogRHNo1ZUqcdY7bzcBGoF4aInQXvfVW6hkEi6Ii9hVJVkSAb30L+O1vmcnQ3b3X1kZj953vJNj2NPZoUFmJOkcdRZ1aWdnzsV276JHYvZuT1LKzmexx333A//4vjUXcaG1lMVOwbUtuLuMLSuIyfjzwwx/SJ7lrF2snKioYcD54ELjySmD27HhLGXd0h6BEnYwMfhfvvZffv8JCKv59+zh/ecAA7iKGDvUeP2gQR+0+9BBnK8elotnanrjdgYezuFwMNCuJzZFHAn/8I9tfbNrE6zp2LAd8p1pSQC9Rg6DEhKFDgTvv5LTEt97i4mzzZg7CmjKFRqEzlnv+009pROLSDjs9ncLt2BG49fKhQ+zb3wusrFVt4REj+tr+IsVRg6DEjAEDgJNP5q2qivGCkhL/ytAqXvvgg14ahMZGYONGZgoNGABMmhR+I7PTT2eW0cCBvgMaLS3cPZxwQsindDpZ0vDGG9wFiXBw0Lx5jNXGa7yxoqhBUOJCYyOTWIKtjDMz6eINC6eTPfyXLOHvFjYbZ3hefHHowcNJk7j6f+kl+rry8ym0MRSssZF1CP6q7rrR1gb87W8s1svP97YA2rmTLrXZs4ErrkieBB4ltdCPnRIXMjO9nQ8CGYX29jALYd1u4LHHGJwoKemqWV0u4L336OK58cbQtK4IWyyXlACvvMLot83G57GMxRFHhCzes88yo2rMmK6vu7iY9mbZMv4899wwXrOiRAg1CEpcGDqUrWXq6wNXMbe3A8ceG8aJt24Fli9nEVl334vdzpz0devYzXPGjNDOKcJjy8qYKtXWxiBkOD34wTT4//yHsRFfRtBm445h8WJg/nwaTUWJJeqtVOKCCHDWWYwldPbqdMbqqjx+fBgnfvttpjD5c8SLMB6waFHYMkOELR1KSsI2BoA3sSVQS530dBrBLVvCF09R+ooaBCVuzJxJb8zu3Vx4u928v6mJmUW5ucBNN4UZZN26NXj3zYICOu1j3La5oSH0bKKWlujKoii+SCiXkYj8E8CZACqNMVPiLY8SXUToK584kcVpn3zC+wsKgG9+kxk3tbVAXR3dSyG1tQhV48YhzzMvL/TmqIHafChKtEgogwDgXwD+AuCJOMuhxAgr5XLiRC7YnU4agZdf5kwTK6HH4QBOOYUdFALWEE2ezBiBVeXmi+pqFkCEMgAmgkyezKd0Ov3Hs9vamAAVlptMUSJEQrmMjDHLAFTHWw4lPtjtTAC66y4WsI0YwQBsSQmzOhcvZklAQ0OAk8ydS3+LP3eQMdxyzJ8fldcQiLw8GrU9e3zvFNxuoLwc+NrXNKCsxIeEMghK/8YY5ugbQ2PQeQGfns7Eob17geefD3CSww5jH/8vvuByuzMdHYwdzJoVt0Z0F17I2MnOnd6AutPJyu1du2jPzjgjLqIpSsK5jIIiItcCuBYASkpK4iyNEkk+/5wr5ECXdfhwYMUK4Bvf8FOfIMIAxMCBwKuvdjUKaWlMbTr77Ji7izqLcN11LEB7803gs88o8qRJtGMTJmgbCyV+JJ1BMMY8AuARACgrK0vR+YX9EyvVMpBCdDi4g/jiiwCLfJuNfpdTT+VJGxsZpR0/PiGitXY72+7389b7SgKSdAZBSV06OkJfuPurXehCRgZ7byuKEhIJFUMQkacBfABgvIiUi8jV8ZZJiR3DhwdX9MYw+FpYGBuZFKU/kVA7BGPMJfGWQYkM9fXeRqO5uUy5DNSiAqALKCPDm3oJAHZXO4bsX4fDPn8TefXlaOlw4OCYGShxnwygNMqvQlH6FwllEJTkp6ODWUDvvMPMT6uOwGYDvlpWjfOG/Rfp2zfz4IkTgeOP/7INRFYW57o/9hjTTXPQiGM/uB+F1Z+hNbMANWmD0druwpyMFZBfvceD58/XKKyiRAg1CErEcLmAv/8d+PBDKvQvi6+MQen2tzD8vmewbSAw6dhctqPYvJltqi+6iEOURTBnDt1Gzz1rcOT6h5HZuBPl2aVwtgnS0oCZJ9pRMGQELc/TT3M2Z1lZPF+2oqQMahCUiLFpE7ByZc/WziW738f0DU+idsgobKtPR2E7MGwYmBra3g78+9/cHsyeDRHahuNH7EbDDzdg/9DRyLYJiotZfPylkUlLY7XaSy8BxxyjuwRFiQAJFVRWkpslSxgv6Kybba4OTN74PBpyh8M40pGZySlhX5Kezmjyc89x1e8hZ/MqDBtpx9HHCKZPZ9fTHu0e8vI4mFkH3CtKRFCDoESMbdt6NhotOrgF6e0NcKaxM11WFlsJdWndkJXFWoHNm733VVcH798gwuBEU1PfhW9sZN+M1ta+nyuWVFez7HnfPm+7WEXpJeoyUiKGzdazR09WSzXS25swuGID0tsb4LKlQWwjANcQwNGt6KC21vt7fn7P1hPdMYaBi74Um23eDLz+Ov1dIrwdfzwHHI8Y0fvzRpvPPwcWLmQql/XGFxUBX/865zvrYGalF6hBUCLGpEnUq0OG8G+HsxVHbH0FQw98jLaMPLhtaXB2uDHJsRfydiZ7CnUeep+W5v195kxOoQ80Y7OhgU/WW8W9eDHwf/9HP5c1xszpZCDkv/8FfvhDvqhEY+1a4M9/piEcNcqr/BsagEceAXbsAC67TI2CEjb6iVHC4uBB4LXX2ITu0Uepm9rb+dhpp9F743YDMAbT1/wduQ170Z6eg/b0XHQ4stAiA5A9vICKfsWKTv8AtqS2KC1lYx9/8QGnk93hzjmndwHlrVuZpTRqFAcaW+dwOBjTyM8H/vQndkZNJOrqgIceoszFxV2Vfm4uI/rvvAOsWRM/GZWkRXcISki4XMALL3BRLcLFqcvFWfb5+cAPfsBWQaecArz7LjA1eweG71+D6sLxyGyrx4D6/ahHPvLzgQEDAEgWq9a2b2fZ8YwZdHlYiADf+x7wxz/SR15YSIXndtMQtLRw3FpYA5c78eabfBGddyUADdWhQ3yOigrm0V53XZAhDDHkww9pDP25yWw274jQsjLNvlLCQg1CClBVxWZvbjfT8ktLI68Hnn+eOmb06J79hurqgP/9X+AXv6CnorAQqP7jMtS1pKMWghr7UTjc2YiitFoMyM6B9bFz2jNg1m1Ey/zzkP3Ny3t+GPPzgdtvB1avpvto924+eVkZ+0QfdljvXmhbG/Dxx9wddH8hq1YxwGyzMevpySfZIO+882JfBFdfT3k++ojbsJISYMMGP21eO1FQwF7aDQ3Bj1WUTqhBSGIOHmQKvzV6UoRGYeRI4PLLgSOOiNzzLF7s2xgA1NstLSwJ+MEPGNfsWF2OA5tysP4LoL4lA58POxGNTTswuPxzHNrTBIcDaHc50Jxegservov2X+Vi/nzq+S6L9sxM4MQTebMi1n1Vyu3t3gwli8ZGYPly3m/FNTo6+JxDhtC95HZzZFssWLeOfrn2dr7BdjvwwQeMbwwezPiLw0GlX1/P/8nJoQGwguMhdQBUFC9qEJKUQ4eA3/yGeqxzXNEYJuv89rfALbdEJia6ciX1S6BOpIMHc9FdU0OPRYdkYOd2F9w2FpSJpKO1aAK2NR2OAztb4GwDikZmYnTaXhSNyYVUb8MX/+9DvP9wDWafPRCOE2b13AFEanWelcX6h/Z2/gS4C3C5uq6oOzqojNPSuDp/8UUaps6B8GiwfTvwwAN0oQ0Y4L0/N5dWeetW73tRVcWfVo+QgQN50dPSeLwSHk4nd2GLF3t3pEcfDcyZE52td4KhQeUk5bnnuDgcPrzrQleEOmHgQODhhyOzSNyzJ3hmp83GW7VnAOp/mmfC1ljfpVDN7QbK99vhzMyBLS8HTZWNaEgvQtlHf8Npq3+Dsub34dz8GfY9swy4+25avGgEdR0ObkUOHODfra0cxdY9TtDeDowdy9/T0qhwP/oo8vJ058UX+YZ3NgYWpaWUf+1aoLKSBquggD/z8xmkf/ttDlvoHh9RAtPcDPzhD8D99zOZobCQC4SVK4Ff/5oDl3zNPk0h1CAkITU1dC0HmiOfl0ddunFj358vPT20mie32+vFePXADKTnZSK9vfHLx5uavAPm7eJGfkcV9h1MQ379btQVlKIxbziksBAba0bAPWo0AyN/+EN0isXmzKE7qqaGiqC7C6m+ngp28GDvfdnZDHBHk4MHWRvROcDemUGDuJOxZm/62kFlZHAUW4orr4jzz39yp1haSmPgcHgr6UeOZCBt5cp4SxlVwjYIIjJAROIzf1ABwMUsEHyYjMPRrU1EL5k+nTGCQLS0cEE7fDiTc9ocA7Dq2BuQ2VKDnMYDEOP+Mlab7azHsNad2JAxA1vaxqAxz1tHkJbGhXlDo7C+YNcuroYjTVERcOut/H3vXhqd9nZardpaWtTjjuv6JlsWL5rU1fFN8ueaaGykIcvP57H19TRojY2UOy2Nk+JqayNz8fsL+/YxVbekxPd773BwcbBwYUpXhAc1CCJiE5FvisjrIlIJYAuA/SKySUTuFZHDoi+m0plAtVqdsYLMfWXqVK/+8SfP/v0s7rW8FCJA1eDJWHrSHagYPAV59eUY2FSOYR274LSl45Vh38VHOBbt6b7TOb+Ue+BA+nOjQWkp8LvfATfc4A3GFhXREMye3bN1RktL9OdepqcHXtnX11PO4mLGCsaO5Wp2+HDKPXeuNw6ya1d0ZU0l1q71BuP9kZvLmM3u3bGTK8aEstx5D8DbAG4H8Kkxxg0AIjIIwBwA94jIQmPMv6MnptKZoUOpMN3uwMWoHR3UeX0lLQ248UbgnnuoEwcP9j5vSwuNwZFHskupJZ+V5FJXMBqrZt6AjNY6lG+qx9YdaXAVDoaBIG/3Cgwb2rUPkaULv9TFVgO7aJGVRffRLbfQR+zvDWts5BYo2iM5hw9nTKCpyXcMwVoNuFwMund2afk6VgmN6mpvgkEgItU7K0EJxWV0qjHmLmPMessYAIAxptoYs8AYcz6AZ6MnotKd4mIq4IoK/8c0N9Pl7XcQfZiMGwfccQcnn5WX87Z7N5/nwgtpMKzvU04OF9id9XhbZj6yx4/CQcdQuIwNHU6Bw+7C+OzyLs/T1ESDkpXlucPp9BkcdbkivHM/4wyutnft6tJ1FcbQr3/oEHD99d5RbtHCbmfe7oEDvl9gTg6tcHa2/ziDRSL3Yko08vO7Xnd/GNPpw5l6BN0hGGM6RGQCgBEAPjTGfBklFJHTjTGLjTEhvJNKJLnoIuCuu7iDLSrqutNtaqKx+MEPIqu/Ro7kOWtquKCyujz4SmY56yxm7+3dy9kHNhsXvOPGMWbqcABnT6jFoOZ9aEwfDoD1Ym43O1Z8SVUVm7V5Hl+1ijVqe/fynFOncmcyaVIfMwKzsoAf/5g+4v/8hxYHoECHHQZcfDF/xoLZs9m8bulSXlzLndXeTpdRXh6LTPxtD+vr6UYaPz428qYCRx/Nax/IH9vUxN3b6NGxlS2GiAmyrRSRGwF8H8BmANMA3GSMednz2FpjzNFRl9IPZWVlZvXq1fF6+rhTXs7OCpar2EpFz89nxXC8B4nV1ABPPMEaKwur84TdDkzK2olzN/waNVkj0OZOQ0YGe9oVFnoO7uig5v/1r9EwaDTuu496ctAg6kRr8d7YyPjFJZdEqJ9bUxO3P04nhRk2LPb55243M1oWLfJaP4eD7q0jjmDRWnZ2z37j9fV843/8Y27nlNAwhm1SNm/myqc7Lhe/aFddBZx8cszFizQissYY00NDhGIQNgA4zhjTKCKlAF4A8KQx5gER+dgYMz0aAodCfzcIAD/HO3eywaXLRd01aVL0k2HCobLSK19hIXvYHTzIwltZ/AYmfPw08scMRNG4AjjSxFtdV10NXHopzLzTcd99TKHt3m0C4Hm/+AK48kr2UkoprPfC5WJQ09ry7djB1cCBA11XtYMGUWmpMQifhgb4XHVUVdE3evrp3CmmQHFaXwzCRmPM5E5/54BGYROAU4wx0yItbKioQUgBjOEW4qWXuCq32bg6Li0Fzj4bmDYN5eXAz3/Onbq/72JzMz0qv/99YhnDqOJ2M7V0927+PmIE/W3B8pEV/7S1eXtn7d3LD9y0aWzlO2FCShgDwL9BCOWrUyEi04wx6wDAs1M4E8A/AUQ5B09JeURY6DBtGrcSVjR88OAvv3zr1gVOzQf4LwcPcqcQK1d/3LHZ6D6KVNMqhTuwE07gze0OnoqaYoRiEC4H0KUBgjHGCeByEXk4KlIp/Q8R72SdbtTVhdaFQSR4AZ2ihEw/HDAUSpZReYDHVkRWHEXpycCBoWUEut2+U/eTjpoaNrBra6Mfe+LE4POlFSUC9Bdvq5LEHH00m/kFyghsbGTAOhKFeHGjuZkjPf/73641CJmZnAw3b16/cl8osUcNgpKQGMPEGrudhWqzZnFYmK9WM04n6y6uuy6Jd/ltbcxw2b6daY+dA8NtbcBTT9HqnX++GgUlaoRsEEREAFwKYKwx5k4RKQEw1BgTsX7AInI6gAcA2AH8wxjzu0idW0kO6uuZjvrGG8y2dDhYmzB7NvXhhg3MvszP5yL64EFmF11wAXD88fGWvg+sWEE30ZgxPRV+RgZTrF57jZbRV568okSAcHYIDwJwAzgFwJ0AGgAsADAjEoJ4Oqj+FcBpAMoBrBKRV4wxmyJxfiXxOXCAozhratieo7SUu4Q1a6gvL7mE2X9vvslsIrud/dxOOSVwSmrC43azAK1TZlUPHA7eli4FLr00tvIp/YZwDMKxxpijReRjADDG1IhICN2gQmYmgO3GmB0AICLPADgbrHdQUpyODhaKtrV17Qxgt7PYrr2d7vWf/MTbtTplaGxkEV5JSeDjCgpYSasoUSIcj2uHZxVvAEBEisEdQ6QYAWBPp7/LPfd1QUSuFZHVIrK6yhofqCQcLhdX+jU13rZAgfj0U5YhFBf7fjw9nQk3r70WWTkVRfESzg7hTwAWAhgsIncD+AaAn0dFqgAYYx4B8AjASuVYP78SmNZW4P336QGx5ifk5wPz5/seMWCxfHnwlNHCQg60qqvjOVOGnBy2Smhs7DnGszO1tZx3kAg0NbGid8MGWvxx4+i/+7IRVQRoaGDDLrebNSrBursqfSYkg+AJKC8DsAbAXAAC4BxjTCT3r3sBdO5UM9Jzn5IkNDXR7fPZZ/z+Wh6Qpibg3/9mltAPf+hb8dfXB+/MahWNtrSkmEGw2YCvfQ147DG+Ob7iCE4nFe9XvhJ7+bqzZg3wyCP07+XkUP5PPgEWLGB67Fln9S2gU1fHc63oVObkdnMWxYUXssWuEhVCchkZNjxaZIzZYoz5qzHmLxE2BgCwCsDhIjLGE5u4GMArEX4OJYo8/TQb7Y0d21XpDxjA+3buZPakLwYNCl5l7HYzHbUvxWetrRxt0NDQ+3NEheOPZ6+c3bt7+tja2thp88wz459htHEj8Oc/0yKXlnLVPmgQrf+IEcALLzBFrLfU1gK/+Q23jMOGsZuhdduyBbjzTmDPnuDnUXpFOC6jtSIywxizKhqCGGOcInIDgDfBtNN/GmMiMAuADNAAACAASURBVCJeiQU1NaynCjSTZcQIppRecEHPrs2zZtHN1NLCZJqiop7upaoqtjzKzQ1fvv37mZ20YgX1rTEcF/C1rwFTpiRAhlJGBnDzzbSqy5d7C9NEOKvhW9/yjqSLF8ZQvvx8No/qjsNBw7BwIXDSSb2z3M8+y1zi7gF2m40FKYcOAQ89xGEgSVt0kriElWUE4FIR2QWgCXQbGWNMhGZyAcaYRQAWRep8SuzYupU6LFCjTbudOmXrVhoAgP+zZAl1yJ49fCw7m3qwpITKOi2NhqKlhQo8XLZtYxdUY+jKcjj4+969wL33starr16OsLDmjhrDKLo1Azkri62rzzuPb0R7O63fxInRn9QWCrt3800LlA2VlsaUsY8/Bk48Mbzz19TQrxhoVVFYyN3S9u3a1C8KhGMQ5kVNiiTDGCqvigqv4go02rY/0N4e/rHG0FX8yiv0hMybxx1EfT31ys6d1BFjx9KYfO97jF2GQ1MT8MADdHVbehfgdSss5GJ3wQKed8qUEE/qcvHEDofvlbI/GhuZJvXee4wJWMyaRd+7lWJVUAAce2zo540VNTXB284CvHgHDoR/fmvSU7D23SK08moQIk7IBsEYsyuagiQLO3dyCtjOnd4gpzGcXfytb/lPm0x1OivbQIh4jy0vB15/na5oSwfMnk1d8vnn3krkuXPZlsJPM9SArFrFFkH+rovDQXkWLQrBINTVsTBsyRKe1BhO+znjDAY8AynKhgZW3e3Zw6CoNYDa5eKqeP164Kc/pd88UbG2VsFwuXrXjM9qNx0Mmy20XGYlbMJpXXGHr/uNMXdGTpzEZvt24He/486+c2Ws2816obvvBn72s/5pFCZO5GK5tdW/Lmht5Xs3cSL/XrqUi8nOC0KHg7sFK3ba0MDFeG/f0xUrgmckFRby+gXM+qys5MWvqfGmQBpD188f/kD//qWX+vdrP/cc3S3du+/Z7XyxlZX0jf/qVwkQ0PDDmDGUt6PDfz9yY/iFsC5yOBQXewM8gd4Dp5Nb8i1b+OHIzuYQjFB6pCsBCcdl1NTp90wAZ4JzlvsFLhfw8MNUGAUFXR+z2bjo27eP1bQ33RQfGeNJRgZd348/TmPZfWqZ00mPwDHHMLvS7WbsNFhqeW4uXdfNzYFT9P3R0hJcT4jwGvp1e7nd9Du1tHQto7b8TgMHctcwejS3ON2pqwsecS8u5htkpWklIgMGMFj89tv+28pWVfF96M1rGDmSRufgQf/1DM3NDCw/8QSzryyys5mF9dWvarC5D4TjMvpD579F5PdgRlC/YOtWftYDtVceOpTTvQ4e7J81NHPncpX98sv8TlqGs7aWC2mnkxXJVvLJxo3UtdOn++7pFgmGDgU2bQqc8NLRwYWv32O2bvW9urew2bhife01BlK7K6SdO7nqDeQbt/yPW7dGxiBYw7bffZd1A04nUzdPP52pWum97Dpz/vlsJPXZZ3RvWdvBjg76+nJyGOzpzcUUod/1N7+hFe/uh2xpoWEdMICPZWV5H2ttZU5zZSVw2WWJu8tKcPrS/jobLB7rF+zYEXzhYcXb9uwJbBA6Orjbra/n9/KII1Kj0EqEsdFjj+Xqf5OnC9XgwXytY8d2/Q6PG8dYwbp11JWdF98WDQ18L8OJ3XZmzhwW1AbyQlRUACefHCCR56OPgitQayuzb1/PWoHqan4oNm3iqjYjg2/GqFFd3xC7PbzovD+MoXFasIByFxfzw3noEPDXvzLu8T//07stV1YW8KMfAW+9xTzeqiqvMTvpJKaB9aVaedw44JZbgAcfpOFJT+e529poEHJygLKynlvQzEyuKt5+m49PmtR7Gfox4cQQNsDTxwisEygGcFc0hEpEgrk1Lawgs79zLF3K2p2mJu85RViAevHFXfVDsjJsGGsNAO4AbruNmVjdX9vo0TS0AwZw59B9DIAx1DdXXtl7L8CECdR/O3bw/N2vYV0dn/O00wKcpLExtBW1zdbVjQFwJf3oo1RuhYV8UYcOccuUnc1WrVaAxOmMTFD5ww8Zs+juuxs4kNu2HTtYaXzzzb1bSWdlMU/39NO5Ine7+doiNa5uwgTGZTZu5I7J5eLu7KOPuJLqbgwsbDYajLfeUoPQS8LZIZzZ6XcngArPbOV+wahRwRMbrKEuQ4f6fvy11/g9HT686w7C5QKWLaOO+NGPEiPlPFJ8/jldaL5W/3l5VNZbt/K9q6z06kOXi1lIEyb0bc6B3Q7ceCPwpz9RN2dlUW91dHDXkpPDBWnADKYhQ5hXHwjr4neumquupmIrLqby//xzCmSz8fjmZuD55+n7HjKEF/7IPpb1uN3ASy9xW+ZLcYrww7x+Pd/gUaN6HhMq6enRq5xOS6Nra9o0731PPBF891FYyP5Koa7glC6Es+663hizy3Pb66ksvidqkiUYkyfzu97U5P+YqioqMF+tVg4c4A7e10rZbuf9W7bQ1ZJKHDrk/3spwoXckUfSU7JjBxfSX3xBl/1XvsJFbF8NZG6ut2321Kk0CMOGsQbsnnuYoBKQY4/l6j1QymV1Na1b54KU5cv5whoaqPxtNiq69HS+KGtF/e67XA1ffHHfZyeXl9MHFqicW4QfutWr+/ZcscbtDr5VFOk6flQJi3B2CKcBuK3bffN93JeSpKVRgdx/Pxd83d2v1dVcIH7zm77/f/lyrz7whQh1yaJF9HunSqJEsNR1ESpky202YQL15YQJPbO5fFFdzZvDwSQef++v3U7j0ytPwsiR9EuvXu17Ek9LC7cb3/9+1/vffZd9fpYu5cXNy+M2sKOj6wVuaOAH6qSTeiFcNyzDE4yMDL5xycS4cTR4gXYJdXXRy1DoBwQ1CCLyPQDXAxgrIus7PZQL4L/REiwRmT6dK9bHHuMq1mbzLkaGD2cnT39V/Z9+2rN/T3dycph5WF8fmjJMBqyEGWs+si+sBJwzzgjdA7F7N9tdfPKJ9zpkZ9Ot/dWv9j6JxiciwNVXU5GvW8cnys31Dn2w2ZhZM2FC1/+rr6el6ujgbiAtjT+t4grAuyNIT+fFHzOmb7JmZ4e2Qm5vD/6BTDTmzWM73UGDfCt8Y3g9Lrss9rKlCKHsEP4PwBsAfgvgJ53ubzDGJNkSo+9Mm0a38KZNTCgRodIbNy4yq/pUW9gMGsTODB995N9dXVFBXRooTb8z27ax6DctjQbEet9bWxmj2bKFcYOIGoWsLBaYbNvGlf+uXVS+c+b4nwOQl0ffV+eLarMxpcxKK+vooGGx2Rhs6atBGDmSW9iGBv9uI2PoAisr69tzxZopU7gq++QTrrw6f+HcbmZyTZ3KqnGlVwQ1CMaYOgB1AC4RkYEADgcL0yAiMMYsi66IiYfDQb93OPG/CROYERfItdvURB3Sm26eicwll1B/7tpF372lqDs66EEpKAC+853QjGFbG7sv5+X1TFO3Mg/Xr+d7fcYZEX4hNhsvZPedgD9OOYVpnoF8Zs3NPF+wOoUQaGgAVq2yocl+DiYu/xuyJmRhyHBHz9jy3r1Urn0JKMcDh4M7saee8gbb7HZvtsesWcDll2vFch8IJ+30OwBuAmsP1gGYBeADAKdER7TUYvZsYPFiLsz8Zc1VVLAup496IeHIywNuv53xkXff9fZ1E2H+/5lncicRCuvXU/H5qxETodF54w2mksZVN5xwAvDMM1y5+sp6aWvjxR4xgtH3QFWPATCGmZbPPUfdmJV5HMpzKzB12UvYkZGJ8ScWY/BQO/3rVrfA7343ObejmZl0351zDt13dXX8gB11VP/sGRNhwgkq3wRgBoCVxpg5IjIBwG+iI1bqMXw48PWvMyNw5MiumTNuN2NlY8cmxkCsaJCbC1x0EdPXKyu9nZ/DTV3/+OPgRWpZWcz4OnAgzovgwkLgl79kj6OKCv7tcNAiNjfTGBx3HEu5Z8wI3Sp24513mJFZUmLtvgQVg89BxxFTMHzzO3D/Zw0ypriQP2UUM5mmT0/+3ObCwsQZJ5pChGMQWo0xrSICEckwxmwRkfFRkywFOfdcKrOXXmJMz/IkiFAfXH554hemNTQwHvDppzRkhx/OOoFQdZnVGLC3dE/Q8YfN1rXDdNwYP54Nnm65hZFwh4OrXCs/uaaGRSn+0tOC0NzMncGoUd1iJiKoLjwc1ScejkOHgKVFBr+4Q5JyU6DEjnAMQrmIFAB4CcBbIlIDQFtih4HNxmHzc+Yw7byujgu18eOTo/fRihXAv/5FRZuTQ0O2YQPw4otscXPGGdH3QowaxdY8gXC5aKx6ueCOPJMnMzVtwQL247YmBdXV0Zd47rm97l3yySc0koEW/IMGATu/EOzdy6detYq7tJwcNhuMVEKEkvyE09zuXM+vvxKR9wDkA1gcFalSnMxMfhGTibVr2e11+PCutVODBlEhPfMMldKpp0ZXjuOOY7ppoDTWigruuBKqP9SQIcD113NHYE1WGj68zxkEe/f6j0lZWLVaDz/McIbNxmvY3s64VmkpcMMNybEoUaJLyOsCId8SkTuMMUvBwPK0YP+nJD9uN0fpFhf7LqS10j+ff56pn5GkqYktJ7Zt44K6uJh1Bl984buVSG0tFeBZZ0VWjogxcCDdRePHRySdLCMjtJYqW7dyNzF6tHfC38iRNAYVFazYrq/vszhKkhOOy+hBAG4wq+hOAA0AFoCBZiWFCdSPyCIjg8bg008jk95eX8/RmkuXUuFZTQNnzfLOP37zTe8MepeLzz9wIJvphVrTkOyMHx+8Dq2ykklMJ5/s26U3ZAgN7LJlzPhS+i/hGIRjjTFHi8jHAGCMqRGRSJb+KAlKTU1ox9ntzO7pK3V1HE5WUcEUUit11OWi/3vDBk6mO/VUYOVKuk0yMlg0OHlylFJNnU5WI65fz3TRUaPol4pzte9hh9H4BZrBsW4ddwOB2iQNGUIDO39+6qU9K6ETjkHoEBE7PC2wRaQY3DEoKY7DEVqw2O2OTHXwM8/QsHRvA2Kl7FdW0h9+xx0xWtHu3An85S/s/ZOeTkGWL6egZ57JnPg4RWWtrhm/+Q0r54cM8Sr09nYW/tlswYt3rVTd5ubUK4xUQiccg/AnAAsBDBaRuwF8A8DPoyKVklCMG0eDEKwfkTF0YfSFmhqu+gP1NCoupo7etavXtVyhs28ftyuZmT19Zk4nI9wA54fGiZEjaRwXLuQOyuqubbez/U9xcXC3knX9dHfQvwmlud2TxpjLABQBuBXAXAAC4BxjTL+Zqdyfyc9nrcGKFf6b9x04wLnqffXd79zJn8GmTQIMNEfdILz0En/6cg05HDQSr71GB30c81yHDuVO4ZJLuNK35nxnZfH28suBB6RZBcwh1cE0NzPt7N13GcXPy+PrnzEjckNylLgQyj73GBEZDuAqABUAngYb3lWISKJkeitR5uKL6TbfubNrJlF7O1fqubnANdf0vQ4hWMaMRUwKz2pr2fI60PQcK+dz5cooCxMaBQUsFhw3zqvcTziB18VfBpjbTYMwf34I12/vXuDnPwf+8Q+vC62ujsV3P/0pPwxK0hKKQXgIwDsAJgBYA2C152b9rvQDBgxg9s5557Faec8ettuoqmIg97jjqDu3bOnbfBIrMBqoHxzA5/A3mS5iWNN9gsUHsrISWhEWFQFXXEHvV01N1/e2qYlGfs4c4Oijg5yosRH4/e+5CigtpfXJzOQWcvRonvjee2lIlaQklG6nfwLwJxH5mzHme9EQQkQuAPArABMBzDTGqKFJQLKzgbPP5kqyqor1AS++yFTTjRt5jNWj6NpruVINl9JSZhbV1fmfCdHSQlmmTOn1SwmNUB3qbnfCd9j8ylf4fr74oneWhzHU5VdcEeJQpo8+orL3l388aBDbc6xYAXztaxF+BUosCCWGIIb4NQbWMX2Q41MA5wF4uA/nUGJEejoXi48/TiXT3Y9fW8s47O23hzCeshsi7PhqzTvo7pJua2PmzDXXRHjegS+ssuzW1sA5my0tXWf/JihTp9KI7t/PXV5GBt2AIQeSlywJHicpLmbr1VAMwr59dLXt2cP3t6yMQkb9wir+CCXL6D0RWQDgZWPMbutOTw3CiQC+DeA9AP/qrRBWcFq081ZS4HYDjz5KY+CrPURBAY/517+Au+4KP64waRIn0z38MPPrMzK8PnCHg6NMZ85kRs3771O5DRzICZSTJwdv5RAy6elM01mwgFbP1wupr2dQNZzhGHHE6pjRK2pqus6M9kVmJgtIAs0/7ugAnnySF89mo9V3Omkc8vM53cgatafElFC+OqeDAeWnRWQMgFpwQI4dwBIA9xtjPo6eiF0RkWsBXAsAJf5SXpSosn07XUaBKpcHDqT3YOfO3n23jzoKuO8+FlVt3kz3xtixXETW1TGuWVXFzJmMDOqgtWuZgnnzzYHH7obFvHksSNu8mZrU6iLndtNatbcDt97aP1a1Awbw9QZKRerooFHwZwyMYa/upUtpZLsfV1vLPhq/+hV9h0pMCSWG0Aq2rXhQRNLA9NMWY0xYkSMReRuArzDgz4wxL4d6HmPMIwAeAYCysrK+uKmUXrJ/f/Cgr7WY3r+/94u9jAzg2GN5s6ivpzupo6Orq8oqptq3j7rkqqt4n3dGQC/JyKCFWbSIrpDWVioxl4v+l298o2/9vJOJk05i/mqghVhlZeA5BXv3cmfgyxgA3F42NzOV95pr+iyyEh5hba6NMR0A9vfmiYwxUe6DqcSKcFxAkfYCrljBHYIvHVxVxdqE8nLuTAYP5mJ2/nwu9Hsd983IYIvqM87gyZ1ObkH6W3vQE04AXn+dASRfRQ3NzVwpnHyy/3OsWEGfXqAI9pAhdB9dckng4gkl4kTK25o0VFdz1QrQAxDnVjRJiTWFzNdUSAur8jWSE8uMYb8dX5MS9+5lEkxmJnV1YyPdS62tHCCzYwdw/QVVcHz0X7b9dLmYBjV7NoUMxXJlZDDBv79SVAT84AfAAw/QKg8eTCvrdNJn53IB3/9+YFfPnj3Bi9esKHdtrRqEGJMQBkFEzgXwZwDFAF4XkXXGmHmRfI6KCrZnXrvW+903hsWVF1zQ/xZ7faG0lCv0Q4f8v29VVdS3gVpQhIvLRR3RPauprY1Dc3JyqJ9cLubXAzQQY0oNml99F1VvPEVdVVDAD8HSpcDbb7NL3iWXaN+GUJg6FbjzTs7tXLaMxsBu5+7h1FODrwBC6dcNMEYTsewAJVR69Y6LyFBjzIFICWGMWQj2SYoKBw4Ad99NxTFypHe36nLRQGzdyu6ZOqM7NEQ45/zuu+kyLi72Gtn2dhantbQAJ57IwHJJSWRcR3a7dyRxZ11RXt61FKB7WcCIfR/h6L3/wua8URg6M90rS24uPwRvvknf0vnn913IRMUYrs63bOG2afBgZkYFG1Dti+HDgcsu49jP1lYq+VCV94wZtN6BVmCNjdzmBctoUiJOb03wIgDB6hoTAmOYIul09ky3s7pn7t/PnPof/zg+MiYjJSXM9HnySfrtAb6PO3dSzx5xBNPWlyxhLcK11/b9+y3CiugPP+x6Lffv7zpCsrmZzw8AYtyYvOl5tBcMQUNL+pdFbV9it3O7s2gRJ++kYqvPgweBRx7hhbIqr10uRtvPO48Blt5YbLs9/N5F06ZxK2el63bHGG7nr7pK53rGgd6+40lTMFBezoraQMpo6FBW2x6I2J6nfzBqFIvP7r6b33Oru+bXv86up5Zrqbwc+O1vGb/pK3PnchfS3u69z+326rOODv5uJcIMrNmB7OZDaE+nL9pndpTDQQX5ySd9FzDRqK3lm797Ny/G6NG8cKWlXKU/9RTw6quxkyczk3UG9fX8wnXuc9LUxGDP8ccztqPEnN4ahL9HVIoosnNn4OAnwMdEvJ02ldAR4Yp7/Xpg+vSe+f8iNLh1dUxQCURzM/XWnj1dFX5nSkuBb3+bRqaykvpk4EC6AxsbqVPKyrwL14zWOhgRuFxccPodRu9wRGa6T6KxeDEt8dChPb8E6ek0EAsXchcRK444AvjlLxmPKC/nBd+9m0b5yiu5ndR4TlzorcvoxYhKEUWs8Yuh0JembP0Zq9FnoLTOYcMYgzz//J5u6+pqpp2//773GmRkAKedxh1H9+PnzKHL6PXXOT0tLY3G4PDDeeucOeaypwPGoLGRriu/rm6Xq3f+9ESmtRV4773AWT9WCugHH3BrFytGjWJGUl0ddzEOB42WGoK4kvIxhECdiztjjMawestnnwV3JaeleYt7O9c1VVZy2ldDA/WBZVRaWzmKYP16xna6n3/8eN4aG7mzWLiQKe7d3dLVgw5Dc3sasgZ0YOxYPxbLypGNere8GHPoEINnwSrzcnJ4EeNBfr7v/idKXEj5GML48VwxNjb6P6aujoajP6eY9wW7PXjlskXn3ZoxwEMPUfmPGtV1h5GZCYwZw67Szz/v/3w5OTTkV18NnH466xF276Z7urwc2L43C+UTT8VXxpQjK9OPkPv2sQlSr5v8JCjhbI21j5iCfhBDsNuByy/3zovtTmMje3ZdfrkmNfSWyZO9ef/+aGujG6jzLmzXLsYQA+3ihg/3NrALhMPBLMh77wUuugiYNYvupp/+FLjwqXOQNesoBok6n6ilhb2gi4oiM90n0SguZjqtv8k4Fo2NSdOcT4kuvXIZGWMejLQg0WT6dCY2PPooXRTWDrq9nSvMm2+mUlN6x4wZwP/9H/Wrv75n+/dzlkLnoO6WLd6Avj8cDu4kduwIPigeYFB7Xo+SxnRW2P73v0wv3e1p2jtgAHsRnXxyalbEpqVx2/T889xu+cJqITtzZmxlUxKSflMKeMwxdBGvX0/lAjAAOWVK/2hUGU0GDAC+8x3gr39lu/zOLmGnkx6Z0aN7Kuq2ttB3ZX0el5mWxuZsX/kKUx6NYc1BqlfDzp3L6sudO1l00/n11tczqHPddalZf6GETYp/G7qSkcHV7IwZ8ZYk9Zg5k37/p5+mK8ha9YuwYvmii3om8QwZEryLgRXvjdj8epvN/yi2VCQri1H5F15gmpeVdmcML8DNN3MLrSgIbWJaKF9Fd7jtsJXQsDoObN7MFXVxMV0niZgheeSRTC3fuZOppA4HW1/7Kki1jk9Pp+vO3y6tvp5Zk937FylhkJ3NINm553J73NHBTIsxYzRwpnQhlB3CPs8tUMTNDkCn1USYgwc5Neyzz7p2HEhLY8eB009PvDioCI1AKDMQsrPpwn/ySaaidq9jaGmhYfnxjxPvdSYlubmhBWJiSGOjN5anrtv4E4pB2GyMCbinFJGYTUzrL1gdB5qa6H/vrBDb2xnEbWsDzjknfjJGgtNO4+t50VPqOGAAd0WNjVQQN9zAXYeSOhjDWN7rr3OxY7MxG/Ckk/h50Hqg+CEmSAK5iGR6pqb16ZhoUFZWZlavXh3rp40JzzzDJpz+hnE5ncy5v/fe1GjdXV3Niuft26kcpkzp2oJCSQ2MofF/+WUmHwwaxMVORwd72jkcwG23+U+KUiKDiKwxxpR1vz/UEZr+TnqlMeaxeBiDVCbUjgMizKQ866zYyRYtBg3iQDIltfn4Y1agl5Z27VKRlsbW9LW1wP33c6xyZmbcxOy39DWi9OuISKF0IdSOA7m58es4oCjhYgx7Vg0c6L9lUUEBOwesWxdb2RQSSpbRen8PAQixU5ASDqEmfhijSSJK8lBbywy0kiDpJ7m53PnOmhUbuZKKxkbOtWhvp8/t8MMjWksTypmGAJgHoKbb/QLgvxGTRPmSoiJm4ASq/AXYhUE7DijJQns7FzDBMsbS0oK3Qul3tLay4nzp0q61JHl5TNU78cSIpOKFYhBeA5BjjOmxiROR//RZAqUHaWms6tWOA0oqYRVDu1yBu1w3NQETJ8ZGpqSgvR144AH2eulebd7cDPz971wdRiAIF9ThYIy52hiz3M9j3+yzBIpPTj2Vu8Fdu3q2baivZzuIK6/UjgNK8pCdTTdQRYX/Y4xhOrUOTOvEihXAxo30tXV3D2Vns1Xw888HfmNDRD3QCUpmJvDDH3IYzIED7Me2ezcNRHo6Ow4cf3y8pVSU8Jg/n922fXWvtaryJ07kYkgB36zXX2dxhj+XUFoafXErVvT56UIJKq81xgQchhPKMUr4WB0HzjsP+Pxz7hQKCrTjgJK8jBzJyvMHHmA2XUEBF71NTfR+TJzIQWr6+fZQX88inWCR+IICjg8877w+PV0oMYSJATKNAAaXdeRRFMnJSbiOA0osaWqiYkhLY3/vJO/jMWECCypXrWI2UWsrW53MmcNxyzpFsxOhTp4CIjIDOBSDMCGEY4L0rFQUJWz27wfeeINa0xh+4YcNA848k874JF5G5+TQAMyZE29JEpy8PAYKm5sDd7Ssq2Npfx8JpVJ5V+e/ReROz/+tA7DOGLOtz1IoitKVHTtYrut2c9i0NSmovh7429+YcXLFFUltFJQQsNvZxfKZZ/ynHLpc9CdHIBIf9qfJGHMHgAcA1AE4V0SSZpymoiQFbW3s35CV1TXNUITFSGPGsLfJBx/EV04lNpx0EmMI5eU93UIdHRwDO39+RGaCh2wQROQBETovjTEVxpg3jTH3GGOu6asQInKviGwRkfUislBE+tEEE0XpxiefcCfgb5CPzcbBGK++Gp6PWUlOsrOBW29lFeru3TQA1s+qKuDCC4ELLohZYZpFA4BXRORiY0yTiMwDcIcx5oQ+SwG8BeB2Y4xTRO4BcDuA2yJwXkVJPj78MPiM59xcKoWqKu0X3R/IzeVg+IoKTstqbWVHyClTIjotK2SDYIz5uYh8E8B/RKQdQCOAn0RCCGPMkk5/rgTwjUicV1GSEqsMPRDWxKT29tjIpCQGQ4bwFiXCcRnNBXANgCYARQBuNMa8HwWZrgLwRgA5rhWR1SKyuqqqKgpPryhxZsQINjELhDWMOl8zvpXIEU5Q+WcAfmGMORlcwT8rIqeE+s8i8raIfOrjdnanY34GWx6JlAAADjtJREFUwAngKX/nMcY8YowpM8aUFRcXhyG+oiQJJ5zArJFA8YGKCuDYY7V3iRJRwnEZndLp9w0iMh/AAgAhNVAwxpwa6HERuQLAmQDmmmBj3BQllSkpYdfCDz/kJJnuwcL6ehoLnSikRJheN9I2xuz3uJH6jIicDuBWACcZY5ojcU5FSVpEgKuvptJftYoVyjk53DU0NHCu6C23sA+EokSQPk1WMMa0REiOvwDIAPCWJ7N1pTHmugidW1GSj4wM4Prr2c3w/fc5QDsjgzuHo48OPChDUXpJ5Ebt9AFjzGHxlkFREg4RuoxKS+MtidJP0Lp3RVEUBYAaBEVRFMVDQriMFCVROXCALvyNGxnjnTCBPcSGD0/6LtSK0gM1CIriA2M4qGrBAir+gQN5/zvvAG++yYzPb3xDm40qqYUaBEXxwbJlwLPPsiQgLc17f24ui4RffZWZoFoKoKQSur5RlG50dAAvvEC3UGdjYGG3c675yy8DLZFKvFaUBEANgqJ0Y9s2thIKlOqfns6xBRs3xk4uRYk26jJSlG7U1YV2nAhQUxNdWRKGtjZg3ToGUaqr6S876SRgxozgrbqVpEENghJzqqqArVupYwoKgMmTgczMeEvlJT09tOOM6ScFw5WVwO9/z5/5+bxYdXXA448z6v6jH/kf76gkFWoQlJjR0AA88QSwejUnAYrwlp4OnHsuMG9eYqRyHnEEs4ecTv9jCSz5J0yIrWwxp6WFxqChoWvFdFYWjUNtLXDvvcBddwGFhXETU4kMGkNQYkJzM/XK2rUMyI4ZQ/0yejQHPz31FLBwYbylJHl5rDUoL/ffgXrvXqCsDCgqiq1sMWftWu4M/A1lKSjgQJ9ly2IrlxIV1CAoMWHpUo6AHTWqZ+5+RgYNwyuvsBAsEbjoIq7+d+7sOqumuRnYsYOv49vfjp98MeOdd/zPdrYYMgR46y2d75wCqEFQoo7TCSxeDAwd6v8Yh4PpnO9HYwZfL8jMBG6+GbjiCv69ezdv7e3At74F3HZbP4mlVlUFn9mbkUHXko7zTHo0hqBEnbo6uqCtal9/5OcDW7bERqZQyMgATjkFOPlkzqQBWJhmt8dVrNiSk0NFHyjS7nTyTfFVtKEkFbpDUKJOIgSK+4LNRq9JQUE/MwYAgymHDgU+prISOP547eORAugVVKJOfj5vTU2Bj6utZQqqkkAcdxy3Sg0Nvh9vbWVp96kBJ+QqSYIaBCXq2O3A/PmcC+8v7tjRwcdOPDG2silBKCgAbrqJPrPycl4ogA2d9u1jFsB3vsOmT0rSozEEJSbMns3xwNu3cxRwZ9dLSwuwfz8zewYPjp+Mih8mTgTuvBN4912mizmd9AMeeyxw2mlalJZCiEniVLGysjKzevXqeIuhhEhzM/DMM8CKFSzsssjJYSvp2bOTP96Q8jiddBNlZGgQOYkRkTXGmLLu9+sOQYkZ2dnAVVcB55/PBnLt7YwtjB+vuiVpcDj6Sb5t/0QNghJz8vPZE01RlMRCg8qKoigKADUIiqIoigc1CIqiKAoANQiKoiiKBzUIiqIoCoAEMQgicpeIrBeRdSKyRESGx1smRVGU/kZCGAQA9xpjjjTGTAPwGoA74i2QoihKfyMhDIIxpr7TnwMAJG/5tKIoSpKSMIVpInI3gMsB1AGYE+C4awFcCwAl2lBLURQlYsSsl5GIvA3A18ysnxljXu503O0AMo0xvwx2Tu1lpCiKEj5x72VkjAm1YfpTABYBCGoQFEVRlMiREDEEETm8059nA0igQYqKoij9g0SJIfxORMYDcAPYBeC6OMujKIrS70gIg2CMOT/eMiiKovR3EsJlpCiKosQfNQiKoigKADUIiqIoigc1CIqiKAoANQiKoiiKBzUIiqIoCgA1CIqiKIoHNQiKoigKADUIiqIoioeEqFRWoo/TCezYATQ1AVlZwLhxQFpavKVSFCWRUIOQ4hgDLF0KvPQSUFcHiPC+7GzgzDOBr34VsNvjLaWiKImAGoQUxhjgxRdpDIYNA0aP9j7W2go8/TRw4ADw7W8DNnUeKkq/R9VACrNzJ/DqqzQE2dldH8vMBEpLgffeAzZujIt4iqIkGGoQUpj33gPS0wGHn32gzQbk5gJvvhlbuRRFSUzUIKQw69YBhYWBjykq4g7B7Y6NTIqiJC5qEFIYtzu02IAxvCmK0r9Rg5DCjBsH1NYGPqa+Hhg1SjONFEVRg5DSnHYa0NDgf/VvDHDoEDB/fmzlUhQlMVGDkMJMmgSUlQG7dvWMERgD7NkDjB/PYxRFUbQOIYWx24Hvfhd46ing/fd5n8PBqmVjaAiuvJKZSIqiKGoQUpyMDOCqq4Czz2bWUU0NkJcHHHUUMGRIvKVTFCWRUIPQTygsBObOjbcUiqIkMhpDUBRFUQCoQVAURVE8qEFQFEVRAABikrhEVUSqAOyK4lMUATgYxfNHk2SVXeWOPckqe7LKDcRf9tHGmOLudya1QYg2IrLaGJOUWfrJKrvKHXuSVfZklRtIXNnVZaQoiqIAUIOgKIqieFCDEJhH4i1AH0hW2VXu2JOssier3ECCyq4xBEVRFAWA7hAURVEUD2oQFEVRFABqEIIiIneJyHoRWSciS0RkeLxlCgURuVdEtnhkXygiBfGWKVRE5AIR2SgibhFJuNS87ojI6SKyVUS2i8hP4i1PqIjIP0WkUkQ+jbcs4SAio0TkPRHZ5Pmc3BRvmUJBRDJF5CMR+cQj96/jLVN3NIYQBBHJM8bUe36/EcAkY8x1cRYrKCLyVQDvGmOcInIPABhjbouzWCEhIhMBuAE8DODHxpjVcRbJLyJiB7ANwGkAygGsAnCJMWZTXAULARGZDaARwBPGmCnxlidURGQYgGHGmLUikgtgDYBzEv09FxEBMMAY0ygiaQCWA7jJGLMyzqJ9ie4QgmAZAw8DACSFBTXGLDHGOD1/rgQwMp7yhIMxZrMxZmu85QiRmQC2G2N2GGPaATwD4Ow4yxQSxphlAKrjLUe4GGP2G2PWen5vALAZwIj4ShUcQxo9f6Z5bgmlT9QghICI3C0iewBcCuCOeMvTC64C8Ea8hUhRRgDY0+nvciSBckoVRKQUwHQAH8ZXktAQEbuIrANQCeAtY0xCya0GAYCIvC0in/q4nQ0AxpifGWNGAXgKwA3xldZLMLk9x/wMgBOUPWEIRXZFCYSI5ABYAOB/uu3kExZjjMsYMw3csc8UkYRy1emAHADGmFNDPPQpAIsA/DKK4oRMMLlF5AoAZwKYaxIsWBTGe57o7AUwqtPfIz33KVHE44NfAOApY8yL8ZYnXIwxtSLyHoDTASRMUF93CEEQkcM7/Xk2gC3xkiUcROR0ALcCOMsY0xxveVKYVQAOF5ExIpIO4GIAr8RZppTGE5x9FMBmY8wf4y1PqIhIsZXtJyJZYCJCQukTzTIKgogsADAezHrZBeA6Y0zCrwBFZDuADACHPHetTIbsKAAQkXMB/BlAMYBaAOuMMfPiK5V/ROQMAPcDsAP4pzHm7jiLFBIi8jSAk8FWzBUAfmmMeTSuQoWAiJwI4H0AG8DvJQD81BizKH5SBUdEjgTwOPg5sQF4zhhzZ3yl6ooaBEVRFAWAuowURVEUD2oQFEVRFABqEBRFURQPahAURVEUAGoQFEVRFA9qEBRFURQAahCUFEFESkWkxdMnxrqvR1tqEcnytDJvF5GiPj5nlogs9XQ8hYjcKCKbRSSsNiEiUiAi1/dFlhCeo0eraxFJF5FlIqIdCxQAahCU1OJzT58Yqy31XwHMBzAJwCUiMskY0+I5Zl8Enu8qAC8aY1yev68HcJox5tIwz1Pg+d+wEBLqd/hfYJuEL/F0Z30HwEXhPreSmqhBUJIGz1CU0zy//z8R+XOAw2PRlvpSAC975HkIwFgAb4jIzSLyLc8wlHUi8nCnXcRLIrLGMyDlWs95fgdgnOfYez27nc4r+R+LyK88v5d6dj1PgD1wRvl7rs4EaHX9kud1KIoaBCWp+CWAn4nIpWDL4/8JcGxU21J7+haNNcZ8AQCetiD7AMwBsBhcdZ/g2Y244FW6VxljjgFQBuBGESkE8BN4djfGmFtCePrDATxojJkMIDvAc4XCpwBmhHG8ksKo71BJGowxyzyNzX4I4GTLVSMid4HNzmJJEdhnyRdzARwDYBXFRRbY/x6gETjX8/soULkfCPO5d3WashXouYJijHF54im5nmEzSj9GDYKSNIjIVADDAByylJeIDAUnT3Un7LbUIvJ9ANd4/jwDwLmd/zbGdI47tADI9HcqAI8bY27vdv6TAZwK4DhjTLOI/MfPOZzounvvfkxTsOcKkwwArX34fyVFUJeRkhQI5+g+BcYBGj3tvQFgGoB1Pv4l7LbUxpi/etw204wx+7r/3e3YGgB2EfGl0N8B8A0RGeyRfZCIjAaQD6DGYwwmAJjlOb4BQG6n/68AMFhECkUkA5xp4Q9/zxUSHpfVQWNMR6j/o6QuahCUhEdEsgG8COBHxpjNAO6Cd0iRT4PgmSd9A4A3wZm7zxljNkZYtCUATvTx3JsA/BzAEhFZD+AtcGezGIBDRDaDgeSVnuMPAVjhmRh3r0c53wngI8//+u2ZH+C5uuBpdf0BgPEiUi4iV3semgPg9d68eCX10PbXSlIjIo+Cbp0SAK8ZY0IaSSgiXwAoM8Yc7MNzHw3gZmPMZb09R7wRkRcB/MQYsy3esijxR3cISlJjjLnaGOMGs2vyOxem+cIqTAPjDu5Ax4bw3GsBvOcrzTMZ8LjSXlJjoFjoDkFRFEUBoDsERVEUxYMaBEVRFAWAGgRFURTFgxoERVEUBYAaBEVRFMWDGgRFURQFgBoERVEUxYMaBEVRFAUA8P8BBYGhJSJXix0AAAAASUVORK5CYII=\n",
|
||
"text/plain": [
|
||
"<Figure size 432x288 with 1 Axes>"
|
||
]
|
||
},
|
||
"metadata": {
|
||
"tags": [],
|
||
"needs_background": "light"
|
||
}
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "SjPlpf2-wHl8"
|
||
},
|
||
"source": [
|
||
"## Обучение простейшего одноуровневого персептрона вручную\r\n",
|
||
"\r\n",
|
||
"Используем возможности tensorflow по вычислению градиента для обучения одноуровневого персептрона.\r\n",
|
||
"\r\n",
|
||
"Для начала, задаём архитектуру сети, в которой будет 2 входа и один выход. Соответственно, матрица весов $W$ будет иметь размерность $2\\times1$, а вектор сдвига $b$ -- $1$.\r\n",
|
||
"\r\n",
|
||
"Функция обучение будет такая же, как в прошлом примере, но функция ошибки будет представлять собой логистическую функцию ошибки. Для этого нам нужно получить на выходе сети значение **вероятности** класса 1, т.е. необходимо привести выход сети $z$ к диапазону [0,1] с помощью передаточной функции `sigmoid`: $p=\\sigma(z)$.\r\n",
|
||
"Далее, если для примера с номером класса $y_i\\in\\{0,1\\}$ был получен выход сети $p_i$, то ошибка вычисляется как $\\mathcal{L_i}=-(y_i\\log p_i + (1-y_i)log(1-p_i))$. \r\n",
|
||
"\r\n",
|
||
"В Tensorflow оба эти этапа (применение сигмоиды и взятие логистической функции ошибки) делается одним вызовом `sigmoid_cross_entropy_with_logits`. Поскольку мы делаем обучение по минибатчам, то необходимо усреднить ошибку по всем компонентам минибатча с помощью `reduce_mean`. "
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"trusted": true,
|
||
"id": "kdDxWeCqwHl8"
|
||
},
|
||
"source": [
|
||
"W = tf.Variable(tf.random.normal(shape=(2,1)))\n",
|
||
"b = tf.Variable(tf.zeros(shape=(1,),dtype=tf.float32))\n",
|
||
"\n",
|
||
"learning_rate = 0.1\n",
|
||
"\n",
|
||
"@tf.function\n",
|
||
"def train_on_batch(x, y):\n",
|
||
" with tf.GradientTape() as tape:\n",
|
||
" z = tf.matmul(x, W) + b\n",
|
||
" loss = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(labels=y,logits=z))\n",
|
||
" dloss_dw, dloss_db = tape.gradient(loss, [W, b])\n",
|
||
" W.assign_sub(learning_rate * dloss_dw)\n",
|
||
" b.assign_sub(learning_rate * dloss_db)\n",
|
||
" return loss"
|
||
],
|
||
"execution_count": 22,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "zAAgw0h6KzUd"
|
||
},
|
||
"source": [
|
||
"Далее, разбиваем входные данные на минибатчи по 16 элементов, и по-очереди проводим обучение, подстраивая веса $W$ и $b$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"trusted": true,
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/"
|
||
},
|
||
"id": "PfyqjVb2wHl8",
|
||
"outputId": "308850b8-fe17-4cda-ac27-8bcda210f113"
|
||
},
|
||
"source": [
|
||
"# Create a tf.data.Dataset object for easy batched iteration\n",
|
||
"dataset = tf.data.Dataset.from_tensor_slices((train_x, train_labels.astype(np.float32)))\n",
|
||
"dataset = dataset.batch(16)\n",
|
||
"\n",
|
||
"for epoch in range(15):\n",
|
||
" for step, (x, y) in enumerate(dataset):\n",
|
||
" loss = train_on_batch(tf.reshape(x,(-1,2)), tf.reshape(y,(-1,1)))\n",
|
||
" print('Epoch %d: last batch loss = %.4f' % (epoch, float(loss)))"
|
||
],
|
||
"execution_count": 25,
|
||
"outputs": [
|
||
{
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Epoch 0: last batch loss = 0.5500\n",
|
||
"Epoch 1: last batch loss = 0.5502\n",
|
||
"Epoch 2: last batch loss = 0.5504\n",
|
||
"Epoch 3: last batch loss = 0.5506\n",
|
||
"Epoch 4: last batch loss = 0.5508\n",
|
||
"Epoch 5: last batch loss = 0.5510\n",
|
||
"Epoch 6: last batch loss = 0.5513\n",
|
||
"Epoch 7: last batch loss = 0.5515\n",
|
||
"Epoch 8: last batch loss = 0.5516\n",
|
||
"Epoch 9: last batch loss = 0.5518\n",
|
||
"Epoch 10: last batch loss = 0.5520\n",
|
||
"Epoch 11: last batch loss = 0.5522\n",
|
||
"Epoch 12: last batch loss = 0.5524\n",
|
||
"Epoch 13: last batch loss = 0.5525\n",
|
||
"Epoch 14: last batch loss = 0.5527\n"
|
||
],
|
||
"name": "stdout"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "s4_Atvn5K4K9"
|
||
},
|
||
"source": [
|
||
"Для демонстрации того, как сработало обучение, построим граничную прямую $W\\times x + b = 0.5$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"trusted": true,
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/",
|
||
"height": 283
|
||
},
|
||
"id": "PgRTHttLwHl9",
|
||
"outputId": "e4407e1b-edf5-48e5-fdc2-da28120a3c6b"
|
||
},
|
||
"source": [
|
||
"plot_dataset(train_x,train_labels,W.numpy(),b.numpy())"
|
||
],
|
||
"execution_count": 26,
|
||
"outputs": [
|
||
{
|
||
"output_type": "display_data",
|
||
"data": {
|
||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEKCAYAAAASByJ7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydd3hU1dbG3z2T3gtJgJCQQhFpAlFERJqCYPeq2CufBa9YsNBROna59nLtF70qliuo9A7SkY4ISYA0SkhPJpPZ3x8v46RMz7Qk+/c884hzzpyzZjKz115dSCmhUCgUCoXG2wIoFAqFwjdQCkGhUCgUAJRCUCgUCsU5lEJQKBQKBQClEBQKhUJxDqUQFAqFQgEA8PO2AI2hVatWMiUlxdtiKDzIieITyCvNw/lx5yPYP9jb4igUTZJt27adklLG1X++SSuElJQUbN261dtiKDxIYUUh0uanITUpFT/f/rO3xVEomiRCiCxzzzdphaBoeUQHR2N8//EYv3w81matxYD2A7wtUtPm1Clg40bgyBFAqwW6dwcyMoDQUG9L5hxSAllZwOrVwMGDgEYD9OwJDBgAtG7tbel8HtGUK5UzMjKkshBaHuXV5egwvwPSotOw9r61EEJ4W6Smh8EA/Pgj8NNPgBBAWBifKysDAgKAhx4Cevf2tpSOUVMDfPEFsHIl4O8PREVRQRQW8r3dfDMwYgTfbwtHCLFNSplR/3kVVFa4jaoq4PffgU8+AT7+mJu2srLGXzfEPwTTBk7D+mPrsejPRY2/YEtk0SJg4UIgMRFITgZiYoBWrYD27YHISGD+fGD/fm9L6Rg//AAsX873k5hIKycsDEhKAtq2BRYsANav97aUPo2yEBRuYdcu4L33gPJyICiIm7KKCm7c7rwTuOyyxm3Uqmuqcf7b5yPILwg7H9oJrUbrOuGbOyUlwJNPAgkJ/IOYo7AQiI4Gpk1rGjvqkhLgiSeANm0APwue8LIyQK8HXn6Z7rEWjM9bCEKIJCHESiHEPiHEXiHE496WSeEc+/YBr73GDVpKCl23CQn8d6tWwEcfNX6j5q/1x8zBM7GnYA8W7FngCrFbDjt20L1iSRkAdLdkZgLHj3tMrEbxxx98T5aUAcAv5NmzwOHDnpOrieEzCgGAHsA4KeX5AC4G8KgQ4nwvy6RwECmBL7/kemIuLhkYyE3cf/4D6HSNu9fNXW9Gr9a9MGXlFOhqGnmxlkRurnVlANAq0GhoKTQFzpyxb9cvBFBc7H55mig+oxCklLlSyu3n/l0CYD+ARO9KpXCUzEwgJ4duaEsEB9N9tGdP4+6lERrMGToHmWcz8f629xt3sZZEUBB30/ZgbcftanJygG++oXn51lsMQFVV2ffakBD73pOU3JUozOKTaadCiBQAvQD8bubYgwAeBIDk5GSPyqWwzalT3ITZcjsLAeTlNf5+w9KHYVDKIMxYMwP3XnAvwgLCGn/R5k7XrsD333NxtPSHqq6mMvBE4Wd1NfD558CaNbyncXHfupVB4bFjgY4drV/j/HPOBFvvyd8f6NDBtfI3I3zGQjAihAgD8B2AJ6SUDWw7KeX7UsoMKWVGXFyDQjuFl9Fq+Zu0hV4P/PUXMHs28PjjwHPPMQPy9GnH7ieEwJyhc1BQVoDXN73unNAtjfR0ZuKcOmX+uJTAiRPAkCFcnN3NF18Aq1ZRpnbtmPEUF8eMJz8/4MUXbccy2rQBevWi3OaQkte4/HLPvKcmik8pBCGEP6gMvpRSLvS2PArHMW4orVnv5eXMQlq1ilZCSIgpLX78eGD7dsfueXG7i3H9edfjpQ0v4VS5hUVOYUIIYMwYau9jx6idjVRUAEePAp06Addf735ZcnJoGaSkMGZRn8hIPv/TT7avdd99VAxHj5rym6VkzODIEeCCCzzznpowPqMQBKuLPgKwX0r5qrflUThHTAxw0UX8nZvDYGDdkFYLdOvG8wMDGYBOSmKm45tvMhbhCDMHz0SprhRz181t9HtoEbRuDUydCgwcSK187BiQnU1tfcstwFNPMdbgbjZs4IJvThkYiY+n+6ioiLL+9BPw/vvMXti/n18qAIiIACZMAO64g0ouO5uPoCAW2v3znyy6U1jEZ+oQhBCXAlgLYDeAc39hTJRSLrb0GlWH4JsUFQFz5gAFBdywGRNaamr4+927Fxg5EggPN//6vDx2G3jkEcfue9+P92HB7gX487E/kRSZ1Lg30ZIoK2OWjkbDxddWBpIref117t5jYqyfl5UFdOnCnGatlrsIvZ5B53btgMceq9uawmAASktNVdhNoZbCg/h8HYKUcp2UUkgpe0gpLzj3sKgMFL5LZCQwcSIwaBCVwrFjdN+eOMG1pm9fy8oAoPt461bHq5qfH/g8JCReWP1Co+RvcRjNs8REzyoDgLv32i4rc0gJHDgAbN7MOENSEhVX27ZAaipTY+fOrZsiq9HQYggPV8rAAXxGISiaFxERwN13cwM4fjzw7LPAq69ykxcba/21xnRyRxVC+6j2GJMxBh/v/BgHTh1wTnCFZ+nTh24qa+TkMA7QpYt511J8PI+vWOEeGVsQSiEo3EpoKDMGO3VisVp4uO3Ucilp8Tvjwp44YCJC/EMwecVk5wRWeJaePbl7KCoyf1xK+hhTU60XniUkAMuWMbVU4TRKISg8yqWX0rVrjdOnmVYeEeH49eNC4/B0v6fx3f7vsOXEFueEVHiOgADmHZeV0RKonZ5WWsr4QlQUdxTWCAzkTkNVITcKpRAUHqVHD7qMLKXAV1ezT9lVVzl/j6f6PYVWIa0wYfkE5y+i8Bzp6Wyil5FBpWDMeAKYSnrxxbaLW6Tkw5OV1c0Q9ekpPEpgIDMa583jbz4+3tRJoaCAm7w77jAVnjpDeGA4Jg+YjCd+ewLLjizD5WmXu+4NKNxDYiLw4IPAbbexAZ2fH78cWi13CV99Zb0fSnExs42cMSsVf6MsBIXHSUwEpk8HbryRXoGsLGYgXXABMHkyMGxY4+/xcMbDSI5Mxvhl4+ErqdUKOwgPZxZRmzammEHfvvx3RYX510hJk/Oqq1RGUSNRFoLCK0RFAVdfzd9wVRWzHV3Zoj7QLxDTB03HvT/ei+/2f4ebzr/JdRdvSRiLR5YtY+5wQAAX6P792cvcE0RFsbDszTdpAcTEmBb+8nJ2bx0wgBWRikbhM4VpzqAK0xTWqDHUoMe7PaA36LF3zF74adT+xyHKyrgI79vH4q6wMCoIYxHb6NH073uKAweA777jPAOtlqloERHcVQwZ0uKH3jiCpcI09QtRNFu0Gi1mD5mN67++Hp/s/ASje4/2tkhNBynZHuLgQfYZqu2KCQuj++add9hrpHNnz8h03nmseCwoYJpqQADjBiqQ7DJUDEHRrLm287W4uN3FeH7V86iotuCDVjQkK4sdCJOSzPvlg4Pp7//xR8/KJQRrDjp1oqJyVBlIyeZ3q1ezqVbtXkgKZSEoPI+Upnb71nqauQIhBOYOnYtBnw7CW1vewtOXPO3eGzYXNm7kH8hakDY2lgvq6dO2y899gWPHOL81M7Pu3IRWrZje2rWrV8XzBZRCUHiMwkJg7VpgyRK6p7Va4JJLgKFD2freXQxMGYgrO1yJ2WtnY3Tv0YgKinLfzZoLBQW0AqxhHLNZUuL7CuHECWDWLH7p2revq+iKi4GXXwbGjWML3haMchkpPEJ2NjBlCgd1hYXxN9m6NbBpE2uS1q517/1nD5mNwspCvLzhZffeqLkQHm67DYSxx0hTGEn55ZdUAnFxDa0eY+bShx/abrTXzFEKQeF2KiqAV17hZrJ9e1OPIj8/Nqxs04aW/F9/uU+GXm164dZut+K1Ta8hr9QFszubOxddBFRWWj+nuJh/vNptp32R3FxmSsXHWz4nPJwFcfv3e04uH0QpBIXb2baNa4ellveBgVQSv/3mXjlmDJ4BXY0OM9fMdO4Cej1THvfsYdC1Cads26RLF2rrggLzx2tqWAx2zTW+XwyWm8vdiC05tVr+XVswKoagcDtr1tjuKGCcgVBZ6b5BXR1iOmB0r9F4b9t7eKrfU0iLTrPvhQYDC7MWLaK/XAg+17o18I9/sAdPc0OrZdO5efO4SMbHM6YgJYPIxcXM//dkHYK7ac4K3k6UhaBwO8XFtt3MxpoiW62xG8uUgVPgr/HH1JVT7XuBwQB88gnw+ed8E8YBLcnJFHb+fEbJmyMJCQzw/OMfrAjOyuKjfXvgmWeAUaN83zoA2CvFGO+whsHANtstGGUhKNxObCyDyiEhls/R62nV20psaSxtw9vi8b6PY976eXjmkmfQs3VP6y/YtQtYtYoLRe0cWSFo9gQFAQsWAN2705/e3IiMZI+RkSMZDPLzaxpB5NokJDB76M8/Lcc7ior4RT3vPM/K5mMoC0HhdgYNoqfFGvn5bI/jiaLTZ/s/i8igSExaMcn2yb/8wkXRUsFEQACPrV7tWiF9DY2G046amjIwcscd/HLl59d1DUnJfOiiInZbbeHtL5RCULidnj25McvPN3/8+HEmgSxZAtx/P/Dcc9yU25qs6CzRwdEY3388Fv25CGuzrOS7VlUBhw7ZHgAfGwvs2OFaIRWupXVrYNIkuruys1mcZnSBhYUBEyZ4rgWHD6Oa2yk8wsmTrP3Jz2fzytBQrre7dvE32aOHaWRuaSnjlm3a0FVtaz12hvLqcnSY3wFp0WlYe99aCHO+8IoKYMwY21VzlZXM2X/lFdcLqnA9J05QKUhJd1JaWtOIhbgQS83tlIWg8AhxccDzzwP/93/sh1ZUxIxGvZ4u6m7daK0LwZTwlBQqhX/9yz2tZkL8QzBt4DSsP7Yei/5cZP6koCAKW1Zm/WJFRRRY0TRITAT69WOZfHp6i1MG1lAKQeExgoMZJ5g2jQt9aqppxro52rZlH7I//3SPPPf3uh8dYjpgwvIJqDHUNDxBCGDECMu5+AB3meXlwOVqKpui6aMUgsIr5OfTarc2FRFgzPb33xtxIykBnc5sjrm/1h8zB8/EnoI9WLBngfnX9+/PFssnTjS8hsFAf1dGhsP+5+pqhh2+/x744Qdg9+4W3zVB4QOotFOFV6ioMLmIrBEYyI4CDnPyJDN/li2jj9/fn+lOgwfT9DjHzV1vxrz18zBl5RTc0vUWBGgD6l4nJISBjHff5YAWjYZayqhkLr0UuOsuh9q2bt8O/PvfjJX4+fEyNTWMrTz8cIvPfFR4EaUQFF4hNJSLYO0uxOaorHSikeahQwzw6vWssA0M5JZ85Uo+Hn0U6NULAKARGswZOgdXfnkl3t/2Pv550T8bXi8qiqlP2dnc1hu7e2ZkWO+PY4adO4E33uDL6k+gLC4GXnyRCS8dOzr4nhUKF6BcRgqvEBfH5I7CQsvnGOcmONQd4exZ4LXXqHGSkkx58/7+dP3ExABvvcX+NucYlj4Mg1IGYcaaGSjVlZq/rhDMNrr+eloEI0c6rAz0ehY9x8VRvPpERDCg/umnqouCwjsohaDwCkIA111HhaDTmT/nxAm6T9LsbDkEANiwgfmsliLVxnLpVatqySIwZ+gcFJQV4PVNrztwM8c4cIAJSWFhls+JimJdRmam28RQKCyiFILCa/ToAdx7Lzfrx49zHdfrqSSOHuWGfswYB7MCV62y7WNKSOB5tbbhF7e7GNefdz1e2vASTpWfcuLd2CYvz/bO3zhzxlIRn0LhTpRCUHiVoUOBGTMY7y0tZSw4NpZu/okTbXdJbUBJCYO+1vD3p/apqZtqOnPwTJTqSjF33VwHb2ofGo19riAp3T9aVKEwh08FlYUQ/wZwNYACKWXLnmXXgmjXDrjzTj7qY1xA7bYSYmJMWUWWqKqi66he35qu8V1xd8+78ebmN/F438eRFJlk503tIyWF78NaIF1KPpKTXXprhcIufG0f8gmAK70thMK7VFdzNsKMGcADD7C6+Y032O/I5g576FDgzBnr5xQUAFdcYXZVfn7g85CQeGH1C86/AQukpjIufcqKRyovj41TfX0ImaJ54lMKQUq5BoCNX7OiOVNRAbz+OscMnDpF66FNG+DgQc5q+fxzG60sLrqI1W6nT5s/XlRE6+Gyy8webh/VHmMyxuDjnR/jwKkDjX9DtRACGD2a8ufl1X0fBgOD6EFBTGJSKLyBTykEheLzz2kJpKayjZBGQ89OfDx318uWAUuXWrlAWBjw9NN8UWYmk/urqxmgyMqiu+jpp60GnicOmIgQ/xBMXjHZ5e8vKQmYPBno0IFlDcbHsWO0DCZPdjibVaFwGT7X7VQIkQLgZ0sxBCHEgwAeBIDk5OQ+WS18Bmpz4uRJ4NlnuWhaCqpWVLB10KuvWg8ToLQU2LIFWL6caUsREaxS7tvXdr8MAC+segHPr34em0dvxoWJFzr3hmyQn09LQQj2W3O4AE+hcBJL3U6bnEKojWp/3bxYtgz48kvb3aazsqg4unRxnywlVSVIm5+Gngk9sezuZe67kULhBVT7a4XPU1xs38Q0Idw3PMdIeGA4Jg+YjOVHl2PZEaUQFC0Dn1IIQogFADYC6CyEOC6EeMDbMik8R2Qk3f32YK71g6t5OONhJEcmY/yy8fA1S1qhcAc+pRCklLdJKdtIKf2llO2klB95WyaF5+jZk7t/a1lEFRVUBunp7pcn0C8Q0wdNx7bcbfhu/3fuv6FC4WV8SiEomheFhWxJYasswEirVhw/YJxuaCSk/BQ6Hfwf+mx6C51Wv487O2+Bv6HKPULX484ed+L8uPMxecVk6A1qYIGieeNTlcqK5sGBA8BPPwH79zP7s6aG7Zyvuw7o2uYMkJPDE9u2bTAw+c47GUvYuROICq/BJbnfoeNfv6CySqBchmBAkgEdtmwADoQBjz3m9sHoWo0Ws4bMwg1f34BPd36KB3orL6ai+eJzWUaOoLKMfI/Vq4GPPmKWZ2ysqVWD7sRJdP7jG1wRvRWJ7c5VCBsMQJ8+wKhR7Al9Dr0e2LMHOPLiN2iz5SecCU9BfBst0tKoP4QAtUZxMTBliu20pEYipUS/j/rhRMkJHPrnIQT7B7v1fgqFu1FZRgq3c+IE+/0nJtL9Y+wMEVaWj+t2z8D5uh3YmtcOZ8OTWGyQlATs2sUeFbXae/r5ARckn8GNQb+g360puOpaLS680KRgAFDj+PsDCxe6/X0JITD38rk4Xnwcb2952+33Uyi8hVIICpexahVdRMaZNAAAKdFr58fQ1uhQHpUIjZ8GR4+eO6bRUHvodMDHH9cNHBgHKddrQFeHuDjgjz8st6lwIYNSBmF4+nDMXjcbRZVFbr+fQuENlEJQuIyNG+t4fgAA4SU5iD11AGWhCQCYIXT8eL0XJiQw8GCMLQDs5RAUZP2GGg0f9katG8nsobNxpuIMXt7wskfup1B4GqUQFC5Dp2tYWBZ1NpP/OOfrEcI0S/lvjH6g2mPCAgMbzCswi5T2VbPZorycaVFVlrOXerfpjVFdR+HVTa8iv9RHJticOcPPLSfHRtc/hcI2KstI4TJat+Z8mtqtgoQ0wE9fiZhThxCgK0WV9EdgeGsItAJQr/107QXtgguAlSut37CyEggOptvJWQ4eBH75hbEMIeiiGjCA7bHbtGlw+ozBM/Dtvm8xc81M/Gvkv5y/b2P56y/ghx8Yfddq+dnFxQFXX83cXTVhR+EE6lujcBnDh9f13mhqqpGUvQ5tc7Yi7tR+hJfmIvJMJroXrQdWrKD2MCJEXX9T165MKSosNH8zKTl7c/hw2xPSLLF0KTBrFnDoEAPcycl0X61ZA0ydSjdWPTrGdsTo3qPx3rb3cKTwiHP3bSw7dwIzZ1IpJCWxR3hyMtOz3n8f+OIL+0azKRT1UApB4RBSMpto9262qa7dU6hPH65NOTk8sefOTxF3ch/KQ+Kg9wtCGUJRExKOkLZR3N2vW8fS4+JiphB16mS6mJ8f6wwqK3nB2u6jigoOXe7WDbjSyXlKf/7JhbNdO/abNu6o/fz4XEQEp/IUFzd46dSBU6HVaDFt1TTn7t0YioqAt99mGldtuQEgPJx9w5cuBbZt87xsiiaPchkp7ObwYeCrr7gxFcI0EH7QIOCGGziV8umnOdymeHc24g6tQ15UKooiwpCStxGBIQJtkwOYOBQaysVt714qg6efbujmSE0Fpk1jldvmzaahxCEhwC23AMOGOW8d/Porr2Pu9VLy+dxcRsqHD69zuG14Wzze93G8uP5FPHPJM+iR0MM5GZzh999pCVhq5qTR0LJatIga2u7ZowqFE4VpQohQAJVSSjsifu5FFaZ5jj17OIMgNLRWcRjYjC4nh2v3M8/QpW8wALkvfY7in9fgdGAigoOB1KBcRGXugKZaZ1r4q6t58scfc9KZNYqK6I/SaunbtzoMwQZVVcDDDzccvGA0fw4fBs6e5cIbFARMnAgMGQJERf19amFFIdLmp+HS5Evxv9v+57wstqip4aAIvZ4Tg157jWm21mY6SMn+H2+8QUtHoaiHpcI0mxaCEEID4FYAdwC4EEAVgEAhxCkAiwC8J6U87GJ5FXZy9iywaRPdyno9J3ENGMCuEK7aHFZWAu+8Q0UQFlb3mL8/C4WPHGFs9sYbz5UX6DKReGE4yrRcm3bktIFBG482/vlICi9ESLDEaUM0KoqrcehgCtJj2LDOosyRkXYNtrELnc5k3hiRkh/i0aPUapGR/ECrq4Gff2ZcYfz4v4cdRwdH47n+z2HC8glYl70OlyZf6hrZjOj1vOeiRYyjGGXNz6dLy9pnYXxvetV7SeEY9riMVgJYBmACgD1SSgMACCFiAAwGME8I8b2U8gv3iakwx/r13FzX1HB90Gg4ZOa33zhr/rbbrNd12cuuXUBZWcMag9q0aQMsWQJcddW5wjR/f5zIrsHWv3g8OBgQWi0yq9pi+59todMBMdES8VXH8PMvWpQs54Z9zBizyT2uJTiYLiGdzuQyysykMoiOrmv+REYyYFtQQF/YjBl/f6hj+47F/N/nY/yy8Vh731oIV2lgvZ4aePNmBrmTk03PHzxIn92wYdTOubm0GKSkxm7blv/292+ovRX2U1PDqXtaLc3iFuJ6s0chXC6lbNClXkp5BsB3AL4TQjTCflc4wx9/AO+9x99/7fqtiAh+l3/7jQvzzTc3/l67d3MNtUZgINfX3FwgJQU4GpuBYzu+RGib6DplAsXF/J0JAaC0BDIxHgmpIeietwlxy1Zj9+KzCB8ZhbCrB7Eftq3iNGfw8+M4zSVLqIUMBmYahYXV/eHrdEBaGv8dH89RbYcO/T2qLcQ/BFMHTsUjix7B4j8X46pOV7lGvuXLqQzS0urK4+fHdNxly/gw1moYP+CsLH4xEhKA++5zPr7SkikpYUOu335jxoTBQIU8ciSQkeGaHZYPYzPLSEpZLYQ4TwgxVAhRZ8shhLjSeI67BFQ0RErg66+5ITS3Xmq1dOP88gtd741Fr7cvrd04y0BK4KsjF0Eb7I8gaUpD0uvpDg8KAoICJfzPnsTB6H4YsnIyLtz2LtoYTsBQXYPMDSeAd99l47rc3Ma/AXMMGcJd9NmzXAQqK+suoCUl1K61J977+wP1YlYP9HoA6dHpmLB8AgzSBYVhej2weDHNJHO70pgYPrKyaMFERVGRhYXx31LSf6diB45z+jQtwG+/ZcKBMRW5qAh4803gww/tK5Zswtj8mQshxgL4EcBjAPYIIa6rdXi2uwRTWCY7m4Fca25kPz8uztu3N/5+qam2R1Yaq49bteLg+IO5Edjd7yGElhUguJwuDWMGZ5CsQNuqo9gTciFOHzgJv+pKFEWloDI4BgFRIfjrTAyqWrfnTV96iSaFq4mL42Dmmhp+oDodH+XlVBJhYUC/fnV3hP7+9J3Vwl/rj5lDZmJ3wW4s2L2g8XLl5FAZhYRYPkdKppgWFtLkqqjgZ3T2LE25wYOB//3P/vFzCn6mb7/NxT8lxWQSC0FFm5ZGH+2SJV4V093YU4fwfwD6SCmvBzAIwBQhxOPnjrUMx5qPUVRkSvu0RkAAXd+NpW9f3sva+pKXB1x8MTemJSVcR/MS+2DdgIkoCW+DyKJshBYeQ9uabATKSiyLvx0rQq5GSYUW5aGm4ITREikvBxftwkK6T9xBWhowbx5wzz10v2g0jCFcfDEwcGBDP1llZV2L4Ry3dL0FF7S+AFNWToGuRtc4mfR663/YwkJ+OImJHDKRlkaZ27alAhs6lC6j0lKm9Crs4+hRxmbOJQ00QAhabYsXN2tFa08MQSOlLAUAKWWmEGIQgG+FEO2hFIJXsDfjUq+3vtG0l6goZg99/TUt6Pr3P3WKyue6c7ZjYKCpUPZ0bCesGzARoaX5OLytCHmn/VEanYQa4Yfw/ZvgF2nez/33mhgbS3/ukCGNfyPmCAkBrr2WebW5ubyfOaSkNXHxxQ0OaYQGc4bOwYgvR+CDbR/g0YsedV6eqCiadgaDeT+d0VTT66mczjvP/HWEcM1uoKWwcyd3MdaUcXAwfZ5ZWUzna4bYYyHkCyEuMP7POeVwNYBWALq7SzCFZVJTuSjrrGxGpeSa0q2ba+551VXA7bcz69HYS+3YMf47PByYMIEbU4BZkVFRdT09ZWEJ0Kd1QpYmFTXCjwJWVyM9sm7rar2ev8u/E2RCQjzS3hr/+AdNm4qKhsek5CJw8cXciZthePpwDGw/EDPWzECprhEurpgYoEcPLjzmMC5YUvKDtoQx00hhH6Wl9n1eQlhtgNjUsUch3A0gr/YTUkq9lPJuAJe5RSqFVYKCWDx74oTlljX5+czrT0lxzT2FAEaMAF5/HXjgAW7Yr7qKimDWrLprk1bLTXd+ft1+da1bM7ah0wGlZQIJQcVIDqjz1UJpKeX+OzNJp3NPplF9OncGHn2UxW9ZWfTNl5XRasjKYtXvffdZfLkQAnOGzkF+WT7e2PRG42S54Qa+79q9noxERvL59u0tp5UavxTp6Y2ToyURF2d9hwWYdlnNOGBv02Ukpazfvb72sfWuFUdhL1dfzd35jh3cmRvXhqoq+vNjYoBHHnF9+nRYGAvfbHHZZYzVLlvGNSwmhhuwbt3YwigyEujXqxJhOQUoiWwHg4FrcFRUPWv85MkGrSPOnqWbSqvlhr3OQHCVzBwAACAASURBVJ7GcOGF9Mtv3Ahs2cIFIiODvTk6dLD5YfZL6ofrOl+HFze8iIczHkZsiAX3ky1SUlj2PX8+raOwML7ZsjKaUP36WX/TBQWUNynJufu3RC68EPjvfy276gAG79q1s26ZNXHUTOUmTHU1sHYt41xnznC98vdn5+bLL3ddYa+zSMmitsWL2UsO4OaqSxcqs8qjubhxx2SUBraC3j8Y7dvz2N/ZnxUVXBBnzgRat0ZuLjs+b9liWpuN73fkSNu1Ep5gb8FedH+nO8b1G4eXhr3UuItVVFDjb91K5dS+PVtbBwcDc+ZQK7ZtazKnampo0YSGst2G0YensI9PPmHL9ZSUhsq/qoom+bhxrI9p4lhqXaEUQjPAYGDyicFAJeCL9UiVlaYgt7FH3fHjQPnqzYj79h2ExwTAPzGeO+GaGu5ydTq6cTIykJ3NNbCmhuucMRu0qorxjPR09sdzRRC9sdz7w734as9XODz2MNpFuGk3WVQE/Pgj21sYf8NS0nq44Qbm/yocQ6cD/v1vWohBQdy9GH9cQgB3383ss2ZAoxWCYF3+HQDSpJTThRDJAFpLKd2UE2gbpRCaCZmZzO/+/XcuakIwgDtsGNC+PQwGthGqqLCcBJSZSc/Sbbd5UnDzZJ7NROc3O+PuHnfjg2s/cO/NSktpFUhJTelts7CpIyXTT1etYiqqnx/Quzcts2akZF2hEN4BYAAwRErZRQgRDWCJlPJC14pqP0ohNDOqqrjqBwfX8ZHv389yAWsB8upqhhtef903rIQnfn0C/9r8L+wdsxfntbKQGqpQeAlLCsGRATl9pZSPAqgEACllIQAfdE4omiyBgYwq1wuY7t1re2yyvz9dUllZbpTPASYOmIgQ/xBMWTnF26IoFHbjiEKoFkJoAUgAEELEgRaDQuFWdDr7eooJ4TutZuJD4zGu3zh8u+9bbDmxxdviKBR24YhCmA/gewDxQohZANZB9TJSeIDERPtTxC3FGLzBU/2eQquQVpi4YqK3RVEo7MKuEZrnAsprAGwDMBRsWXG9lHK/G2VTKAAwpvf554wTWComPX2aJQRun6XgABGBEZg0YBKe/O1JLDuyDJenXW77RUVFnHi0di3bVMTHsz9Rz56+mT6maFY4ElTeLaX0qVYVKqjccli8GFiwwHwvpZISZgZOmuR7xbmV+kp0frMz4kPjsXn0ZutDdPbtYzFaVZWpkq+8nBV7iYnAU0/5lgmkaLK4Iqi8XQjh1owiIcSVQoiDQojDQojx7ryXwrfR602dnQG2zRg1ytRJ4uRJU1+lqioW9vqaMgCAIL8gvDDoBWzN2YqF+xdaPjEnh/OSQ0NZgBYezlz4mBimV506xePNuNOmwvs4YiEcANABQBaAMtBtJKWUPVwiCAPWhwBcAeA4gC0AbpNS7rP0GmUhND9OnwZWrODQMJ2OsYGuXakQzj+fHpXNm00p4t2705visvYVbqDGUIMe7/ZAjaEGe8bsgZ/GjKf200/pJrLWFiEzE3j8caBXL7fJqmgZWLIQ7IohnGO47VMaxUUADkspjwCAEOIrANcBsKgQFM2L7GzgxRfpJWndmi5zKbn4z5vHAtzrr2e9WlNCq9Fi1pBZuOHrG/Dpzk/xQO8H6p6g01EZWOrFbyQsjK0VlEJQuAm7XUZSyixzDxfKkgjgWK3/P37uuToIIR4UQmwVQmw9aalFsKLJUVXFojKNhnECY/xUCDaiTE4Gvv+evZGaItd1vg59E/vi+dXPo6K6Xott4+xeW8UWISGW22IrFC7AboUghJhq7uFO4cwhpXxfSpkhpcyIi4uz/QJFk2DXLgaGY2LMH/fzY83azz9bbvntywghMPfyuThefBxvb3m77kHjRCGDjbIenc5yy2uFwgU4ElQuq/WoATACQIoLZTkBoHa/3nbnnlM0MUpKGAd4+20+Vqww39q/Nhs2MJ5qjehotpk5e9Z1snqSQSmDMDx9OGavm42iyiLTgeBguoFs7f6LithX3FfQ6TglKTu7waxpRdPE7hiClPKV2v8vhHgZwG8ulGULgI5CiFRQEdwK4HYXXl/hAdasAT77jBXDxp5CW7YA//kPcNddlptFlpXZTrMXgi6lpjywavbQ2ejzfh+8vOFlzBgyw3TgyiuBbdu4yJr7IM6epXXQp4/nhLVERQWwdClHm1ZUmAZ8DxjAQR2uSI09c4a7hD/+4JepUyde38LEOoVrcCSoXJ8QcBfvEqSUeiHEP0ElowXwbymlmhLehNi0CfjgAybK1M/6qaoCPvyQz5sZS4z4eG42rQ2jMralCA93ncyepneb3hjVdRRe3fQq/nnRP5EQdm5mQadOwD33UJsGBvID0Wr5wRUUUEk8+6z3O/eVlwOvvsoBF23aMMADME94zRpg+3aO0bMVILfGqlX8HAB2bxWCCujXX9nS9pZbLA+xUTQKR2IIu4UQf5x77AVwEEAjZwXWRUq5WErZSUqZLqWc5cprK9yLXs/CsdatzaeABgby2IIFPLc+l13GmQnW4gMFBRxsZcu1ZAkpOeNk3z66nryV0j9j8AxU6aswc83MugeGDAGmTAEuuICCZmXRMhg5Epgxg8O0vc3ChfzwUlPrjjb18+OEtupq4J13nA/0bN3KmQQJCcwkiIzkLsE4qWzRIuCnn1zzXhQNcMRCuLrWv/UA8qWUZn7aipbIwYMsJGvf3vI5oaF0kx84wFGatUlIoNJYt44bz4SEupZAaSkVyciRzsm3Zw/w7bdcY40DekJDeb1hw+xrnucqOsZ2xOjeo/HetvfwZL8nkRadZjqYns7Zp6NHc3ENDPSscNYoKwNWr7butomL44d85IjjlYIGA8dYxsWZ31VotVQSP//MkYAqwO5yHLG7xtRKNz1xzsUzz22SKZoUp0/bvyk8c8b0b72ea8C4cVz0ja18Fi3i2nPyJGOWpaXAE09wPXCUDRtY31BYyNcnJfG/ISG0WN5/38NdUvV6TO3yMLRCi2nLJ5s/x9+fAvqKMgBYEFJTY7mhFGAK9OxzonzoyBH+wa35BI19zptq/rGP44iFcAWA5+o9N8LMcy2C8nIugkLQ3dvS+47ZSqE3d66UwBdfMAspOZnPp6ezS8ORI2xTsXs3R2MOHOjcMLAzZ+iBaNu2rocDYHJPaioVRq9e5mMbLqWyEli2DPjtN7QtK8PjUR3x4p4FeKa0J3pc/xDzan0Zvb7hrGFzGGMfjlJUZN/1/fzoP1S4HJs/YyHEIwDGAEgTQvxR61A4gA3uEsxXOXOGjdZWrzaljQcGMtY1bJhvDHr3BkbvgMFgOd5n/LyM5x49yvhhSorpNRoNFWx8vOmcoCDnJ0Nu2MD71lcGRoRgUsyiRUDfvjbWo5oa+ruWL6fZEhDAF11yiSm4aomKCgZjDx6kTyw2Fs8hDu/hT0za/yb+t7uAwVhfHtMYHc0P0zjm1BLV1c4Fle3dVdVOYVO4FHtcRv8BcA2An8791/joI6W8w42y+RwFBYztrVjBBSspiY+ICMbajG0XWiIJCdxl5+ZaPic3l32HEs4l1qxaxTXAWsJIfDzwyy+2a7YssX277Y13ZCRw/LiNWomKCjaXe/FFKgWtlimiP/7Igc8bbOyNvv0WOHyYJsm5XUM0gvEc+uPnkONYpztM35UvV90lJzOwa60QxNij3Jn2Gunp/FytRfulNDW4UrgcmwpBSlkkpcyUUt4GoBhAAoD2ALoJIXyoSsa9SAm89x7XhfotmIOC+DvPzAS++cZrInqdu+/mJjI7u+5vurqaz0VHsxbh9Gm6hQ4etL3zDw2lJ8HZuie93naGotHtbTGOICXzaffupTkTH89FPSyMX4a4OODddzn82RylpTQpExMb7KzHoi/ayDCMj90Beeggc299FSGAW2/lH8Tczkev5x/6hhucSwULCeHshxMnLCvG3Fz+2Pbto0X1yCPAc8+xJqKoyPxrFHbjSNrpaHBIzm8AXjj33+fdI5bvkZXFbDvj7tYciYlMxbZVldtciY7mTIIrrmBs8NgxPk6eZAygXz9g1iy2qn72WXpeDh+2Pg3NuCG0x7VsjpQUZj9Zo7KSSt1iLPP4cZoaycnmBTH6tBYuNL+QHTlCbWMm0BICf0zFQKwXx7A45IRlpeIs5eW0aPbuZb/wxtK1K/DYY1x8s7Ko2U+fpiLIyQFuvpmtaZ3lhht4jyNHqEiNVFZyx6XR8O+xYAF3GrGx/Gy/+gqYOJGvUziNI0HlxwFcCGCTlHKwEOI8tKARmgcPmgoyLeHnx/XgyBG6RloikZHAbbexK6kx7hcezg308uWm9HKA68iePfxv//7mMw2Ns2GcrT0YNMgU77FkKeTnU16LgfFNm3jQ2h8/JobFWidPmgIgRqqr+dqKCpNLpVaw6QH0wstyAyZEb8eIinKHUv8sUl5Od9aKFSZ/m8EAdOnCwq6UFOev3acP3WfbtpkqidPSGEux1IzKXgID2eJ77Vr6CrOz+dkFBTFIt2oVLYnoaNNrwsL4OHsWePll7jpqH1fYjSMKoVJKWSmEgBAiUEp5QAjR2W2S+RiVlfYXR5orvKrNmTP8LRUU8LvdsyetYGd3wb5IcLCpJuHTT7lW1n+PHTpwk1lcDOzY0TDLR0puQG+6yfnPJjWV69T69XWD18br5+dzDRs0yMpFCgosR6WNCEGlUVLSUCGcPMmF0+i/MhgYPO7cGYiLgz+0mIkhuC3gOyww7MIduMG5N2ukvBx46SXuqNu2Nfk3peRzM2fSzdKxo/P3CAlhK4kBAxonqzkCAug6GjyYucJScqexbBkVqqU6iKgoKpD169lCQ+EwjiiE40KIKAA/AFgqhCgEh+W0CBISbOeqGxtWWtqc6PXA11/zew3we6/XcyNnrEfy5SQTZyguphvNjPsc4eH0DvzxB70ApaWmWiOdjq7kjAwm8jiLEMD993PjuXo1/z8wkOuKXk8l8eijNmIZERHW/VpGzKUzrVgBfPklj/n78zrl5dzNZmbyzXXsiFt0HTFPxGJK9qe4uWYiArSNyGP+3/947fpVgsYc6aIi4M03uZu2VlPgbTSaun2Rli+3/QOJi+MPTCkEp3CkuZ1x2/K8EGIlgEgAv7pFKh+kRw8u4JZ6jwFc/Nq0Md9hQErulFev5u+0dr2RsaXCvHnsXGCtn09T488/rbf679CBC/Tvv9N9ZNz8+fsD117LhyM1Dubw9wfuvRe46io22isooAuqVy8qYpvWR0YGFyNrlJRwsWrTxvRcdjbw+efMzDlzBti4kW/G+IbKy9mfR6+HJiAAc0aOxYhD0/DBtg/w6EWPOvdmKyqohGrLUZ/ISCqMPXua1rCds2etvy+ACrmgoHGBpxaM3T81wengdwBIk1JOF0IkA7gAwGZ3CedLhIQAN97IzZ65Qe8VFfzNjxtn/nuYlcWdcn23BcDzW7fmb3TVKi6CzQVb/YKE4OdpMND7cMEFVLgdOri+piMuzsnWF507M784P998VkFNDRehBx+s+8dduZKLf24uo+tJSTzPYOCOwFh1u3IlMHEiho8ai4GfrcCMNTNwzwX3ICzAidYMx4/zQ7eV0x8YyEBzU1II4eHckVn7Yuh01PZKGTiFI/GrtwH0A3Dbuf8vAfCWyyXyYYyNFnNyuPkzpk9mZdHV+eijtCTMYU/OfevWzJ6zFYNoSjhSUNajB+OV3bv7WIGfRgOMHctdQVYWtT9gCnJkZTGz5tJL675u0yb6D/fsodkXHU2TJCGBO9mgIP67VSsgPx9CCMwZOgf5Zfl4Y5OTfSOtRc/rvydvdfdzlsGDbc+MKChgk0CFUzhijPeVUvYWQuwAAClloRCiRTVsEAK45hpmxGzcyDRUjYaN2jIyrPfaOnrU9uJotHZLSppPkkTHjnzfZWWWM4V0Om5Y6ze8s4Zez1T0Zcu4+TYWDV96acOYrkuIiwOmTWP2y2+/mVKozjuPyqB794a70qoqU7DC6Cby8+Mf1/gHlpK+xtxc4MgR9Evvh+s6X4cXN7yIhzMeRmyIg7MF4uKoFGwphspK650IfZH+/VlSXlJiPke4tJSWlzsC3S0ERxRCtRBCC0ACgBAiDoCT9aNNm5gY+qMdQau1XW1rzLlvTq3e/fyYJfTee/SY1Pdk6PVc0G+/3X6roKyMMdF9+6iEw8PptVm0iI/77+fa4XIiIviHHzGCVoJWaz37qHVrCmnNfWEciykEfY7p6Zg5ZCZ6vNMDc9fNxUvDXnJMxpgYpq3t32+5fUR1Nb9kGRmOXdvbxMayw+FrrzGeEB/PnYROR3eeRsOUVTVa12kcWXrmA/geQLwQYhaAdWhBdQiNpXdv26MfjUHp5hRUBrg433UXkJdHV1tREd/rsWN0eV9/PQeG2YOU7PBw8CDjMXFxXJNDQ6lw4uKofFxd31UHjYY3tJWKOnw4A8fW2lGUlzNgIsTfmQbd4rvhrp534c0tb+J48XHH5bvpJt6zsLDhMZ2Of4R//KNpftG6dGHa7MiR/EFlZVGRDh/O5x0xMxUNsKe53edSyrsAtALwLIChAASA66WU7vzZNSv69WMhq7Eqtj5Gd/RDDzW/eJgQrCnq3Zstf3bvprXUrx+te0f6oGVlATt3UhlYKxr+/nuuHV7lootMxRbGyV+1KSnhopyQQBdUmmkuwguDXsCC3QswffV0vH/N+zZvVVPDGPGSJcDRo4mIqxyPGzPfQkpYJsKj/SG0GioDrZbmmL0a2BeJj6dCu/FGvnGttvn9aLyEkDaaaQkh9gG4HMAvAAaByuBvpJRnzLzMI2RkZMitW7d66/YOs3YtW+LExNRdHyor6ULOyADGjPGtFvi+xldfMW7QzsrwVim5CX7xRR/wHpw6xXYOOTn0bfn50U+m1/NLcPHFPKdvX2Yp1eLxXx7HW1vewt4xe9G5leUa0KoqDinbsYPep6gofganC2rQ6uR+XJO+F317VUMktbMd7FK0CIQQ26SUDXyG9sQQ3gWwHEAagG21rwnGE9LMvUjRkAEDuCZ8+y0XLOPkrqAgWvkjRihlYIuTJ23HGozN6kpLfUAhtGoF/Oc/bN505Ah93mFh9G+Fh9OP1q4d+33UY9Jlk/DRjo8wZeUU/Pfm/1q8xRdfmLea2rTTQt+6G97O7oaqZPaTUiisYVMhSCnnA5gvhHhHSvmIB2Rq1lxwAWN+x47Rjx4QwB9yUxmwU13NLMoDB+j2SU/ne7LlTncV9hQNGyvGzfVG8gpt2gCvv84K4nXr+FxlpWkm6MiRZnft8aHxGNdvHKavmY6tOVuR0bZhEPjUKV7SUt89Pz+65L7/nhlYasOhsIY9LiMhbZxkzznuoKm5jJo6e/YwYFtaypoqIeiuCAwE7rmHPYPczf79rOi21putuJgyzZ7tg67l0lKaOcZqRBuatLiqGGlvpKF3m95YcteSBseXLWOxpK0M0uxsdotOTWWH2ZISfkYdOqhZMy2RxriMVgohvgPwo5Qyu9YFAwBcCuAeACsBfOIiWRU+yMGDwCuvMP5Rf/GpqGA3U43G/WMojUXDeXnmg9E1NVxvx4zxQWUAmDpz2klEYAQmDZiEp5Y8heVHlmNo2tA6xwsL7WvtISUr5d96yzSpUkoq9iuuAK67rulYqQr3YU/a6ZUAagAsEELkCCH2CSGOAPgTrFp+XUr5iRtlVHgZKdmSJyLCfD1QcDAX5y++cH/xq7FoOCyMrT6Mc1oMBiqCrCwWD7p9PrIHeeTCR5AUkYQJyyegviEeEWG76SJAF+WPP1IBtG9PF1P79gxx/Pwz8PbbzatCXuEc9kxMq5RSvi2l7A9OShsKoLeUsr2U8v+klDvcLqXCq2Rmsl7A2ijKkBB6Q/budd19S0rYp23+fOCNN1h0duaMqWh41Ci6rLKyuOClpTF2e/PNPmodOEmQXxBeGPQCtuRswfcHvq9zrGdP05hjSxQUsHni+ec3rBY3xrC2bWODQUXLxmYMwZdRMQTPsHkz0xrt8VM3dmCWkQ0bgI8/5q7VWMhbWsqF76abGIcVgothVRWDpc3Z5aE36NHjnR6QkNj9yG74aUx+ojff5EC3pKSGitBgYO+8kBDWfViiqIjnzJzZvJSpwjyNiSEoWjj2ZqYYm3g2lh07GJNo27ZuzDUmhi6pr77i4n/FFXQheaQRXkEBB69s3sw0p6Qk4PLLWf3mgdQdP40fZg2ZhRv/eyM+2/UZ7u91/9/H7ruPC/rBg6YaFynZfLGkhJZdZxujrCIiqNDLy52fTqdo+jSjrjkKd2HM6LHWi8loaHbo0Lh7GQwcl9uqlfkEHH9/pu1/+y0zNz3CunXA+PFMG9XrKcShQ5xK9tpr5gfOu4Hrz7seFyVehGmrpqFSb3rzoaGcU/3ww4zxGF1oqal0oXXtars/lnE8bBN2GChcgFMKQQjhQLMBRVMnNpZtJ3JzLZ9z6hRdSuaGAznCkSMMDltrsxMYSDfR7t2Nu5dd7N3L5knx8YzEGnsYxcdTU+7dC3z0kUdWUiEE5g6di+PFx/H2lrfrHAsIYNrvjBkU56OPgKefZmufTp1s99EqK6MloVJQWzbOWgiLXSqFwue58066Io4dq5vVYjCYFMWDDzbe/3zmjH3X0GhMHajdhpSs6IqKMm+uGKf7bNvGqLsHGJw6GMPSh2H22tkoqiwye45WW9ciGDyYCtSahVdQwEaudnXaNRhYmfjxxxzD+eGH7OpqT7qTwqdxViGosFMLIzoamDSJu9CcHCoG46NbN47+TExs/H38/e3bbHukErmggEMvrA2nEIKFAJs2uVkYE7OHzMbpitN4ZeMrdp2fksKZMZmZDVNLpaQuS0mxs7Dw7FlGnufO5Xs+dgzYupWNo55/noELRZPF2aDyBy6VQtEkiI4GHniAmUQnTnAxSUhgILOigq704ODGWQnp6dylGptYmsOoMGwFShtNSQmFsfWGgoJsT/JyIX3a9sEtXW/BqxtfxaMXPoqEMDNjPWshBC280FDgl19Mn60xXbV3bwambQbnq6pYnZiXZ75UPC+PimHaNOV7aqI4qxAWulIIIcTNAJ4H0AXARVJKlUvqw0REmHoKbdoELF7MtUAItu0ZOZLNO+vPnbb32v37m/rzmCM/n8rAWsdTlxAUZJ+5otOZr9hzIzMGz8B3+77DrLWzMH/EfJvna7VM1x0+HPjjD7rmQkJo3ZkbE22W7duZimQpUGQcDP777/RTKZocvhJD2APgRgBrXHxdhZuoqABefZXuY52OG8bkZD7/3nssJKuqcu7ao0bxWkeP1s0kqq7mehQaCvzf/3kgX75tW6Y7lZRYP0+n4+wDD9IpthMe6PUA3t36Lo4WHrX7deHhVLjXXAMMHeqAMgA4OtTWbNfYWODXXx24KOjHsjVOUOERfCKGIKXcL6U86MprKtzLf/7DvPfUVFNGkBAMPKelsRHefy13bLZKaCjTJW+4gY3qsrPpqj55krUHU6ZwnTbitgQfjQa49lrGEiwFTPPyqL06dnSTEJaZOnAqtBotpq2a5pkb5ubatoTCwmjC2VrgS0uBpUuBceOA0aP5ePdddt5TeI0mF0MQQjwI4EEASLbkU1C4lcJC1mi1a2d+ly4Ej61ezfGYznhTQkL42pEjqQik5ObT6Oc+eZIDh1as4NoSFcU6sf79bW9iHaJ/f2qkX3+l5ouN5RssL6eiiIsDHnvMK4OwEyMSMfaisXhpw0t45pJn0D2hu3tvGBjI3by1QryaGlMrXEucPMlYw8mT/PxSUvi6HTuAjRtpIo4YoUqmvYBT32Ip5du2z6qLEGKZEGKPmcd1Dt77fSllhpQyI87r009aJsZZCNbWBT8//sYPHGjcvQICmL3Urp1JGezZw4ynRYu4IW3fnmvVwoXA5Ml0NbkMITi85umnaQlkZ7Pyq7ISuOUWYOpUr07hee7S59gRdcUk99+sb1/bwfOCAp5naTGvqeFsiJISKgJjWbRWywBUu3YsRd+1y6WiK+zDY60rpJSXe+peCvdSUWHfeVI6H0ewRH4+4xORkXW7SAcHc70+e5YZkU8/zeynmBgXbDSFAHr04KOqiotaUJBXrIL6xATH4Ln+z2HiiolYn70e/ZP7u+9mgwZxAINxCEZ9qqt5bOjQhseM7N/PFDVLAy38/Wnu/fQTO/cpK8GjeP8brWhyWKsiro0Qrk++WbWK1om5kQIlJax0XrmSXpxnnmEG5ObNLowzBAbSn+UDysDI2L5j0TqsNcYvH9+gPbZLSUzkJKQTJ1iabryXlExbys6mNWWtXH3jRtv5rdHRNPNUTYPHsWkhCCFi7LiOQUppozje6j1uAPAvAHEAFgkhdkophzt7PYV76dqVrhydznKH0aoq/u7PO89195WSi725zJjTp9khFaBVUFrK/nNFRcC//gVceSVw+20S4vgxunyMRRQdO/rU4u4MoQGhmHrZVIxZPAa/HP4FIzuOdN/NBg1i246ffmJWgVZLiyk9nalf3W3EMYqKbLelNQ7FttcUVbgMe1xGOece1mw3LQCnI7xSyu8BfG/zxEZgrLZfupTfY4D94S+/nDntyjK1n+Bgpi3+97+0/OvHEmpqWP16xx2urSbW66lo6q8n1dWshwgI4P1qaujiF4Leh/BwYNPCHAxc+xHaVf5leqEQ9P/fey+/DE2Y0b1H45WNr2DC8gm4ssOV0Ag3Krnzz+fjzBk2QQoONgXbbREbC/z5p/VzjEOxHZgsp3AN9iiE/VLKXtZOEEL49JAcvR7497+ZGRMaaspC2bsX2LKFNTR33aUGkDvCVVcxJXTpUrp9Y2L4Oy4s5AJ99dXAsGGuvaefH1339S2T3Fze0xifrKmpq4giy3Mx7M9ZyMmSSBzZHkJTa+EqKmLX0nHjWKXVRPHX+mPG4Bm4feHt+GrPV7i9++3uv6kxSOMIl1zC9DNrnD7NtuIuTRdT2IM92wgrYzUcOsdrLFzIytf27Wnt+vvzU8U2mwAAG4NJREFUkZDA51asYMaKwn40GuD22+mjN/bAMRiYJBIfz/m9Tz8NfP219S6pjiAEe/Lk59d9/vjxugqgvJy1EEa67/4SQf41yNHHo7Kq3i42MpKL2ocfNvkZkqO6jULPhJ6YsnIKdDW6ugfz8oBvvqHie/RRYNYs7oZ0OvMXcxcdO/JhqRlgZSV3Gtc5lHyocBF2jdC0dEwIcZ+tc7xNaSmwZAkzUMy5ijUaLmKLFnmwv34zQQjGD++7j+meUVEM6lZWcp319+dnP2kSFYQrGDiQllzt4mG93vS3ragwpaoCQGhpPuIL9qAsLAFCWKgvCw+npbBvn2uE9BIaocGcoXNwpPAIPtz+oenAunXAxImspfDz4x8qPx946y1g9my+d48JqQH++U/+gTIzTSZlRQWrDwsK2DbX7Y2qFOZorKPxBZdI4UaMXXn9rDjHjAHS/fs9J1dzQkoOaT96lDGFmBh+psHBDOwmJJg6JDeW+HjgySep6LOzuY6EhtKVXVREWS65xGQxhJfkQAoNDFJACCsxDY2GweYmzpUdrsRl7S/D9NXTUaYrY8Dsgw/4wSUl8Y9i9PGlpHCn/uabnm0dERnJXcKYMYzhnD3LH+Dw4bRcLr3Uc7Io6mBPltEflg4BcKQTilcwzuG1h7Iy98rSXDlyhMq0fXvzccXAQK4BP/xgPXar03Ft0GjoPrYU0+nShRvb9euZdRQcTKXfvTvXPHOjC4xZR8403GtKCCEwZ+gc9P93f7zx+xuYuMmfwVlzHwrAnfqff7JlRKdOnhM0IID9nzzcA0phHXuCygkAhgMorPe8ALDB5RK5GEe68HpkNm8zZONG/r6tJZnExHDdOXWqbh8igO6fZcsYoK6qogKPimL3goEDzWcpxsayzdC11/L8119ncWt9C6AkIhH6KgNEkAEdOlgxiA0Gy8VSTYxLki7BtZ2vxbx1c/HQ4asQ287KQm80m9av96xCUPgk9riMfgYQJqXMqvfIBLDKrdK5gPPP507TWrywupoupS5dPCdXc+L0acsbUCNC8O9QWlr3+TNnOG/lxx9pRSQlMd6j1QKff86RxbZiO0JwnvAFF9Drk5ND99Hp08Dek/E4HtMDl3UusFwkV1xMDdSMvgCzhsxCia4U86L22jfP4dQpzwim8GnsCSo/IKVcZ+GYB3LbGkdEBGtpjh0z7zqSkseuuELN9HCWyEjbLSqkpFunthUmJd3bhYXcnNfe3YeGMmC9fz+nWNoiOBh4/HF2Qr3kErqmU1OpKEZ+cTsiY/0YsKz/JTh7lgKMHm090NTE6BbfDXd1vhn/CtuH49JG0Fin4x9R0eJpPr8AK4waxZ3o1q30TRvTm8+c4U6yXz+2WlY4R9++bClhjaIi7vzj403PHT9uij2YQwi6uFesoGvIWGdgCSFYMJueXv9IawYx//1vU1GUEFQOrVtTa7iypNpHeGH4XCw4+B2mVy7F+4E3WT6xstLO+ZmK5o49QeXtUsrejT3HmwQEMPX6jz84QtDYcr1zZ46E7N5dFaU1hs6dudjn5XF9rY9eT/fNXXfV9V7s38//t+bR8Pfn6//6i73lnKZNG6ZeHj9Ok9BgYPqTcWZnMyQlOhWPpN2Ct/76CuOq89DZ38wfJz+fn00zcpcpnMceC6GLlUwjgMFln7c3/fw4O7Z3b64FthYihf1oNMDYsSz4zcykuyY0lJ9zQQFTQ2+6iZ99bSoq7FPEQjDO02iEYJAiKckFF2saTLrhNXz02veYUvYr/nv6Sv5xtFqm1Bkj/E88oXZECgD2KQR7bGkL46R8k2a6IfQqrVrRf79pE60wY0p/Rgb7RXXq1FABx8XZVxwspXJxO0t8WALGXfospq+Zjq3J0cjYcYIfaHQ0O5NeconqGaT4G5sKQUpZp1pHCDH93Ot2AtgppTzkJtkUTYywMC7+Q4eaBmtZU749etByq662XB9QUkLFUbsVhcIxxl0yDm9teQsTo7ZhyYe/mio1lYmsqIfDe2Up5VQAbwAoAnCDEMJr4zRbCidPsrXz6tWMg7jEfeJGhOACb8sSCwtjsDg723xLiaoqupxuuUVZdY0hIjACkwZMwtIjS7E8c6XtEZeKFouwd6CGEOINAE9It07gcIyMjAy5detWb4vhNoqKgM8+A7Zvr5stGRbGRXLAgKb/uzYYgG+/BRYvpkUREcH3WlzM93b33SxOUzSOSn0lOv2rE1qHtcbvo3+HaOpfHEWjEEJsk1Jm1H/ekbTTEgA/CSFulVKWCSGGA5gqpXTjzL6WS2kpMG8ed8hJSXV3yBUV7A1UWen6FtOeRqMxKbd165gVqtHwffXrpzogu4ogvyC8MOgF3P/T/fj+wPe4scuNXpXn7FlWuG/YwO9x69Z0NXbr1qzKQZocdlsIACCEuB3AkwB0AEoBzJRSrnWTbDZpzhbCwoUcSmWpm4JOxzTPV15Ri6bCPvQGPXq80wMSErsf2Q0/jXdW3j/+YKNVnY4tTfz8mPRUUsIs4CeesH9Mq8I5LFkIdntmhRBDAfwfgDIArQCM9aYyaM7odOzr06aN5XMCAuha2bTJc3IpmjZ+Gj/MGjILB04dwGe7PvOKDFlZ7DsVEcGCxPBwVpm3asXK8uxsYP58zzZfVZhwJFQ3CcAUKeUgADcB+FoIMcQtUrVwTp2iUrA1fjI8XLXsVjjG9eddj4sSL8K0VdNQqff8AJBffmFM21LVeWIiC0eNY24VnsVuhSClHGLsaSSl3A1gBICZ7hJMoVC4HiEE5g6di+PFx/H2lrc9eu+yMmDz5rrtS+ojBHvtuWqgksIxnE7mk1LmAhjqQlkU52jVitaBrYZxJSWq44DCcQanDsaw9GGYvXY2iio9Ny2ttNTU9dYaISFMplCYITcXWL4c+PlnRuTrtw9uJI2KKkkpK1wliMJEQAC7r/74o/WgskbDTByFwlFmD5mNjA8y8MrGVzB98HSP3DMwkLEBKa2nS+t0qni6AUVFbM64axf/X6s1FRhedRULelzQfkSV+/goV1xBf+rx4w0DbOXlfP7229nGX6FwlD5t++CWrrfg1Y2vIr803yP3jIoCOnZkl2FrlJYC/VUyu4myMjYK27ePkfiUFOaip6SwQePChcDXX9s/GtIKSiH4KGFhwHPPAb16MfMiM9P0qKjgHPIhKqSvaAQzBs9Apb4Ss9bO8tg9r7mGm11LPazOnGEqas+eHhPJ91m9mh16ExMbmlb+/lQMS5YAJ040+laqBMSHiYhg2+5Tp5h5UV3NmoPOnZv/bGCF++kU2wkP9HoA7259F09e/CRSo1Pdfs9u3ViI+M03zDRq1Yquz6oqduIOCQGefNJ2hl2LoaYG+O03833ljWi1dB2tXg3ccUejbqcUQhOgVauGc4gVClcwdeBUfPbHZ5i2aho+u8H9tQlC0OWdns51btcu01jnq6/mdMOYGLeL0XQoLqYPzVb1aVQUcKjxfUaVQlAoWjCJEYkYe9FYvLThJTxzyTPontDd7fcUgtlxXbrQMqiuZnGaGslgBuNkP1tI6ZIOkCqGoFD4MtXV3EavXs00QzfkYz536XPsiLpiksuvbYvAQMbLlDKwQEQE3QMlJdbPO3uWox8bibIQFApfREpg7Vpmj5SV1c3V7N0buOcel00NigmOwbP9n8WkFZOwPns9+ierFB+fQaMBRo5kymlYmPl8XZ2O349LL2387Rp9BYVC4XqWLgU++ICLQEoKG/2kpHB49e7dbIXrwqKkx/s+joTQBExYPgE+1OFeAXCqXffubARVfxhKWRkzkEaNsl4CbidKISgUvkZhIfDVV8w1Dw6ue0yjYfphTg5TDV1EaEAopg6cirXZa/Hr4V9ddl2FCwgI4NDy4cPpMszKMj30euDhh13WB9+h9tfuQgjxEoBrwLbafwG4T0p51tbrmnP7a0ULZvFiTg1KTrZ8TlUVE/pff50LhgvQ1ejQ5a0uCA8Ix/aHtkMj1H7R5ygrYw66Tsf4QocOTgVgGt3+2s0sBdBNStkDwCEAE7wsj0LhPQ4cYCtbaxibXZ065bLbBmgDMGPwDOzK34Wv93ztsusqXEhoKKv2LryQBUkujsb7hEKQUi6RUhprFzcBaOdNeRQKr+Mly/3WbreiR0IPTF45GboanVdkUHgPn1AI9bgfwC+WDgohHhRCbBVCbD158qQHxVIoPESXLrbTDKuqaCW4uGJRIzSYM3QOjhQewUfbP3LptRW+j8cUghBimRBij5nHdbXOmQRAD+BLS9eRUr4vpcyQUmbExcV5QnSFwrP068fgsc7KDj0nhx0QXRQ/qM2IDiMwIHkApq+ZjjJdmcuvr/BdPKYQpJSXSym7mXn8CABCiHsBXA3gDukLkW6FwltERbGV7bFjbG1bG4OBzycluSyzpD5CCMwZOgd5pXmY//t8t9xD4Zv4RGGaEOJKAM8CGCilLLd1vkLR7BkyhC6hb74BTp6sW5h24YXAXXdZnkPpAvon98c1na7BvPXz8FDGQ4gJVg2GWgK+knZ6GEAggNPnntokpXzY1utU2qmi2aPXM+vo7Fl2tOzYEYiN9citd+fvRs93e+KZS57BvCvmeeSeCs9gKe3UJywEKWUHb8ugUPgkfn7sGe0Fuid0x5097sT8zfMxtu9YJEYkekUOhefwxSwjhULhI7ww6AXUGGowfbVnxmwqvItSCAqFwiKp0al4OONhfLTjIxw63fh++wrfRikEhUJhlUkDJiHILwhTVk7xtigKN6MUgkJhg8pKZnpmZ3OedUsjISwBT/V7Cv/d+19sy9nmbXEUbsQnsoycRWUZKdxJaSnwyy/AsmWmrsN+fsDgwcCIESwXaCkUVRYhfX46+rTtg9/u/M3b4igaia83t1MofIriYmDOHGDRIs74TU7mIy6OXadnzQLOnPG2lJ4jMigSEwdMxJK/lmDF0RXeFkfhJpRCUCjM8OWXQF4eZ9IEBpqe9/cH2rdn5+mPWlirnzEXjkG7iHZqiE4zRikEhaIeZ84AW7ZwDo0l2rQB9u1jS6GWQpBfEF4Y9AI2n9iMHw784G1xFG5AKQSFoh5//cVOERorvw4heM7hw56Tyxe4u+fdOK/VeZi0YhL0Br3tFyiaFEohKBT1qD+21hrWGpI2O/Ly4Lf4V8wSQ7H/1H58vuI1r81tULgHn2hdoWh5VFVxMQ0OZuaOLxHjQB83F48j8E0qK4FPPgE2bQI0GtwQ4I8LY1ph2uoXcNvWKgQ98hgQGeltKRUuwMd+iormzoEDwG+/ATt30u0SEAAMHcpUTl9ZXDt2BKKjmXYaFmb+nIoKICQEOP98z8rmcWpqgHfeAXbtYjRdCAgAczESQ8VneCfvZzz5chUwcSK1u6JJo1xGCo+xeDFTOQ8eZDv/5GTuxn/9FZg2DcjK8raERKvlOIKCAm6O66PTMZh8661umU/jW+zfT+19ThkYGYJUXCHTMCvqDxQfO0zrQdHkUQpB4RH27gW++gpo1w6IjzcFbAMCqBz8/IDXXjO/AHuDjAzgoYeA06eBzEzOsj99mkorPx+45x5gwABvS+kBli6lmVRLGRiZjaE4LSrwSttMVvCpeEKTR7mMFB5h0SIgIoJ5/OaIjubCu3MncPHFHhXNIv37Az16MAV13z6ud507A337tiCXeWYm/3BmyEBb3CzPxysB2/BoXjLidbq6RRuKJodSCAq3U1JCz0NysvXzIiKAtWt9RyEAQHg4h5cNGeJtSbyEnx/HdlpgBgZjIfZjVvRuvKHVelAwhTtQLiOF26mspIvIjNehDv7+QJma6e5b9O5NX5kFOqMV7q/sgnciDiGz9LgHBVO4A6UQFG7HmKlTU2P9vIoK9gpS+BADB/IPZ6k4w2DA1PzzoNX4YdqqaZ6VTeFylEJQuJ3gYLqB8vOtn1dezvVH4UO0aweMGsXe38XFdQPHZWXA0aNoN/g6PNb3MXy+63PsKdjjPVkVjUYpBIVHGDGCG83SUvPHT5wA0tKALl08K5fCDoYPB8aOZcA4O5vDIbKyaDXcd9//t3fvMVaUdxjHvw8gLKKgiHdAhKLWC6UCpoq0KErF1FrSNtVAjVFrDBov1FpwVQqUWELSNBCbaoKpGtJKIqWNRcQKYtECCgWLogStN6wX8ApCKfDrH/OuLutezpHdnTPL80k2ObPMOe+zu+z89p2Z83th7FgmDJtI105dqV5cnXda2we+qGytomdPGD8eZs3KTkl3755dr9y2LSsSffrADTdk7wGwCiPBkCHZvbivv5790KqqsvcmpB9Y987duWXoLVQvrubpN57mrF5n5RzavgwvkGOt6qOPYMUKWLYsu2Zw1FHZO5VPOaXhW1KtGLbt3Ea/mf044bATWHr5UtTUXQSWm4YWyPEMwVpVt24wcmT2YW1Ll45duONbd3DtgmtZuHEho/qPyjuSlcnXEMys2Vx1+lX0PbQvEx+fyJ5o+P0LVplcEMys2XRs35Gp50xl7TtreXDdg3nHsTK5IJhZs7rk1EsYcOQAbltyGzt3708LRhSfC4KZNat2asedI+7klQ9eYfbq/Wzh6YJzQTCzZjfqK6MY1nsYU56cwrad7kdSFC4IZtbsJHHniDt5e+vbzFwxM+84ViIXBDNrEUN7D+WiEy5i+lPTeX/7+3nHsRK4IJhZi5l27jQ+/u/HTF82Pe8oVoKKKAiSpkp6TtIaSYskHZN3JjPbd6cdeRpjB4xl5sqZbPp4U95xrAkVURCAGRExICIGAg8Dd+QdyMyax+Thk9m9ZzdTlk7JO4o1oSIKQkR8XGuzC1DcBktmtpfjDz2eawZfw+x/zmbDlg15x7FGVERBAJA0TdIbwBgamSFIulrSs5Kefe+991ovoJl9adXDqqnqUMXtS27PO4o1otUKgqS/SVpXz8fFABFRHRG9gDnAdQ29TkTcExGDI2Lw4V5ey6wQjjzoSMafOZ65z89l1Vur8o5jDai49teSegMLIuLUpvZ1+2uz4vhox0f0m9mPQccM4tGxj+YdZ7/WUPvrijhlJKl/rc2LgRfzymJmLaNbVTduHXYri15exOJ/L847jtWjIgoC8Kt0+ug5YCRwQ96BzKz5jRsyjp5dezLx8YlU2tkJq5CCEBHfj4hT062nF0WEb1g2a4OqOlQxefhkVm5ayfwX5+cdx+qoiIJgZvuPy752GSf1OInqxdXs2rMr7zhWiwuCmbWqDu06MO3caazfvJ4H1j6QdxyrxQXBzFrd6JNGM+SYIUx6YhI7du3IO44lFXfbaTkkvQe81oJD9AA2t+Drt6SiZnfu1lfU7EXNDflnPy4ivvBGrkIXhJYm6dn67tUtgqJmd+7WV9TsRc0NlZvdp4zMzAxwQTAzs8QFoXH35B1gHxQ1u3O3vqJmL2puqNDsvoZgZmaAZwhmZpa4IJiZGeCC0KSirvcsaYakF1P2P0k6JO9MpZL0Q0nPS9ojqeJuzatL0gWSXpK0UdKEvPOUStK9kt6VtC7vLOWQ1EvSEkkvpP8nhWiGKalK0kpJa1PuyXlnqsvXEJogqWvNEp+SrgdOjohrco7VJEkjgcURsUvSdICI+HnOsUoi6avAHuBu4OaIqNhFLyS1BzYA5wNvAs8Al0bEC7kGK4GkbwJbgftLWX+kUkg6Gjg6IlZLOhhYBXyv0r/nkgR0iYitkg4AlgE3RMTynKN9xjOEJhR1veeIWBQRNZ3DlgM988xTjohYHxEv5Z2jRGcAGyPilYjYCfyRbE2PihcRTwLv552jXBHxn4hYnR5/AqwHjs03VdMiszVtHpA+Kup44oJQglLXe65gVwCP5B2ijToWeKPW9psU4ODUVkjqA3wdWJFvktJIai9pDfAu8FhEVFRuFwSab73n1tZU7rRPNbCLLHvFKCW7WWMkHQQ8BNxYZyZfsSJid0QMJJuxnyGpok7Vdcg7QCWIiPNK3HUOsACY1IJxStZUbkmXA98BRkSFXSwq43te6TYBvWpt90yfsxaUzsE/BMyJiHl55ylXRHwoaQlwAVAxF/U9Q2hCUdd7lnQBcAvw3Yj4NO88bdgzQH9Jx0vqCFwC/CXnTG1aujg7G1gfEb/OO0+pJB1ec7efpM5kNyJU1PHEdxk1QdJDwIlkd728BlxThCU+JW0EOgFb0qeWF+HuKABJo4FZwOHAh8CaiPh2vqkaJulC4DdAe+DeiJiWc6SSSPoDMJysFfM7wKSImJ1rqBJIOhv4O/Avst9LgFsjYkF+qZomaQBwH9n/k3bA3IiYkm+qvbkgmJkZ4FNGZmaWuCCYmRnggmBmZokLgpmZAS4IZmaWuCCYmRnggmBthKQ+kranPjE1n/tCW2pJnVMr852SeuzjmJ0lLU0dT5F0vaT1kspqEyLpEEnj9iVLCWN8odW1pI6SnpTkjgUGuCBY2/Jy6hNT05b6LmAUcDJwqaSTI2J72uetZhjvCmBeROxO2+OA8yNiTJmvc0h6blmUKfV3+PdkbRI+k7qzPg78qNyxrW1yQbDCSIuinJ8e/1LSrEZ2b4221GOAP6c8vwP6Ao9IuknS2LQYyhpJd9eaRcyXtCotkHJ1ep1fAf3SvjPSbKf2X/I3S/pFetwnzXruJ+uB06uhsWprpNX1/PR1mLkgWKFMAqoljSFreXxjI/u2aFvq1Leob0S8CpDagrwFnAMsJPure2iajezm84PuFRExCBgMXC/pMGACaXYTET8rYfj+wG8j4hTgwEbGKsU6YEgZ+1sb5nOHVhgR8WRqbDYeGF5zqkbSVLJmZ62pB1mfpfqMAAYBz2Rx6UzW/x6yIjA6Pe5FdnB/u8yxX6u1ylZjYzUpInan6ykHp8VmbD/mgmCFIek04GhgS83BS9JRZCtP1VV2W2pJ1wI/SZsXAqNrb0dE7esO24Gqhl4KuC8iJtZ5/eHAecCZEfGppCcaeI1d7D17r7vPtqbGKlMnYMc+PN/aCJ8yskJQto7uHLLrAFtTe2+AgcCaep5SdlvqiLgrnbYZGBFv1d2us+8HQHtJ9R3QHwd+IOmIlL27pOOAbsAHqRicBHwj7f8JcHCt578DHCHpMEmdyNa0aEhDY5UknbLaHBH/K/U51na5IFjFk3QgMA/4aUSsB6by+SJF9RaEtJ70dcCjZGvuzo2I55s52iLg7HrGfgG4DVgk6TngMbKZzUKgg6T1ZBeSl6f9twBPpRXjZqSD8xRgZXpugz3zGxlrL6nV9T+AEyW9KenK9E/nAH/9Ml+8tT1uf22FJmk22Wmd3sDDEVHSkoSSXgUGR8TmfRj7dOCmiPjxl32NvEmaB0yIiA15Z7H8eYZghRYRV0bEHrK7a7rVfmNafWremEZ23WFPY/uWMPZqYEl9t3kWQTqVNt/FwGp4hmBmZoBnCGZmlrggmJkZ4IJgZmaJC4KZmQEuCGZmlrggmJkZ4IJgZmaJC4KZmQHwfyDHwTuSEudxAAAAAElFTkSuQmCC\n",
|
||
"text/plain": [
|
||
"<Figure size 432x288 with 1 Axes>"
|
||
]
|
||
},
|
||
"metadata": {
|
||
"tags": [],
|
||
"needs_background": "light"
|
||
}
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/",
|
||
"height": 282
|
||
},
|
||
"id": "oEQswfCGrmHw",
|
||
"outputId": "3cf61882-60e1-4baa-8e51-0c31ea80875c"
|
||
},
|
||
"source": [
|
||
"pred = tf.matmul(valid_x,W)+b\r\n",
|
||
"fig,ax = plt.subplots(1,2)\r\n",
|
||
"ax[0].scatter(valid_x[:,0],valid_x[:,1],c=pred[:,0]>0.5)\r\n",
|
||
"ax[1].scatter(valid_x[:,0],valid_x[:,1],c=valid_labels)"
|
||
],
|
||
"execution_count": 27,
|
||
"outputs": [
|
||
{
|
||
"output_type": "execute_result",
|
||
"data": {
|
||
"text/plain": [
|
||
"<matplotlib.collections.PathCollection at 0x7f4cc3440ad0>"
|
||
]
|
||
},
|
||
"metadata": {
|
||
"tags": []
|
||
},
|
||
"execution_count": 27
|
||
},
|
||
{
|
||
"output_type": "display_data",
|
||
"data": {
|
||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd5xU9b3/8ddnZnZnC0WaIqKCXSxE3UiIGsSKFTXq1dxr7MRekvvLtXujN15bOpZwNZY8jDUxoqLYg8aoLIoCIoqFACIsSN02Mzuf3x+7moWdhZWZnTMz5/18PHiw8z2H832rn/l4OHPmfM3dERGR0hcJOoCIiOSHGr6ISEio4YuIhIQavohISKjhi4iERCzoAOvTv39/HzJkSNAxpERNmzZtqbsPyPe8qmvpTuur64Ju+EOGDKG2tjboGFKizGxeEPOqrqU7ra+udUlHRCQk1PBFREJCDV9EJCTU8EVEQkINH2hY3ciMV2czf87CoKOI5NSiTxYz49XZrF6+JugoUgAK+i6dfHj0FxO558oHcRxPO4N33IKbn7+avgP7BB1NZKOtWVHP1UffyOw3PsIihqed4y4+nLNvPgUzCzqeBCTUZ/hTJ0/nrsseIJlIkUq00JJKM2/WfC7e56qgo4lk5boTfsHM1z6gJdVCKpGiJdXCo794kifGPxt0NAlQqBv+PVc+SLol3WH8i0+X8NE7nwSQSCR7q75czfSXZmbcdu81D+U5jRSSUDf8RZ8u7nRb7eTpeUwikjtrVtTT2ToX9Ssb8pxGCkmoG/5mW3f+rfpB2w7MYxKR3Nl0q/6dbovGQv2WD71Q/9c//fqTIMPnV2XxMkYcsVf+A4nkQCwWY5d9dsq4bd/vfyfPaaSQhLrhjzhiL75/6ZFEYxHMjEg0QllFGddP/C8qquJBxxPZaNf99acMHLppW21DrCzKVjsP5pI7xgUdTQIU+tsyz7n1VMaeN4baydOpqK7gu2NrqO5dHXQskaz06teT+z76HW+/MIMFcz5nq2GD+dboXYhEQn2OF3qhb/gAm2+zGUede2jQMURyKhKJUHPIcGoOGR50FCkQ+t+9iEhIqOGLiISEGr6ISEio4YuIhIQavohISKjhi4iERE4avpn9wcyWmFnGJzaZ2f5mttLMprf9uiYX84p0J9W1lJpc3Yd/LzAeuH89+7zq7kfmaD6RfLgX1bWUkJyc4bv7FODLXBxLpFCorqXU5PMa/kgze9fMnjGzXTrbyczGmVmtmdXW1dXlMZ7IRlFdS9HIV8N/G9ja3YcDvwP+2tmO7j7B3WvcvWbAgM4fXyxSAFTXUlTy0vDdfZW7r2n7eRJQZmadP7RbpAiorqXY5KXhm9lAa1s52cz2bpt3WT7mFukuqmspNjm5S8fMHgT2B/qb2QLgWqAMwN3vBI4HzjWzFNAInOSdrcEmUiBU11JqctLw3f3kDWwfT+vtbSJFQ3UtpUbftBURCYmSafirvlzN3OmfUr+yPugoIjnj7iz4aBGfzZpPOp0OOo4UuaJf8SqVTPHrcyfw0gOvURaPkUqkOPr8MZx9039oOTcpav/8YCH/fdwtLJlXh0WMql5VXPGnixk+qtPb/UXWq+g74t1X/IlXHvw7yeYkDasaSTQlefKO53j8t5OCjiay0ZKJJD/Z/xoWzFlIc2OCpvpmvly0nKuO/F+WLVoedDwpUkXd8NPpNE/d+RzNjYm1xpsbmnn0F08GlEoke29NeodEY5J17/lJt6R5/v5XAskkxa+oG34q2UJinWb/ldXLVuc5jUjufLloOS2plg7jiaYkS+brVn/ZOEXd8MvjZQzabmDGbTt+e7s8pxHJnV333Qms43hljwr2GL1r/gNJSSjqhg9w4fiziFeV0/aFRyIRo6I6zjm/PDXgZCIbb+huW/OdI2uoqI5/PVZeWc7gHQcx8uiaAJNJMSv6u3T2PGh3fvm36/jTDX9h3vsL2GGvbfnBFcey9bAtg44mkpXLH7iIyfe8wtO/f45kIsUBP9iPYy48jFhZ0b9tJSBWyN8Er6mp8dra2qBjSIkys2nunvfTZdW1dKf11XXRX9IREZGuUcMXEQkJNXwRkZBQwxcRCQk1fBGRkMhJwzezP5jZEjOb2cl2M7PfmtlcM3vPzPbMxbwi3Ul1LaUmV2f49wJj1rP9MGD7tl/jgDtyNK9Id7oX1bWUkJw0fHefAny5nl3GAvd7qzeATcxs81zMLdJdVNdSavJ1DX8LYH671wvaxkSKmepaikrBfWhrZuPMrNbMauvq6oKOI5ITqmspBPlq+AuB9g+3Gdw21oG7T3D3GnevGTBgQF7CiWwk1bUUlXw1/InAD9vuavgOsNLdF3X3pJ9//AX/eLKW+XMyvgdFshVIXbe0tPDelPd5c9Lb1K9q6O7ppITk5LF7ZvYgsD/Q38wWANcCZQDuficwCTgcmAs0AKfnYt7OJJqT/PykX1E7eTqx8hipZAu77bcz//2X/0dFVXzDBxCh8Ooa4ON3P+Pyw35OU30TZkYq2cL5vzmdw886qLunlhKQk4bv7idvYLsD5+dirq64/9qHqX3uXRJNSRJNSQBmTHmfO39yH5fcMS5fMaTIFVpdt6RauOyQ61lRt2qt8dsvuYcdarZlu28NzVcUKVIF96FtLjz9fy90WPow0ZTk+fteoZAfBy2yPu+8NJNEc7LDeLI5xdMTng8gkRSbkmz4zQ3NGceTzSnS6XSe04jkRv3KBshwvpJuSbNq2Zr8B5KiU5INf/fvDcMyrAe604jtiEaj+Q8kkgO7jxpGKpnqMF5RHWffY0cEkEiKTUk2/PN+cwZVvaooi5cBECuPUdmjgotuPzvgZCIbr8+mvfmPa04gXhX/+oSmojrOtt8ayn7fV8OXDSvJxTG32mkL7n7/10y87VnmTJ3LNsOHcMwFY9h0K93/LMXt5MuOZdjIHXh6wgvUr2xg1AkjGX3yPlrnVrqkZKuk3+Z9OP1/1nuThUhRGj5qF4aP2iXoGFKESvKSjoiIdKSGLyISEmr4IiIhoYYvIhISavgiIiGhhi8iEhJq+CIiIaGGLyISEmr4IiIhoYYvIhISOWn4ZjbGzOaY2VwzuyzD9tPMrM7Mprf9OisX84p0N9W2lJKsn6VjZlHgNuBgYAEw1cwmuvv76+z6sLtfkO18Ivmi2pZSk4sz/L2Bue7+ibsngIeAsTk4rkjQVNtSUnLR8LcA5rd7vaBtbF3fN7P3zOwxM9uys4OZ2TgzqzWz2rq6uhzEE9loOatt1bUUgnx9aPskMMTddweeB+7rbEd3n+DuNe5eM2CAnl8vBa9Lta26lkKQi4a/EGh/VjO4bexr7r7M3b9aaPYuYK8czCvS3VTbUlJy0fCnAtub2VAzKwdOAia238HMNm/38mhgdg7mFeluqm0pKVnfpePuKTO7AJgMRIE/uPssM7sOqHX3icBFZnY0kAK+BE7Ldt5cSSaS/OXXT/PMXS+SSrUw+qR9OPny46jqWRl0NAlYsdf2go8Wcf+1DzPj1dn0G9SHky8/jn2O2TvoWBIgc/egM3SqpqbGa2tru+347s7lY/6Hma99QHNjAoCyeBlbbD+QO6bdrHVCS5yZTXP3mnzP2911DfD5x19w7l4/pWlNE+l063u8oirO6TeczHEXHdGtc0uw1lfXof6m7QdvzWXW63O+bvYAyeYkiz+r4/UnpgaYTCQ7f7zuUZrqm79u9gBNDc3ce9VDJJqTASaTIIW74b/5ES2pdIfxxjVNzPz7BwEkEsmNma99QLqlY21jsOiTxfkPJAUh1A1/wJb9iJVHO4zHK8sZOHTTABKJ5MZmW2e+9TOVbKHPZr3znEYKRagb/ogj9qSyRyUWsbXGo2VRDvr37wWUSiR7J19+LPGq+Fpj5RVl7DP22/Tq2zOgVBK0UDf8svIyfv3q9Wy/51DK4mWUV5QxeMdB3PLitfTqpzeFFK+9Dh7OhePPpEefaiqq45TFy9j3uBH85O7zgo4mAQr9bSibb7MZt711E8sXryCVbGHA4H5BRxLJiUNPG81B//E9lsxfSq9+PanuVRV0JAlY6Bv+V/pstknQEURyLhqLsvnQzYKOIQUi1Jd0RETCRA1fRCQk1PBFREJCDV9EJCTU8EVEQkINX0QkJHRbZh698dQ0nhj/DKtX1LPf90dw9LmHUtlDj2GW4ubp1XjDH6HpRYj0wapPxeL7BR1LMlDDz5N7rnmIv/zqKZrqWxdH+nTGP3nu3r9x29QbqVjnK/AixcLTa/Blx0DLEqC1tj0xFe9xPpEe44INJx3okk4eLF+8gkdvnfh1swdINCZYPK+O5+//W4DJRLLjDQ9DSx1fNftWjbDmd3h6VVCxpBM5afhmNsbM5pjZXDO7LMP2uJk93Lb9TTMbkot5i8X7//iQsvKOf5lqbmjmrUlvB5BIukq1vQHNLwFNHcetDJIz8h5H1i/rhm9mUeA24DBgGHCymQ1bZ7czgeXuvh3wK+CmbOdtz92ZU/sxbzw1jS+/WJ7LQ+dE7wG98HTHlcUi0Qh9N+8TQCLpikKo7cb6JqY++w7TX55JKpnK5aFzI7opYBk2tEBEtV1ocnENf29grrt/AmBmDwFjgffb7TMW+O+2nx8DxpuZeQ7WV1z6+Zdcdsj1LJ5XRyQaIdmcYuwFYxh38ymYZSrE/Bs2cgd69e9JU0PzWo2/LB7j6PMODTCZbECgtf23R17n1jNvJxJtPS+LlkW5/onL2OW7O2Z76Jyxqh/iTS+y9ll+BCJbQGznoGJJJ3JxSWcLYH671wvaxjLu4+4pYCWQ8bGUZjbOzGrNrLaurm6Dk193/K3Mn/M5TfXNNKxqJNmc5Kk7n+OVh1/fmH+WbhGJRLjlhWsZvMMgKqrjVPWqpKpnJZdOOIdthw8JOp50Lme1/U3retEni7n59Nu+ruuGVY2sXraGKw7/OY31GS6hBMTK94BeV4NVgvUAKiG2Pdb37oI54ZJ/Kbi7dNx9AjABWhd7Xt++S+Yv5ePpn3VYyq2pvpnHf/s0o0/ap/uCfkObb7MZd8/6FfPeX0DDqga223MbyuNlOTn26uVr+OCtuWwyoBfb7TFUb7QC9E3qGuC5+18hnWrJdBzefGoa+/9b4dR2pOoEvPIoSL4Pkd5YbNucHdtTc6Hlc4jtjEUzr+IlXZeLhr8Q2LLd68FtY5n2WWBmMaA3sCzbiRtWNRKNRYGOizKvWdGQ7eFzzswYssuWG97xG3jopsf5488epSxeRkuqhU236s//PnMlm26lN0cOBFbbq7+sJ5Xs2PDTLWnqVxZibVdA+Z45O56nV+LLx0FydusHwN6MV/0b1vMqndBkIReXdKYC25vZUDMrB04CJq6zz0Tg1LafjwdeysU1zi13HEQsw90vZeUx9jnm29kevuBNnTydB/7nzySaktSvbKCpvpkFHy7iqqNuDDpaqQistkccvgcVPSo6jHva2ePA3bI9fMHzFT+F5EygCXw1kICGx/CGR4KOVtSybvht1y0vACYDs4FH3H2WmV1nZke37XY30M/M5gI/Bjrc3rYxorEoP/6/c4hXlX/9wVa8spw+m2/CCf959Ab+dPF7/LdPr3VvP7SeAX7+8RfMm70goFSlI8ja3uuQ4ez+vZ2pqP7Xl/IqquMcdd6hDNp2YC6mKFieXgWJ1+j4N/dGaLgviEglIyfX8N19EjBpnbFr2v3cBJyQi7nWte+xI/jdP27gr+OfZfG8OmoOGc5hZx0YiuXcVi5dnXE8GouyZnl9ntOUpqBqOxKJcN0T/8WUR9/gpQdfpTxexpgzD6TmkOG5nqrweD2dnov6yrxGKTUF96Htxhi629Zc+vsfBR0j7/Y5Zm8+m/FPEk1rnwml0852ewzJ6VzuSWiZB9YHi2rd33yIRqOMPmmfgrr5IC8iAyGyCaQXr7MhCvH9cz6dp5dDehlEt6L1yl3p0qMVitjY88fQf3A/4pWtRWpmxKvinPfr04hX5u75POmGJ/AlI/Flx+N1o0h/eZa+Ni/dxsyw3jcAFUC0bbQcrDfW48KczePeRHrFJfiS/Vpre8kI0vX35+z4hagkzvDDqrpXFXdMu5lJ//cCbzw1jX6D+nDMhYez84jtczaHJ6bCqqtZ64s1iX/gKy7E+up6qnQPi+8H/f6MN9wLqXlQPgKr/ncs0jdnc/jKK1qf8EkCPNE6uPoXeHQQVnFQzuYpJGr4Ra6qZyXH//gojv/xUd1yfK+/m47PSklC4m28ZSEWXfd7SCK5YWXbY71/3i3H9vRqaHoOSKyzpRGvv7NkG35JXtJZvXwNX3y2hHQ6veGdZf1aPs88bmXQsjS/WULO3Vk8r46VS3U5LWvp5fzrctE6Wtb97KB0lNQZ/urla7jp1PG8/fx7RKJGVa8qLv39jxh5VE3Q0YpX+Xch9TEdbpHzFOTwG5WyftNfnsnNp41n1dLVpNNpho3ckSv+dDF9B+oBZRslOqjtC12N62yIQHnpfoenpM7wrz3mZqY99y7J5iTNDQmWf7GCn5/8K+a+82nQ0YqWVZ/R9oyU9ucGldDjAizSI6hYobLok8VcfdSN1M1fRnNjgmRzipmvfcBPD76OHHzHK5TMYtDzclo/GP5KBKwK63FxULG6Xck0/AUfLeLD2o9JJdZ+hGyiKcljv3wyoFTFz6KbYv2fgMoTILollO2BbXKLVjPKoyduf7bDo5FbUi0smbeU2W9+FFCq4hep+j7W53Yo2xuig6HiaKzf41hs66CjdZuiuqSzdOEyJt4+mbnvfMoOe23DUeeNoV/b8+Tr5i8lVh6juXHtD2E87Sz6pHSvyeWDRQdivX8WdIyS9u4rs5h014s01Tcx6sTvMuqEkW3PiYJFcxdnfK6OmVE3fxl8J99pS4fF98Xi+wYdI2+KpuF/OvOfXLLvVSSbkySbU0x/eRZ/Hf8sv3n952y982C2HT6EZHPHh6iVxWN8a/SuASQW6Zr7f/YIj9wykURjM+7w9gvvMfnel7lh0hVEo1G+dcAuTHvhXZob1j6ZSSVT7FCzTUCppRgVzSWd351/V9vz7lv/aptsTtKwqoHbLvoDAL369WTsBWPWevZINBahqmclx1x0eCCZRTakbsEyHr7przQ3tDZ7aH289/v/+JA3n2pd/vKQ00bTq19PYmX/uqskXhVn/5P2YfOhmwURW4pUUTR8d2fm3z/IMA7v/W3W16/PvukULrrtbIbuvhX9t+jLIaeN5o63b6HPpr3zGVeky6a/PJNIrOPtgU1rmnh94lSg9Qt2t9fexJHnHMKALfux1c6DGXfLKfzkrnPzHVeKXFFc0jEzyivKaW5o7rCt/SMEzIyDfziKg384Kp/xRDZada8qIhme7x6NRejZ9193QW0yoDfn/+YMzv/NGfmMJyWmKM7wAQ49bX/KK9ZeIaq8oowxZx4QUCKR7NUcOvzrR3u3Fy2LMeb00QEkklJWNA3/7JtPYbfvDSNeWU5Vr0rKK8sZPnpXzrzhB0FHE9lo5RXl/O+zV9Krf0+qelZS1auSeGU5F912FlsPy+3qaCJFcUkHoKIqzo3PXsW82QtYMOdzttp5C7bcUc9xkeK3097b8/DCCcx4dTbNDQl2HzWMqp6VQceSEpRVwzezvsDDwBDgM+BEd1+eYb8WYEbby3+6+0YvR7X1zoPZeufBG/vHRbok37UdK4uxxwGlv3ShBCvbSzqXAS+6+/bAi3S+vFuju3+r7Vfprz0opUC1LSUn24Y/Fvjqoej3AcdkeTyRQqHalpKTbcPfzN0Xtf38BdDZt0AqzKzWzN4ws/W+ccxsXNu+tXV1dVnGE9loOa1t1bUUgg1ewzezF4CBGTZd2f6Fu7uZdfbovq3dfaGZbQO8ZGYz3P3jTDu6+wRgAkBNTY0eBSjdaQczm5lhPOe1rbqWQrDBhu/unS79YmaLzWxzd19kZpsDSzo5xsK23z8xs1eAPYCMDV8kjz5094yLJai2pRRle0lnInBq28+nAk+su4OZ9TGzeNvP/YF9gPeznFeku6m2peRk2/BvBA42s4+Ag9peY2Y1ZnZX2z47A7Vm9i7wMnCju+tNIYVOtS0lJ6v78N19GXBghvFa4Ky2n18HdIOxFBXVtpSionm0goiIZEcNX0QkJNTwRURCQg1fRCQk1PBFREJCDV9EJCTU8EVEQkINX0QkJNTwRURCQg1fRCQk1PBFREJCDV9EJCTU8EVEQkINX7qdN79OetnxpBfvRXrpsXjzlKAjiWTNPUF6zXjSS/YjvfjbpFf8FG9ZHHSs9VLDl27lzVPw5edA8j3w1ZCahS+/gHTj80FHE8mKr7gY1kyA9GLwldD0JL7sODy9Ouhoncqq4ZvZCWY2y8zSZpZxqbi2/caY2Rwzm2tml2UzpxQXX3Uj0LTOaBOsuTGIOF2m2pb18dRcaP47a9d2C6RX441/DirWBmV7hj8TOA7o9O/oZhYFbgMOA4YBJ5vZsCznlWLR8mkn4/NxT+c3yzej2pbOJWeDRTNsaILE23mP01VZNXx3n+3uczaw297AXHf/xN0TwEPA2GzmlSISGZB53PpiVrhXFFXbsl7RLQHPsKEcYtvmO02X5eMdtwUwv93rBW1jGZnZODOrNbPaurq6bg8n3azH+WCV6wxWQo9zAomTY12ubdV1iSkbDtGt6LBKrMWwqn8LJFJXbLDhm9kLZjYzw69uOZNx9wnuXuPuNQMGdHJ2KEXDKk+EHpeC9QTKW3/vcT5WdWrQ0QB2yFdtq65Li5lhfe+D+CigDIhBbAes7x+x6MCg43Vqg4uYu/tBWc6xENiy3evBbWMSAmaGVZ+GV50CvgqsJ2YbLLt8+dDdO/1AtgtU2yFmkT5YnztwbwRPYZGeQUfaoHxc0pkKbG9mQ82sHDgJmJiHeaWAmEVb3yCF0+xzQbUtmFUWRbOH7G/LPNbMFgAjgafNbHLb+CAzmwTg7ingAmAyMBt4xN1nZRdbpHuptqUUZXW65e6PA49nGP8cOLzd60nApGzmEskn1baUosK9L05ERHJKDV9EJCRK6hM06bqlC5cx5bE3SDQlGXHEngzddaugI4lkzd2Z8epsZrw6mz6b9mbUiSOp7l0ddKyCoYYfQi8//HduPf12wGlJpfnjdY9y9HmH8qNbfhh0NJGNlkqmuGbsTcx4dTaJxgTlleX8/j/v56bnr2anvbcPOl5B0CWdkFmzop5fnHE7iaYEiaYkLakWEo0JnrrzOWa9vqEnCYgUrmfufon3psymqb6ZdNppqm+mYXUjPzv+VtwzPQYhfNTwQ+atZ94hEuv4n725IcGLD+g59VK8Jt/zMs0NzR3G16xo4LOZ/wwgUeFRww8bnelI6Kjmv6KGHzLfPmwPWlIdH0tcXlnOAT/YL4BEIrlx6OmjiVfFO4z36F3NEN2UAKjhh07PPj34yd3nUl5RRnlFGdFYlPLKco4852B23WenoOOJbLTDzjyA3fbdiYrqOBYxKqrjVPWs5JrH/hMzCzpeQdBdOiF0wEn7svt+OzPl0TdINCVab8vcbeugY4lkJVYW44ZnruS9Ke8z89UP6LOZbstclxp+SPXfoh/HXXJE0DFEcsrMGD5qF4aP2iXoKAVJl3REREJCDV9EJCTU8EVEQkINX0QkJNTwRURCItsVr04ws1lmljazTtcGNbPPzGyGmU03s9ps5hTJB9W2lKJsb8ucCRwH/L4L+45296VZzieSL6ptKTnZLnE4G9C32KTkqLalFOXrGr4Dz5nZNDMbt74dzWycmdWaWW1dXV2e4olstC7VtupaCsEGz/DN7AVgYIZNV7r7E12cZ193X2hmmwLPm9kH7p7xWbzuPgGYAFBTU6PH3El32sHMZmYYz3ltq66lEGyw4bv7QdlO4u4L235fYmaPA3sDevh6EXN3SH0AnoCyYZiVBR1pY3zo7p1+INsVqu3S4+k1kPoQIv2xWGk9ZbPbn6VjZtVAxN1Xt/18CHBdd88r3ceTH+LLfwS+HDAgBpvcisVHBR0tr1TbpSe95m5Y8xuwGHgSL9sF63M7FukbdLScyPa2zGPNbAEwEnjazCa3jQ8ys0ltu20GvGZm7wJvAU+7+7PZzCvBcU/gX54C6YXgDeD14Cvx5RfiLQuDjpczqu3w8aaXYc1vgSbwNUAzJN/Dl18YdLScyfYunceBxzOMfw4c3vbzJ8DwbOaRAtI8BUhk2NCCN/wZ63lRvhN1C9V2+HjDPUDjOqOp1qbfsgiLbh5ErJzSN23lm0kvA++4YhYkIa27T6SItXRSvxaD9PL8ZukmavjyzZTvDWRo+FaFxffNexyRnImPAjq5+SC2XV6jdBc1fPlGLDYUKscCle1GKyC2A8QPDCqWSNas+myI9AbK241WQM8rMSvv7I8VFa14Jd+Y9boeykfiDQ8BTVBxNFZ1ImYqJyleFu0H/Z/C6+9r/awqOhCrPgMr/3bQ0XJG71D5xswMKo/AKrVEopQWi/TFel4KPS8NOkq30CUdEZGQUMMXEQkJNXwRkZBQwxcRCQk1fBGRkFDDFxEJCTV8EZGQUMMXEQkJNXwRkZBQwxcRCQk1fBGRkMh2xatbzOwDM3vPzB43s0062W+Mmc0xs7lmdlk2c4rkg2pbSlG2Z/jPA7u6++7Ah8Dl6+5gZlHgNuAwYBhwspkNy3Jeke6m2paSk1XDd/fn3D3V9vINYHCG3fYG5rr7J+6eAB4CxmYzr0h3U21LKcrlNfwzgGcyjG8BzG/3ekHbWEZmNs7Mas2stq5OS+ZJQci6tlXXUgg22PDN7AUzm5nh19h2+1wJpIAHsg3k7hPcvcbdawYMGJDt4UTWZ4d81bbqWgrBBhdAcfeD1rfdzE4DjgQOdHfPsMtCYMt2rwe3jYkE7UN3r+lso2pbSk22d+mMAX4KHO3uDZ3sNhXY3syGWuvCkCcBE7OZV6S7qbalFGV7DX880BN43symm9mdAGY2yMwmAbR98HUBMBmYDTzi7rOynFeku6m2peRktaatu2/XyfjnwOHtXk8CJmUzl0g+qbalFGkRcyk57mlIvIY3vQSRXljlMVhsm6BjiWTN06vwxschNRdiu2CVR2GR6i7/eTV8KSnuLfiK8yHxBngDEMPr78V7XU+kSrfIS/Hy1N54wcoAAAOCSURBVKf4shPBE0AjUInXj4d+j2HRgV06hp6lI6Wl+YV2zR5a76hsglVX4+k1QSYTyYqvvAp8Fa3Nntbf08vwVTd2+Rhq+FJSvPGpds2+HYu2/o9ApAi5JyE5DVj37uAWSLzc5eOo4UtpsQrAMm0Ai+c7jUiOROi8XZd9o6OIlAyrPB6oyLAlAuUj8h1HJCfMohA/kI4fu5ZDRdc/m1LDl5Ji8RFQfRoQByrBqsGqsT530vrdKJHiZL1/BtEhrTVNBVgVlA3Dev64y8fQXTpSciI9L8UrT4DE661vjvhoLFIVdCyRrFikL/R/ChJvQss8iO0AZXtglukSZmZq+FKSLDYYYicGHUMkp8wiEB8JjNyoP69LOiIiIaGGLyISEmr4IiIhoYYvIhISavgiIiFhmRfyKQxmVgfM68Yp+gNLu/H4uVAMGaE4c27t7nlfb1B1/TXlzJ0u1XVBN/zuZma161virhAUQ0ZQzkJSLP+Mypk7Xc2oSzoiIiGhhi8iEhJhb/gTgg7QBcWQEZSzkBTLP6Ny5k6XMob6Gr6ISJiE/QxfRCQ01PBFREIi1A3fzE4ws1lmljazgrvtyszGmNkcM5trZpcFnScTM/uDmS0xs5lBZ+mMmW1pZi+b2ftt/70vDjpTdyvk2i6GuobSrO1QN3xgJnAcMCXoIOsysyhwG3AYMAw42cyGBZsqo3uBMUGH2IAU8BN3HwZ8Bzi/QP9d5lJB1nYR1TWUYG2HuuG7+2x3nxN0jk7sDcx190/cPQE8BHR9LbM8cfcpwJdB51gfd1/k7m+3/bwamA1sEWyq7lXAtV0UdQ2lWduhbvgFbgtgfrvXCyjxJpUPZjYE2AN4M9gkoaW67iZdqe2SX/HKzF4ABmbYdKW7P5HvPBIcM+sB/Bm4xN1XBZ0nW6pt+UpXa7vkG767HxR0ho20ENiy3evBbWOyEcysjNY3xAPu/peg8+RCkda26jrHvklt65JO4ZoKbG9mQ82sHDgJmBhwpqJkras83w3MdvdfBp0n5FTXOfRNazvUDd/MjjWzBbSuCPy0mU0OOtNX3D0FXABMpvWDmEfcfVawqToysweBfwA7mtkCMzsz6EwZ7AOcAhxgZtPbfh0edKjuVKi1XSx1DaVZ23q0gohISIT6DF9EJEzU8EVEQkINX0QkJNTwRURCQg1fRCQk1PBFREJCDV9EJCT+P2uLgQUAFhskAAAAAElFTkSuQmCC\n",
|
||
"text/plain": [
|
||
"<Figure size 432x288 with 2 Axes>"
|
||
]
|
||
},
|
||
"metadata": {
|
||
"tags": [],
|
||
"needs_background": "light"
|
||
}
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/"
|
||
},
|
||
"id": "HUjdeIefsIsg",
|
||
"outputId": "f267f505-8ba4-43ef-9ebe-df124c3c05a1"
|
||
},
|
||
"source": [
|
||
"tf.reduce_mean(tf.cast(((pred[0]>0.5)==valid_labels),tf.float32))"
|
||
],
|
||
"execution_count": 28,
|
||
"outputs": [
|
||
{
|
||
"output_type": "execute_result",
|
||
"data": {
|
||
"text/plain": [
|
||
"<tf.Tensor: shape=(), dtype=float32, numpy=0.4>"
|
||
]
|
||
},
|
||
"metadata": {
|
||
"tags": []
|
||
},
|
||
"execution_count": 28
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "_95qF9lY2kHp"
|
||
},
|
||
"source": [
|
||
"## Используем оптимизаторы TensorFlow\r\n",
|
||
"\r\n",
|
||
"Tensorflow достаточно плотно интегрирован с библиотекой Keras, которая содержит в себе множество полезного. Например, мы можем использовать оптимизаторы, реализующие немного другие алгоритмы обучения, чем градиентный спуск.\r\n",
|
||
"\r\n",
|
||
"Также попробуем выводить точность на всех этапах обучения."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/"
|
||
},
|
||
"id": "ups7nlV22ofp",
|
||
"outputId": "aa4dff06-82b9-4b2f-ca00-33970ea2b989"
|
||
},
|
||
"source": [
|
||
"optimizer = tf.keras.optimizers.Adam(0.01)\r\n",
|
||
"\r\n",
|
||
"learning_rate = 0.05\r\n",
|
||
"\r\n",
|
||
"W = tf.Variable(tf.random.normal(shape=(2,1)))\r\n",
|
||
"b = tf.Variable(tf.zeros(shape=(1,),dtype=tf.float32))\r\n",
|
||
"\r\n",
|
||
"@tf.function\r\n",
|
||
"def train_on_batch(x, y):\r\n",
|
||
" vars = [W, b]\r\n",
|
||
" with tf.GradientTape() as tape:\r\n",
|
||
" z = tf.sigmoid(tf.matmul(x, W) + b)\r\n",
|
||
" loss = tf.reduce_mean(tf.keras.losses.binary_crossentropy(z,y))\r\n",
|
||
" correct_prediction = tf.equal(tf.round(y), tf.round(z))\r\n",
|
||
" acc = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))\r\n",
|
||
" grads = tape.gradient(loss, vars)\r\n",
|
||
" optimizer.apply_gradients(zip(grads,vars))\r\n",
|
||
" return loss,acc\r\n",
|
||
"\r\n",
|
||
"for epoch in range(40):\r\n",
|
||
" for step, (x, y) in enumerate(dataset):\r\n",
|
||
" loss,acc = train_on_batch(tf.reshape(x,(-1,2)), tf.reshape(y,(-1,1)))\r\n",
|
||
" print('Epoch %d: last batch loss = %.4f, acc = %.4f' % (epoch, float(loss),acc))"
|
||
],
|
||
"execution_count": 30,
|
||
"outputs": [
|
||
{
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Epoch 0: last batch loss = 6.1202, acc = 0.6667\n",
|
||
"Epoch 1: last batch loss = 5.9872, acc = 0.6667\n",
|
||
"Epoch 2: last batch loss = 5.8651, acc = 0.6667\n",
|
||
"Epoch 3: last batch loss = 5.7551, acc = 0.8333\n",
|
||
"Epoch 4: last batch loss = 5.6570, acc = 0.8333\n",
|
||
"Epoch 5: last batch loss = 5.5700, acc = 0.8333\n",
|
||
"Epoch 6: last batch loss = 5.4931, acc = 0.8333\n",
|
||
"Epoch 7: last batch loss = 5.4255, acc = 0.8333\n",
|
||
"Epoch 8: last batch loss = 5.3660, acc = 0.8333\n",
|
||
"Epoch 9: last batch loss = 5.3140, acc = 0.8333\n",
|
||
"Epoch 10: last batch loss = 5.2685, acc = 0.8333\n",
|
||
"Epoch 11: last batch loss = 5.2289, acc = 0.8333\n",
|
||
"Epoch 12: last batch loss = 5.1945, acc = 0.8333\n",
|
||
"Epoch 13: last batch loss = 5.1649, acc = 0.8333\n",
|
||
"Epoch 14: last batch loss = 5.1394, acc = 0.8333\n",
|
||
"Epoch 15: last batch loss = 5.1177, acc = 0.8333\n",
|
||
"Epoch 16: last batch loss = 5.0993, acc = 0.8333\n",
|
||
"Epoch 17: last batch loss = 5.0840, acc = 0.8333\n",
|
||
"Epoch 18: last batch loss = 5.0714, acc = 0.8333\n",
|
||
"Epoch 19: last batch loss = 5.0612, acc = 0.8333\n",
|
||
"Epoch 20: last batch loss = 5.0532, acc = 0.8333\n",
|
||
"Epoch 21: last batch loss = 5.0472, acc = 0.6667\n",
|
||
"Epoch 22: last batch loss = 5.0430, acc = 0.6667\n",
|
||
"Epoch 23: last batch loss = 5.0404, acc = 0.6667\n",
|
||
"Epoch 24: last batch loss = 5.0391, acc = 0.6667\n",
|
||
"Epoch 25: last batch loss = 5.0391, acc = 0.6667\n",
|
||
"Epoch 26: last batch loss = 5.0403, acc = 0.8333\n",
|
||
"Epoch 27: last batch loss = 5.0424, acc = 0.8333\n",
|
||
"Epoch 28: last batch loss = 5.0454, acc = 0.8333\n",
|
||
"Epoch 29: last batch loss = 5.0491, acc = 0.8333\n",
|
||
"Epoch 30: last batch loss = 5.0535, acc = 0.8333\n",
|
||
"Epoch 31: last batch loss = 5.0584, acc = 0.8333\n",
|
||
"Epoch 32: last batch loss = 5.0638, acc = 0.8333\n",
|
||
"Epoch 33: last batch loss = 5.0696, acc = 0.8333\n",
|
||
"Epoch 34: last batch loss = 5.0756, acc = 0.8333\n",
|
||
"Epoch 35: last batch loss = 5.0818, acc = 0.8333\n",
|
||
"Epoch 36: last batch loss = 5.0882, acc = 0.8333\n",
|
||
"Epoch 37: last batch loss = 5.0946, acc = 0.8333\n",
|
||
"Epoch 38: last batch loss = 5.1010, acc = 0.8333\n",
|
||
"Epoch 39: last batch loss = 5.1073, acc = 0.8333\n"
|
||
],
|
||
"name": "stdout"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "dvAiaj_JndyP"
|
||
},
|
||
"source": [
|
||
"**Задание 1**: Постройте графики ошибок на обучающей и тестовой выборке в процессе обучения\r\n",
|
||
"\r\n",
|
||
"**Задание 2**: Попробуйте решить задачу классификации на датасете MNIST с помощью этого кода. Подсказка: используйте `softmax_crossentropy_with_logits` или `sparse_softmax_cross_entropy_with_logits` в качестве функции ошибки. При этом в первом случае на выход сети необходимо подавать целевые значения в формате *one hot encoding*, а во втором - в виде целочисленного номера класса."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "995iCprDrgYQ"
|
||
},
|
||
"source": [
|
||
"## Keras\r\n",
|
||
"### Deep Learning for Humans\r\n",
|
||
"\r\n",
|
||
"* Раньше работал поверх Tensorflow, CNTK или Theano, сейчас включен в состав Tensorflow\r\n",
|
||
"* Оперирует нейросетями на уровне слоёв\r\n",
|
||
"* Включает упрощённый \"обучатель\", средства работы с типовыми данными (картинками, ...)\r\n",
|
||
"* Много готовых примеров\r\n",
|
||
"* Functional API vs. Sequential API\r\n",
|
||
"\r\n",
|
||
"Keras даёт более высокоуровневое API для реализации нейросетей, позволяя определять нейросети как комбинации слоёв и оперировать понятиями \"модель\", \"слой\", \"алгоритм обучения\".\r\n",
|
||
"\r\n",
|
||
"Книжка от создателя Keras: [Deep Learning with Python](https://www.manning.com/books/deep-learning-with-python)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/"
|
||
},
|
||
"id": "QJWplVfy34Eo",
|
||
"outputId": "9be976f2-4f9a-495c-bddc-a7f9ec30989a"
|
||
},
|
||
"source": [
|
||
"inputs = tf.keras.Input(shape=(2,))\r\n",
|
||
"z = tf.keras.layers.Dense(1,kernel_initializer='glorot_uniform',activation='sigmoid')(inputs)\r\n",
|
||
"model = tf.keras.models.Model(inputs,z)\r\n",
|
||
"\r\n",
|
||
"train_x_norm = train_x-np.min(train_x) / (np.max(train_x)-np.min(train_x))\r\n",
|
||
"\r\n",
|
||
"model.compile(tf.keras.optimizers.Adam(0.1),'binary_crossentropy',['accuracy'])\r\n",
|
||
"model.summary()\r\n",
|
||
"h = model.fit(train_x_norm,train_labels,batch_size=8,epochs=15)"
|
||
],
|
||
"execution_count": 32,
|
||
"outputs": [
|
||
{
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Model: \"model_1\"\n",
|
||
"_________________________________________________________________\n",
|
||
"Layer (type) Output Shape Param # \n",
|
||
"=================================================================\n",
|
||
"input_2 (InputLayer) [(None, 2)] 0 \n",
|
||
"_________________________________________________________________\n",
|
||
"dense_1 (Dense) (None, 1) 3 \n",
|
||
"=================================================================\n",
|
||
"Total params: 3\n",
|
||
"Trainable params: 3\n",
|
||
"Non-trainable params: 0\n",
|
||
"_________________________________________________________________\n",
|
||
"Epoch 1/15\n",
|
||
"9/9 [==============================] - 0s 1ms/step - loss: 1.1443 - accuracy: 0.3283\n",
|
||
"Epoch 2/15\n",
|
||
"9/9 [==============================] - 0s 2ms/step - loss: 0.6759 - accuracy: 0.5823\n",
|
||
"Epoch 3/15\n",
|
||
"9/9 [==============================] - 0s 1ms/step - loss: 0.4717 - accuracy: 0.7888\n",
|
||
"Epoch 4/15\n",
|
||
"9/9 [==============================] - 0s 1ms/step - loss: 0.3932 - accuracy: 0.9044\n",
|
||
"Epoch 5/15\n",
|
||
"9/9 [==============================] - 0s 1ms/step - loss: 0.4254 - accuracy: 0.8513\n",
|
||
"Epoch 6/15\n",
|
||
"9/9 [==============================] - 0s 1ms/step - loss: 0.3866 - accuracy: 0.8534\n",
|
||
"Epoch 7/15\n",
|
||
"9/9 [==============================] - 0s 1ms/step - loss: 0.4578 - accuracy: 0.8344\n",
|
||
"Epoch 8/15\n",
|
||
"9/9 [==============================] - 0s 1ms/step - loss: 0.4432 - accuracy: 0.8192\n",
|
||
"Epoch 9/15\n",
|
||
"9/9 [==============================] - 0s 1ms/step - loss: 0.4172 - accuracy: 0.8408\n",
|
||
"Epoch 10/15\n",
|
||
"9/9 [==============================] - 0s 1ms/step - loss: 0.3778 - accuracy: 0.8819\n",
|
||
"Epoch 11/15\n",
|
||
"9/9 [==============================] - 0s 1ms/step - loss: 0.4008 - accuracy: 0.8800\n",
|
||
"Epoch 12/15\n",
|
||
"9/9 [==============================] - 0s 3ms/step - loss: 0.3769 - accuracy: 0.9019\n",
|
||
"Epoch 13/15\n",
|
||
"9/9 [==============================] - 0s 1ms/step - loss: 0.3995 - accuracy: 0.8336\n",
|
||
"Epoch 14/15\n",
|
||
"9/9 [==============================] - 0s 1ms/step - loss: 0.3816 - accuracy: 0.8719\n",
|
||
"Epoch 15/15\n",
|
||
"9/9 [==============================] - 0s 1ms/step - loss: 0.4211 - accuracy: 0.8244\n"
|
||
],
|
||
"name": "stdout"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/",
|
||
"height": 282
|
||
},
|
||
"id": "K2Kf60IrZcqs",
|
||
"outputId": "b60b868d-3562-4715-f5d5-1f9764e45f09"
|
||
},
|
||
"source": [
|
||
"plt.plot(h.history['accuracy'])"
|
||
],
|
||
"execution_count": 37,
|
||
"outputs": [
|
||
{
|
||
"output_type": "execute_result",
|
||
"data": {
|
||
"text/plain": [
|
||
"[<matplotlib.lines.Line2D at 0x7f4cbdcce910>]"
|
||
]
|
||
},
|
||
"metadata": {
|
||
"tags": []
|
||
},
|
||
"execution_count": 37
|
||
},
|
||
{
|
||
"output_type": "display_data",
|
||
"data": {
|
||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAbKElEQVR4nO3dfXRU933n8fdXT0iAAAkE2EJIGAtsbKd+EPhpm+MmddZJeiDd7Db41E+JU7a7cZJts921d3u8OW67p/vUdM+ptz3eOMa1U7Neb9LQLq3tk6Yn3fgBCRs/AJYgWAIBGgQSYiTQ43z3jxnhsRBoECPdufd+Xudw0Ny5zHyA0Ud3fvd3f2PujoiIhF9R0AFERCQ/VOgiIhGhQhcRiQgVuohIRKjQRUQioiSoJ16yZIk3NDQE9fQiIqG0a9euE+5eM9l9gRV6Q0MDLS0tQT29iEgomVnHhe7TkIuISESo0EVEIkKFLiISESp0EZGIUKGLiESECl1EJCJU6CIiERHYPHSZvv6hUX74VidXLKzglvoqquaVBR1JRAqACj1kegeGeeiZnbzT2XduW+PS+TQ1VNFUX836hmrqqiswswBTikgQVOgh0tU3yP1Pv0lHzxn+9NdvpnpeGS0dvTS39/DX7x7jhZ2HAaipnMP6rIK/9opKSoo1uiYSdSr0kGg/McB9T79J78Awz355A7evXgzArVelf0+lnLbjSZrbe9nV3kNzey873usCYG5ZMTetXERTfTVNDVXctLKK+XP0Xy8SNRbUR9A1NTW51nLJzb5jp7n/6Z2MpVI8+5UNfGLFopz+3NFTZ2np+KjgP+g6TcqhyGDdlQvOFfz6hmqWLSif4b+FiOSDme1y96ZJ71OhF7ZdHb18+ZmdzC0r4bmHN9C4rHLaj5UcHOHtQ6doyRT87sOnODsyBkBddQVN9enhmeKi/A/PLK2cQ1NDFVcsrMj7Y+fL4MgY7x3pY+/R04ym9Fm7M6Gs2LhhxSKuu3IBpQU6DOjuHO45y65DPfQMjMzIc9yxejHXXrFgWn/2YoWu990F7B/2d7Plz3exbMEcnnv4Vuqq517W41WWl/LJNTV8ck165c2RsRR7j56mub2HXR29/MP+E/zw7SP5iH5BtYsq0idwG6pZ31DFmqWVFBUFcwK3d2CYXR29NHf00NLey3udfQyPpQLJEjflpUXcWLeI9Q3V3FJfxc31VSwoLw0ky+hYin3HkrRkXgfN7T0cTw7N6HP+/heun3ahX4yO0AvU37x3jG9se5vVNfP584c3sLRy5odE3J3k0Ch5f0k4HOo5c+4HR/Y3TGV5CbfUp4d9muqr+IW6RZSXFuc5QPrvdqjnDC3tvbR0pN+hHDjeD0BpsXFD7UKaZjiDwMDQKLsPn6K5PV2ee4+dZizlmME1yxewvqHq3OvhykUz825uYGiUtw+dOvd6fOtQL2eG0+9Usw84muqrZixDeWkRc0qm9xrTkEvIvNh8mEd/8C43raziew+tZ2FFMEcuM2X8Le14sba097A/q1yvr11IU/1H31SL58+55OcYHUux99jp9EnizPN0Z/0QyX58FXhwsgt+V0cvb3X0MpBVrulyr+KW+mrWLq+keBrv5hKnB88debd09LDvWPLcD5Frly+YlQLPJxV6iPzPnx7kD3bs45Nraviz+25mblk8RsVOnckMf2QK+J3DHw1/XLVkXtYwTTUNi+eeN8++f2iUtw/1nvsBsfvwqXNHXSuqKs4V+PqGahqXzg9smEcubnQsxQddyfR5no70/2XidOYH8ZwSbq6vOvd/eWPdIirKPv6DOJVyDnT3p39AtKeH0w73nAXSR8U31WV+QDRUc9PKRYEN81wOFXoIuDv/7ZU2/uQnB/j8DVfwnS/dSFlJYZ40mg2DI2O8f6SPlsw3dUtHL6fOpE9QLZ5XRlNDFTfWVZE4PUhzew/7jn00g+faKxacG5st9BOxcnHuTmfvR+/mdrX30ppIAlBSZFxXu5D1mauld3X0squjl76z6dfJkvlzMuWfHsJZV8AnYi+FCr3ApVLOf9i+h+fe6GDz+jr+4FdvmNZbyyhLpZyDJ/ppzrx13tXRS8fJM1SUZubYZ06yao599PWdGeGtQ73nxuF3d55ieDTF6pp56XMxmeGT+kneyUWBCr2AjYyl+Nf/+x1+tPso//yTV/HoZ6+J5ItwJvQODDO/vCQSR10yfUOjYwwOp1g4N3zDJ9OhaYsFanBkjK99/y1+/MFxfucfr+Vf3rVaZX4JtCiZAMwpKZ72jJGoUaEHJDk4wlefbWFnew+/94Xruf+2+qAjiUjIqdADcLJ/iAef2ckHx5L88ZduZNONtUFHEpEIUKHPsqOnznL/02/S2XuWpx64hU9dsyzoSCISESr0WfThiQHu++6bnD47wnMP38qGVdVBRxKRCFGhz5I9R/t48Hs7cYcXttzG9bULg44kIhGjQp8Fze09fGVrM5VzSnjuq7eyumZ+0JFEJIJU6DPs71uP85vP7+LKhRU899VbqQ3BWhEiEk4q9BnUOzDMv3j+La5akl4xcck0FpkSEcmVLrGbQduaD3N2ZIzvfOlGlbmIzLicCt3M7jGzVjM7YGaPTnL/SjP7iZm9bWbvmtnn8h81XEbHUjz3ejt3rF7M2uXT/5QhEZFcTVnoZlYMPAl8FlgH3Gtm6ybs9rvAi+5+E7AZ+B/5Dho2r+5NcLRvkIfuaAg6iojERC5H6BuAA+5+0N2HgW3Apgn7ODD+eUoLgaP5ixhOW19rZ0VVBZ++VhcOicjsyKXQa4HDWbc7M9uyfRu4z8w6gR3A1/OSLqT2Hj3Nmx/28MDt9VoGV0RmTb5Oit4LbHX3FcDngOfM7LzHNrMtZtZiZi3d3d15eurC8+xr7VSUFvOlppVBRxGRGMml0I8AdVm3V2S2ZXsYeBHA3V8HyoElEx/I3Z9y9yZ3b6qpqZle4gLXMzDMX+4+wq/eXBub9ZlFpDDkUujNQKOZrTKzMtInPbdP2OcQ8GkAM7uWdKFH9xD8IrY1H2JoNKWToSIy66YsdHcfBR4BXgb2kZ7NssfMnjCzjZndvgX8hpm9A7wAPORBfRRSgEbHUjz/egd3rF7MmmWaqigisyunK0XdfQfpk53Z2x7P+novcGd+o4XP+FTFb2+8LugoIhJDulI0j57RVEURCZAKPU/2HO1j54c9PHh7g6YqikggVOh5Mj5V8dea6qbeWURkBqjQ8yA9VfEo/0RTFUUkQCr0PNjWfIjh0RQPaqqiiARIhX6Z0qsqdnDn1ZqqKCLBUqFfplf2JjjWN8hDd6wKOoqIxJwK/TJt/Vk7ddUVfOqapUFHEZGYU6Ffhj1H+9jZrqmKIlIYVOiXYXyq4j/TVEURKQAq9Gn62FTFCk1VFJHgqdCn6YWd6amKWlVRRAqFCn0aRsZSPP9GB//o6iU0aqqiiBQIFfo0vLJnfKpiQ9BRRETOUaFPw9bXPqSuuoJf0lRFESkgKvRL9P6RPprbezVVUUQKjgr9EmmqoogUKhX6JTjZP8SP3jnKF2/RVEURKTwq9EuwrflwelXF2xuCjiIich4Veo5GMqsqaqqiiBQqFXqOXtmToOu0piqKSOFSoedo62sfsrJ6rqYqikjBUqHnYHyq4gO312uqoogULBV6Dra+1s7cMk1VFJHCpkKfwon+IbZrVUURCQEV+hS27TzE8JimKopI4VOhX0R6VcVD/GKjpiqKSOHLqdDN7B4zazWzA2b26CT3f8fMdmd+tZnZqfxHnX0v7+nSVEURCY2SqXYws2LgSeBuoBNoNrPt7r53fB93/62s/b8O3DQDWWfd1p+1U794Lr+0VlMVRaTw5XKEvgE44O4H3X0Y2AZsusj+9wIv5CNckN7r7KOlo5cHbm+gSFMVRSQEcin0WuBw1u3OzLbzmFk9sAr4uwvcv8XMWsyspbu7+1KzzqqPpiquCDqKiEhO8n1SdDPwkruPTXanuz/l7k3u3lRTU5Pnp86fE/1D/NU7R/nizStYUK6piiISDrkU+hEg+4qaFZltk9lMBIZbzk1VvKM+6CgiIjnLpdCbgUYzW2VmZaRLe/vEnczsGqAKeD2/EWfXyFiK597o4Bcbl3D1Uk1VFJHwmLLQ3X0UeAR4GdgHvOjue8zsCTPbmLXrZmCbu/vMRJ0df/t+F4nTQ3z5zoago4iIXJIppy0CuPsOYMeEbY9PuP3t/MUKztbX0lMV71qjqYoiEi66UjTLe5197NJURREJKRV6Fk1VFJEwU6FnjE9V/Ke3aKqiiISTCj3jhTfTUxUf0KqKIhJSKvSMH39wnKb6Kq5eOj/oKCIi06JCB1IpZ38iyfW1C4OOIiIybSp04MipswwMj7FGa56LSIip0IG2RBKAtcs13CIi4aVCB1ozha5PJRKRMFOhA21dSa5cWK7piiISaip0oDXRz5rlOjoXkXCLfaGPjqX4+fF+1mq4RURCLvaF3tFzhuGxlMbPRST0Yl/obV2ZGS4qdBEJudgXemsiiRm6QlREQi/2hd6WSFJfPZeKsuKgo4iIXJbYF3prV1JXiIpIJMS60AdHxmg/eYa1mrIoIhEQ60I/2D3AWMp1hC4ikRDrQv9oDRcVuoiEX6wLvTWRpLTYaFg8L+goIiKXLdaF3taV5Kol8ykrifU/g4hERKybrDWR1BouIhIZsS30/qFROnvPsnaZLigSkWiIbaHv1xroIhIxMS70fkBruIhIdORU6GZ2j5m1mtkBM3v0Avv8mpntNbM9ZvYX+Y2Zf62JJOWlRdRVzw06iohIXpRMtYOZFQNPAncDnUCzmW13971Z+zQCjwF3unuvmS2dqcD50pZI0ri0kuIiCzqKiEhe5HKEvgE44O4H3X0Y2AZsmrDPbwBPunsvgLsfz2/M/NMaLiISNbkUei1wOOt2Z2ZbtjXAGjP7mZm9YWb35CvgTOgdGOZ4coi1yzXDRUSiY8ohl0t4nEbgLmAF8FMzu8HdT2XvZGZbgC0AK1euzNNTX7rxS/51hC4iUZLLEfoRoC7r9orMtmydwHZ3H3H3D4E20gX/Me7+lLs3uXtTTU3NdDNfNq3hIiJRlEuhNwONZrbKzMqAzcD2Cfv8Jemjc8xsCekhmIN5zJlXrYkkleUlLF9QHnQUEZG8mbLQ3X0UeAR4GdgHvOjue8zsCTPbmNntZeCkme0FfgL8jrufnKnQl6utq5+1yyox0wwXEYmOnMbQ3X0HsGPCtsezvnbgtzO/Cpq705pI8vlPXBF0FBGRvIrdlaLHk0P0nR3RFaIiEjmxK/TWLs1wEZFoil2hfzRlUXPQRSRaYlnoS+aXsXj+nKCjiIjkVewKvTXRr+EWEYmkWBV6KuXsT2gNFxGJplgV+pFTZzkzPKYrREUkkmJV6JrhIiJRFq9C1wwXEYmwWBV6WyJJ7aIKKstLg44iIpJ3sSr09Ida6OhcRKIpNoU+MpbiYPcAa3RCVEQiKjaF3nFygOGxlNZwEZHIik2ht3b1A5rhIiLRFZ9CTyQpMrh6qcbQRSSaYlPobV1J6hfPo7y0OOgoIiIzIj6FflwzXEQk2mJR6IMjY7SfGNAJURGJtFgU+s+7+0k5mrIoIpEWi0If/1ALHaGLSJTFotBbu/opLTYalswLOoqIyIyJRaG3JZKsrplPaXEs/roiElOxaLj0Gi4abhGRaIt8oScHRzhy6qw+1EJEIi/yhb7/uC75F5F4iHyht3VphouIxEPkC701kaSitJgVVRVBRxERmVE5FbqZ3WNmrWZ2wMweneT+h8ys28x2Z359Nf9Rp6ctkb7kv6jIgo4iIjKjSqbawcyKgSeBu4FOoNnMtrv73gm7/i93f2QGMl6W1q5+7lpbE3QMEZEZl8sR+gbggLsfdPdhYBuwaWZj5UfPwDAn+oc0fi4isZBLodcCh7Nud2a2TfRFM3vXzF4ys7rJHsjMtphZi5m1dHd3TyPupRm/5F9ruIhIHOTrpOhfAQ3u/gngVeDZyXZy96fcvcndm2pqZn4YRGu4iEic5FLoR4DsI+4VmW3nuPtJdx/K3PwucEt+4l2e1q4kC8pLWLZgTtBRRERmXC6F3gw0mtkqMysDNgPbs3cwsyuybm4E9uUv4vS1JZKsXV6JmWa4iEj0TVno7j4KPAK8TLqoX3T3PWb2hJltzOz2DTPbY2bvAN8AHpqpwLlyd63hIiKxMuW0RQB33wHsmLDt8ayvHwMey2+0y5M4PcTpwVGt4SIisRHZK0Vbx2e46AhdRGIisoU+voaLCl1E4iKyhd6aSFJTOYfqeWVBRxERmRWRLfS2RFLzz0UkViJZ6KmU05ZI0rhsftBRRERmTSQL/XDvGQZHUjpCF5FYiWShtyUyn1KkKYsiEiMRLfT0DJfGpRpyEZH4iGSht3YlqV1UQWV5adBRRERmTSQLfXwNFxGROIlcoY+Mpfh5d78uKBKR2IlcobefGGBkzFm7XOPnIhIvkSt0reEiInEVuUJv60pSZLC6RkfoIhIvkSv01kSShiXzKC8tDjqKiMisilyhtyX6dYWoiMRSpAp9cGSM9pMDGj8XkViKVKEfON6Pu06Iikg8RarQWzMfaqEpiyISR5Eq9LbjScqKi6hfPC/oKCIisy5ahd6V5KqaeZQWR+qvJSKSk0g1X1uiX2u4iEhsRabQk4MjHDl1VidERSS2IlPo4x9qoTnoIhJXESr08RkuKnQRiafIFHprV5K5ZcXULqoIOoqISCByKnQzu8fMWs3sgJk9epH9vmhmbmZN+YuYm7ZEksZllRQV2Ww/tYhIQZiy0M2sGHgS+CywDrjXzNZNsl8l8E3gzXyHzEVbIsnaZbqgSETiK5cj9A3AAXc/6O7DwDZg0yT7/R7wn4DBPObLyYn+IU70D2uGi4jEWi6FXgsczrrdmdl2jpndDNS5+//NY7ac6YSoiEgeToqaWRHwR8C3cth3i5m1mFlLd3f35T71OW1d+pQiEZFcCv0IUJd1e0Vm27hK4Hrg782sHbgN2D7ZiVF3f8rdm9y9qaamZvqpJ2hN9LOwopSllXPy9pgiImGTS6E3A41mtsrMyoDNwPbxO929z92XuHuDuzcAbwAb3b1lRhJPYn8iydpllZhphouIxNeUhe7uo8AjwMvAPuBFd99jZk+Y2caZDjgVd6c1kWSNlswVkZgryWUnd98B7Jiw7fEL7HvX5cfKXdfpQZKDo7rkX0RiL/RXirbqhKiICBCBQh+fsqhCF5G4C32ht3b1s7RyDlXzyoKOIiISqNAXelsiqQuKREQIeaGPpZz9x5MabhERIeSFfrjnDIMjKc1wEREh5IXeOn5CVEMuIiLhLvTxNVwal+qiIhGRUBd6ayLJiqoK5s3J6fooEZFIC3Wht2XWcBERkRAX+vBoioPdAxo/FxHJCG2ht58cYDTlOkIXEckIbaFrDRcRkY8LbaG3JZIUFxlX1cwLOoqISEEIbaG3diVpWDyX8tLioKOIiBSE0Ba61nAREfm4UBb62eExOnrOaPxcRCRLKAv9wPF+3NEMFxGRLKEsdK3hIiJyvlAWelsiSVlJEfXVc4OOIiJSMEJZ6K1dSVbXzKekOJTxRURmRCgbMb2Gi1ZYFBHJFrpC7zs7wrG+QY2fi4hMELpCP3A8fUJUM1xERD4udIXe2tUPaA0XEZGJQlfoS+aXcfe6ZdQuqgg6iohIQQndR/185rrlfOa65UHHEBEpODkdoZvZPWbWamYHzOzRSe7/TTN7z8x2m9n/M7N1+Y8qIiIXM2Whm1kx8CTwWWAdcO8khf0X7n6Du98I/Gfgj/KeVERELiqXI/QNwAF3P+juw8A2YFP2Du5+OuvmPMDzF1FERHKRyxh6LXA463YncOvEnczsa8BvA2XApyZ7IDPbAmwBWLly5aVmFRGRi8jbLBd3f9LdVwP/FvjdC+zzlLs3uXtTTU1Nvp5aRETIrdCPAHVZt1dktl3INuALlxNKREQuXS6F3gw0mtkqMysDNgPbs3cws8asm58H9ucvooiI5GLKMXR3HzWzR4CXgWLge+6+x8yeAFrcfTvwiJn9MjAC9AIPzmRoERE5n7kHMyHFzLqBjmn+8SXAiTzGmWlhyhumrBCuvGHKCuHKG6ascHl569190pOQgRX65TCzFndvCjpHrsKUN0xZIVx5w5QVwpU3TFlh5vKGbi0XERGZnApdRCQiwlroTwUd4BKFKW+YskK48oYpK4Qrb5iywgzlDeUYuoiInC+sR+giIjKBCl1EJCJCV+hTrc1eKMyszsx+YmZ7zWyPmX0z6Ey5MLNiM3vbzP466CwXY2aLzOwlM/vAzPaZ2e1BZ7oYM/utzOvgfTN7wczKg86Uzcy+Z2bHzez9rG3VZvaqme3P/F4VZMZxF8j6XzKvhXfN7IdmtijIjOMmy5p137fMzM1sSb6eL1SFnuPa7IViFPiWu68DbgO+VsBZs30T2Bd0iBz8d+Bv3f0a4Bco4MxmVgt8A2hy9+tJX3G9OdhU59kK3DNh26PAj929Efhx5nYh2Mr5WV8Frnf3TwBtwGOzHeoCtnJ+VsysDvgMcCifTxaqQieHtdkLhbsfc/e3Ml8nSRdObbCpLs7MVpBei+e7QWe5GDNbCHwSeBrA3Yfd/VSwqaZUAlSYWQkwFzgacJ6PcfefAj0TNm8Cns18/SwFsujeZFnd/RV3H83cfIP0IoKBu8C/K8B3gH9Dnj87ImyFPtna7AVdkgBm1gDcBLwZbJIp/THpF1kq6CBTWAV0A89khoe+a2bzgg51Ie5+BPivpI/GjgF97v5KsKlysszdj2W+7gKWBRnmEnwF+JugQ1yImW0Cjrj7O/l+7LAVeuiY2Xzg/wD/asInOxUUM/sV4Li77wo6Sw5KgJuBP3X3m4ABCmc44DyZsedNpH8QXQnMM7P7gk11aTw9v7ng5zib2b8nPdz5/aCzTMbM5gL/Dnh8Jh4/bIV+qWuzB8rMSkmX+ffd/QdB55nCncBGM2snPZT1KTN7PthIF9QJdLr7+Duel0gXfKH6ZeBDd+929xHgB8AdAWfKRcLMrgDI/H484DwXZWYPAb8C/LoX7gU2q0n/YH8n8722AnjLzJbn48HDVuhTrs1eKMzMSI/x7nP3gv/QbHd/zN1XuHsD6X/Xv3P3gjyKdPcu4LCZrc1s+jSwN8BIUzkE3GZmczOvi09TwCdxs2zno6WwHwR+FGCWizKze0gPF2509zNB57kQd3/P3Ze6e0Pme60TuDnzmr5soSr0zEmP8bXZ9wEvuvueYFNd0J3A/aSPdHdnfn0u6FAR8nXg+2b2LnAj8B8DznNBmXcSLwFvAe+R/r4rqEvVzewF4HVgrZl1mtnDwB8Cd5vZftLvMv4wyIzjLpD1T4BK4NXM99qfBRoy4wJZZ+75CvediYiIXIpQHaGLiMiFqdBFRCJChS4iEhEqdBGRiFChi4hEhApdRCQiVOgiIhHx/wHO1VhtMoQ9YwAAAABJRU5ErkJggg==\n",
|
||
"text/plain": [
|
||
"<Figure size 432x288 with 1 Axes>"
|
||
]
|
||
},
|
||
"metadata": {
|
||
"tags": [],
|
||
"needs_background": "light"
|
||
}
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "iJruFXmb_dur"
|
||
},
|
||
"source": [
|
||
"Выше мы использовали **функциональный** способ задания модели, когда мы сначала описываем входную переменную, затем - происходящие с ней преобразования, и потом определяем объект `Model`.\r\n",
|
||
"\r\n",
|
||
"Мы можем также задавать модель как последовательности слоёв с помощью `Sequential`:"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"colab": {
|
||
"base_uri": "https://localhost:8080/"
|
||
},
|
||
"id": "iWc_kSr8_YXt",
|
||
"outputId": "345dbe65-629d-468f-ed75-1d412c966340"
|
||
},
|
||
"source": [
|
||
"model = tf.keras.models.Sequential()\r\n",
|
||
"model.add(tf.keras.layers.Dense(5,activation='sigmoid',input_shape=(2,)))\r\n",
|
||
"model.add(tf.keras.layers.Dense(1,activation='sigmoid'))\r\n",
|
||
"\r\n",
|
||
"test_x_norm = test_x-np.min(train_x) / (np.max(train_x)-np.min(train_x))\r\n",
|
||
"# это не ошибка, мы нормируем тестовые данные так же, как нормировали обучающие!\r\n",
|
||
"# (поэтому min и max считаем для обучающих данных)\r\n",
|
||
"\r\n",
|
||
"model.compile(tf.keras.optimizers.Adam(0.1),'binary_crossentropy',['accuracy'])\r\n",
|
||
"model.summary()\r\n",
|
||
"model.fit(train_x_norm,train_labels,validation_data=(test_x_norm,test_labels),batch_size=8,epochs=15)"
|
||
],
|
||
"execution_count": 45,
|
||
"outputs": [
|
||
{
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Model: \"sequential_6\"\n",
|
||
"_________________________________________________________________\n",
|
||
"Layer (type) Output Shape Param # \n",
|
||
"=================================================================\n",
|
||
"dense_14 (Dense) (None, 5) 15 \n",
|
||
"_________________________________________________________________\n",
|
||
"dense_15 (Dense) (None, 1) 6 \n",
|
||
"=================================================================\n",
|
||
"Total params: 21\n",
|
||
"Trainable params: 21\n",
|
||
"Non-trainable params: 0\n",
|
||
"_________________________________________________________________\n",
|
||
"Epoch 1/15\n",
|
||
"9/9 [==============================] - 1s 19ms/step - loss: 0.6962 - accuracy: 0.5596 - val_loss: 0.5533 - val_accuracy: 0.7333\n",
|
||
"Epoch 2/15\n",
|
||
"9/9 [==============================] - 0s 5ms/step - loss: 0.5057 - accuracy: 0.8035 - val_loss: 0.4924 - val_accuracy: 0.8000\n",
|
||
"Epoch 3/15\n",
|
||
"9/9 [==============================] - 0s 5ms/step - loss: 0.4173 - accuracy: 0.8726 - val_loss: 0.5273 - val_accuracy: 0.6667\n",
|
||
"Epoch 4/15\n",
|
||
"9/9 [==============================] - 0s 4ms/step - loss: 0.4405 - accuracy: 0.8145 - val_loss: 0.5641 - val_accuracy: 0.6667\n",
|
||
"Epoch 5/15\n",
|
||
"9/9 [==============================] - 0s 5ms/step - loss: 0.3523 - accuracy: 0.9039 - val_loss: 0.5498 - val_accuracy: 0.7333\n",
|
||
"Epoch 6/15\n",
|
||
"9/9 [==============================] - 0s 5ms/step - loss: 0.3239 - accuracy: 0.9012 - val_loss: 0.5666 - val_accuracy: 0.6667\n",
|
||
"Epoch 7/15\n",
|
||
"9/9 [==============================] - 0s 5ms/step - loss: 0.4043 - accuracy: 0.8568 - val_loss: 0.5883 - val_accuracy: 0.6667\n",
|
||
"Epoch 8/15\n",
|
||
"9/9 [==============================] - 0s 5ms/step - loss: 0.3351 - accuracy: 0.8903 - val_loss: 0.5906 - val_accuracy: 0.6667\n",
|
||
"Epoch 9/15\n",
|
||
"9/9 [==============================] - 0s 5ms/step - loss: 0.3624 - accuracy: 0.8761 - val_loss: 0.6198 - val_accuracy: 0.6667\n",
|
||
"Epoch 10/15\n",
|
||
"9/9 [==============================] - 0s 4ms/step - loss: 0.4015 - accuracy: 0.8622 - val_loss: 0.6087 - val_accuracy: 0.6667\n",
|
||
"Epoch 11/15\n",
|
||
"9/9 [==============================] - 0s 5ms/step - loss: 0.3387 - accuracy: 0.8846 - val_loss: 0.6604 - val_accuracy: 0.6667\n",
|
||
"Epoch 12/15\n",
|
||
"9/9 [==============================] - 0s 5ms/step - loss: 0.3579 - accuracy: 0.8694 - val_loss: 0.6177 - val_accuracy: 0.7333\n",
|
||
"Epoch 13/15\n",
|
||
"9/9 [==============================] - 0s 5ms/step - loss: 0.3304 - accuracy: 0.8830 - val_loss: 0.6274 - val_accuracy: 0.6667\n",
|
||
"Epoch 14/15\n",
|
||
"9/9 [==============================] - 0s 4ms/step - loss: 0.3193 - accuracy: 0.8807 - val_loss: 0.6472 - val_accuracy: 0.6667\n",
|
||
"Epoch 15/15\n",
|
||
"9/9 [==============================] - 0s 4ms/step - loss: 0.3552 - accuracy: 0.8546 - val_loss: 0.6596 - val_accuracy: 0.6667\n"
|
||
],
|
||
"name": "stdout"
|
||
},
|
||
{
|
||
"output_type": "execute_result",
|
||
"data": {
|
||
"text/plain": [
|
||
"<tensorflow.python.keras.callbacks.History at 0x7f4cbdbf7890>"
|
||
]
|
||
},
|
||
"metadata": {
|
||
"tags": []
|
||
},
|
||
"execution_count": 45
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "BmHNhUU8bqEX"
|
||
},
|
||
"source": [
|
||
"## Функции ошибки при классификации\r\n",
|
||
"\r\n",
|
||
"При использовании Keras для классификации важно правильно указать функцию ошибки и передаточную функцию на последнем слое. Основные правила:\r\n",
|
||
"* Если у сети один выход, то передаточная функция - сигмоида, если несколько - softmax\r\n",
|
||
"* Если на ожидаемый выход подается в виде one-hot-encoding, то функция ошибки - cross entropy loss (categorical cross-entropy), если номер класса - sparse categorical cross-entropy, для бинарной классификации с одним выходом - binary cross-entropy (она же log loss)\r\n",
|
||
"\r\n",
|
||
"В целом бинарную классификацию можно рассматривать как частный случай мультиклассовой, подавая на выход one hot encoded вектор.\r\n",
|
||
"\r\n",
|
||
"| Классификация | Формат входных данных | Передат.функция | Функция ошибки |\r\n",
|
||
"|---------------|-----------------------|-----------------|----------|\r\n",
|
||
"| Бинарная | Вероятность 1-го класса | sigmoid | binary crossentropy |\r\n",
|
||
"| Бинарная | One-hot encoding (2 выхода) | softmax | categorical crossentropy |\r\n",
|
||
"| Мультикласс | One-hot encoding | softmax | categorical crossentropy |\r\n",
|
||
"| Мультикласс | Номер класса | softmax | sparse categorical crossentropy"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "yX6hqiafwHl9"
|
||
},
|
||
"source": [
|
||
"## Выводы\n",
|
||
"\n",
|
||
"* Tensorflow позволяет более гибко определять структуру графа вычислений, описывать свои функции и конфигурации.\n",
|
||
"* Есть более удобные средства для работы с данными (`td.Data`), со слоями (`tf.layers`)\n",
|
||
"* Для массового использования нейросетей Google рекомендует **Keras**, который позволяет собирать нейросети как конструктор\n",
|
||
"* При этом возможно реализовать свой слой для Keras, и потом использовать его в своих моделях.\n",
|
||
"* Для типовых задач имеет смысл использовать Keras\n",
|
||
"* Также стоит посмотреть на PyTorch, это \"восходящая звезда\"\n",
|
||
"\n",
|
||
"Хороший Notebook про Keras и Tensorflow 2.0 от создателя Keras - [тут](https://t.co/k694J95PI8)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "gZ-kWx84bMDH"
|
||
},
|
||
"source": [
|
||
"**Задание 3**: \r\n",
|
||
"Используйте Keras для обучения классификатора на сети MNIST. При этом:\r\n",
|
||
"* Обратите внимание, что в keras заложены типовые датасеты, включая MNIST. Для обращения к нему достаточно пары строчек кода (см, например, [тут](https://www.tensorflow.org/api_docs/python/tf/keras/datasets/mnist))\r\n",
|
||
"* Попробуйте несколько конфигураций сети с несколькими полносвязными слоями, передаточными функциями, и разным количеством нейронов\r\n",
|
||
"\r\n",
|
||
"Какой точности вам удалось достичь?\r\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": {
|
||
"trusted": true,
|
||
"id": "MG64NKzawHl-"
|
||
},
|
||
"source": [
|
||
""
|
||
],
|
||
"execution_count": null,
|
||
"outputs": []
|
||
}
|
||
]
|
||
} |