path: root/logistic_regression_notebook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic Regression\n",
    "\n",
    "Logistic regression is a *binary classification algorithm*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'Data')"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEWCAYAAABsY4yMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAASeElEQVR4nO3de5Cdd13H8fcn2SaQmjYiMQNNIOJAC9YxbWJDBUGLrJVLqlgvYBSVGVoviHhBaGcQ61RRB6+AtHJTGooIYldQGphiocq0JDVCLxQ6WCEFmwD2RqElydc/npPJZtnL2eyePbu/vF8zz5zzPOf3e57veXbzybO/c87vpKqQJLVn2bALkCQNhgEvSY0y4CWpUQa8JDXKgJekRhnwktQoA16SGmXA67iT5I4kX0tyX5K7k/xHkguTzPjvIcnGJJVkZCFqlebCgNfx6rlVtRp4LPAa4HeANw+3JGl+GfA6rlXVPVU1BvwU8MIkpyd5dpL/THJvks8nefW4Lh/p3d6d5P4kZyf5ziTXJPlyki8l2ZFkzYI/GWkCA14CquoGYC/w/cBXgZ8D1gDPBn4pyY/2mj6td7umqr6lqj4GBPhD4NHAE4ENwKsXrnppcga8dMQXgEdU1b9V1Ser6lBVfQK4Enj6VJ2q6vaq+mBVPVhV+4E/na69tFB8oUg64hTgK0m20o3Lnw6sAFYC/zBVpyTrgL+gu/pfTXfh9H8Dr1aagVfwEpDke+kC/jrgHcAYsKGqTgbeSDcMAzDZ9Kt/0Nv+3VV1ErB9XHtpaAx4HdeSnJTkOcA7gSuq6pN0V+FfqaqvJzkLeMG4LvuBQ8Djxm1bDdwP3JPkFOC3F6Z6aXpxPngdb5LcAawDDtCF9S3AFcAbq+pgkvOB1wKPAK4F7qB7UXV7r/8lwC8BJwDnAvcBfwecCtwOvB14WVWtX7hnJX0zA16SGuUQjSQ1yoCXpEYZ8JLUKANekhq1qD7o9MhHPrI2btw47DIkacnYvXv3l6pq7WSPLaqA37hxI7t27Rp2GZK0ZCT5n6kec4hGkhplwEtSowx4SWqUAS9JjTLgJalRBrwkNcqAl6RGDSzgk7wlyb4kNw3qGJKkqQ3yg05vA15HN0/2QF18MYyNwbZtcOmls+s7NgY7d8LoaNd/bAwuuwzuuutIm3XrYNMm2LOn275uHVxwwZH2v/EbcOedsGZN1/7++2H9ehgZgdNOgw99CO6+G1auhGXL4KGH4MQTYfXqbn8rV8KhQ1AFZ57Zbbvjjq7teed1+/zAB7rjnnEGXHst3HMPHDjQPXa4/4MPwooVXV3r1sHJJ3ftRkfh+uvh8sshgac/HR54AG67Db7xDdi+vdvPjh3dPg4ehMc8Bl71qm77ZZd1txdc0O3n8LneuvXoc9fvOT5W87Uf6bhRVQNbgI3ATf2237x5c83WRRdVddHYLRdd1H/fq66qWrWq67dqVdd35cqj9zfVsmJF137Zsv7aD3MZGTm2fsuXH913+fLJ97tqVXcu+znHU7Wb7c/qWPcjtQbYVVNk6tDH4JO8OMmuJLv2798/6/5jY9OvT2fnzu5KFrrbsbHuCrYfDz3UtT90qP/jDcvhK/3ZOnjw6L4HD06+3wce6M7lZCae46nazWS+9iMdT4Ye8FV1eVVtqaota9dOOl/OtCb+qT6bP91HR2HVqu7+qlVd35Ur++t7eChk2dDP4MxGjnEgbvnyo/suXz75flet6s7lZCae46nazWS+9iMdTxbVZGPH4vCY+7GMwW/bBldeefS47tatsxuD37rVMfjpxsQnO8fHYr72Ix1PBvqdrEk2Au+rqtP7ab9ly5ZyNklJ6l+S3VW1ZbLHBvk2ySuBjwGnJtmb5EWDOpYk6ZsNbIimqp4/qH1Lkma2BF4ilCQdCwNekhplwEtSowx4SWqUAS9JjTLgJalRBrwkNcqAl6RGGfCS1CgDXpIaZcBLUqMMeElqlAEvSY0y4CWpUQa8JDXKgJekRhnwktQoA16SGmXAS1KjDHhJapQBL0mNMuAlqVEGvCQ1yoCXpEYZ8JLUKANekhplwEtSowx4SWqUAS9JjTLgJalRBrwkNcqAl6RGGfCS1CgDXpIaZcBLUqMMeElqlAEvSY0y4CWpUQa8JDXKgJekRhnwktQoA16SGmXAS1KjDHhJapQBL0mNMuAlqVEGvCQ1yoCXpEYZ8JLUKANekhplwEtSowx4SWqUAS9JjTLgJalRBrwkNcqAl6RGGfCS1CgDXpIaZcBLUqMMeElqlAEvSY0y4CWpUQa8JDXKgJekRhnwktQoA16SGmXAS1KjDHhJapQBL0mNMuAlqVEGvCQ1yoCXpEYZ8JLUKANekhplwEtSowx4SWqUAS9Jjeor4JO8NMlJ6bw5yY1JRgddnCTp2PV7Bf+LVXUvMAp8K/CzwGsGVpUkac76Dfj0bp8FvL2qbh63TZK0CPUb8LuT7KQL+KuTrAYODa4sSdJcjfTZ7kXAJuCzVfVAkm8DfmFwZUmS5qqvgK+qQ0k2AtuTFHBdVb13kIVJkuam33fRvAG4EPgkcBNwQZLXD7IwSdLc9DtEcw7wxKoqgCR/C9wysKokSXPW74ustwOPGbe+AfjM/JcjSZov/V7BrwZuTXIDUMBZwK4kYwBVtW1A9UmSjlG/Af+qgVYhSZp3/b6L5tpBFyJJml99BXyS++iGZgBWACcAX62qkwZVmCRpbvq9gl99+H6SAOcBTx5UUZKkuZv1dMHV+SfghwdQjyRpnvQ7RPO8cavLgC3A1wdSkSRpXvT7Lprnjrt/ALiDbphGkrRI9TsG78RikrTE9DsXzfok702yr7e8J8n6QRcnSTp2/b7I+lZgDHh0b/nn3jZJ0iLVb8Cvraq3VtWB3vI2YO0A65IkzVG/Af/lJNuTLO8t24EvD7IwSdLc9P2l28BPAv8LfBE4H7/RSZIWtRnfRZNkOfA8Z4yUpKVlxiv4qjoIPH8BapEkzaN+P+j070leB/w98NXDG6vqxoFUJUmas34DflPv9vd6t6GbXfKcea9IkjQv+g3499EFenrrBdybZFNV7RlIZZKkOen3XTSbgQuBR9F90OkCutkk/ybJywdUmyRpDvq9gl8PnFlV9wMk+V3g/cDTgN3AHw+mPEnSser3Cv7bgQfHrX8DWFdVX5uwXZK0SPR7Bb8DuD7JVb315wLvSHIicMtAKpMkzUm/0wX/fpJ/BZ7S23RhVe3q3f+ZgVQmSZqTfq/g6QX6rhkbSpIWhVl/J6skaWkw4CWpUQa8JDXKgJekRhnwktQoA16SGmXAS1KjDHhJapQBL0mNMuAlqVEGvCQ1yoCXpEYZ8JLUKANekhplwEtSowx4SWqUAS9JjTLgJalRBrwkNcqAl6RGGfCS1CgDXpIaZcBLUqMMeElqlAEvSY0y4CWpUQa8JDXKgJekRhnwktQoA16SGmXAS1KjDHhJapQBL0mNMuAlqVEGvCQ1yoCXpEYZ8JLUKANekhplwEtSowx4SWqUAS9JjTLgJalRBrwkNcqAl6RGGfCS1CgDXpIaZcBLUqMMeElqlAEvSY0y4CWpUQa8JDXKgJekRhnwktQoA16SGmXAS1KjDHhJapQBL0mNMuAlqVEGvCQ1yoCXpEYZ8JLUKANekhplwEtSowx4SWqUAS9JjTLgJalRBrwkNcqAl6RGGfCS1CgDXpIaZcBLUqMMeElqlAEvSY0y4CWpUQMN+CTnJrktye1JXjHIY0mSjjYyqB0nWQ68HngmsBf4eJKxqrplUMeUNLWxMbjssu7+BRfAtm1w8cWwYweMjMCpp8KmTbBnz5E2ADt3wuho1x66PmNjR9avuALuvbdbDh2CFStgwwZYswbWrev2ec89cNddcO21XZ/HPx4+8xl48MHu/sqVcOONkMCjH931Bbjppq7NKafAG94Al1wCn/scnHRS1+bOO2Hfvu64D384PPax8OlPd+sAy5bBE54Az3hG9xyuv76r/bTT4LrrYP9+WLsWnvpU+NSnuue0des3n6fD5++SS7o+27fDpZd22yaen4nnfKbHJzvWvKmqgSzA2cDV49ZfCbxyuj6bN28uSfPvqquqVq6sgm5ZsaLq/POPrE+2jIwc6bNqVbePiy6avs9iXkZG+mu3bNnR5+mqq7plYv/zz+/Oy/jzM/Gcz/T4xJ/JxDb9AHbVFJk6yCGaU4DPj1vf29t2lCQvTrIrya79+/cPsBzp+LVzZ3clfNhDD8E110zf58CBI30eeKDbx9jY4GoctAMH+mt3+OofuvO0c2e3TOx/zTXdeYEj52e8nTtnfnziz2Rim7ka+ousVXV5VW2pqi1r164ddjlSk0ZHu2GQw1asgHPOmb7PyMiRPqtWTT3MsFSM9DkgvWxcKq5Y0T3v0dFv7n/OOd15gSPnZ7zR0Zkfn/gzmdhmrgY2Bg/cCWwYt76+t03SAtu2Dd71rrmPwY8fjz7exuDf857ZjcFv2wZXXjn945P9TOZTuiGc+ZdkBPg08Ay6YP848IKqunmqPlu2bKldu3YNpB5JalGS3VW1ZbLHBnYFX1UHkvwqcDWwHHjLdOEuSZpfgxyioar+BfiXQR5DkjS5ob/IKkkaDANekhplwEtSowx4SWqUAS9JjTLgJalRA/ug07FIsh/4KvClYdcyC49kadULS6/mpVYvWPNCWGr1wmBqfmxVTTrPy6IKeIAku6b6VNZitNTqhaVX81KrF6x5ISy1emHha3aIRpIaZcBLUqMWY8BfPuwCZmmp1QtLr+alVi9Y80JYavXCAte86MbgJUnzYzFewUuS5oEBL0mNWnQBn+Qnktyc5FCSRf0WqCTnJrktye1JXjHsemaS5C1J9iW5adi19CPJhiQfTnJL73fipcOuaSZJHpbkhiT/1av594ZdUz+SLE/yn0neN+xa+pHkjiSfTLInyaL/lqAka5K8O8mnktya5OyFOO6iC3jgJuB5wEeGXch0kiwHXg/8CPAk4PlJnjTcqmb0NuDcYRcxCweA36yqJwFPBn5lCZzjB4Fzqup7gE3AuUmePOSa+vFS4NZhFzFLP1hVm5bIe+H/AvhAVZ0GfA8LdK4XXcBX1a1Vdduw6+jDWcDtVfXZqnoIeCdw3pBrmlZVfQT4yrDr6FdVfbGqbuzdv4/uH8Upw61qetW5v7d6Qm9Z1O9kSLIeeDbwpmHX0qIkJwNPA94MUFUPVdXdC3HsRRfwS8gpwOfHre9lkYfPUpZkI3AGcP1wK5lZb7hjD7AP+GBVLfaa/xx4OXBo2IXMQgE7k+xO8uJhFzOD7wD2A2/tDYO9KcmJC3HgoQR8kg8luWmSZVFfAWs4knwL8B7g16vq3mHXM5OqOlhVm4D1wFlJTh92TVNJ8hxgX1XtHnYts/TUqjqTboj0V5I8bdgFTWMEOBP466o6g26+rQV5zW6g38k6lar6oWEcd57dCWwYt76+t03zKMkJdOG+o6r+cdj1zEZV3Z3kw3SveyzWF7afAmxL8izgYcBJSa6oqu1DrmtaVXVn73ZfkvfSDZku1tft9gJ7x/0l924WKOAdojl2Hwcen+Q7kqwAfhoYG3JNTUkSunHLW6vqT4ddTz+SrE2ypnf/4cAzgU8Nt6qpVdUrq2p9VW2k+x2+ZrGHe5ITk6w+fB8YZfH+B0pV/S/w+SSn9jY9A7hlIY696AI+yY8l2QucDbw/ydXDrmkyVXUA+FXgaroX/95VVTcPt6rpJbkS+BhwapK9SV407Jpm8BTgZ4Fzem+H29O70lzMHgV8OMkn6C4CPlhVS+Kth0vIOuC6JP8F3AC8v6o+MOSaZvISYEfv92IT8AcLcVCnKpCkRi26K3hJ0vww4CWpUQa8JDXKgJekRhnwktQoA17NS/JrvRn8dsyy38YkLxhUXdKgGfA6Hvwy8Myq+plZ9tsIzDrgezONSkNnwKtpSd4IPA741yQX9+bEv6E36dN5vTYbk3w0yY295ft63V8DfH/vA1YvS/LzSV43bt/vS/IDvfv3J3lt78M3ZyfZnOTa3mRYVyd51MI+c8mAV+Oq6kLgC8APAifSfRT/rN76n/Q+6r6P7gr/TOCngL/sdX8F8NHenON/NsOhTgSu780Dfz3wV8D5VbUZeAtw6Tw/NWlGQ5lsTBqSUbqJtX6rt/4w4DF0/wG8Lskm4CDwhGPY90G6SdEATgVOBz7YTafDcuCLc6hbOiYGvI4nAX584hfKJHk1cBfdN+0sA74+Rf8DHP1X78PG3f96VR0cd5ybq2pBvpZNmopDNDqeXA28pDdLJUnO6G0/GfhiVR2im9zs8Iuk9wGrx/W/A9iUZFmSDXRT1E7mNmDt4e/dTHJCku+a12ci9cGA1/Hk9+m+Qu8TSW7urQO8AXhh7wXS0+i+kAHgE8DB3hdovwz4d+C/6aZ6/UvgxskO0vsKx/OBP+rtcw/wfZO1lQbJ2SQlqVFewUtSowx4SWqUAS9JjTLgJalRBrwkNcqAl6RGGfCS1Kj/B6ua70hcje7mAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "from sklearn.datasets import make_blobs\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "\n",
    "# create linearly separable data\n",
    "sep = False\n",
    "while not sep:\n",
    "    X, Y = make_blobs(n_samples=200, n_features\n",
    "                      =1, centers=2, cluster_std=0.5, center_box=(0, 10))\n",
    "    sep = True\n",
    "    for x_1 in X[Y == 1]:\n",
    "        for x_0 in X[Y == 0]:\n",
    "            if Y[X.argmin()] == 0 and x_0 > x_1:\n",
    "                sep = False\n",
    "            elif x_0 < x_1:\n",
    "                sep = False\n",
    "         \n",
    "X = np.hstack([X, np.ones((X.shape[0], 1))])\n",
    "\n",
    "                \n",
    "X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.33)\n",
    "\n",
    "plt.scatter(X[:, 0], Y, s=10, c='b')\n",
    "plt.xlabel('feature')\n",
    "plt.ylabel('groups')\n",
    "plt.yticks([0, 1])\n",
    "plt.title('Data')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Binary target\n",
    "\n",
    "### Sigmoid function\n",
    "\n",
    "We have to fit a **model** trough our data.  \n",
    "The *sigmoid* (also called *logistic*) function is used:\n",
    "$$\\boxed{\n",
    "\\sigma(x) = \\frac{1}{1 + e^{-x}}\n",
    "}$$\n",
    "\n",
    "It produce an 'S' shape like function that stays between 0 and 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3deXhU9d338fc3O5CQQICwE/ZVURZxpSCo4FJ4vOt2o1atdWnV2vZpK3VptZe3bW+3urQ+3Irb7VpKKVVLFTF1QZFd9n0LhiUJhISsM/N7/pjBBgiQhMmcmcnndV1zzXZmzmdg8skvvzlzjjnnEBGR2JfgdQAREQkPFbqISJxQoYuIxAkVuohInFChi4jECRW6iEicUKFLxJjZFDN7P9rWa2Z5ZnbzMe4zM3vRzPaZ2ZdNl7LOdf/DzL4byXVKbDNthy7hZGbnAr8HBgN+YA1wt3NuoafBjsPM8oD/dc49X8d95wFvAP2dcwebMMOvgT7OuWubah0S/5K8DiDxw8xaA+8AtwNvAynAeUCVl7lOUg9ga1OWuUi4aMpFwqkfgHPuDeec3zlX4Zx73zn3FYCZ3WBmnx5a2MwuNLN1ZlZiZn80s38dmvoILfuZmT1hZvvNbLOZnR26fYeZ7ak9HWFmmWb2ipntNbNtZnafmSUcY70XmNna0HqfAayuF2Nm3wOeB84yszIze/DI5wot58ysT+jyS2b2rJm9a2alZrbAzHrXWnawmX1gZsVmttvMfmlmE4BfAleF1rM8tOw3U0FmlhB6TdtCr/0VM8sM3ZcbyvBdM9tuZoVmdm+j/xclZqnQJZzWA34ze9nMJppZm2MtaGbtgBnAVCAbWAecfcRio4CvQve/DrwJjAT6ANcCz5hZemjZp4FMoBfwLeB64MZjrHcmcB/QDtgEnFNXRufcC8BtwOfOuXTn3K9O9A8QcjXwINAG2Ag8HFp3BjAXmAN0Dr2OD51zc4D/At4KrWdoHc95Q+g0NvQa04FnjljmXKA/MA54wMwG1jOvxAkVuoSNc+4AwVJxwP8Ae81stpnl1LH4xcAq59xM55wPeArYdcQyW5xzLzrn/MBbQDfgIedclXPufaAa6GNmiQRLdKpzrtQ5txV4DLjuOOud4ZyrAZ6sY70n66/OuS9Dr+s14LTQ7ZcCu5xzjznnKkNZF9TzOacAjzvnNjvnygj+IrzazGpPmz4Y+qtoObAcqOsXg8QxFbqElXNujXPuBudcV2AIwZHok3Us2hnYUetxDsg/YpndtS5XhJY78rZ0giPtZGBbrfu2AV3qud4ddSx3Mmr/gigPZYTgL6RNjXzOzhz9+pKA2r8sj7VeaSZU6NJknHNrgZcIFvuRCoCuh66YmdW+3kCFQA3BDzAP6Q7sPMZ6ux2x3m51LHcsB4GWtR7fsQGP3UFwuqQuJ9rc7GuOfn0+Dv+lJ82cCl3CxswGmNlPzaxr6Ho34BrgizoWfxc4xcwmh6YNfgg0pBy/EZqSeRt42MwyzKwH8BPgf4+x3sFmdnlovXc1cL3LQ48/zczSgF834LHvAJ3M7G4zSw1lHRW6bzeQe+iD3Dq8AfzYzHqGPjc4NOfua8D6Jc6p0CWcSgl+kLnAzA4SLPKVwE+PXNA5VwhcQXCb9SJgELCIxm/ieCfB0fNm4FOCH6JOP856fxtab1/gs/quxDm3HniI4IebG0Lrqu9jS4ELgMsITo9sIPghJ8CfQ+dFZrakjodPB14FPga2AJUEX7PIN/TFIokKoZFpPjDFOfeR13lEYpFG6OIZM7vIzLLMLJXgdthG3dMzIlIPKnTx0lkEt/ooJDgNMdk5V+FtJJHYpSkXEZE4oRG6iEic8GznXO3atXO5ublN9vwHDx6kVatWTfb8TU35vRPL2UH5vdbU+RcvXlzonGtf132eFXpubi6LFi1qsufPy8tjzJgxTfb8TU35vRPL2UH5vdbU+c1s27Hu05SLiEicUKGLiMQJFbqISJyIqiMW1dTUkJ+fT2Vl5Uk/V2ZmJmvWrAlDKm9EMn9aWhpdu3YlOTk5IusTkaYRVYWen59PRkYGubm5BHeC13ilpaVkZGSEKVnkRSq/c46ioiLy8/Pp2bNnk69PRJrOCadczGx66JBXK49xv5nZU2a20cy+MrNhjQ1TWVlJdnb2SZe51J+ZkZ2dHZa/ikTEW/WZQ38JmHCc+ycS3GNdX+AW4E8nE0hlHnn6NxeJDyeccnHOfWxmucdZZBLwSujIL1+EdrbUyTlXEKaMIiJRwTmHL+Co9gWo9gWo8QeoCp0fun3zfj/pW4up8Tv8AYcvEMDnDz7u0PWAc/j8joBz+APgd45A6P7gbY6Ag0Do9kOXT7SrlnDMoXfh8EN45YduO6rQzewWgqN4cnJyyMvLO+z+zMxMSktLwxAJ/H5/2J7LC5HOX1lZedT/x8koKysL6/NFUixnB+U/xBdwHKyBgzWOcp+josZR4YeKGkelHyp9jgofVPkdVf5/n1f7HTV+qAoEz2sCUFPrcr32fvXF5yedvzEi+qGoc24aMA1gxIgR7shvU61ZsyZsHwTqQ9GGSUtL4/TTTw/b88Xyt/1iOTvEb37nHAcqfew5UMnuA1XsPlDJ3rIqisqqKCyrprCsin3l1ew7WMO+8mrKq/0nXFdqUgKtUpNomZJIy5REWqQl0jYlkRbJiaSFTqlJCaQlJ5KSlEBqUgIpiQmkJP37lJwQOk9MICnRWLNqJcNOG0pSopGUkBA6NxITgteD58HrCQlGohkJCZBoh9+WmGCYQYJZ6BScHrXfHfv1hKPQd3L4MRm7UvexHCWMNm/ezMMPP0xJSQkzZszwOo5IWFRU+9l2wM8/VhSwvbic7cXl5O+r4Ov9wdPBOko6LTmBdumpZLdKoX16Kv06ZJDVMoU2LZNp3SKZzBbJtG6RROu0ZDLSkslIS6JVahKtUhJJSgz/V3FS965ldL86d7XS5MJR6LOBO8zsTYKHHyvR/HnT69WrFy+88ALf+c53vI4i0mAV1X7W7S5l/a5S1u4qZcOeUjbtKePrktDWVvODR+HLaplM1zYt6NW+Fef2bUfnzBbkZKaRk5FKTus02mWk0iolUR/sh5yw0M3sDWAM0M7M8oFfAckAzrnngPeAi4GNQDlwY1OF9VpFRQUTJkxg3rx5JCYm1rlMdXU148ePZ968eSQlRdVm/iKe8PkDrN1VyuJt+/gqv4SVO0vYsKeUQGgyOi05gb4dMhjVK5te7VpRsXcbl4weSbe2LWmdpi+7NUR9tnK55gT3O4JHbI9706dP5/LLLz9mmQOkpKQwbtw43nrrLaZMmRLBdCLRocYfYPmO/czfVMQXm4tYtmP/N/PZ7dJTOaVLay4anMOgzpkM6JhBt7YtSUz49wg7L28ngztnehU/pmkIWYfly5dz5513UlhYyNq1a3HOcf/99zN37lxef/31b5YbO3Ysv/zlL7ngggu47777KCkp4emnn2by5MlMnTq1SQu9qKiIe++9l6VLl/LII48wderUJluXyIkUlFQwb+0e5q3ZwxebizhY7ccMBnZszRXDuzI8ty3De7Shc2aapkeaUNQW+oN/X8Xqrw80+vF+v/+okfSgzq351WWDj/u4yspKrrrqKl555RXOOOMM7r//fiorK7n33nuZNm0atQ/K8eCDD/LAAw+wZ88eli5dyuzZswEYMmQICxcuPOq5zzvvvDo3RXz00UcZP358g15fdnY2zz33XIMeIxJOWwsP8u6KAt5bUcCq0M9qt7Yt+D/DunBO73ac2SubNq1SPE7ZvERtoXtl7ty5DBs2jDPOOAOAU089lTlz5lBUVERWVtZhy44ePRrnHI8//jh5eXnf/AJJTEwkJSXlqE0PP/nkk8i9EJEmUFRWxV+X7mTWsp2s3Bks8WHds7hn4gDGDehAnw7pGoF7KGoL/UQj6RNp7HbcK1eu5JRTTvnm+pIlSxg2bBgtWrQ4an8nK1asoKCggOzs7KPWVVVVRVpa2mG31WeEHskfBh0gXOojEHB8vGEvb365g7lrduMLOIZ2zeS+SwZy8Smd6JzVwuuIEhK1he6V7Oxs5s2bB8D69euZOXMm8+fPp02bNvj9fiorK0lLS6OgoIApU6bwt7/9jbvuuos5c+YwYUJwlzdFRUW0a9fuqN3R1meEfqhkw/3Forlz51JcXExRURG333572J5X4tfBKh8zl+Tz4mdb2Vx4kOxWKdx4Ti5XjOhGv5zY/dJePFOhH+Gaa65h9uzZDBkyhHbt2vHGG2+QnZ0NwIUXXsinn37K2WefzeWXX85jjz3GwIEDuf/++/nFL37xTaF/9NFHXHLJJY1a/6EvDBUVFTFr1qyj7l+xYgWZmZl07979uM8za9asbx4/ceJELr74Yi666CI+++yzRuWS5mN/eTXTP9vKS59t4UClj6FdM/nD1acxcUgnUpJ0TJxopkI/Qnp6On//+9/rvO+HP/whTzzxBOPHj+fzz/+9r4bRo0cfdv3111/nt7/9baPWf+gLQ5MnT67z/sWLF5Obm3tYodfU1PCrX/2K8vJyAoEATz31FJMnT/7mOQKBAA8//DA/+9nPWLRoESNHjmxUNolv+w5W8/ynm3l5/jbKqnxcNDiHW0b3Zlj3LM2LxwgVegMMGzaMsWPH1rkFzSHV1dVMnjyZfv36hXXdq1ev5qmnnmLdunVkZGTw5ptv8uCDD5KTk8O0adOoqKggKyuLLVu2HPXYhIQE7r///rDmkfhR5fPz8vytPD1vI2VVPi4e0ok7x/VhQMfWXkeTBlKhN9BNN9103PtTUlK4/vrrw77eQYMG8dxzz/HSSy+Rm5t72M6Lli5dyrPPPktqamrY1yvxyznHuysK+O0/1pK/r4Ix/dszdeJA+nfU/His0oRYlCkqKuK2225j+fLlPPLII/V6zKRJk7jhhhv4+c9/zpw5c5o4ocSD7UXlXD/9S+54fSnpqUm8+r0zeOnGM1TmMU4j9Chz6AtDx9rK5YYbbjjqtssuu4zLLrssAukk1tX4A7zw6RaenLuepIQEHvz2YK49s8dhX72X2KVCF2kmthQe5O63lrF8x34uGpzDr789mE6Z2oY8nqjQReKcc463Fu7goXdWk5yYwLP/OYxLTu3kdSxpAip0kThWWlnDz2d8xT9W7uLs3tk8duVQjcrjWNQVunNO27xGmHYBEJ827inlllcXs62onKkTB/D983qRoLnyuBZVhZ6WlkZRURHZ2dkq9QhxzlFUVHTUfmckts1ZuYufvr2MFimJvHbzKM7sle11JImAqCr0rl27kp+fz969e0/6uQ7tcyVWRTJ/WloaXbt2jci6pGk553hnUzUz5ixmaLcsnrt2mKZYmpGoKvTk5GR69uwZlufKy8sL61HsIy3W80vk+fwBfjV7FTM21DDptM78/junkpp07KNrSfyJqkIXkcYpr/Zx1xtLmbtmD5f0TOaJK0/TfHkzpEIXiXGllTXc+OJClmzfx28mDaZb1VaVeTOlr/6LxLADlTVcP/1Llu3Yz9PXDOO6s3K9jiQe0ghdJEaVlNdw/fQFrC44wLNThnHR4I5eRxKPqdBFYtCByhqufWEB63aV8ty1wxk3MMfrSBIFVOgiMaayxs/NLy1i7a4DTLtuBGMHdPA6kkQJFbpIDKnxB7jj9SUs3FbMU1efrjKXw+hDUZEYEQg4fjHjK+au2cNDk4Zw2dDOXkeSKKNCF4kR//3+OmYu3clPL+jHdWf28DqORCEVukgMmLE4nz/lbeKaM7pzx/l9vI4jUUqFLhLlFm4tZurMrzi7dzYPTRqsHdfJManQRaLYjuJybn11MV3btOSPU4aRnKgfWTk2vTtEolRFtZ/vv7IInz/AC98dQVbLFK8jSZTTZosiUcg5x72zVrBudykv3jCSXu3TvY4kMUAjdJEo9NbCHcxcspM7z+/LmP7a1lzqR4UuEmVW7izhgdmrOK9vO340rq/XcSSGqNBFosiByhp+8NoSslul8ORVp5Go3eBKA2gOXSSKPDBrJTv3V/D2rWeSnZ7qdRyJMRqhi0SJvy3byaxlX3Pn+X0Y3qOt13EkBqnQRaLAzv0V3DdrJcO6Z3HHWH0TVBpHhS7iMX/A8ZO3lhEIOJ686nSS9OUhaSTNoYt47PlPNrNgSzGPXjGU7tktvY4jMUxDAREPbdpbxmMfrOeiwTn8x7AuXseRGKdCF/HIof2bt0hO5DeTh2inW3LSVOgiHnnl860s2raPBy4dRIeMNK/jSBxQoYt4YEdxOb+bs45v9WvP5ZpqkTBRoYtEmHOOe2Z+RWKC8V+Xn6KpFgkbFbpIhM1e/jWfbSziFxP60yWrhddxJI6o0EUiqKSiht+8s4ahXTP5z1E6LqiEl7ZDF4mgx99fR/HBKl68YaR2vCVhpxG6SISsyC/h1S+2cd2ZPTila6bXcSQOqdBFIsAfcNw3awVtW6Xykwv7ex1H4pQKXSQC3l60g+X5Jdx/6UAyWyR7HUfilApdpIkdqKzh0X+u44zctnx7aGev40gc04eiIk3s6Q83UFxezcuXDdI259KkNEIXaUJbCg/y0vytXDm8G0O66INQaVoqdJEm9PC7q0lNSuT/XqQPQqXpqdBFmsjH6/cyd80e7ji/D+0zdHxQaXoqdJEm4A84Hn53DT2yW3LjOblex5FmQoUu0gT+siSfdbtL+cWEAaQmJXodR5oJFbpImFVU+3n8/fWc1i2LiUM6eh1HmhEVukiYTf9sC7sOVDJ14gBtpigRpUIXCaOisir+lLeJ8QNzGNUr2+s40syo0EXC6Ol5Gymv9nHPRG2mKJGnQhcJkx3F5by2YBtXjexGnw4ZXseRZkiFLhImT8xdT4IZPxrXz+so0kyp0EXCYMPuUv66dCffPTuXjplpXseRZkqFLhIGj3+wnlYpSdz2rd5eR5FmTIUucpK+yt/PP1bu4ubzetK2VYrXcaQZU6GLnKRH319Pm5bJfO/cnl5HkWZOhS5yEhZsLuLj9Xv5wZg+ZKTpSETiLRW6SCM553jsg/V0yEjlurN6eB1HRIUu0lifbyriyy3F/GBMb9KStQMu8Z4KXaQRnHM8MXc9HVuncfUZ3b2OIwKo0EUa5dONhSzcuo8fjtXoXKKHCl2kgZxzPPHBejpnpnHlyG5exxH5hgpdpIE+3lDIku37+cHYPjp4hUQVFbpIAxwanXfJasGVIzQ6l+iiQhdpgI83FLJsx35+MLY3KUn68ZHoonekSD055/jD3ODc+XeGd/U6jshRVOgi9TR/UxFLtu/n9jG9NXcuUUmFLlJPf/hwAzmtU7lCc+cSpVToIvXwxebgt0Jv/5a2O5fopUIXqYenPtxA+4xUfStUopoKXeQEFm0tZv6mIm4d3Uujc4lqKnSRE3h63kayW6UwZZT2qCjRTYUuchzLd+znX+v3cvN5vWiRotG5RDcVushxPPPRRjJbJGt/5xITVOgix7Cm4AAfrN7NTef0JD01yes4IiekQhc5hmc+2kh6ahI3nJ3rdRSRelGhi9Rh454y3ltRwPVn9SCzpY4VKrFBhS5Shz/mbSQtKZHvndvT6ygi9VavQjezCWa2zsw2mtk9ddx/g5ntNbNlodPN4Y8qEhl7ygP8bdnXTBnVnez0VK/jiNTbCT/pMbNE4FngAiAfWGhms51zq49Y9C3n3B1NkFEkot7bXEOiGd8f3cvrKCINUp8R+hnARufcZudcNfAmMKlpY4l4o6Ckgk93+rhyZFdyWqd5HUekQeqzLVYXYEet6/nAqDqW+w8zGw2sB37snNtx5AJmdgtwC0BOTg55eXkNDlxfZWVlTfr8TU35vfHamioCzjE0ZW9M5ofY/bc/RPkbL1wb1/4deMM5V2VmtwIvA+cfuZBzbhowDWDEiBFuzJgxYVr90fLy8mjK529qyh95hWVVfPLhPM7ukswVFx/19o0ZsfhvX5vyN159plx2ArV3AN01dNs3nHNFzrmq0NXngeHhiScSOS98uoUqX4BLe2kzRYlN9Sn0hUBfM+tpZinA1cDs2guYWadaV78NrAlfRJGmt7+8mlc/38Ylp3SiYyttzSux6YTvXOecD7gD+CfBon7bObfKzB4ys2+HFrvLzFaZ2XLgLuCGpgos0hRemr+Vsiofd5zfx+soIo1Wrzl059x7wHtH3PZArctTganhjSYSGaWVNUz/dAsXDsphQMfW7FrrdSKRxtHfltLsvfrFNg5U+rjz/L5eRxE5KSp0adbKq308/8kWxvRvzyldM72OI3JSVOjSrL2+YDvFB6u5U3PnEgdU6NJsVdb4mfbxZs7unc3wHm29jiNy0lTo0mz9edEO9pRWacsWiRsqdGmWqnx+/pi3iZG5bTirV7bXcUTCQoUuzdKMxfkUlFRy17i+mJnXcUTCQoUuzU61L8AfP9rE6d2zOLdPO6/jiISNCl2anb8uzWfn/gqNziXuqNClWanxB3jmo42c2jWTMf3aex1HJKxU6NKszFq6kx3FFdx1vkbnEn9U6NJs+EKj80GdWjNuYAev44iEnQpdmo2ZS3eyraicH1/QT6NziUsqdGkWavwBnp63gVO6ZDJeo3OJUyp0aRb+sjifHcUV/PgCzZ1L/FKhS9yr9gV4et5GhnbLYmx/jc4lfqnQJe7NWBzc7vzu8RqdS3xToUtcq/YFePajjZzePUvbnUvcU6FLXHtz4XZ27q/gJ9qyRZoBFbrErfJqH099uJEze7XVPlukWVChS9x6ef42Csuq+NlF/TU6l2ZBhS5xqaSihuf+tYnzB3TQ0Yik2VChS1x6/pPNlFTU8NML+3kdRSRiVOgSdwrLqnjh0y1cemonBnfO9DqOSMSo0CXuPDNvI5U1fn58gUbn0ryo0CWubCs6yGsLtnHVyO70bp/udRyRiFKhS1z573+uIykhgR+P7+t1FJGIU6FL3Fi2Yz/vfFXA90f3okPrNK/jiEScCl3ignOOR95bQ7v0FG4Z3cvrOCKeUKFLXPho3R4WbCnmR+P7kZ6a5HUcEU+o0CXm+fwBHnlvLb3ateLqkd28jiPiGRW6xLzXv9zOhj1l3DNxAMmJektL86V3v8S0/eXVPP7Bes7pk80Fg3K8jiPiKRW6xLQn527gQEUN9186SDvgkmZPhS4xa/3uUl79YhtTRvVgQMfWXscR8ZwKXWKSc47fvLOaVimJ+oq/SIgKXWLSB6t388mGQu4e34+2rVK8jiMSFVToEnPKq308+PfV9M/J4LqzengdRyRq6BsYEnOe+nAjO/dX8OfbztJmiiK16KdBYsr63aU8/8lmrhjelZG5OhKRSG0qdIkZzjnum7WSVqlJ3DNxgNdxRKKOCl1ixswlO/lySzH3TBxAdnqq13FEoo4KXWLC3tIqfvPuak7vnsVVI7S/FpG6qNAlJvx69irKq/z8/j9OJSFB3wgVqYsKXaLenJUFvLuigLvG9aFvTobXcUSilgpdotr+8mrum7WKQZ1ac+u3ensdRySqaTt0iWoPvbOa/eXVvHzTSG1zLnIC+gmRqDVn5S5mLtnJbd/qzeDOmV7HEYl6KnSJSrsPVDJ15lcM6dKau8b19TqOSExQoUvUCQQc//fPy6mo8fPkVaeTkqS3qUh96CdFos6L87fyyYZC7rtkEH06pHsdRyRmqNAlqqwpOMDv5qxl3IAOTBnV3es4IjFFhS5Ro7Syhh+8toTWacn87jun6pByIg2kzRYlKjjnuOcvK9heXM5rN4+infbVItJgGqFLVHhp/lbeXVHAzy7qz5m9sr2OIxKTVOjiucXb9vHwu2sYPzCHW0f38jqOSMxSoYundpVU8oPXFtMpK43HrhyqeXORk6BCF8+UV/u4+ZWFlFX6+J/rR5DZItnrSCIxTR+KiicCAcdP3lrOqq8P8Pz1IxjQsbXXkURinkbo4onHPljHnFW7uPfigYwbmON1HJG4oEKXiHt9wXae/WgTV4/sxvfO7el1HJG4oUKXiHpvRQH3zlrBmP7teWjSEH0IKhJGKnSJmE83FHL3m8sY3r0Nf5oyXDvdEgkz/URJRCzdvo9bXl1Er/ateOG7I2mRkuh1JJG4o0KXJrd42z6uf+FL2qWn8spNZ5DZUpsnijQFFbo0qYVbi7n+hQVkp6fw1q1n0qF1mteRROKWtkOXJvPF5iJuemkhHTPTeOP7Z5KjMhdpUip0aRKLd/uYNvdLurdtyWvfH0WHDJW5SFPTlIuE3cvzt/LM0ioGdmrNm7ecqTIXiRCN0CVsAgHH7+as5f99vJnTOyTy+vfP1NYsIhGkQpewOFBZw0/eWs7cNbu59szunJ9ZqDIXiTBNuchJ27C7lMnPfMZH6/bwwKWD+M2kISToG6AiEacRupyUd776mp/P+IqWKYm8fvMoRuloQyKeUaFLo5RW1vDr2av5y5J8Tu+exZ+mDKdjpj78FPGSCl0abPG2Yu5+axk791Vw5/l9uGtcX5ITNXsn4jUVutRbWZWPx95fx8vzt9I5qwVv33oWI3Lbeh1LREJU6FIvc1fv5oG/raTgQCVTRnXnFxMGkJGmfbKIRBMVuhzXxj2lPPLeWj5cu4f+ORk8/Z/DGN6jjdexRKQOKnSpU1FZFU/O3cDrX26nZXIiUycO4KZze2quXCSKqdDlMEVlVUz7ZDOvfr6NKl+AKaO686NxfclOT/U6moicgApdACgoqeDFz7by6ufbqPT5uezUztw1ri99OqR7HU1E6kmF3swt3b6P6Z9t5b0VBTjnmHRaF344to+KXCQGqdCboZKKGmYv28nbi/JZsbOEjNQkbjonl+vPyqVb25ZexxORRlKhNxNVPj8fry/k78u/5p+rdlHlCzCwU2semjSYy4d1JT1VbwWRWKef4jhWVuXj0w17eX/1bj5YtZvSKh9ZLZO5ckQ3rhrZjSFdMr2OKCJhpEKPI4GAY93uUj7bWMi/1u/li81F1PgdrdOSmDCkI5cO7czZvbO16aFInFKhxzCfP8CaglIWbStm0dZ9fLG5iKKD1QD0at+KG8/pyfkDOjC8RxuVuEgzoEKPETX+AFsKD7JyZwkrdx4Inn9dQnm1H4DOmWl8q197zu7TjrN7Z9M5q4XHiUUk0lToUaa82sfWwnK+LPDx1Ycb2JdqRDgAAAb4SURBVLinjPW7S9m0t4wavwMgLTmBQZ1ac+WIbgzv0YbhPdqowEVEhR5JgYCj6GA1uw9UsvtAJV/vr+DrkuD5juJythdXUFhWVesR6+mS1YJ+OemM6d+B/h3TGdQpk97tW5GkKRQROUK9Ct3MJgB/ABKB551zvz3i/lTgFWA4UARc5ZzbGt6o0aXaF6CsykdpZQ0lFTUcqPBRUlHD/opq9h2sZl95DfsOVlN4sJrC0iqKDlZRWFaNP+AOe57kRKNjZhrd2rRk/MAOdGvbkh7ZLSneuoYrJozRcTlFpN5OWOhmlgg8C1wA5AMLzWy2c251rcW+B+xzzvUxs6uB3wFXNUVgAOcc/oDDF/j3uc8fwB9w1IQuf10WYE3BAXx+R7U/QLUvQE3ovNofoMrnp9oXoMoXoKomeL2yJkBFjZ+KGj+V1X7Kq/2U1/ipqPZxsMrPwWofZZU+yqp8VPkCx83YMiWRNi1TaJeeQqfMNIZ0aU37jFRyWqfRISONnNapdMlqQbv0VBISjj7+Zl7xepW5iDRIfUboZwAbnXObAczsTWASULvQJwG/Dl2eATxjZuacO3w4Wsv63aWc/2geAecIOILngeBlf+iyP1Tchy4HAnxzW718+kn9lgtJMGiRnEiLlERSkxJpmXLolETnrGRapSbRKjWJ9NApIy2JjLRkWqclkdkimcyWyWS1SCGrZTJpySpjEYms+hR6F2BHrev5wKhjLeOc85lZCZANFNZeyMxuAW4BSM/JpX1SJWZgQIJZrctHnDASEhJIOOK+xARINPv39dBtSQY11VW0apFGUkLw9uQEIymB0MlITiB0MpITg5cTDeywo9U7wBc61aEmdCqFCoKnXfX4B62PsrIy8vLywvRskRfL+WM5Oyi/17zMH9EPRZ1z04BpACNGjHBv3T2hydaVl5fHmDFjmuz5m5ryeyeWs4Pye83L/PXZVGIn0K3W9a6h2+pcxsySgEyCH46KiEiE1KfQFwJ9zaynmaUAVwOzj1hmNvDd0OXvAPOON38uIiLhd8Ipl9Cc+B3APwlutjjdObfKzB4CFjnnZgMvAK+a2UagmGDpi4hIBNVrDt059x7w3hG3PVDrciVwRXijiYhIQ+jrhiIicUKFLiISJ1ToIiJxQoUuIhInzKutC81sL7CtCVfRjiO+qRpjlN87sZwdlN9rTZ2/h3OufV13eFboTc3MFjnnRnido7GU3zuxnB2U32te5teUi4hInFChi4jEiXgu9GleBzhJyu+dWM4Oyu81z/LH7Ry6iEhzE88jdBGRZkWFLiISJ+K+0M3sTjNba2arzOz3XudpDDP7qZk5M2vndZb6MrP/Dv27f2VmfzWzLK8z1YeZTTCzdWa20czu8TpPQ5hZNzP7yMxWh97vP/I6U0OZWaKZLTWzd7zO0hhmlmVmM0Lv/TVmdlYk1x/XhW5mYwke73Soc24w8KjHkRrMzLoBFwLbvc7SQB8AQ5xzpwLrgake5zmhWgdEnwgMAq4xs0HepmoQH/BT59wg4EzghzGWH+BHwBqvQ5yEPwBznHMDgKFE+LXEdaEDtwO/dc5VATjn9nicpzGeAH5O8ACnMcM5975z7tDBWL8geKSraPfNAdGdc9XAoQOixwTnXIFzbknocinBMunibar6M7OuwCXA815naQwzywRGEzw+BM65aufc/khmiPdC7wecZ2YLzOxfZjbS60ANYWaTgJ3OueVeZzlJNwH/8DpEPdR1QPSYKcTazCwXOB1Y4G2SBnmS4OAl4HWQRuoJ7AVeDE0bPW9mrSIZIKIHiW4KZjYX6FjHXfcSfH1tCf75ORJ428x6RdPh8U6Q/5cEp1ui0vGyO+f+FlrmXoJTAa9FMltzZmbpwF+Au51zB7zOUx9mdimwxzm32MzGeJ2nkZKAYcCdzrkFZvYH4B7g/kgGiGnOufHHus/Mbgdmhgr8SzMLENxxzt5I5TuRY+U3s1MI/sZfbmYQnLJYYmZnOOd2RTDiMR3v3x7AzG4ALgXGRdMv0eOozwHRo5qZJRMs89ecczO9ztMA5wDfNrOLgTSgtZn9r3PuWo9zNUQ+kO+cO/RX0QyChR4x8T7lMgsYC2Bm/YAUYmQvbs65Fc65Ds65XOdcLsE3y7BoKfMTMbMJBP98/rZzrtzrPPVUnwOiRy0L/uZ/AVjjnHvc6zwN4Zyb6pzrGnqvX03wQPOxVOaEfjZ3mFn/0E3jgNWRzBDzI/QTmA5MN7OVQDXw3RgZKcaDZ4BU4IPQXxhfOOdu8zbS8R3rgOgex2qIc4DrgBVmtix02y9DxwSWyLgTeC00INgM3BjJleur/yIicSLep1xERJoNFbqISJxQoYuIxAkVuohInFChi4jECRW6iEicUKGLiMSJ/w+a+bLGSkCgbAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def sigmoid(x):\n",
    "    return 1 / (1 + np.exp(-x))\n",
    "\n",
    "xs_sig = np.linspace(-7, 7, 100)\n",
    "ys_sig = sigmoid(xs_sig)\n",
    "plt.plot(xs_sig, ys_sig, label=r'$\\sigma(x) = {\\frac{1}{1 + e^{-x}}}$')\n",
    "plt.yticks([0, 0.5, 1])\n",
    "plt.xlim([-7, 7])\n",
    "plt.legend()\n",
    "plt.grid()\n",
    "plt.title('Sigmoid function')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3dd5xU1f3/8ddnZwu7LL0LyEpEFAttEYyKoCaC2BNr7IXYicbeomgQlagRNQTFrwGjGKLmZ6yJUSxYIigWBJQSZJGO9O17fn+cWVlwYXdhd86U9/PxuI/dmXu59zMDvOfMufeeY845REQk8aWFLkBEROqHAl1EJEko0EVEkoQCXUQkSSjQRUSShAJdRCRJKNBFqjCzjWbWNUbHGmdmt8biWJIaTNehSyyZ2VSgJ9DeOVccuJyYMbNzgQudc4eErkWSl1roEjNmlgccCjjguAY6RnpD7FckESjQJZbOBj4EngTOqXzSzJ6Mdj/828w2mNnbZtalynpnZlea2QIzW2Vm95lZWnTduWY2zcweMLPVwO1mlmZmt5jZIjNbYWYTzaxZdPtTzWyhmTWNPh5qZsvMrE2VY+1Zpa5HzezVaFfMNDNrb2YPmtn3ZjbHzHpXqfMGM5sffQ1fmdmJ0ef3AcYBB0X3s7bK/u+q8ucvMrN5ZrbGzF40s922eQ8uNrNvzGytmT1iZlbPfz+S4BToEktnA3+NLkeZWbsq634F3Am0BmZGt6nqRCAf6AMcD5xfZV1/YAHQDvg9cG50GQx0BXKBhwGcc88C7wMPmVkrYAK+K2Tldmo+BbglWlcx8AHwSfTx34H7q2w7H/8NpBlwB/CUmXVwzs0GLgY+cM7lOueab3sQMzscuDt6vA7AImDyNpsdA/QDDohud9R2apYUpUCXmDCzQ4AuwN+cczPw4XdGlU1eds69E+1Xvxnfmu1cZf09zrk1zrlvgQeB06us+845N9Y5V+acK8R/ONzvnFvgnNsI3AicVqU75jLgcGAq8E/n3Es7KP0F59wM51wR8AJQ5Jyb6JwrB54FfmihO+emOOe+c85VRD84vgEOrOVb9CvgCefcJ9H34Mboe5BXZZvRzrm10ffgLaBXLfctKUKBLrFyDvAv59yq6OOnqdLtAiyu/CUawmuA3apbj2+9bm8d0XWLttk+Hd+Cxzm3FpgC7Af8oYa6l1f5vbCax7mVD8zsbDObGe0SWRvdf+sa9l9tzdH3YDXQsco2y6r8vrnqsUXA/yMXaVBmlo3vIoiYWWUoZQHNzaxn9HHnKtvnAi2B76rspjMwK/r77tus2/ZSre/w3wYq7Q6UEQ1jM+uF77J5BngIGLJTL6yKaJ//Y8AR+K6VcjObCVT2c9d0OdlWNZtZY6AVsGRXa5PUoRa6xMIJQDnQA99N0AvYB3gX368OcLSZHWJmmfi+9A+dc1Vb3teaWYtoN8wIfHfH9jwDXGVme0Q/HEYBzzrnysysEfAUcBNwHtDRzC6th9fYGB/aKwHM7Dx8C73ScqBT9PVtr+bzzKyXmWVFa/7IOfe/eqhNUoQCXWLhHOD/nHPfOueWVS74E5W/wn9TfBr4Hb6rpS9w5jb7+H/ADPwJ05fxJzO35wlgEvAOsBAoAq6IrrsbWOyc+1O0r/pM4C4z67YrL9A59xW+++YDfHjvD0yrssmb+G8Yy8xsVTV//g3gVuA5YCnwE+C0XalJUo9uLJLgzOxJoMA5d8t21jugm3NuXkwLE0kwaqGLiCQJBbqISJJQl4uISJJQC11EJEkEuw69devWLi8vL9ThRUQS0owZM1Y559pUty5YoOfl5TF9+vRQhxcRSUhmtmh769TlIiKSJBToIiJJQoEuIpIkFOgiIklCgS4ikiQU6CIiSUKBLiKSJGoMdDN7IjrR7pfbWW9m9lB0ctvPzaxP/ZcpIiI1qc2NRU/ix62euJ31Q4Fu0aU/8Kfozwbz6qvwwQdw0EEwdGjt/9zcufDNN9CtG3TvvvW+OneGJUtg5Uro1Qs6dIClS2Hx4i3Hqfzzn34Kb7wBubmw557+uZYt4ZBDYPZsv66sDDIyoHFjaNMGNm3y++7ZE1q3hmnTYO+94bDD4K23/D5ycvyxV64E5+Dww31NU6f6n2aw227QpAnMmQPFxXDooXDmmVBeDpHIlp8vvwwLF8IJJ8DAgTB27NaPn3nGv57WraFVK2jfHgYPhgULtry3Xbv62sCvgx+/f7V9n3dGfe1HJFXUanCu6ES1Lznn9qtm3Z+Bqc65Z6KP5wKDnHNLd7TP/Px8tzN3ir7/wEd0uu50KixCORHadojQpFnEp1gkAunp1S4bitKZPS+DikgGZWSwX58M1hVm8ua7mRS7TL7fnEWxZVFqWRSTRfP2jfh2RSMiTbIpizTihDNy+GR2NvOX5vDB543Z4BqzkVw20RjSIjgH2dmweXPdXk96ug/hqn8NZpCW5heA0tId76N5c+jXD+bPh5/8BD75BNau9ftJT/cfTosXbzlely7w7bdQUgIVFZCV5T90OnaEggJo1AiKiqBTJ//hApCX5z/AWrb09d50U/UhO3cujBq15cNle9vVpL72I5JszGyGcy6/unX1cet/R7aepLcg+tyPAt3MhgPDAdq1a8fUqVPrfLCNZYto2XtP0inHlVXwfXoFRVkVWEUFVl7ufxYWYmVlW54rL6eipIy9SstJKy7HyspIf7OMdqWlnFFaRqS8lEhFKVY1VStfUXH05wMwbDs1lUayKMnKobhRY0pa+p/F2bkUZzehODuXwpwmFOU0pTCnGZtzm1OY25zNuc0pymnqU7caZluH/I6Y+W8CpaX+W8HQoT6o09L8Pir3Vfmz8vnKxQwyM7c8rgz0yg8E8M+np/tvByUl/hvC0mo+stet898sMjN3vF1N6ms/IqkkpmO5OOfGA+PBt9AHDRpU5328WgjDxp1DWpoPrYcfrl23S3UtvgUL4PLLfffIiuWOdMrITismi2J6dC1i6cIi2jUtpHFaIRecUcicTwv5vmATBV9vJtttIpeN5LKRpqUbaFK6gRaF68kpX09z1tKMNbRgAS1ZQw6F1dZUQgbLaM8y2lNApx+Wb+nCorQ9+DYtj5W0obSs+tCvpBa6iEACdrlA4vWh79ayiPR1q6lYsYq+XVbRMWMFK2etYK+my9i72VKKFi6l8doltCtdTG75+q1qLs5swqLMPfmGbizM3JvV7Xqwsk0P3lqyFxtKstSHLpJidtTlUh+BPgy4HDgafzL0IefcgTXtc1cCPamtX++bzwsX+mX+fJ9q33zj07aiwm+Xng49evhPnz59oH9/6N3bN7dFJGntUh+6mT0DDAJam1kBfmb2DADn3DjgFXyYzwM2A+fVT9kpqmlT2G8/v2yrqAi+/hpmzYIvvoCZM+Ff/4KJ0QuQMjJ8uA8c6C+fOeQQaNYstvWLSDDBpqBTC70eLVkCH33kl2nT4L//9WdIIxHfcj/qKN9n1LfvlktnRCQh7XKXS0NQoDegwkL48EP4z3/g9ddhxgx/mUrHjnD88XDSSTBokA98EUkoCvRUt2qVP/v7wgvw2ms+8Dt0gNNPh7PO8v3wIpIQdhTo+v6dClq39sH9/PM+3KdM8V0xY8f6E6n9+sGECf52VhFJWAr0VJOTA7/8pW+tL1vmQ72wEC680HfJXH89fPdd6CpFZCco0FNZy5b+zqovvoD33vMnT8eM8XcRnX8+zJsXukIRqQMFuvhbSg8+GJ591l/vPnw4TJ7sRw8bPnzLbaYiEtcU6LK1rl39eArz58Oll8Jf/uJv1bzxRti4MXR1IrIDCnSpXocO8NBD/kamU0+F0aNhr71g0qTajxomIjGlQJcd69LFt9I/+MCfND37bDjiCD8MgYjEFQW61M6AAf5O1HHjYPp02H9/34KvHFtGRIJToEvtpaXBr3/tx5I57DAYMQJ+9jMNVC4SJxToUnedO/vxeSdM8EMM9Ozp70AVkaAU6LJzzPy16tOn+8HUhw6Fm29WF4xIQAp02TX77OP71i+80E8xdMIJsGFD6KpEUpICXXZddjaMH++vX3/lFT/dka6CEYk5BbrUDzO47DI/4cbSpfDTn/q5+kQkZhToUr8OP9xPspGV5cdcnzo1dEUiKUOBLvVv7719qHfqBEOGwD/+EboikZSgQJeG0akTvPuuH2/95JMV6iIxoECXhtOypZ8Cr29fOOUU+Oc/Q1ckktQU6NKwmjb1od6rF/ziF/6GJBFpEAp0aXjNmvmrXw44wM+W9P77oSsSSUoKdImN5s39RNWdO8Oxx8KcOaErEkk6CnSJnTZt/JgvGRl+ujvNXSpSrxToEltdu/q7SdesgWHDYNOm0BWJJA0FusRenz4wZQp8/jmcd55mQBKpJwp0CWPIELjnHh/so0aFrkYkKSjQJZzf/hbOPBNuuQVefDF0NSIJT4Eu4Zj5URrz832wz5sXuiKRhKZAl7Cys+G55yA9HU49FYqLQ1ckkrAU6BLe7rvDk0/CJ5/AtdeGrkYkYSnQJT4cdxxcdRWMHQvPPx+6GpGEpECX+DF6NPTr5+cqXbw4dDUiCUeBLvEjMxOeeQbKynyoa8JpkTpRoEt8+clP4A9/gDfegD/9KXQ1IglFgS7xZ/hwf+PRtdfC11+HrkYkYSjQJf6YwYQJ0KgRnHMOlJeHrkgkISjQJT7tths88gh8+KG/8kVEaqRAl/h12mlw9NF+aIBFi0JXIxL3FOgSv8y2nBi95BKNyihSAwW6xLfdd4ff/97PdjR5cuhqROKaAl3i3+WX+xuORozwE2OISLUU6BL/IhE/KuPq1XDbbaGrEYlbCnRJDL16waWX+j71mTNDVyMSlxTokjhGjoSWLeGKK3SCVKQaCnRJHC1a+AG83nsPnn46dDUicUeBLonlvPP8CdJrroENG0JXIxJXFOiSWNLS4OGHYdky31oXkR8o0CXxHHggnHEG3H+/xk0XqUKBLolp1Ch/YvTmm0NXIhI3FOiSmLp08VPWTZoEM2aErkYkLijQJXHdcAO0bg2//a0uYxRBgS6JrFkzuOMOePtteOml0NWIBKdAl8R20UWw556+L11zkEqKU6BLYsvIgDvvhC++8BNMi6QwBbokvlNO8WO93HYblJSErkYkGAW6JL60NH8Z44IF8PjjoasRCUaBLslhyBA49FDf/bJpU+hqRIJQoEtyMIO77/ZDAjzySOhqRIJQoEvyOPhgOOoouO8+2LgxdDUiMadAl+Ryxx2wapUfwEskxSjQJbn07w9Dh/pWuobXlRSjQJfkc8cdfjLpsWNDVyISUwp0ST79+sExx8CYMbBuXehqRGJGgS7J6fbb4fvv1UqXlKJAl+TUty8MGwYPPqgrXiRlKNAled1yC6xeDePGha5EJCYU6JK8BgyAI4/0femFhaGrEWlwCnRJbrfeCsuXa4wXSQkKdEluAwf6MV7uvReKi0NXI9KgFOiS/G69FQoK4C9/CV2JSINSoEvyO/JIf236vfdCWVnoakQajAJdkp8Z3HgjzJ8Pf/976GpEGowCXVLD8cfD3nvD6NHgXOhqRBqEAl1SQ1oaXH89fPYZvPpq6GpEGoQCXVLHGWdA585+IgyRJKRAl9SRmQnXXAPvvecXkSSjQJfUcuGF0Lq170sXSTIKdEktOTlwxRXw8svw5ZehqxGpVwp0ST2XXeaDfcyY0JWI1CsFuqSeVq1818tf/wqLF4euRqTeKNAlNV11lb8e/cEHQ1ciUm8U6JKa8vLgtNNg/Hg/s5FIElCgS+q69lo/m9Gjj4auRKReKNAldfXsCUcd5ecdLSoKXY3ILlOgS2q77jo/AcakSaErEdllCnRJbYMHQ58+/hLGiorQ1YjsEgW6pDYz30r/+mt48cXQ1YjskloFupkNMbO5ZjbPzG6oZv25ZrbSzGZGlwvrv1SRBvKLX/irXu67L3QlIrukxkA3swjwCDAU6AGcbmY9qtn0Wedcr+iiGXklcaSnw9VXw/vvw7RpoasR2Wm1aaEfCMxzzi1wzpUAk4HjG7YskRg7/3xo2VKtdElo6bXYpiNQ9f7oAqB/Ndv9wswGAl8DVznnfnRPtZkNB4YDtGvXjqlTp9a5YJGGkjdsGF2eeoqPJ05k8+67hy5HpM5qE+i18U/gGedcsZn9GvgLcPi2GznnxgPjAfLz892gQYPq6fAi9aBHD5gyhQPffRceeyx0NSJ1VpsulyVA5yqPO0Wf+4FzbrVzrjj68HGgb/2UJxJDbdvCuefCxImwbFnoakTqrDaB/jHQzcz2MLNM4DRgq+u7zKxDlYfHAbPrr0SRGLr6aigt9XePiiSYGgPdOVcGXA68jg/qvznnZpnZSDM7LrrZlWY2y8w+A64Ezm2ogkUaVLducOKJfnyXDRtCVyNSJ+acC3Lg/Px8N3369CDHFtmhjz6CAQPggQfgN78JXY3IVsxshnMuv7p1ulNUZFv9+8Ohh8L99/vuF5EEoUAXqc511/nZjJ59NnQlIrWmQBepztFHw777wr33+pmNRBKAAl2kOmlpfgKML76A114LXY1IrSjQRbbn9NOhUye4557QlYjUigJdZHsyM/1k0m+/7a98EYlzCnSRHbnoImje3Peli8Q5BbrIjjRpApddBi+8AHPmhK5GZIcU6CI1ufJKyMrS0LoS9xToIjVp2xYuuMBPJF1QELoake1SoIvUxjXX+Emk778/dCUi26VAF6mNvDx/GeP48bB6dehqRKqlQBepreuvh02b4OGHQ1ciUi0Fukht7bcfHHssPPQQbNwYuhqRH1Ggi9TFTTfBmjUwblzoSkR+RIEuUhcDBsCRR8KYMVBYGLoaka0o0EXq6pZbYPlymDAhdCUiW1Ggi9TVwIFwyCF+0K6SktDViPxAgS5SV2a+lV5QABMnhq5G5AcKdJGd8fOfQ9++cPfdUFYWuhoRQIEusnPM4LbbYMECeOqp0NWIAAp0kZ137LHQpw/ceacmk5a4oEAX2VlmcPvtaqVL3FCgi+yKY47xfel33aVWugSnQBfZFVVb6ZMmha5GUpwCXWRXDRsG+flqpUtwCnSRXWUGI0fCwoW6e1SCUqCL1IchQ/zdoyNHaowXCUaBLlIfzGDUKFi6VOOlSzAKdJH6cuihvqU+ejSsWxe6GklBCnSR+vT73/vx0jX3qASgQBepT336wMkn+0Bfvjx0NZJiFOgi9e3OO/2J0TvuCF2JpBgFukh9694dLr4Yxo+HOXNCVyMpRIEu0hB+9zvIyYHrrw9diaQQBbpIQ2jTxk8o/eKLMHVq6GokRSjQRRrKiBHQuTNccw1UVISuRlKAAl2koWRn+5uNZszQVHUSEwp0kYZ0xhkwYADccINuNpIGp0AXaUhpaTB2LKxY4cd5EWlACnSRhpafDxdcAA89BLNnh65GkpgCXSQWRo2Cxo39iVLnQlcjSUqBLhILbdr4Lpd//xueey50NZKkFOgisXLppdC7N1x5JaxdG7oaSUIKdJFYSU+Hxx7zg3bdeGPoaiQJKdBFYqlvX9+PPm4cTJsWuhpJMgp0kVgbORJ23x2GD4eSktDVSBJRoIvEWm4uPPoofPUV3HVX6GokiSjQRUIYNgzOOstfzjh9euhqJEko0EVC+eMfoX17OPtsKCoKXY0kAQW6SCgtWsCECf7u0dtuC12NJAEFukhIRx0Fv/41jBkD774buhpJcAp0kdDuuw+6dvUjM65eHboaSWAKdJHQmjSByZP9DUfnn6+xXmSnKdBF4kF+Ptx7r5+ybuzY0NVIglKgi8SLESPguOPg2mv9LEcidaRAF4kXZvDEE9CuHZx4op8UQ6QOFOgi8aRVK3jhBVi5Ek45BUpLQ1ckCUSBLhJv+vb1ozK+/TZcfXXoaiSBpIcuQESqceaZ8OmncP/9cMABcNFFoSuSBKAWuki8uucef+PRJZfAq6+GrkYSgAJdJF6lp8OUKbD//nDyybryRWqkQBeJZ02awCuv+JOlw4bBwoWhK5I4pkAXiXcdOsBrr/nJMI48EgoKQlckcUqBLpII9tnHh/rKlXD44bB0aeiKJA4p0EUSxYEH+lD/7js44gjdeCQ/okAXSSQ//anvU//f/2DQIHW/yFYU6CKJZuBAfxljQQEcfDB8/XXoiiROKNBFEtFhh8HUqVBYCIccAp98EroiiQMKdJFE1acPvPceZGf7Vvs//hG6IglMgS6SyPbaCz74AHr0gJNOgrvv1gQZKUyBLpLodtvND+R16qlw001+HJiNG0NXJQEo0EWSQXY2PP003HUXPPOMnwHp889DVyUxpkAXSRZmcPPN8J//wLp10L8/jBunLpgUokAXSTaDB8Nnn/kTpZdcAkOGwLffhq5KYkCBLpKM2rb116o/+ihMmwb77Qd//jNUVISuTBqQAl0kWaWl+Rb6F19Av35w8cVw0EHw8cehK5MGokAXSXZ77AFvvAGTJvmul/794cILYcmS0JVJPVOgi6QCM38549y5fp7SiRNhzz3hmmtg1arQ1Uk9UaCLpJKmTWHMGD/+y6mnwgMP+Bb8tdf6URwloSnQRVJRXh48+aTvXz/2WD8ZdV4eXHABzJwZuDjZWQp0kVTWo4e/IWnePBg+3N+U1Lu3H6Z30iTYvDl0hVIHCnQR8d0uDz/sT5Q+8ACsXg1nnw3t28N558Gbb0J5eegqpQYKdBHZokUL+M1vYM4ceOstOPlkeO45P0NShw5w0UX++vaiotCVSjXMBbotOD8/302fPj3IsUWkDjZvhpdfhuefh5de8gN/ZWf7O1KHDPETV++9t7+SRhqcmc1wzuVXu06BLiK1VlTku19ee8231OfN88+3besn3Tj4YBgwAHr1gqyssLUmqR0FenqsixGRBNaoERx9tF8AFizwMydVLlOm+OczM2H//f0J1t694YADYN99fZeONBi10EWk/ixZAh99BB9+6KfF+/RTWLNmy/oOHWCffaBbNz85x557+hOyeXnQpEmwshPJLne5mNkQ4I9ABHjcOTd6m/VZwESgL7AaONU5978d7VOBLpICnIPFi+HLL2HWLL/MnetvbKoa9AAtW0KnTtC5s//ZocOWpW1bv7RpA40bp3R//S51uZhZBHgE+BlQAHxsZi86576qstkFwPfOuT3N7DTgHuDUXS9dRHbG3Ln+IhXw5y67d/dd3i+95DOxf39/FWIkAosW+e26dPHPdevmt6/06qt+lrvOnf36d9/1c2esXg0ZGX6O6kGDfG537uzzt1s3eOcdeOEFo2vX3enZc3fe+vRo1q+HPfJhTVdYt2A1vZstIM8tJGf5QpqtXUTOgsV0+KaATvYRjTdXPyRBEVmsS29FcW4r1rgWrCprzqqyFqxPa0ajtk3p1rcZexzQhGWbmjBrUS7NO+XSbLfGvDy1MZ/MyWGvXjn0+mkO5RmN6Nw1g/IKIxLZ+rXPnesvyV+xwt93NXTolvf1m29+/B5Vfd93tL7yvTzooC37rE81ttDN7CDgdufcUdHHNwI45+6uss3r0W0+MLN0YBnQxu1g52qhizSMuXPhhhv8TaDgR84dNgxGjvTzXpSX+xEA9t8fZs/2gzJW/k/t1w9yc/1MdpUfApdfDmVlfsiXSAQ2bPjxMTMy/AfF+vU+rDZs8DeclpX5EXvT0vxP534830bluqrMoFWTEhpvXE57ltE+bQWtylfQ2q2gFat/WJqzlhZ8T3PW0ox1NGU9adS+G7mcNErTGlFIIyoyG1GenkVOiyyWf5/J2s2ZlFgWLj2Dbvtmkt0kg7kLMqiIZFBqGex7QDrNWmf4NyU9ne83pPPR9AguEqHcRTjo0HRatY34FxiJMHdehH/8Mw1nEZ5vfBZ3jGu3U6G+S10uZvZLYIhz7sLo47OA/s65y6ts82V0m4Lo4/nRbVZts6/hwHCAdu3a9Z08eXLdX42I7NC6db5lWVbmH0ciPnArg7i83AdmdjYUF/u8AR+qTZr4x82bQ7NmfniXNWv8c8XF1QdypcxMf8ycHCgpgdJSf5ydHYJ9216VWp3uc47MkkJy3WYimzeTXVaI21hIRkkR6cVFpJcUkVFaTEZpMZmuhEhJMVkVxVhxKY0oIVJWSnpFCeWFZaSVlJJWUUakvIwsSkl3ZVSUlBOpKMPKykh35URcOVbuF1de4X+vqMAqKkir8L9XZ/J146no2Y3ddqv7+zJ48OD4uMrFOTceGA++hT5o0KBYHl4kJcydu2WYFtjSQr/vvp1rod96a5gWetOmW+a6jkS27KsmTZv6EQ0KCvy+16zxH1AFBVtv07y5398++8D8Jf5Ky9xcOO00+MMf/LldM//BNn48dO0Ko0bxQ/dM5XtU9X2vdn1FBZSX8/qrFVw9opz0tAo2T8nmoUER6jsCaxPoS4DOVR53ij5X3TYF0S6XZviToyISY927w+jRP+5D79Sp7n3oQ4f6EQF2vg/dB2HPnr6e9ev9RS1r1mzZvkUL/3jjRn8VZHY2HHecr3PWLH+cvfaCjh3hscdg4UI/IkHPnn4fS5fC2rW+3n339YNIDh7s91VZd4cO8Le/wYwZ0LevH6qm6mvetg+9a9fq+9Bvumn7feTdu29nfVoapKVx1HEwJiN8H3o68DVwBD64PwbOcM7NqrLNZcD+zrmLoydFT3LOnbKj/aoPXUSk7nbpKhfnXJmZXQ68jr9s8Qnn3CwzGwlMd869CEwAJpnZPGANcFr9lS8iIrVRqz5059wrwCvbPHdbld+LgJPrtzQREakLjbYoIpIkFOgiIklCgS4ikiQU6CIiSUKBLiKSJBToIiJJIth46Ga2EtgEVD+kWvxqTWLVnGj1QuLVnGj1gmqOhYaqt4tzrk11K4IFOoCZTd/eHU/xKtFqTrR6IfFqTrR6QTXHQoh61eUiIpIkFOgiIkkidKCPD3z8nZFoNSdavZB4NSdavaCaYyHm9QbtQxcRkfoTuoUuIiL1RIEuIpIkgge6mZ1sZrPMrMLM4vaSJDMbYmZzzWyemd0Qup6amNkTZrYiOt9r3DOzzmb2lpl9Ff33MCJ0TTUxs0Zm9l8z+yxa8x2ha6oNM4uY2adm9lLoWmrDzP5nZl+Y2UwzS4hZccysuZn93czmmNlsMzsoFscNHujAl8BJwDuhC9keM4sAj0fanBkAAAKtSURBVABDgR7A6WbWI2xVNXoSGBK6iDooA37rnOsBDAAuS4D3uBg43DnXE+gFDDGzAYFrqo0RwOzQRdTRYOdcrwS6Dv2PwGvOub2BnsTo/Q4e6M652c65uaHrqMGBwDzn3ALnXAkwGTg+cE075Jx7Bz97VEJwzi11zn0S/X0D/j9Ax7BV7ZjzotMYkxFd4voqAzPrBAwDHg9dS7Iys2bAQPxMbjjnSpxza2Nx7OCBniA6AourPC4gzsMmkZlZHtAb+ChsJTWLdl/MBFYA/3bOxXvNDwLXARWhC6kDB/zLzGaY2fDQxdTCHsBK4P+iXVuPm1njWBw4JoFuZm+Y2ZfVLHHdypXYM7Nc4DngN8659aHrqYlzrtw51wvoBBxoZvuFrml7zOwYYIVzbkboWuroEOdcH3yX52VmNjB0QTVIB/oAf3LO9caPWRWT8261mlN0VznnjozFcRrQEqBzlcedos9JPTKzDHyY/9U593zoeurCObfWzN7Cn7eI1xPRBwPHmdnRQCOgqZk95Zw7M3BdO+ScWxL9ucLMXsB3gcbtOTf8N/iCKt/W/k6MAl1dLrXzMdDNzPYws0zgNODFwDUlFTMzfJ/jbOfc/aHrqQ0za2NmzaO/ZwM/A+aErWr7nHM3Ouc6Oefy8P+G34z3MDezxmbWpPJ34OfE7wcmAM65ZcBiM+sefeoI4KtYHDt4oJvZiWZWABwEvGxmr4euaVvOuTLgcuB1/Mm6vznnZoWtasfM7BngA6C7mRWY2QWha6rBwcBZwOHRy9NmRluS8awD8JaZfY7/0P+3cy4hLgVMIO2A98zsM+C/wMvOudcC11QbVwB/jf7b6AWMisVBdeu/iEiSCN5CFxGR+qFAFxFJEgp0EZEkoUAXEUkSCnQRkSShQBcRSRIKdBGRJPH/AZg50RKotGwHAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#  helper function for approximation\n",
    "def sigmoid_mu_s(x, mu, s):\n",
    "    return 1 / (1 + np.exp(-(x + mu) / s))\n",
    "\n",
    "plt.scatter(X[:, 0], Y, s=10, c='b', alpha=0.5)\n",
    "sig_x = np.linspace(X[:, 0].min(), X[:, 0].max(), 100)\n",
    "y_flip = 1 if Y[X[:, 0].argmin()] == 0 else -1\n",
    "sig_y = sigmoid_mu_s(sig_x, -X[:, 0].mean(), y_flip * 0.5)\n",
    "plt.plot(sig_x, sig_y, c='r')\n",
    "plt.yticks([0, 0.5, 1])\n",
    "plt.grid(axis='y')\n",
    "plt.title('Approximation')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hypothesis and Prediction\n",
    "\n",
    "We can tweak the sigmoid function by passing the dot product of $x$ (vector of feature) and $\\theta$ (the vector of weights).\n",
    "\n",
    "$$h(x) = \\sigma(x \\cdot \\theta) = \\sigma(\\theta_0 x_0 + \\theta_1 x_1 + \\cdots + \\theta_n x_n)$$\n",
    "$x_0$ being equal to 1.  \n",
    "$\\theta$ is not a parameter of $h$ because it's considered as a constant.\n",
    "\n",
    "$h$ is our hypothesis function. It return a probability of an $x$ to belong to one of the group.  \n",
    "To predict some new $x$ we input it to $h$ with the weights and see if the returned value if above or below 0.5.\n",
    "\n",
    "$$\n",
    "\\text{prediction} =\n",
    "    \\begin{cases} \n",
    "        1, & h(x) \\ge 0.5 \\\\\n",
    "        0, & h(x) \\lt 0.5\n",
    "    \\end{cases}\n",
    "$$\n",
    "\n",
    "This is equivalent to:\n",
    "\n",
    "$$\n",
    "