Skip to content

Instantly share code, notes, and snippets.

@kevincruzai

kevincruzai/00_cqf_ml_elective.md

Forked from yhilpisch/00_cqf_ml_elective.md
Created February 18, 2018 05:35
Show Gist options
  • Save kevincruzai/003757da393c62b6f73f49ba9512215b to your computer and use it in GitHub Desktop.
Save kevincruzai/003757da393c62b6f73f49ba9512215b to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. @yhilpisch yhilpisch revised this gist May 23, 2017. 3 changed files with 1204 additions and 2 deletions.
    4 changes: 3 additions & 1 deletion 00_cqf_ml_elective.md
    View file
    Original file line number Diff line number Diff line change
    @@ -54,6 +54,8 @@ If you have either Miniconda or Anaconda installed there is not need to install
    Installing `TensorFlow`: https://www.tensorflow.org/install/




    Finance
    -------

    @@ -87,7 +89,7 @@ Here some resources to get a first overview of basic **machine learning** concep
    Neural Networks
    ---------------

    Here some resources to get a first overview of **neural networks**
    Here some resources to get a first overview of **neural networks**:

    * [A Neural Network in 11 Lines of Python](https://iamtrask.github.io/2015/07/12/basic-python-network/)
    * [Introduction to Deep Neural Networks](https://deeplearning4j.org/neuralnet-overview)
    2 changes: 1 addition & 1 deletion 01_neural_network_python.ipynb
    View file
    Original file line number Diff line number Diff line change
    @@ -11,7 +11,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
    "# Neural Network in Python"
    "# Simple Neural Networks in Python"
    ]
    },
    {
    1,200 changes: 1,200 additions & 0 deletions 04_stock_market_prediction.ipynb
    View file
    1,200 additions, 0 deletions not shown because the diff is too large. Please use a local Git client to view these changes.
  2. @yhilpisch yhilpisch revised this gist May 23, 2017. 1 changed file with 301 additions and 13 deletions.
    314 changes: 301 additions & 13 deletions 03_lsm_mcs_ml.ipynb
    View file
    301 additions, 13 deletions not shown because the diff is too large. Please use a local Git client to view these changes.
  3. @yhilpisch yhilpisch revised this gist May 23, 2017. 1 changed file with 549 additions and 0 deletions.
    549 changes: 549 additions & 0 deletions 03_lsm_mcs_ml.ipynb
    View file
    549 additions, 0 deletions not shown because the diff is too large. Please use a local Git client to view these changes.
  4. @yhilpisch yhilpisch revised this gist May 23, 2017. 2 changed files with 421 additions and 86 deletions.
    120 changes: 64 additions & 56 deletions 01_neural_network_python.ipynb
    View file
    64 additions, 56 deletions not shown because the diff is too large. Please use a local Git client to view these changes.
    387 changes: 357 additions & 30 deletions 02_python_data_analysis.ipynb
    View file
    357 additions, 30 deletions not shown because the diff is too large. Please use a local Git client to view these changes.
  5. @yhilpisch yhilpisch revised this gist May 23, 2017. 1 changed file with 3 additions and 0 deletions.
    3 changes: 3 additions & 0 deletions 00_cqf_ml_elective.md
    View file
    Original file line number Diff line number Diff line change
    @@ -49,6 +49,9 @@ If you have either Miniconda or Anaconda installed there is not need to install
    conda install numpy pandas=0.19 scikit-learn matplotlib
    conda install pandas-datareader pytables
    conda install ipython jupyter
    jupyter notebook

    Installing `TensorFlow`: https://www.tensorflow.org/install/


    Finance
  6. @yhilpisch yhilpisch revised this gist May 23, 2017. 1 changed file with 389 additions and 0 deletions.
    389 changes: 389 additions & 0 deletions 02_python_data_analysis.ipynb
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,389 @@
    {
    "cells": [
    {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
    "<img src=\"http://hilpisch.com/tpq_logo.png\" width=350px align=right>"
    ]
    },
    {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
    "# CQF Elective\n",
    "\n",
    "**Machine Learning for Finance**"
    ]
    },
    {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
    "## Python Import"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 1,
    "metadata": {
    "collapsed": true
    },
    "outputs": [],
    "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from pylab import plt\n",
    "plt.style.use('seaborn')\n",
    "%matplotlib inline"
    ]
    },
    {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
    "## Random Data"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 2,
    "metadata": {},
    "outputs": [],
    "source": [
    "rn = np.random.standard_normal(50)"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 3,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "-0.35456894548352658"
    ]
    },
    "execution_count": 3,
    "metadata": {},
    "output_type": "execute_result"
    }
    ],
    "source": [
    "rn.mean()"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 4,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "0.98524976961888933"
    ]
    },
    "execution_count": 4,
    "metadata": {},
    "output_type": "execute_result"
    }
    ],
    "source": [
    "rn.std()"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 5,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "[<matplotlib.lines.Line2D at 0x10e7e0ba8>]"
    ]
    },
    "execution_count": 5,
    "metadata": {},
    "output_type": "execute_result"
    },
    {
    "data": {
    "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW8AAAD3CAYAAADSftWOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEwZJREFUeJzt3X9sXeddx/HPjU/rNMiZrhSvC9vE/kD+BiusSExaIzYc\nCSq2IegMlkjQBk1WLZP4I1mCNhqt+2OCVBOktDAqonah0K0KqocR/3RUAjJGw6D8GiFOnmgI/tgC\nwmlMbebMyr25/HFt147Pje89v5/nvF9SpPj4+N7nuT8+5znPeZ7nNDqdjgAAftlWdgEAAIMjvAHA\nQ4Q3AHiI8AYADxHeAOChqKgnmptbTDyspdncofn5pSyL4wXqXS/Uu176rffo6EgjbrsXLe8oGiq7\nCKWg3vVCveslbb29CG8AwEaENwB4iPAGAA8R3gDgIcIbADxEeCO14ZlpNSf2adfuppoT+zQ8M112\nkYDgFTbOG2EanpnWziOH136OLl/SziOHtSBpeXKqvIIBgaPljVR2PHU6fvvTTxZcEqBeCG+kMnT1\nykDbAWSD8EYq7bE9A20HkA3CG6ksHTsRv/3o8YJLAtQL4Y1UlientHDmrFrje9WJIrXG92rhzFku\nVgI5Y7QJUluenCKsgYLR8gYADxHeAOAhwhsAPER4A4CHCG8A8BDhDQAeIrwBwEOENwB4iPAGAA8R\n3gDgIcIbADyUKLzN7B4ze8HMvm5mf29mP5t1wZANblEGhClpy/sjkl53zr1f0gckfSG7IiErq7co\niy5fUqPdXrtFGQEO+C9peL8k6fGV/zcktbIpDrLELcqAcDU6nU7iPzazEUl/JulZ59yLd9u31Wp3\nomgo8XMhgSiS2u347bduFV8eAEk04jYmXs/bzN4paUbSM1sFtyTNzy8lfSqNjo5obm4x8d/7Km29\nm2N7FF2+tGl7a2yP5iv8evJ+1wv13nq/OEkvWN4v6RVJn3bOnU3yGMgftygDwpW05X1SUlPS42a2\n2vf9QefczWyKhSwsT05pQd0+7qGrV9Qe26Olo8e56w0QgETh7Zw7KuloxmVBDrhFGRAmJukAgIcI\nbwDwEOENAB4ivAHAQ4Q3AHiI8AYADxHeAOAhwhsAPER4A4CHCG8A8BDhDQAeIrwBwEOENwB4iPAG\nAA8R3gDgIcIbADxEeAOAhwhvAPAQ4Q0AHiK8AcBDhDcAeIjwBgAPEd4A4CHCGwA8RHgDgIcI7xIN\nz0yrObFPu3Y31ZzYp+GZ6Q3bFUUbtgPAqqjsAtTV8My0dh45vPZzdPmSdh45rKXX/k47njuzafuC\npOXJqRJKCqCKaHmXZMdTp2O33/fC8/H7P/1kjqUB4BvCuyRDV6/E/2J5ebD9AdQS4V2S9tie+F8M\nDw+2P4BaIrxLsnTsROz2mx99JH7/o8dzLA0A33gZ3r1GafhkeXJKC2fOqjW+V50oUmt8rxbOnNV3\nT/3m2nat287FSgDrNTqdTuI/NrP3Svq8c27/VvvOzS0mfqLR0RHNzS1K2jxKY1WIAbe+3nVCveuF\nem+5XyNue+KWt5l9StJzkrYnfYwkeo3SYDQGgDpJ3PI2s5+X9K+SXnDOPbjV/q1WuxNFQ4mea4Mo\nktrt+O23bqV/fADZOndOOnVKmp2VxselkyelAwfKLpVPYlveiSfpOOe+Ymbv6nf/+fmlpE+14fSi\nObZH0eVLm/Zpje3RfGCnXpxO1kuI9d7UzXnxonTwoBYWbq51c4ZY734M0G0Su927C5a9RmkwGgOo\nHro58+NdePcapRHaxUogBL0mlzHpLD0v1zZZnpwirAEPtHt0czLpLL1ULW/n3H/2c7ESQD3RzZkf\n77pNAPiDbs78eNltAsAfdHPmg5Y3AHiI8AYADxHeAErB7f7Soc8bQOF63QaQ2/31j5Y3gMIx8zI9\nwhtA4Zh5mR7hDaBwvWZYMvOyf4Q3gMIx8zI9whtA4dbPvOR2f8kQ3h4J4d6dwKrlySnNn78g3bql\n+fMXCO4BEd6eWB1aFV2+pEa7vTa0igDPFwdMVBXh7QmGVhWPAyaqjPD2BEOriscBE1VW6fBm+uyb\nGFpVPA6YqLLKhvf6U1ZxysrQqhJwwESVVTa8OWXdiEXti8cBE1VW2fDmlHWz1aFV16/dYGhVAThg\n+iPLUUG+jDCq7KqC3LgUVcBdYKovyxUK7/ZYUrdHYOjqFbXH9mjp2IlSPxuVbXlzygqgH1l2sfZ6\nrO/73GcrN2y0suHN9FkA/ciyi7XX32z7zrdjt5d5Da6y4S0xfRaoq0H6nbMcFTTo35R5Da7S4Q2g\nfgad2ZplF2uvx7r99nfEbi/zGhzhDaBSBu3DznJUUK/H+u5nPxe7f5nX4Co72gTVMjwzXakr7QhX\nkj7sLEcF9XqsBXUPIGvfgaPHS/0O1Ca8CZ/kuFksilTVYcJVGzZai24TVodLh9muKBLDhPtTi/Am\nfNJhtms8X2bi+YaZrf2pRbcJ4ZNOVU9jy0RXUr6q1kVRRbVoebM6XDqcxm7G2RzKlqjlbWbbJD0j\n6QFJy5Iedc59K8uCZWnp2IkNraS17TUOn0EsT05V7kp72TibQ9mSdpt8WNJ259w+M3tQ0mlJD2dX\nrGwRPulxGrsRXUkoW9Lwfp+kr0qSc+4bZvae7IqUD8IHWeJsDmVLGt47Jb2x7ue2mUXOuVavP2g2\ndyiKhhI+nTQ6OpL4b31GvSvq44eknfdJTzwhzc5K4+PSY49p54EDvf/m3Dnp1Kk39z95Urpj/8rX\nOyfUe3BJw3tB0vpn3Xa34Jak+fmlhE/VreDc3GLiv/cV9a64n/jp7r/1epT7ztEpunhROnhQCws3\n184Ival3xqj31vvFSTra5FVJH5KklT7viwkfp7IYw4ssMToFWUva8p6R9JCZXZDUkHQouyKVjzG8\nyBqjU5C1ROHtnLst6RMZl6UytmolsUZKPlbXn9HVK2oG9toyOgVZq8UknUH1bCVdmWWNlJysX39G\nAb62dZjoRFdjsQjvGD1bQ/fcE7uZfsv0Qu8TDn29Dt8WfwvhQNPodDqFPNHc3GLiJyr6avSmkQEr\nOtu2qXH79ubtUaTr125kXo46XYXftbupRru9aXter20V+fx+Nyf2xXYLtcb3dm9leBdV+X4XfTAd\nYLRJI247Le8YvVpJbfuh2P3pt4xX1n0IUTyfLshmfZZXViu+FqsKJtFrRiaz6voz6IgdZiz6zacL\nslkeaMocmUbLewCh91tmKc19CMVr6x2fLshmeZZX5rUaWt4DYo2U/qS5D+Ho6IjmPe37rSufFn/L\n8iyvzO4iwhu58Ok0GtnwpWGT5YGmzM853SbIhU+n0aif5ckpzZ+/oOvXbmj+/IXEB50yP+e1D+8Q\nxntWEdcHUAdlfs5rPc67KuM9e/F53G8a1LteBqn36hIKISxPwTjvFEKf1QfkoayzVd9mceat1uHt\n08QCoArKDFAaWxvVOryZ1QcMpswApbG1Ua3DmxERwGDKDFAaWxvVOrzrPCKCUTZIoswApbG1Ua3D\nW8puvKdPfLvww4GmOsoM0Do3tuIww7KG7tZvWbUvArekq5ayp8H7MouzCLVvedeRTxd+6jDCYPXM\nQlHkxZlFHc9Wq4jwriGfLvz4dKBJIvTbvyE/hHcN+XThx6cDTRJ1OLNAPgjvGvLpwo9PB5okQj+z\nQH64YFlTvlz4KfsCWd5YOhdJ0fJG5RVxgays4Yihn1kgP7S8UXtlDkdcf2YRXb2iVmBnFsgP4Y3a\nK3vcO7d/QxJ0m6D2uGgIHxHeqL3QhyMiTIQ3ao+LhvAR4Y3a82ncO7CKC5aA/Bn3Dqyi5Q1kjCVs\nUYRU4W1mk2b2YlaFCRVf5vxU7bX1ba10+CtxeJvZ05KeSPMYdcCXOT9VfG1ZaApFaXQ6nUR/aGa/\nIOl/JB1xzh3Yav9Wq92JoqFEz9W3c+ekU6ek2VlpfFw6eVI6sGXR8vXud0sXL8Zv/+Y3iy9PSKr4\n2kaR1G7Hb791q/jyrKridwP9asRt3PKCpZl9TNIn79h8yDn3x2a2v99nn59f6nfXTUZHRzS3xcyz\nO6c46+JF6eBBLSzcLPVC1K7Z2dhXvjM7q+tb1Kmfeoeo33qneW3z0uyx0FRrbM+Wsyfzer+r+t1Y\nxed86/3ibNnl4Zz7onNu7x3/Xhu8qPmq6ukqE0DyU8XXtopjxqv63ShC1a6JZCmY/uqqTnEu4ssc\n8gf0bqoYlFmOGc/qfb3bdyPkz04Vr4lkKZhx3lVdFznv9ajrfIPeqq71ncWY8aTv6/DMtHY8dfrN\n1+PYiZ7fjdv3vy3oz07ZC47lLfEFy0HNzS0mfqJEfd4rfJ4p10+9mxP74vtYx/dq/vyFvIqWK/pA\nk72vvb4DS48e0Y7nzmza3n77OzT0nW8P9Bx5yOv93rW7qUbMxeNOFOn6tRuZP9+gBujzjr1gGUy3\nSV2nOFe1uwjpJHlfe7U0773waux3Y9t//9fAz+GTKl4TyVIw3SZSPac4V7W7COkkeV/vFvhx340d\nT50O+rOzdOxE/JlIIAuOBdPyrqsqXrRDekne10FbmqF/dkI/Gw+q5V1HVb1oh3SSvK+DtjTr8NkJ\n+Ww8mAuWIaLe2YsbjVGVL3cW9R6emfYujPmcb7lfshmWQCjqMKwy5JYmNqLPGxuEPGkjpJmGIb9P\n6A/hjTVFzUgrK3hCGVYZ+sxB9IfwxpoiWqZlBk8o435DOoNAcoQ31hTRMi0zeEIZGhfKGQTSIbyx\npoiWaZnBE8q431DOIJAO4Y01RbRMyw6e5ckpzZ+/oOvXbmj+/AXvglsK5wwC6RDeWFNEy5TgSS+U\nMwikwzhvbJD3OOE6zOorAuO5QXijcAQPkB7dJgDgIcIbADxEeANADvKeSUyfNwBkrIhF0Gh5A0DG\niphJTHgDiMXKhckVMZOY8AawCSsXplPETGLCG8AmrFyYThEziQlvAJuwcmE6RSxhwGgTAJu0x/Yo\nunwpdjv6k/dMYlreADZhAbHqI7wBbMLKhdVHtwmAWCwgVm20vAHAQ4Q3AHiI8AYADyXq8zazt0j6\nkqSdku6VdNw597dZFgwA0FvSlvdxSX/hnJuQ9Iik38usRACALSUdbfLbkpbXPcb3sikOAKAfjU6n\nc9cdzOxjkj55x+ZDzrnXzOxtkl6WdMw597W7PU6r1e5E0VCqwlbauXPSqVPS7Kw0Pi6dPCkdOFB2\nqQD4rxG7cavw7sXMfljSOUm/6px7eav95+YWkz2RpNHREc3NLSb989zdufD6qrSTGqpe77xQ73qh\n3lvuFxveifq8zWxc0kuSfrGf4A4dK7ABKFrSPu8nJG2X9LSZSdIbzrmHMyuVZ1iBDUDREoV3nYM6\nDiuwASgak3QywApsAIpGeGeAFdgAFI1VBTPCCmwAikTLGwA8RHgDgIcIbwDwEOENAB4ivAHAQ4Q3\nAHiI8AYADxHeAOAhwhsAPER4A4CHCG8EaXhmWs2Jfdq1u6nmxD4Nz0yXXSQgU6xtguDceWej6PIl\n7TxyWAsS688gGLS8ERzubIQ6ILwRHO5shDogvBGcXncw4s5GCAnhjeBwZyPUAeGN4HBnI9QBo00Q\nJO5shNDR8gYADxHeAOAhwhsAPER4A4CHCG8A8FCj0+mUXQYAwIBoeQOAhwhvAPAQ4Q0AHiK8AcBD\nhDcAeIjwBgAPEd4A4KHKripoZtskPSPpAUnLkh51zn2r3FLlz8zeK+nzzrn9ZvaDkp6X1JH0b5J+\nxTl3u8zyZc3M7pF0VtK7JA1L+nVJswq/3kOSnpVk6tbzE5K+p8DrvcrM3irpHyU9JKmlGtTbzP5J\n0sLKj/8h6TeUot5Vbnl/WNJ259w+Sb8mKf7GhAExs09Jek7S9pVNT0r6jHPu/ZIakh4uq2w5+oik\n11fq+AFJX1A96v0zkuSc+zFJn1H3i1yHeq8esM9IurmyKfh6m9l2SQ3n3P6Vf4eUst5VDu/3Sfqq\nJDnnviHpPeUWpxD/Lunn1v38o5K+tvL/lyX9ZOElyt9Lkh5f+X9D3VZY8PV2zv2ppI+v/PgDkv5X\nNaj3it+S9PuSrq38XId6PyBph5m9YmZ/aWYPKmW9qxzeOyW9se7ntplVtpsnC865r0i6tW5Twzm3\nun7BoqS3FF+qfDnn/s85t2hmI5Km1W2FBl9vSXLOtczsDyX9rqQvqwb1NrNHJM055/583ebg6y1p\nSd2D1k+p20WW+v2ucngvSBpZ9/M251yrrMKUZH3/14i6rbPgmNk7Jf2VpBeccy+qJvWWJOfcL0sa\nU7f/+751vwq13oclPWRm5yX9iKQ/kvTWdb8Ptd5XJX3JOddxzl2V9Lqk+9f9fuB6Vzm8X5X0IUla\nOcW4WG5xSvHPZrZ/5f8flPT1EsuSCzO7X9Irkj7tnDu7srkO9f6omT228uOSugesfwi93s65H3fO\nTTjn9kv6F0m/JOnl0Out7kHrtCSZ2fer27PwSpp6V7kbYkbdI/QFdftCD5VcnjKckPSsmd0r6bK6\n3QqhOSmpKelxM1vt+z4q6XcCr/efSPoDM/trSfdIOqZuXUN/v+PU4XP+RUnPm9nfqDu65LCk60pR\nb5aEBQAPVbnbBADQA+ENAB4ivAHAQ4Q3AHiI8AYADxHeAOAhwhsAPPT/36Hb2z4IoPIAAAAASUVO\nRK5CYII=\n",
    "text/plain": [
    "<matplotlib.figure.Figure at 0x1059788d0>"
    ]
    },
    "metadata": {},
    "output_type": "display_data"
    }
    ],
    "source": [
    "plt.plot(rn, 'ro')"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 6,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "[<matplotlib.lines.Line2D at 0x10e829dd8>]"
    ]
    },
    "execution_count": 6,
    "metadata": {},
    "output_type": "execute_result"
    },
    {
    "data": {
    "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD3CAYAAADxJYRbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8VdW9///XmTKRGU7mBJKQLOYwBEhAhiqgqC2iVr1q\n9Wq19Ve/7a23c7W/3l7be3+9v2rba3+2vbZWvWprHcChIggIBDDMAcKwkpCEhEwkIWQgZD6/P07A\nGDKeDCc5+/N8PHyUc85ee39Wj77PZu291zI5HA6EEEIYg9ndBQghhBg9EvpCCGEgEvpCCGEgEvpC\nCGEgEvpCCGEgVncX0JfKyvoh3VoUEuJHTU3jcJUzbki/jUX6bSwD6bfdHmDq7TOPPtO3Wi3uLsEt\npN/GIv02lqH226NDXwghxOdJ6AshhIFI6AshhIFI6AshhIFI6AshhIFI6AshhIFI6AshhIEYLvQ/\nPlDM0bwqd5chhBBuYajQz8qt4q/bcvnjeyeoa2xxdzlCCDHqDBP6La3tvL41B4Cmlnbe3V3g5oqE\nEGL0GSb0P8w8S1VtE6tSYwgP9WPnkVJKqy65uywhhBhVhgj98zWNfJhZRLC/F+uXJXDXykQ6HA7e\n2nHG3aUJIcSoMkTo/3VrLm3tHdxzQxK+3lbmJk1CxQaTlVfFqbM17i7PsLYfPsd3/r89bMzIp+Fy\nq7vLEcIQPD70s3KrOHqmmumTQ1g4LQwAk8nEXddPBeCN7bl0yOLwbrH7WBk19c28t6eQ7/1+L29+\nkkftJbnALsRI8ujQb+68eGsxm7hvdTIm02dTTMdHBpI+M5yiigY+zS53Y5XGdLm5jbMV9cRHBnDP\n9VPx8bKwaV8R3//9Xl77OIcLdU3uLlEIj+TRof/WtlyqaptYvTCWqEkTrvn8jhWJ2Kxm3tmVT3Nr\nuxsqNK68klocDpgxJZQ1i+L4r8fS+cqaZAL9vNh26Bw//OOn5JXUurtMITyOx4b++ZpG3v4kl5AA\nb764ZEqP24QG+rBmYSw19c1sOVA8ugUaXE7xRQBUbDAANquFL8yP4T+/nsb9a5Jpa3fw4adn3Vmi\nEB7JI0Pf4XDw+tZcWts6uPv6qfh6974q5M1pkwn0s/Fh5llqG5oBaGvvoLC8jh1ZJby06TQvvH+S\n8xcvj1b5hqCLL2I2mUiMDvrc+1aLmevnx5AQFcjRvCrOG3A5PCFG0pheI9dVF+qaOXammpSkSVcv\n3vbG19vKumUJ/O9mzW/fOgbAucoG2to/f3E3K6+SB2+axqLp4SNW91jT2taObQSWpGtubaegtI7J\nEf69/iCvSo3hf947yfbDJdxzQ9Kw1yCEUXnkmX5ooDePrZvJ9+5P/dzF294sT4kk2j6BwvJ6zlU2\nEBvmz8p50fzz2mn820ML+eot0+nogD+8e4KXNp02xPj/pyfK+cazu9hxpGTY951fWkd7h4PkzqGd\nnqSqMIL8vcg4Vsrl5rZhr0EIoxrQmb5SajHwS631SqXUVOAlwAFkA49rrTuUUo8CXwfagJ9rrT/o\nto8e2w1XR7oymUwsmh5OkL83lZf7vwXQYjbzg3vnc6GuiahJE7BaPv9bGBceQGJ0EH/YmM2uo6Xk\nldTy2LqZxNj9R6L8MeHT7HLaOxy8slnT0trOmkVxw7ZvXeR8NkLFhvS6jdVi5gvzotmYUcDe7HJu\nWBAzbMcXwsj6PdNXSn0f+BPg0/nWs8BTWutlgAlYp5SKAL4FLAVuBP5TKeXdbVfXtBueLgwPf18b\nceEB1wT+FRGhfjz5wAJuWBBDadUlnn75IDuOlNDWPiK/W251ubmN00U12IN9CPb34m/b83h/TwGO\nYXqeIaf4IiYgKTaoz+1Wzo3GajGx7dA5eZZCiGEykOGdM8DtXV4vAHZ2/nkTsApYBOzRWjdrrWuB\nPGBOt/301G5csVkt3Lc6mW/ePhsvq5lXNmu+8ewunn75IK9u0ew5XkZJ1SU6OsZ3QJ0ouEBbu4O0\nGRH88P4FTAryYUNGAW/vzB9y8Le2dXCmtI6YMH8m+Nj63DZwgheLp4dTfqGREwUXhnRcIYRTv8M7\nWuu3lVJTurxl0lpf+S+/HggCAoGuN1VfeZ9+2vUpJMQP6xAvJNrtAUNq35M19gDmzojg7e255BTV\nUFhWR0FZHeAc//b1tjArcRILZ0SwcHo4k4J9h72G/gyl36e35gKwcmEcyXEh/Nc3l/OTP+7hw8yz\nmK1mHl03G7O5/2slPTlZUE1rWwcpyfYB1XjnasWe7HJ2HSvj+sVT+t1+JL7v8UD6bSxD6bcrd+90\nHc8IAC4CdZ1/7v5+f+36VDPE2/Xs9gAqK+uHtI/emIA7lycAzrtcis9foqCsjsLyOvJL6zhwsoID\nJysAiAv3JyVxEilTJzElMgDzAC4uD8VQ+t3R4WD/iXKCJngR5GO5up/v3jOPX/3tCB/sLqC2rokH\nb5rmUvDvP14KQNykCQOqMcjbQlJMEIdOn+e4riAi1K/XbUfy+x7LpN/GMpB+9/Wj4EroH1FKrdRa\n7wDWAp8A+4FfKKV8AG9gOs6Ltf218wg2q4WEqEASogKvvld58TLHzlSTlVeFLqqhqKKB9/cWsmRW\nBI/cOsON1fbtTGktDZdbWZ4S+bkfp6AJXvzg3vk880YWGcfKCAvx5Zb0KYPevy5y/tb3dedOd6tS\nY8k9V8u2Q+e4b3XyoI8phPiMK7dsfgf4mVLqU8ALeEtrXQ78N5ABbAee1Fo3KaVmKKWe763d0Msf\nu+zBvtywIIbv3D2X335rGY+vn01YiC+ZJypobBq7tyBmdS4lOXeq/ZrP/H1tfPeeufh6W9l68Nyg\nL2K3d3SQW1JL5EQ/Aid4DbjdvKRJhAR4s/t42Zj+/06I8WBAZ/pa60IgrfPPOcCKHrZ5AXih23sn\ngW/01c4IfL2tLFB2Siob2Li7gJOFF0jt56Gx4VZ58TImE0wK6vv6wtG8amxWM9On9Hw75QQfG8vm\nRLLlQDEHTp0nfVbEgGsoqmiguaX96tQLA+V8Sjeat3fms+d4GasXxg6qvRDiMx75cNZYNTtxIgDH\nzlSP6nE/PVHOky9k8vOXD/b5oNP5mkZKqy4xY3II3rbeL6DfsCAGkwm2HCge1N08rgztXLE8JQqb\n1Sy3bwoxRBL6o2hyRACBfjaO5VePSnB1OBy8syufF94/SVu7g7rGVj4+2PvEcll5zh+juUmT+tyv\nPdiX+Ul2zlbUk3tu4DNhXplkzZXQD/DzIm1GOOcvXmZT5lkuNcmiK0K4QkJ/FJlNJmYnTKTuUgtF\nFSN710Fzazt/2JjNB3sLsQf78JMHUwnws7F5f1Gvq1Qd7RzPT5nad+gDV4dYtvbxI9JVh8NBTvFF\n7ME+hAb69N+gl2ParGbe3pnPt/97N8++kcXOrBLqZOEVIQZMQn+UzekM1JEc4rlQ18QvXzvMQV1J\nckwQTz2QSnxkILekT+FyczsfZl47ZXFjUxs5xReZEhFAsH/3h6mvlRQTxOTwAA7lVFJV2/8MpOfO\nN9DY3Nbn1Av9ibH78/RXF3H78gRi7P5kF1zg5Y80T/xuN7987TAHTspiOEL0R0J/lM2cEoLZZOL4\nCIX+2fJ6vvObnRSW17N0dgTfuWceAX7OO2W+MC+KkABvth06R0198+faZRdU097hYO4AzvLBOb/R\nqtQYHA7Yfqj/SdmGMrTTVViIH7cumcJPH1rILx9L5+7rp5IYFYQuvsjPX9zH/lMVQ9q/EJ5OQn+U\n+fnYmBoTRH5pHfWNwzssUVPfzC9fP0x1XRNfXpnIwzdPx2b97Cu2WS2suy6e1rYO3t9b+Lm2Wbmd\nt2r2M57f1aLp4QRO8GLn0VKaWvq+lfJq6McNLfS7sgf7cuOiOH78lQU8+cACfLyt/M97Jzlw+vyw\nHUMITyOh7wZzEic6pxod5vlkPthbSFNLO4+um83atMk9Tiu9dHYE4SG+ZBwtvbpASXtHB8fzqwkN\n9CY2bOAzh9qsZq6fF83l5jb2HO99aMXROZ4fEuCNPci18fz+JEYF8bOvpeNlM/PHd09wUIJfiB5J\n6LvBnIThv3Wz8uJldh0tJSzEl7W9LA8Jzmmk1y9PoL3DwcbdBQDknavlUlMbKYmTBrT+QFcr5zln\nwtx6sLjXO5LKLzRS19iKig0e9P4HY9rkUP71rrnYbGb++N4JDunKETuWEOOVhL4bRNsnEBLgTXZ+\n9bDNyPnengLaOxzcdl18r9NDX5E6LYy4MH/2najg3PmGz57CHcTQzhWBE7xImxFBRc3lXq9TXL0/\nfxiHdnozNSaIJ76cgtVi5g/vZnMkV4JfiK4k9N3AZDKRkjiRS01t5JfWDXl/ZdWX2JtdTrR9Aotm\n9L+co9lk4vYVCTiAd3blk5VbhbfNwjQXQ3lVqnOBk+7PADgcDkoqG66OsQ/2SVxXJccG8+0vz8Fi\nMfH8huyrP2pCCA9dI3c8mJ04kR1ZpRzLr2JqTL+zTPdpY0YBDgesX5Yw4Bk8ZydMJCkm6Gogzk+2\nu7weblx4ANPigjlZWMPRvCpqGpo5fbaG02drqGt0PhMwMdC7zxkyh5uKC+Hbd6bwmzeP8oeN2fz8\n0cX9TkEhhBHImb6bTJ8cgtViGvK4flFFPQdOn2dKRADzBjE8YzKZuGNF4tXXKVMnDqmO1anOh7V+\n+9YxXvlIs//UeUxmE+kzw3no5mk89eDCER3P78m0ySF85UZFS1sHf9+eN6rHFmKskjN9N/HxsqJi\ngzlRWENNfTMhAf0/ENWTjRnOi7G3L08YdKgmxwYzd+okTp2tISVx8OP5XaVMncSSWRE0t7YzY3II\n0yaHEBHqN+pB3136rAh2ZJVwUFdysvACM6aEurUeIdxNzvTdaHZn0Gbnu3a2f6aklqy8KpJjgpgZ\n71qYPbZuJv/xtbRBTXXcE7PZxCO3zuDx9bP5wvwYIidOcHvgg/P6xf2rFSbgtY9zPHJNYyEGQ0Lf\njeYMcdbNDRn5AKx34Sz/Ci+bxeW/ZYwXkyMCWDE3irLqRrYdOufucoRwKwl9N4oI9SMsxJcThRcG\nfQZ6+mwNJwtrmBkfiopzfT4bo7h9RSITfKy8u7uA2obm/hsI4aEk9N1sTsJEmlraBzVFsaNzymRw\njuWL/vn72rh9eQJNLe28ueOMu8sRwm0k9N3syhDPYCZgO55fTV5JLfOSJhEfGdh/AwHAirnRxIX5\nsze7nLxB/MgK4UlcuntHKeUN/AVIAOqAx4GngStr500BMrXW93Rrd7hze4ACrfVDrhzfk6i4YLys\nZo6eqeKu66f2u/2VhVFMOO/LFwNnNpu4b00y//nqYV79WPN/P7gQs9n9F5uFGE2u3rL5KNCgtU5T\nSingd1rrGwGUUiHAJ8ATXRsopXwAk9Z65RDq9Tg2q4XZCRM5lFNJVm5Vv1MhHNaVFFU0kDYjnJhB\nTI4mnJJigkmfGcGnJ8rZdbSUlfOi3V2SEKPK1eGdGcAmAK21BqZ3+exnwHNa67JubVIAP6XUFqXU\ndqVUmovH9jjrlydgMZt4fWsOrW3tvW7X0eFgQ0Y+ZpOJddfFj2KFnuXLX0jEx8vC2zvP0NjU95TQ\nQngaV8/0s4BblVIbgcVAtFLKAkwEbqDbWX6nRuBXwJ+AJGCTUkpprXv9ry4kxA+ri1MDXGG3Bwyp\n/Wiw2wP40vJENuzIY1d2BfesVj1ut/1gMWXVjaxeFMcs1fccO+Oh3yNhIP222wNYtzyRN7bmUFJz\nmSVzokahspEl37exDKXfrob+izjP7jOAPcAhrXW7UupO4HWtdU+nqzlAntbaAeQopaqBSKDXRVZr\nOud7d5XdHkBl5ciuRTtcVs2LYvvBIv6+NYeUKSFMCv78PDFt7R28uukkVouJNQti+uzXeOr3cBpM\nv6eETQDg4IlykiLHd3DI920sA+l3Xz8Krg7vLAS2aa2vA94E8jvfX0XnsE8PHgaeAVBKRQGBQPch\nIMPy9bZy1xem0trWwd96mCdm97EyKi82sWJuNBNHaCESI4mPDMRiNpF77qK7SxFiVLka+rnAt5VS\nn+K8a+dfO99XfPYD4HxDqVeUUnHAn4FgpdRu4A3g4b6GdowobUY4STFBHM6p/NzUDK1t7by/txAv\nq5lb0ye7sULP4WWzEB8ZSFFFQ79LPQrhSVwa3tFaV+E8q+/+/swe3nugy8t7XTmeUZhMJu5bnczP\nXjrAa1tzefqrIVgtZj45XEJNfTNrF8cR5O/ZUyaMpqTYIPJKajlTUufy3EVCjDfycNYYExcewPXz\nY6i40MiWA8U0tbTxj8yz+HpbWJsmZ/nDKSnGuajLlUXbhTACmVp5DFq/LJ79pyp4f08h1XVN1De2\nsu66ePx9be4uzaMkxQRhAhnXF4YiZ/pjkJ+PjTtXJNLc2s4nh0vw97WxZmGsu8vyOBN8bETbJ5Bf\nWidTLgvDkNAfo5bOiSQhyjmvztq0OHy95S9lIyEpNpiWtg7Olhvv1j9hTBL6Y5TZZOJrX5zBbcvi\nWbUgxt3leKzkK+P6MsQjDEJCfwwLC/HjS0vjXV6wXPQvqXNR+tximXVTGIOEvjC00EAfJgX5kHvu\nIh0Oh7vLEWLESegLw0uKCeZSUxtlVZfcXYoQI05CXxhecqxziCdHFlYRBiChLwwvOdZ5MTdXHtIS\nBiChLwwvItQPf1+b3MEjDEFCXxieyWQiKSaIC3XNVNc2ubscIUaUhL4QfDbEI2f7wtNJ6AuBjOsL\n45DQFwKIDfPHy2aWO3iEx5PQFwKwWswkRgVRWnWJhsut7i5HiBEjoS9Ep6tDPDKuLzyYhL4QnZJl\nHh5hAC7N16uU8gb+AiQAdcDjgD/wAc71cwF+r7V+o0sbM/A8kAI0A49ora9dAVwIN0mICsJiNskd\nPMKjuTpJ+6NAg9Y6TSmlgN8BbwLPaq2f6aXNbYCP1jpdKZUGPAOsc/H4Qgw7by8LceEBnC2vp7ml\nHW8vmd1UeB5XQ38GsAlAa62VUtOBBYBSSq3Debb/ba1115UprgM+6myTqZRK7e8gISF+WIc4rbDd\nHjCk9uOV9Ns1Kcl2CsrqqG5sJSU6eJiqGnnyfRvLUPrtauhnAbcqpTYCi4Fo4CDwJ631IaXUk8BP\nge92aRMIdB0sbVdKWbXWbb0dpKam0cXynOz2ACorjbcikvTbdfHh/gB8tLeAqGCf4ShrxMn3bSwD\n6XdfPwquXsh9EedYfgawHjgEvK21PtT5+QZgXrc2dUDXSsx9Bb4Q7jAzPpTwEF8yT5RzsaHZ3eUI\nMexcDf2FwDat9XU4x/Lzgc1KqUWdn9+A84egqz3AzQCdY/rHXTy2ECPGbDJx46I42todbDt0zt3l\nCDHsXB3eyQWe7hzGuQh8FYgAnlNKtQLlwNcAlFKvAE/hPPtfrZTaC5iAh4ZYuxAjYsmsCDZk5PPJ\n4RJuSZ+Mj5csSi88h0v/Nmutq4BV3d4uBZb2sO0DXV4+5srxhBhNXjYLN8yPYePuAjKOlbE6Ndbd\nJQkxbOThLCF68IX50XhZzXx8oJj2jg53lyPEsJHQF6IHAX5eLJ0TSVVtE4d0pbvLEWLYSOgL0Ys1\nC2MxAZv2FeFwONxdjhDDQkJfiF6Eh/gxX9k5W16PLpKpGYRnkNAXog83LYoD4KP9RW6uRIjhIaEv\nRB8So4OYGhPEsTPVlFRdcnc5QgyZhL4Q/Vjbeba/Wc72hQeQ0BeiHylJkwgP9ZOpGYRHkNAXoh/O\nqRliaWt38I9Pz7q7HCGGREJfiAFYOiuC8FA/th06R1ZulbvLEcJlEvpCDIDNauEbt83CZjXz53+c\npOri5UG1dzgc1De2UFBWx8HT58k8UU5Hh9z7L0afzCQlxADFhvlz3+pkXtp0mt+/m82P7l+A1dL7\neVPmiXIyT1ZQVdtEVe1lWlo/P51DfWMrqxfKvD5idEnoCzEIy+ZEklN8kb3Z5fx9ex73rk6+ZpvW\ntg7+ui2XHUdKAPDzthIR4sfEIB/nP4E+fLC3kA0Z+SycHkawv/dod0MYmIS+EINgMpn4yhpFYXk9\nWw+dIzk2mNRpYVc/v1DXxPMbs8kvrSPG7s/j62cRHup3zX68bRZe2az5+/Y8vvalmaPZBWFwMqYv\nxCB5eznH971sZl788BQVnct6nj5bw7+/dID80jrSZ4bz5AMLegx8gOUpUcRHBpB5soJThRdGs3xh\ncBL6QrggatIEHrxxGk0t7Ty/IZtN+87yq79lcampjftWJ/PIrTPwtll6bW82m7h/jcIEvPpxDm3t\nMn2zGB0S+kK4KH1WBCvmRlF8voE3PzlDgJ+N7/3TPG5YEIPJZOq3fXxkICvnRVNW3ciWA8WjULEQ\nMqYvxJDcuyqJ8zWXMZtNfPWW6YO+KHv7igQO6vO8t6eAxdPDmRjkM0KVCuHkUugrpbyBvwAJQB3w\nODABeA5oB5qBB7TWFd3aHe7cHqBAay3r5IpxzWa18L1/mudy+wk+Nr68ciovfniKv23L5fHbZw9j\ndUJcy9Uz/UeBBq11mlJKAb8DfIBvaq2zlFJfB34A/OuVBkopH8CktV45xJqF8ChLZkeQcayUQzmV\nHM+vZnbCRHeXJDyYq6E/A9gEoLXWSqnpwGKtdVmX/TZ1a5MC+CmltnR+/mOtdWZfBwkJ8cNq7f1i\n2EDY7QFDaj9eSb/Hl2/dM59/eXYHf9uWx8LZUQT4eQ2q/Xjt91BJvwfP1dDPAm5VSm0EFgPRwHkA\npdQS4P8Ay7u1aQR+BfwJSAI2KaWU1rqtt4PUdN4K5yq7PYDKyvoh7WM8kn6PPxOsJlYtiGHLgWLu\n/ckm/H1thIf4EhbiS1iIH2EhvqQkTsTPx3ZN2/Hc76GQfve9TW9cDf0XgelABrAHOKS1bldK3Q08\nCdyi9TWrSecAeVprB5CjlKoGIgG5bUEIYP3yBLxsZs6WN3C+ppHC8nrOlNZd/Txyoh8/un8B/r7X\nBr8QA+Vq6C8Etmmtn1BKpQKTlVL3A18HVmqte3ra5GFgNvANpVQUEAiU9bCdEIbkbbNw+/LEq6/b\nOzqormvmfE0j+0+dZ/exMv77rWN89565ePXxDIAQfXE19HOBp5VSTwIXgUeA40AR8I7z2i47tdY/\nVUq9AjwF/Bl4SSm1G3AAD/c1tCOE0VnMZsKCfQkL9mXGlFDa2jrIPFnBH949weO3z8JilsdsxOC5\nFPpa6ypgVbe3Q3vZ9oEuL+915XhCGJ3ZZOLhW6ZT19hCVl4Vr27J4YEb1YAeAhOiKzlVEGKcsFrM\nPL5+NnFh/uzMKuX9PYXuLkmMQxL6Qowjvt5WnrgrhUlBPmzcXcDOrBJ3lyTGGQl9IcaZIH9v/vXu\nufj72nhls2b/iXJ3lyTGEQl9IcahiFA//uXLc7BZzTz718Ncamp1d0linJDQF2KcSowKYt118Vy6\n3MpH+4rcXY4YJyT0hRjHrp8fQ2igNx8fLKb2Uou7yxHjgIS+EOOYt83C3asVLa0d/OPTQneXI8YB\nCX0hxrnViyYzKciHHUdKqK7tPs+hEJ8noS/EOGezmll3XTxt7Q7e21Pg7nLEGCehL4QHSJ8ZQeRE\nP/YcL6es+pK7yxFjmIS+EB7AbDZx+/IEOhwO3t0tZ/uidxL6QniI+cl2JkcEsP/UeYoqjDfPvBgY\nCX0hPITJZOKOFQkAbNiV7+ZqxFgloS+EB5k5JZTk2GCOnqkm71ytu8sRY5CEvhAexGRyju0DvL3z\nDA6Hw80VibFGQl8ID5McG8zshIno4oucKanrv4EwFAl9ITzQ2sVxAHx8UJagFp/n0spZSilv4C9A\nAlAHPI5zCcSXOv83G3hca93RpY0ZeB5IAZqBR7TWeUMpXgjRMxUXTIzdn0O6kgt1TYQG+ri7JDFG\nuHqm/yjQoLVOA74J/A54FnhKa70MMAHrurW5DfDRWqcDPwSecfHYQoh+mEwmVqfG0OFwsO3wOXeX\nI8YQV0N/BrAJQGutgenAAmBn5+ebuHYN3euAjzrbZAKpLh5bCDEAaTPD8fe1sSurlObWdneXI8YI\nl4Z3gCzgVqXURmAxEA2c11pfuVWgHgjq1iYQ6HoPWbtSyqq1buvtICEhflitFhdLdLLbA4bUfryS\nfhtLb/2+ZWk8b2zNIfvsRW5KnzK6RY0C+b4Hz9XQfxHn2X0GsAc4BER1+TwAuNitTV3n+1eY+wp8\ngJqaRhfLc7LbA6isNN6TidJvY+mr34uUnbe257JhRx7zE0MxmUyjXN3Ike+772164+rwzkJgm9b6\nOuBNIB84opRa2fn5Wpw/CF3tAW4GUEqlAcddPLYQYoBCArxZOC2M0qpLnCyscXc5Ygxw9Uw/F3ha\nKfUkzjP6rwL+wAtKKS/gFPAWgFLqFeApYAOwWim1F+eF3oeGWLsQYgBWpcaSebKCjw8WMzM+1N3l\nCDdzKfS11lVce6EWYEUP2z7Q5eVjrhxPCOG6hKhAEqMDOXammooLjYSH+rm7JOFG8nCWEAawOjUW\ngK0H5fZNo5PQF8IA5ifbCQnwZnd2GY1Nfd4/ITychL4QBmC1mLl+fjTNLe3sPlbq7nKEG0noC2EQ\nK+ZG42U1s/XQOTo6ZPZNo5LQF8Ig/H1tpM2MoKq2iUM5le4uR7iJhL4QBrJ2cRxmk4l3dxfI2b5B\nSegLYSDhoX4smRVBadUl9p+qcHc5wg0k9IUwmC8unYLF7Dzbb+/o6L+B8CgS+kIYjD3Yl2UpUVTU\nXGZvdrm7yxGjTEJfCAO6NX0yVouZ9/cU0tYuZ/tGIqEvhAGFBvqwcm4UVbVNZBwrc3c5YhRJ6Ath\nULekT8bLauaDvYW0tskiK0YhoS+EQQX5e3P9ghhq6pvZkSVP6RqFhL4QBrZ2cRzeXhb+8elZWVLR\nICT0hTCwAD8vVqfGUnephe2ygLohSOgLYXA3LorF19vKpswiLjfLDJyeTkJfCIOb4GPjxkWxNFxu\nZcuBYneXI0aYhL4QgtWpsQT62fhoXxEXG5rdXY4YQS4tl6iUsgEvA1OAduBR4N+AiM5NpgCZWut7\nurU7DNR1vizQWss6uUKMAb7eVm5blsArmzUbduXz0M3T3V2SGCGuLox+M2DVWi9RSq0GfqG1vgNA\nKRUCfAIaob5BAAARK0lEQVQ80bWBUsoHMGmtVw6hXiHECFmWEsnWQ+fYfayMVamxxIb5u7skMQJc\nHd7JAaxKKTMQCLR2+exnwHNa6+6P+aUAfkqpLUqp7UqpNBePLYQYARazmbuvn4oD+Pv2XBwOmXrZ\nE7l6pt+AcwjnNDAJuBVAKRUG3EC3s/xOjcCvgD8BScAmpZTSWvd6u0BIiB9Wq8XFEp3s9oAhtR+v\npN/GMlz9vt4ewI6sUo7kVFJUfZnU6eHDst+RIt/34Lka+k8Am7XWP1JKxQLblVKzgTuB17XWPT3l\nkQPkaa0dQI5SqhqIBHq9XaCmptHF8pzs9gAqK+uHtI/xSPptLMPd7/XXxZOVW8kLG48TE+qDxTw2\n7/eQ77vvbXrj6rdZA9R2/vkCYAMswCpgUy9tHgaeAVBKReEcFpKZnoQYY2LC/Fk2J5LSqktkHJX/\nRD2Nq6H/a2C+UioD2A78WGt9CVBAftcNlVKvKKXigD8DwUqp3cAbwMN9De0IIdxn/bIEvG0WNmbk\nywNbHsal4R2tdQNwVw/vz+zhvQe6vLzXleMJIUZXkL83a9Pi2JhRwIeZZ7ljRaK7SxLDZGwO1gkh\n3O7GRXGEBHiz5UAx1bVN7i5HDBMJfSFEj7xtFm5fnkBrWwfv7Drj7nLEMJHQF0L0Kn1WBLFh/mSe\nqOBcZYO7yxHDQEJfCNErs8nE7csTcAAbduX3u70Y+yT0hRB9mpM4kanRQRzJraKgrK7/BmJMk9AX\nQvTJZDJxx4oEAN7ZKWP7452EvhCiXyouhJlTQjhRWMOpszXuLkcMgYS+EGJAbu+8V/+dXWdkMrZx\nTEJfCDEg8ZGBzE+2c6akjmNnqt1djnCRhL4QYsDWL4vHBLyzK58OOdsflyT0hRADFm33J21mOMXn\nGzh4+ry7yxEukNAXQgzKuuvisZhNbMgooL2jw93liEGS0BdCDEpYiB/LUqKouNDI3uPl7i5HDJKE\nvhBi0L64ZAo2q5n39hTK2P44I6EvhBi0kABvFs8Ip7quidzii+4uRwyChL4QwiWLZzjXz913Si7o\njicS+kIIl0yPCyFwghcHT5+nrV0u6I4XEvpCCJeYzSYWTguj4XIrJwtlaobxwqXlEpVSNuBlYArQ\nDjwK+AIfALmdm/1ea/1GlzZm4HkgBWgGHtFa57lcuRDC7RbPCGfboXPsP1XBnMSJ7i5HDIBLoQ/c\nDFi11kuUUquBXwCbgGe11s/00uY2wEdrna6USgOeAda5eHwhxBiQGBXIxEAfDudU0tLajpfN4pY6\nHA4HheX17DlexrnKS9x9/VTiIwPdUstY52ro5wDWzrP3QKAVWAAopdQ6nGf739Za13dpcx3wEYDW\nOlMplep62UKIscBkMrFoRhibMos4dqaa1Glho3r82kstZJ4oZ/fxMkoqL119/z9fPcS9q5JZMTcK\nk8nUa/vm1nb2n6xAxQUTFuI3GiW7nauh34BzaOc0MAm4FVDAn7TWh5RSTwI/Bb7bpU0gUNvldbtS\nyqq1buvtICEhflitQztzsNsDhtR+vJJ+G4s7+712aQKbMovIyq9m7bLEUTlmTlENf3xxHwdPVdDe\n4cBqMbF0ThQ3LIwF4Nd/PcwrmzXnqhv5v+6Yg4/X56Ouo8PBjsPF/O+Hp6iqbSI+KpDfPLESs7n3\nH4ixZCjft6uh/wSwWWv9I6VULLAdWKa1vvJ43gbguW5t6oCulZr7CnyAmppGF8tzstsDqKys739D\nDyP9NhZ393uC1UTkRD/2n6ig6FwNvt6uxsrAnC2v5/95/TDNLe3Ehftz3exI0mZG4O9ru7rNTx5M\n5fcbs9l+sJicszU8vn4W4aHOM/lTZ2t4Y3suRRUN2Kxmou0TKCit44NdeaTPjBjR2ofDQL7vvn4U\nXL17p4bPztovADbgfaXUos73bgAOdWuzB+e1ADrH9I+7eGwhxBhiMplYPCOctvYOjuRWjuixqmub\n+M1bR2lpaef796fybw8tYlVq7OcCH2BSkC8/vG8BX5gXzbnKBv795QN8cqSE3755lP/3r0coqmgg\nfWY4//FoGv9yxxysFhMbduUb4tZTV3+Sfw28qJTKALyAH+Mc6nlOKdUKlANfA1BKvQI8hfPsf7VS\nai9gAh4aYu1CiDFi8fRwNmYUsO/keZbMihyRY1xqauXXbx6ltqGFe25IYtm86D7PeG1WM1+5UTE1\nOoiXPzrN/27WAKjYYO6+YSpTIj670LtyXjRbD55jx5ESVqXGjkj9Y4VLoa+1bgDu6uGjpT1s+0CX\nl4+5cjwhxNgWHurH5IgAThZeoL6xhQA/r2Hdf2tbB797+zilVZdYnRrLmoUDD+b0WRHEhvuzKbOI\n1Gl25k6ddM3F3VuXTGH3sTLe31vI0tmRIz5E5U7ycJYQYlgsnh5Oe4eDg3p4h3g6HA5e/PAUuvgi\nC5Sdu2+YOuh9xNj9efSLM5iXZO/xbp5APy9uWhRHfWMrm/cXDUfZY5aEvhBiWCyaHoYJ2HeyYlj3\n+/bOM+w7WcHU6CAevXUG5j5uwRyKNYtiCfSzsflAMXWXWkbkGGOBhL4QYliEBvqQFBtMbvFFLtQ1\nDcs+Pzl8jk2ZRYSH+PKtO+eM6MNfPl5Wvrg0nuaWdt7fWzgs+2xt62DHkRJ++dph3tpxhpr65mHZ\n71B47sCVEGLULZ4RTk7xRQ6cPs+Ni+KGtK/i8w28vjWXAD8bT9yVcs0dOiNhxdwothwoYseRElYv\njCUs2Nel/bS2tbPraBkfZp69GvS6+CKb9xeRPjOCGxfHET1pwnCWPmBypi+EGDapyo7ZZBryEE9H\nh4OXNp2ivcPBV2+ZPmpPy1otZtYvT6C9w8HGjPxBt29ubWfL/iK+//tPee3jHC41tXLTojj+67F0\nHrxJMSnYl93Hy/jJn/bxmzePcvpsDY5RXoRGzvSFEMMmwM+LGfEhZOdfYMuBYlanxvQ5DUJvPj5Y\nTEFZPWkzw5mTOGkEKu3dounhfLSviH0nKrhpURxx4f0//XplGOcfnxZS19iKt5eFm9Mmd14ncN7J\ntGJuNMtSojiaW8Wm/c5pK46dqWZtWhxfXjn4i9OuktAXQgyrO1ckUlzRwN+25XKusoGvrFHYrAMf\nVDhf08iGXfn4+9r4pxuSRrDSnplNJu5ckcizfz/KH987wc1pk0mdFoZ3D9cTOhwO9p2oYENGPlW1\nTfh4Wbh1yRTWLLz2gbEr+56XbGdesp28klp+vzGbTw6X8KWl8T3ufyRI6AshhlVceAA/eTCV5945\nzu5jZZRfaOTx9bMJmtD/vfsOh4OXP9K0tHXwzzdPG/b7/QdqZnwoy+ZEknGsjD//4xSvb80lfWY4\ny1OiiAsPwOFwcDz/Am/vPEPx+QasFhNrFsZyS/rkAdc8NTqIpbMj+GDvWY7kVpI2Y3SmgJDQF0IM\nu9BAH35433z+8uEp9p86z9MvH+Bbd8zpd6gk41gZp87WMCdxIounh49StdcymUw8dPN0blkyhYyj\npew+Xsb2wyVsP1zClIgAvGwWcoovYgLSZ0awflk8k1y46Js+0xn6n2ZXSOgLIcY3b5uFr39pJtF2\nfzbsyuc/Xj3EI7fM6HX65Zr6Zt7YnoePl4UHblQuXQsYbmHBvtyxIpHblsVz7Ew1GUfLOHqmCocD\n5iRO5I4VicSG+bu8/8iJE4iPDOBEwQVqL7UM6G9DQyWhL4QYMSaTiS8umULMpAn8z/sneX5jNkkx\nQSyZFcHCaWH4+Xw27v3axzlcbm7jK2uSCQ30cWPV17KYzcxLsjMvyU5NfTOXm9uIGqZbLpfMiqSg\nLId9JysGNb2Eq+SWTSHEiJuXbOfJryxg+uQQ8s7V8vJHmm8/t4fnN2aTlVvFvpMVHM6pJDkmiBXz\not1dbp9CAryHLfDB+SSzxWxib3bZsO2zL3KmL4QYFTFh/nzvn+ZRXdtE5sly9maXc/D0eQ6ePg84\n75F/cO20EZtmYawK8PNidsJEsvKqKKlsINru+nDRQEjoCyFG1cQgH25Jn8LNaZMpLK/n0+xysvKq\nuHFRHJET3fOUqrulz4ogK6+KvSfKR/yefQl9IYRbmEwm4iMDiY8M5N7Vye4ux63mTp2Ir7eVzBMV\n3LE8cUSXbZQxfSGEcDOb1cLCaWHU1DdzuqhmRI8loS+EEGPAklnO+/Q/zS7vZ8uhkdAXQogxYGpM\nEJOCfDiYU0lzS/uIHcelMX2llA14GZgCtAOPAj7Ac52vm4EHtNYV3dodBuo6XxZorWWdXCGEwDkv\nT/rMCN7fW8jh3ErSZ47ME7qununfDFi11kuAfwd+AfwW+KbWeiXwDvCDrg2UUj6ASWu9svMfCXwh\nhOhiNIZ4XL17JwewKqXMQCDQCtyjtb7ydIEV6L50Tgrgp5Ta0vn5j7XWmS4eXwghPE54qB8JUYGc\nKLzAxYZmgv29h/0YroZ+A86hndPAJODWK4GvlFoC/B9gebc2jcCvgD8BScAmpZTSWrf1dpCQED+s\n1qFNN2q39z8XtieSfhuL9NtzrFk8mT9sOE722Yus7+We/aH029XQfwLYrLX+kVIqFtiulJoNrAOe\nBG7RWld2a5MD5GmtHUCOUqoaiASKeztITU2ji+U52e0BVFbWD2kf45H021ik355lemwQFrOJTw4W\nc93Ma2caHUi/+/pRcDX0a3AO6QBcAGzA3cAjwEqt9YUe2jwMzAa+oZSKwjksNDqTTQghxDgR4OfF\n7csTaGnrGJH9uxr6vwZeVEplAF44z+6fA4qAd5RSADu11j9VSr0CPAX8GXhJKbUbcAAP9zW0I4QQ\nRrU2bfKI7dul0NdaNwB3dXv7tV62faDLy3tdOZ4QQojhIQ9nCSGEgUjoCyGEgUjoCyGEgUjoCyGE\ngUjoCyGEgUjoCyGEgUjoCyGEgZgcDoe7axBCCDFK5ExfCCEMREJfCCEMREJfCCEMREJfCCEMREJf\nCCEMREJfCCEMREJfCCEMxNVFVMaszsXan8e5EHsz8IjWOs+9VY0spdRi4Jda65VKqanASzgXqskG\nHtdaj8wSPG6klLIBL+Jcq9kb+DlwEg/vu1LKArwAKJz9fAxowsP7fYVSKgw4BKwG2jBAv5VSh4G6\nzpcFwC8YQr898Uz/NsBHa50O/BB4xs31jCil1PdxLjbv0/nWs8BTWutlgAnnusWe6H6gurOfNwG/\nwxh9/yKA1nopzhXpfoEx+n3lh/6PwOXOtzy+30opH8CktV7Z+c9DDLHfnhj61wEfAWitM4FU95Yz\n4s4At3d5vQDY2fnnTcCqUa9odLwJ/KTzzyacZ30e33et9Ubga50vJwMXMUC/O/0K+ANQ2vnaCP1O\nAfyUUluUUtuVUmkMsd+eGPqBQG2X1+1KKY8bxrpCa/02ny1SD86zgitza9QDQaNf1cjTWjdoreuV\nUgHAWzjPeo3S9zal1Ms416V+DQP0Wyn1z0Cl1npzl7c9vt9AI84fuxtxDuUN+fv2xNCvAwK6vDYb\nbAH2rmN7ATjPBD2SUioW+AT4X6316xio71rrB4FknOP7vl0+8tR+PwysVkrtAOYCrwBhXT731H7n\nAK9qrR1a6xygGgjv8vmg++2Job8HuBmg869Cx91bzqg7opRa2fnntUCGG2sZMUqpcGAL8AOt9Yud\nb3t835VSX1FK/ajzZSPOH7qDnt5vrfVyrfUKrfVKIAt4ANjk6f3G+WP3DIBSKgrnSMaWofTbE4c9\nNuA8I9iLc6z3ITfXM9q+A7yglPICTuEc+vBEPwZCgJ8opa6M7f8L8N8e3vd3gL8opXYBNuDbOPtq\nhO+8OyP8u/5n4CWl1G6cd+s8DFQxhH7L1MpCCGEgnji8I4QQohcS+kIIYSAS+kIIYSAS+kIIYSAS\n+kIIYSAS+kIIYSAS+kIIYSD/P0CkUsAfb73vAAAAAElFTkSuQmCC\n",
    "text/plain": [
    "<matplotlib.figure.Figure at 0x10e89d780>"
    ]
    },
    "metadata": {},
    "output_type": "display_data"
    }
    ],
    "source": [
    "plt.plot(rn.cumsum() + 100)"
    ]
    },
    {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
    "## OLS Regression"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 7,
    "metadata": {
    "collapsed": true
    },
    "outputs": [],
    "source": [
    "x = np.linspace(-5, 5, 25)"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 8,
    "metadata": {},
    "outputs": [],
    "source": [
    "y = 1.5 * x ** 2 - 2 * x + 4 + np.random.standard_normal(len(x))"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 9,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "[<matplotlib.lines.Line2D at 0x10e96bef0>]"
    ]
    },
    "execution_count": 9,
    "metadata": {},
    "output_type": "execute_result"
    },
    {
    "data": {
    "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW8AAAD3CAYAAADSftWOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADqhJREFUeJzt3W+IZfddx/H3dG+2cZnJMg8mEbEaobvfxCItrKERTXZo\nk6b7oK6KKEjVNIoUIrS2Yk3YPBBaREgjSi2pqXH9g5R2baRP1g0EN020ada1FtPOfGejVihFGMOk\nznVbZdzxwT2D22Qy987de+45v3veLwice+/s3O+Pufncc37n92due3sbSVJZXtd0AZKk/TO8JalA\nhrckFcjwlqQCGd6SVKDeNN5kfX2zyCEti4uH2Ni43HQZU2Wbu8E2l2FpaWHutV7zzHsPvd6BpkuY\nOtvcDba5fIa3JBXI8JakAhneklQgw1uSCmR4S1KBDG9JKlD7w7vfp3fxAvT7TVciSa3R7vDu91m8\nZ5nFE29n8Z5lA1ySKq0O716u0Lu0Nji+tEYvVxquSJLaodXhvRW3snXk6OD4yFG24taGK5KkdpjK\n2iZjm59n49x5erkyCO75+aYrkqRWaHd4A8zPs3XstqarkKRWGSm8I+IfgP+sHv4r8BHgNLANvADc\nn5lX6ihQkvRqQ8M7Iq4H5jJz+arnPgecyszzEfEocBJ4orYqJUnfYZQz7zcDhyLiyernHwSOAU9X\nr58F3sEe4b24eKjY5RiXlhaaLmHqbHM32OayjRLel4GHgU8CRxiE9Vxm7mywsAkc3usXlLYA+o6l\npQXW1zebLmOqbHM32OYy7PVlM0p4rwEvVmG9FhEvMTjz3rEAvHxNFUqS9mWUcd73AR8FiIjvAW4A\nnoyI5er1E8AztVQnSdrVKGfefwScjohnGYwuuQ/4D+CxiDgIrABn6itRkvRKQ8M7M/8H+LldXjo+\n+XIkSaNo9fR4SdLuDG9JKpDhLUkFMrwlqUCGtyQVyPCWpAIZ3pJUIMNbkgpkeEtSgQxvSSqQ4S1J\nBTK8JalAhrckFcjwlqQCzV549/v0Ll6Afr/pSiSpNrMV3v0+i/css3ji7Szes2yAS5pZMxXevVyh\nd2ltcHxpjV6uNFyRJNVjpsJ7K25l68jRwfGRo2zFrQ1XJEn1GGUPy3LMz7Nx7jy9XBkE9/x80xVJ\nUi1mK7wB5ufZOnZb01VIUq1mqttEkrrC8JakAhneklQgw1uSCmR4S1KBDG9JKpDhLUkFMrwlqUCG\ntyQVyPCWpAIZ3pJUIMNbkgo00sJUEXEjcBG4G9gCTgPbwAvA/Zl5pa4CJUmvNvTMOyKuAz4BfKt6\n6hHgVGbeAcwBJ+srT5K0m1G6TR4GHgW+UT0+BjxdHZ8F7qqhLknSHvbsNomIe4H1zDwXEQ9UT89l\n5nZ1vAkcHvYmi4uH6PUOXFOhTVlaWmi6hKmzzd1gm8s2rM/7PmA7Iu4C3gL8KXDjVa8vAC8Pe5ON\njctjFzgV/f6uu+8sLS2wvr7ZYGHTZ5u7wTaXYa8vmz27TTLzzsw8npnLwD8CvwCcjYjl6kdOAM9M\npsyGuOO8pAKNM1Twg8BvRcQXgIPAmcmWNF3uOC+pRCPvYVmdfe84PvlSmrGz43zv0po7zksqxuxt\nQLxf7jgvqUCGN7jjvKTiOD1ekgpkeEtSgQxvSSqQ4S1JBTK8JalAhrckFcjwlqQCGd6SVCDDW5IK\nZHhLUoEMb0kqkOEtSQUyvCWpQIa3JBXI8JakAhneklQgw3tc/T69ixfcsFhSIwzvcbjjvKSGGd5j\ncMd5SU0zvMews+M84I7zkhrhBsTjcMd5SQ0zvMfljvOSGmS3iSQVyPCWpAIZ3pJUIMNbkgpkeEtS\ngQxvSSqQ4S1JdalxDSTDW5LqUPMaSIa3JNWg7jWQhs6wjIgDwGNAANvAe4FvA6erxy8A92fmlYlW\nJkkF21kDqXdprZY1kEY5834XQGb+KHAK+AjwCHAqM+8A5oCTE61KkkpXrYG0cfYpNs6dn/gaSEPD\nOzP/CviV6uH3Ay8Dx4Cnq+fOAndNtCpJmgU7ayDVsHjdSAtTZeZWRPwJ8JPATwN3Z+Z29fImcHiv\nf7+4eIhe78A1FdqUpaWFpkuYOtvcDba5bCOvKpiZvxgRHwK+CHzXVS8tMDgbf00bG5fHq65hS0sL\nrK9vNl3GVNnmbrDNZdjry2Zot0lE/HxEPFA9vAxcAf4+Ipar504Az1xjjZKkfRjlzPuzwB9HxOeB\n64D3AyvAYxFxsDo+U1+JkqRXGhremflfwM/s8tLxyZcjSRqFk3SmpcZpspK6x/CehpqnyUrqHsN7\nCuqeJitpClp29Wx4T8HONFmglmmykmrWwqtnd4+fhmqabC9XBsFdw2wrSfXZ7ep569htjdbkmfe0\n1DhNVlK92nj17Jm3JA3Twqtnw1uSRrFz9dwSdptIUoEMb0kqkOEtSQUyvCWpQIa3JBXI8JakAhne\nklQgw1uSCmR4S1KBDG9JKpDhLUkFMrwlqUCGtyQVyPCWpAIZ3pJUIMNbkgpkeLdZy3arltQehndb\ntXC3akntYXi31G67VUvSDsO7pdq4W7U0UwrvlnQD4rZq4W7V0syouiV7l9bYOnKUjXPni/t/zDPv\nNtvZrbqwD5XUdrPQLWl4S+qcWeiWtNtEUvfMQLek4S2pm3a6JQu1Z3hHxHXA48DNwOuBDwNfBU4D\n28ALwP2ZeaXWKiVJ32FYn/e7gZcy8w7gncDHgEeAU9Vzc8DJekvUvhQ+/EnSaIZ1m3wGOFMdzwFb\nwDHg6eq5s8A7gCf2+iWLi4fo9Q5cQ5nNWVpaaLqE0fX7cOfbYHUVbrkFLlwYqy+vqDZPiG3uhllq\n857hnZl9gIhYYBDip4CHM3O7+pFN4PCwN9nYuHyNZTZjaWmB9fXNpssYWe/iBRZXVwcPVlfZePb5\nfffpldbmSbDN3VBim/f6shk6VDAi3gD8DfBnmfkXwNX92wvAy9daoCZjFoY/SRrNnuEdETcBTwIf\nyszHq6e/FBHL1fEJ4Jn6ytO+VMOfNs4+VeSMMUmjG9bn/SCwCDwUEQ9Vz70P+P2IOAis8P994mqD\nwoc/SRrNsD7v9zEI61c6Xk85kqRROD1ekgpkeEtSgQxvSSqQ4S1JBTK8JZWvg8tCGN6SytbRzboN\nb0lFm4VdccZheEsqWleXhXAzBkllm4FdccZheEsqXweXhbDbRJIKZHhLUoEMb0kqkOEtSQUyvCW1\nSwdnS47D8JbUHh2dLTkOw1tSa3R1tuQ4DO+u8xJVLdLV2ZLjcJJOl1WXqL1La2wdOTrYtHhpoemq\n1GUdnS05Ds+8O8xLVLXSzmxJg3tPhneHeYkqlctuky7zElUqluHddR1c0EeaBXabSFKBDG9JKpDh\nLUkFMry1f07s0X74eamF4a39ce0J7Yefl9oY3toXJ/ZoP/y81Mfw1r44sUf74eelPo7z1v44sUf7\n4eelNoa39s+JPdoPPy+1GCm8I+KtwO9k5nJEvBE4DWwDLwD3Z+aV+kqUJL3S0D7viPgN4JPA9dVT\njwCnMvMOYA44WV95kqTdjHLD8p+Bn7rq8THg6er4LHDXpIvSDHKsrzRRQ7tNMvMvI+Lmq56ay8zt\n6ngTODzsdywuHqLXOzBehQ1b6uDmBBNvc78Pd74NVlfhllvgwoXW3bjy79wNs9TmcW5YXt2/vQC8\nPOwfbGxcHuNtmre0tMD6+mbTZUxVHW3uXbzA4urq4MHqKhvPPt+qG1j+nbuhxDbv9WUzzjjvL0XE\ncnV8AnhmjN+hDnGsrzR545x5fxB4LCIOAivAmcmWpJnjWN/Z0e/7d2yJkcI7M78G3F4drwHHa6xJ\ns8ixvuXbbcNqA7wxTo9XOzk6pXVcp6RdDG+1jyvRtZL3LtrF6fFqnd3O8OxyaQHvXbSKZ95qHc/w\nWmzn3oXB3TjPvNU+nuFJQ3nmrXbyDK9+3hQumuEtdZE3hYtneEtd9JWvOOyvcIa31EVvepM3hQvn\nDUupi7wpXDzDW+oqlywomt0m0qxw9EinGN7SLHD0SOcY3tIMcNGo7jG8pRngkgLd4w1LdVu/D//y\nVbjx+8oeceHokc7xzFvdVfUTc/vt++snnsaNwXHewyUFOsXw1uzYZ+CN1U88jRuD3nzUCAxvzYYx\nAm+cfuJp3Bj05qNGYXhrJowVeFU/Mc89N/J+jNO4MejNR43CG5aaCTuBt7M57siBNz8PP/BWWN8c\n+edrvzHozUeNwPDWbJhm4E1jWrlT1zWE3SaaHW0dbeG0ddXA8Jbq5MgR1cTwlmrkyBHVxfCWauTI\nEdXFG5ZSnRw5opoY3lLdHDmiGthtIkkFMrwlqUCGtyQVyPCWpAIZ3pJUIMNbkgo0t7293XQNkqR9\n8sxbkgpkeEtSgQxvSSqQ4S1JBTK8JalAhrckFcjwlqQCuSTsCCLiFuCLwE2Z+e2m66lTRBwG/hy4\nATgIfCAzv9BsVZMXEa8DPg68Gfhv4Jcz88Vmq6pXRFwHPA7cDLwe+HBmfq7RoqYkIm4ELgJ3Z+Zq\n0/VMgmfeQ0TEDcBHGfwP3gUfAJ7KzOPAvcAfNFtObX4CuD4zfwT4TQZ/41n3buClzLwDeCfwsYbr\nmYrqS+sTwLearmWSDO89RMQc8IfAg8DlhsuZlt9l8EGHwZXZrF5p/Bjw1wCZ+Rzww82WMxWfAR6q\njueArQZrmaaHgUeBbzRdyCTZbVKJiF8Cfu0VT/8b8KnM/HJENFBVvV6jze/JzAsR8d0Muk/eP/3K\npuIG4JtXPf7fiOhl5swGWmb2ASJiATgDnGq2ovpFxL3Aemaei4gHmq5nklzbZA8R8SLw9erh7cDz\nmXlngyVNRUT8EPAp4Ncz82zT9dQhIh4BnsvMT1ePv56Z39twWbWLiDcATwAfz8zHm66nbhHxeWC7\n+u8twBrw45n5740WNgGeee8hM9+4cxwRXwPe0VgxUxIRP8jg8vpnM/PLTddTo78F3gV8OiJuB/6p\n4XpqFxE3AU8Cv5qZTzVdzzRcfbIVEeeB985CcIPhrVf7beB64PeqrqJvZubJZkuqxRPA3RHxdwz6\nf9/TcD3T8CCwCDwUETt93ycyc6Zu5HWF3SaSVCBHm0hSgQxvSSqQ4S1JBTK8JalAhrckFcjwlqQC\nGd6SVKD/A8oGMtHIkwXFAAAAAElFTkSuQmCC\n",
    "text/plain": [
    "<matplotlib.figure.Figure at 0x10e84e1d0>"
    ]
    },
    "metadata": {},
    "output_type": "display_data"
    }
    ],
    "source": [
    "plt.plot(x, y, 'r.')"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 10,
    "metadata": {
    "collapsed": true
    },
    "outputs": [],
    "source": [
    "reg = np.polyfit(x, y, deg=2)"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 11,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "array([ 1.48880777, -2.01398057, 4.52192885])"
    ]
    },
    "execution_count": 11,
    "metadata": {},
    "output_type": "execute_result"
    }
    ],
    "source": [
    "reg"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 12,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "17.962554517904596"
    ]
    },
    "execution_count": 12,
    "metadata": {},
    "output_type": "execute_result"
    }
    ],
    "source": [
    "y.mean()"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 13,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "[<matplotlib.lines.Line2D at 0x10e8459e8>]"
    ]
    },
    "execution_count": 13,
    "metadata": {},
    "output_type": "execute_result"
    },
    {
    "data": {
    "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW8AAAD3CAYAAADSftWOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4VOX9/vH3ZJYsJEAqgVYERYVTqq0LaEMVQZQlWkVF\nUWQRFRcEBdlUZBUVZRVBcK/oD7Ug2KqVRbEoIiIg8kXEk1gpaK2YQoAMyWRmMvP7Y0INVZKQZObM\nmblf18V1ZWPyeXIyd555zrM4wuEwIiJiLylWFyAiIsdO4S0iYkMKbxERG1J4i4jYkMJbRMSGXLH4\nJoWFxbac0pKdnUFRUYnVZcSU2pwc1GZ7yMnJchztc+p5V8HlclpdQsypzclBbbY/hbeIiA0pvEVE\nbEjhLSJiQwpvEREbUniLiNiQwltExIYU3iIiNhTX4e33wyOPeMjPj+syRURiLq5Tcd8+B7NmpXLP\nYD94vVaXIyISN+I6vH+ZWUz3jA9Yty2b/E53KcBFRCrEdXi7zB3cUzIRgNnfXIvL3GFxRSIi8SGu\nwztotOX8U7/jHD5hGVdhuk+3uiQRkbgQ1+FNZib7V63hjrHphElh/sLGVlckIhIXarQlrGEYnwIH\nK97dCTwEvACEgc+BIaZphqJRIJmZ9Lgzk1avhFi82M2YMX6aNbPlDrMiIvWm2p63YRhpgMM0zc4V\n/24EZgHjTNPsCDiAntEs0umEO+7wU1bm4Nln3dH8ViIitlCTYZMzgAzDMFYZhvGeYRi5QDvg/YrP\nLwcujlaBh/XuHaBJkxB/+pNHk05EJOnVZNikBJgBPAu0JhLWDtM0D49dFAONqnqA7OyMetkI/e67\n4f77YdmyLEaOrPPD1UhOTlZsvlEcUZuTg9psbzUJ73zgq4qwzjcMYy+RnvdhWcD+qh6gvo4euuYa\neOihTGbODHPddYfweOrlYY8qJyeLwsLi6H6TOKM2Jwe12R6q+mNTk2GTm4CZAIZhHA80BFYZhtG5\n4vN5wNq6lVgzjRtD//4B/v3vFJYti8nxmyIicakm4f0c0NgwjA+BPxMJ82HAZMMw1gMe4LXolXik\n22/343KFeeIJD6HozG8REYl71XZfTdP0A9f/zKc61X851WvePMxVVwVZvNjNu+866dat3IoyREQs\nFd+LdI5iyBA/AHPnRnnQW0QkTtkyvNu2DdG1a5ANG1x88oktmyAiUie2Tb6hQyO97yeeUO9bRJKP\nbcM7N7ecdu3KWbHCRUGBbZshIlIrtk09hyPS+w6HHcyfryXzIpJcbBveAD16BDnllBBLlrj5/nuH\n1eWIiMSMrcPb6YzMPPH7HTz9tHrfIpI8bB3eAFdfHaBp0xALF3o4eLD6rxcRSQS2D++0NLj11gDF\nxQ4WLtTMExFJDrYPb4AbbvCTmRnm6afdlJVZXY2ISPQlRHg3agQDBgTYsyeF117T2LeIJL6ECG+A\n227z43aHeeIJtzasEpGElzDh/atfhbn66iBffeVk5UptFysiiS1hwhuO3LAqrDOKRSSBJVR4t2kT\nokePAJs2OVm/vu7HromIxKuECm+A4bcdAGDGowpvEUlciRXeXi8XjTmPPN7mw/WpfLTab3VFIiJR\nkVDh7TJ34CrIZxKTAJgxNaGaJyLyXwmVbkGjLcHWbTiXjeRlrOHD/8tm3ToNn4hI4kmo8CYzk6KV\nayhavpphi04HYMYMLZkXkcSTWOENkJlJsN05nH1eKhdfHGTdOpd63yKScBIvvCsZNSqy0cn06ep9\ni0hiSejwPvvsEBdfHOSjj9T7FpHEktDhDep9i0hiSvjwPvvsEF27qvctIokl4cMbfux9T5umPU9E\nJDEkRXifdVak971+vXrfIpIYkiK84cixb/W+RcTukia81fsWkUSSNOEN6n2LSOJIqvA+66wQ3bpF\net8ffqjet4jYV1KFN6j3LSKJoUaHPRqG0RTYDHQFgsALQBj4HBhimqZtjvw988xI73vVqkjvu2PH\ncqtLEhE5ZtX2vA3DcANPAaUVH5oFjDNNsyPgAHpGr7zoUO9bROyuJsMmM4Ange8q3m8HvF/x9nLg\n4ijUFVWHe98ff+xi7VqNfYuI/VQ5bGIYxkCg0DTNlYZh3FfxYYdpmof7q8VAo+q+SXZ2Bi5XfIXk\nww/DqlXw2GMZXHUVOBw//3U5OVmxLSwOqM3JQW22t+rGvG8CwoZhXAycCbwINK30+Sxgf3XfpKio\npNYFRkvLltC9ezorV7pYtmgvXZpsJWi0hczM/35NTk4WhYXFFlYZe2pzclCb7aGqPzZVDpuYpnmB\naZqdTNPsDHwGDACWG4bRueJL8oC19VNm7B0e+559y04a511EdvfO4PVaW5SISA3UZqrgSGCyYRjr\nAQ/wWv2WFDtnnBGix+8LWedrz3t0wVWQj8vcYXVZIiLVqtFUQYCK3vdhneq/FGuMHAsresIEHuCC\nU7+NDJ2IiMS5pFuk87/O6JBGj64+PuI8Xh2z/ogxbxGReJX04Q0wbmI5KSlhpkxvRDBodTUiItVT\neANt2oTo2zdAQYGTl192W12OiEi1FN4Vxozxk5ERZto0D4cOWV2NiEjVFN4VmjULc/vtfn74IYUn\nn9RhxSIS3xTelQwd6qdJkxDz5nkoLDzKkksRkTig8K4kMxNGjfJz6JCDmTPV+xaR+KXw/h/9+wc4\n+eQQL77oJj/f6mpERH6ewvt/uN1w//1lBIMOxo61uhoRkZ+n8P4Zf/xjkHbtylm6FDZt0o9IROKP\nkulnOBwwcWJk06rJk1N1YIOIxB2F91Hk5pbTsyds2OBi5cr42otcREThXYVHHgGnM8yUKalaNi8i\ncUXhXYVf/xquv17L5kUk/ii8q6Fl8yISjxTe1WjWLMzgwVo2LyLxReFdA0OGaNm8iMQXhXcNaNm8\niMQbhXcNVV42/49/OMDrxbV5ow4sFhFLKLxrqPKy+YcmO8nu3plsnTgvIhZReB+Dw8vm31qRzqaC\nbACdOC8illB4H4PKy+ZHpc0jDARbt9GJ8yIScwrvY5SbW06PHgHW+dqzaPxWilau0YnzIhJzCu9a\nGD/ej9MZZtyLp+NzKbhFJPYU3rXQunWIW24JsGtXCvPmaeqgiMSewruWRo8uo1mzEI8/7mHXLi3c\nEZHYUnjXUlYWTJpUhs/nYPz4VKvLEZEko/Cug6uuCvKHPwRZscLNO+9oz28ROdL+/bBvX3QeW+Fd\nBw4HTJ1ahtMZZuzYNHw+qysSkXhRUgLdujWgT5+MqDy+wruO2rb98eblE0/o5qWIRMye7eGf/0zh\n/POjc5KLwrsejB5dRtOmIebM0c1LEQHTjHTmWjQPMqbL+qhsoaHwrgdZWTB5sm5eigiEw3DPPakE\ngw7mhO7khCsvjMoeSArvelL55uW77+rmpUiyWrzYxUcfubikQyFX/vtJIDp7ILmq+wLDMJzAM4AB\nhIHbAR/wQsX7nwNDTNMM1WtlNnP45mWXLk7Gjk3j/PMPkZZmdVUiEktFRTB5cioZGWEenBYieFMb\nXAX5UdkDqSY978sATNM8DxgHPATMAsaZptkRcAA967Uqm2rbNsSgQQH++U/dvBRJRg8+mMp//pPC\nyJF+TjAyKFq5hqLlq6OyB5IjHA5X+0WGYbhM0wwahnED0AW4GDjBNM2wYRg9gW6maQ452v8PBsvD\nLldyDCUcPAiGEZnfuWMHnHSS1RWJSCx8/DF06ACnnQZbtkTOAKgHR50BUe2wCUBFcC8ErgSuBrqa\npnk49YuBRlX9/6KikhrWGV9ycrIoLCw+5v83YYKLO+5IZ/DgAC++aK/J37Vts52pzckhmm0OBmHQ\noAzAydSpJezfX14vj5uTk3XUz9X4hqVpmjcAbYiMf6dX+lQWsL+2xSWiXr2CdOigm5ciyeLZZ91s\n3+6kT58Aubn1E9zVqTa8DcPobxjGfRXvlgAhYJNhGJ0rPpYHrI1OefbkcMAjj2jlpUgy+O47B48+\nmkp2dpgJE8pi9n1r0vNeBpxlGMYHwEpgODAEmGwYxnrAA7wWvRLtqfLNy/nzdfNSJFGNH5/KoUMO\nJkwo47jjqr+HWF9qdMOyrgoLi2PXonpU1zGy4mLo0KEBBw86+PDDQ7RsGf8/Bo2FJge1uX6sXu2k\nT58MzjmnnDffLCGlnlfO5ORkHfWGpRbpRFHlbWPH3efCtXmjTpoXSRClpXDvvWk4nWGmTfPVe3BX\nR+EdZb16Belwbhkr3kljfd6MqCyTFZEY8HqP6IBF9jJK4bbbApx2WuzXKCq8o8zhgOkDP8NJkLt4\nnGDBrnpfJisiUeb1kt29M9l5F5HdvTMFW0uZO9dD8+YhRo2K3U3KyhTeMdCmR0uGNn6Jf3Aqk7If\nq/dlsiISXS5zB66CfACcBfncO8pNIODgwQfL6nvhZI0pvGMhM5Nha/No2ayU6QduY0tBQ6srEpFj\nEDTaEmzdBoCXmo1k7dZsunYNcskl0dmruyYU3jGS2awBs+eHCYUcDBuWRpk1r7REpDYyMylauYad\ni99nZPBR0tPDPPywD4eF2/crvGOoY8dyBg708+WXTmbO1NxvEVvJzOSBt3P5z14nI0b4OfFEa6f+\nKrxjbMKEMlq2DDF3rofPPtOPX8QuPv00hYUL3bRpU87gwX6ry1F4x1pmJsye7aO8XMMnInbh88Gw\nYWmEww4efbQMTxy8cFZ4W6Bjx3JuuMHPjh1OZs2Kg98CEanS1KmpmKaTm27yc955sdl4qjoKb4tM\nnFhGixYhHn/cw9atugwi8Wr9eidPPummVasQ48fHz0tlpYZFKg+f3HVXGn7rh9BE5H94vXDnnWk4\nHDBvXikNGlhd0Y8U3ha64IJyBgzQ8IlIvJo4MZXdu1MYOtTPOefE1zG9Cm+LTZxYxgknhJgzx8P/\n/Z8uh0i8WL3ayUsvefjNb8oZPTr+XhorLSyWlfXj8Mmdd2r4RCQeFBXB8OFpuN1h5s3zkZpqdUU/\npfCOA506ldO/v4ZPROLFffelsWdPCmPG+Dn99PgaLjlM4R0nJk3S8IlIPHjjDRfLlrlp166cIUPi\n96WwUiJOZGXBrFmafSJipT17HIwZk0p6epgnnijF5bK6oqNTeMeRzp0jwydffOFk9mwNn4jEUjgM\nI0emsW9fChMmlHHyyfF9bKHCO85MmlRG8+aR4ZNt23R5RGLllVdcrFrlomPHIDfeGLC6nGopHeLM\n4eGTYFDDJyKxsnu3g3Hj0sjKCvP447E/j7I2bFBi8rnwwnL69fOzfbuTxx7T8IlINIVCkU2nvF4H\nDz3ko3nz+B4uOUzhHacmTSrj+ONDzJ7tYcMGp9XliCSsZ55xs26di7y8ANdea93JOMdK4R2nGjaE\n+TOLCIfhtltT2bfP6opEEk9BQQoPPZRKkyYhZswos/RknGOl8I5XXi+XTvgDk0Pj+e7fTu66w03Y\nHq/mRGwhGIShQ9Pw+RxMn15GTo69nmAK7zh1+LTq+5jKxbzDqvfSWLDAbXVZIgljzhwPW7Y4ueaa\nAJdeap/hksMU3nHq8GnVTkK80GoiTXPKefDBVDZt0iUTqatPP01h5kwPxzfx8fD9e60up1aUBPGq\n4rTqouWr8axeypNPlREKwa23plNUZHVxIva1d6+Dm29MIxQM8dJ/LuGkazpFNu62GYV3PMvMJNju\nHMjM5Pzzyxk50s+336ZUnKVndXEi9lNeDoMHp/Gvfzt5gAl04e+4CvJxmTusLu2YKbxtZMQIPx07\nBlmxws3TT2v8W+RYzZzpYc0aF10v9DH61KUABFu3IWi0tbiyY+cIx6ALV1hYbMt+Yk5OFoWFxVaX\ncYQ9exx06ZLB/v0O3nyzhLPPrt/tKuOxzdGmNieHzZuzuOSSMC1ahHnnnUNku724zB2R4M7MtLq8\nn5WTk3XUyYvqedtMs2ZhFizwEQxGxr8PHLC6IpH49+23Dvr2BbcbnnuulOxsjhiWtKMqNzw0DMMN\nPA+cBKQCDwJfAC8AYeBzYIhpmvG5W3mCuuCCcu6+28+sWakMG5bGn/7ks9XiApFYKiuDQYPS2bcP\nZswo44wzEiOuqut59wP2mqbZEegBzANmAeMqPuYAeka3RPk5o0f7Oe+8IG+/7ea55zT+LXI0Eyak\n8umnTgYMgP7943+3wJqqLryXAOMr3nYAQaAd8H7Fx5YDF0enNKmK0wkLFvho0iTEpEmpfPZZxaX0\nenFt3mjLqU8i9W3pUhd/+pOHtm3LWbCAhHqFWqMbloZhZAFvAM8AM0zTPL7i412Am0zT7FfV/w8G\ny8MulzZXioZVq6BHD2jVCj79wEuji8+BL7+EX/8aNm607XieSF1t3w7nngsuF2zaBK1bW11RrRz1\nz021h/wYhtECeB2Yb5rmy4ZhTKv06Sxgf3WPUVRUUpMi444d7sifdRYMG+bhscdSGXBdCX/58svI\n1f7yS4o+/CRyQ+YY2KHN9U1tTjxeL1xxRQYlJU6ef76Uxo2DgP3anJOTddTPVTlsYhhGM2AVcI9p\nms9XfHiLYRidK97OA9bWQ41SB2PG+MnNDfLGh02ZlzMJsO/cVZG6Codh+PA0vvrKyeDBfv74R/vt\nW1IT1fW8xwLZwHjDMA6PfQ8DHjcMwwPsAF6LYn1SAy4XPPWUjy5dMhh1YAK/nZvHaZe20JCJJKVn\nnnHzxhtucnODjBtXZnU5UaNFOlWw20vL995zct11GTRvHmLFihKaNTv2H7vd2lwf1ObE8cknKVxx\nRQbZ2WHee+/I54Ad26xFOkmiS5dy7r+/jH/9K4UBA9IpseetBpFaKSx0MGhQOqEQPP20r1adFztR\neCeYu+7yc911AbZscTJ0aBqhxFiPIFKl8nK4/fY0vv8+hfvv93PeeeVWlxR1Cu8E43DAjBk+zjsv\nyFtvuXnoIR1gLInv0Uc9rF3rokePAEOH+q0uJyYU3gnI44Hnny/llFNCzJ2byqJFWoEpievVV108\n9lgqJ50UYu7c5NkqQuGdoLKzYdGiErKzw4wencratVokJYnnvfecjBiRRnbjEH++/xMaOZNnZbHC\nO4GdfHKYhQtLcTjgppvSKSjQ5ZbEsW1bCjffnI4zBV7P7Mu5t3Qgu3vnpNkaQs/mBJebW87s2T4O\nHHBw/fXp7N2bJK8pJaF9842DPn0iM6qeGv05nb59FcC2p+LUhsI7CfTuHWTEiDJ27UrhhhvSKEvc\ndQuSBIqKoE+fdH74IYUHHyzjkpubEGzdBkiulcUK7yRxzz1+rrwywCefuHQGptiWzwc33JBOfn5k\n6fsttwSOOKy7aOWapFlZrPBOEg4HzJnjo337cpYtczNjhqYQir2EQjB0aBoff+yiZ88AEydWeglp\n81NxakPhnUTS0uDFF0tp2TLE9OmpvPZatZtKisSNSZNSeeMNNx06BJk710dKkqdXkjc/+TRpEmbR\nolIaNgwzfHgaGzZoCqHEv6eecvPkkx7atCln4cJS0tKsrsh6Cu8kZBghnnuulPJyGDgwjZ07NQNF\n4tebb7qYMCGVZs1CvPJKKY0bW11RfFB4J6lOncqZNq2MvXtT6Ns3cjirSLz5+GMnd9yRRkYGvPxy\nKS1a6E77YQrvJNa/f4A77vDz1VdOrrkmg6IiqysS+VFBQWR3zPLyyHYPv/2tdlmrTOGd5CZMKKN/\nfz/btjnp3VsBLnHA62XvO1u5rncq+/c7mDXLx4UXJv4ugcdK4Z3kUlJg+vQy+vb1s3Wrk+7d4cAB\nq6uSpOX14up6KX37pvPNv1zcc3cx112XmMeY1ZXCO9l5vXi2bGTm5L306RNg40a49toMDh60ujBJ\nRqVb8rniH7P5lHYM4hnGdN1odUlxSxN9k5nXS3b3zrgK8gm2bsOst9fg8fyKhQudXHttBosXl5B1\n9MOrRerVgQNw3YMd2IyHq1nC3FNnU/zr1VaXFbfU805iLnMHroL8yNsF+aR+tYPnnoNrrgmweXMk\nwIvtdeSf2FRREVx9dQabt3jo1bOU+W81oXjV6qRaMXmsFN5JLGi0/cmGPk4nPP64j169AmzaFDnQ\nOEl22BSLFBY6uPLKDLZuddK3r595Twbh3PYK7mpo2CSZVWzo4zJ3RHZiq3iyOJ0wb56PcBiWLXPT\np086r7xSqueS1Ls9exz06hXZaOrGG/1MnVqW9Mvea0o/pmR3lA19Dgf4FVcE2LDBxfXXp6sHLvXq\nX/9ycPnlGeTnO7n9dj+PPKLgPhb6UclRuVwwf76Pyy8P8PHHLvr1S+fQIaurkkSwa5eDnj0z2Lkz\nheHDy5g8uSxpzp6sLwpvqZLLBQsW+LjssgAffRQJ8JISq6sSO/v660hw796dwr33ljF2rF/BXQsK\nb6mW2w1PPunj0ksDrFvnon9/BbjUjmmmcPnlGXz3XQoTJvgYMcJvdUm2pfCWGnG74amnfOTlBVi7\n1sWAngEOfqcxFKkBrxfX5o18vtHHFVdEji97+GEfQ4cGrK7M1hTeUmMeDzwz+z/0bPAOH2z9BZef\nW8yuHaVWlyXxrGIh2D/yxtDrMjf79jmYMcPHoEEK7rpSeMsxydi5g6WHejCc2Xzhb02PyxrrQAc5\nKpe5g40Fv+AiVnMglMW8u3cwYICCuz4ovOWYBI22hFufymxGMC9nIvsPuenVK50lS7RkQH7qb9+e\nSTfHuxyiAS/9ciTXDM22uqSEofCWY1PppO7eG27nlVciR1INGZLOI494CGnLZSFyWPCMGR763/IL\nylMzeGH8F3T96B6tmqxHCm85dpUW9nTuXM7bb5dw4okhZs1K5dZb0zQTJcl5vXDjjWlMm5bKCSeE\neOtvJXS/s5WCu54pvKXO2rQJsWJFCbm5Qd54w82VV2awZ48m7iajr792kJeXwfLlbs4/P8iqVSU6\nASdKahTehmH83jCMNRVvn2oYxoeGYaw1DGOBYRj6AyAcd1yYJUtKufbaAFu2OOnePYNt2/SrkUze\nfddJt24NME0nt93mZ/HiUpo00ZmT0VLts8swjDHAs0BaxYdmAeNM0+wIOICe0StP7CQ1NbIj4bhx\nZXz3XQqXXZbBihWaiZLowmGYM8dD377plJXBvHmlTJlShkv3sKOqJj/efwBXAS9VvN8OeL/i7eVA\nN+D1qh4gOzsDl8ueT+KcnOQ7jaCubZ4yBc46C/r1c3DDDRlMnw4jbvXi+GI7nHZaXI596jrXTmR8\nG157DU44AV5/Hdq3T6+H6qIjka5zteFtmuZSwzBOqvQhh2mah18LFQONqnuMoiJ73sHKycmisDC5\nTiOorzZ37AhvvJFCv37pjBqVwhcPvM2Cg/1Iad2KopVr4irAdZ1rZ+dOBwMHprNjh5Pc3CDPPuuj\nadMwhYX1VGQ9s+N1ruqPTW0GJSvffcgC9tfiMSQJnHFGiJUrS/jdKcU8f7A3nXifrwvCuMwdVpcm\ndfT3vzvp3r0BO3Y4uflmP0uXltK0qca3Y6k24b3FMIzOFW/nAWvrrxxJNMcfH+avfz3EdVlv8jEd\nONOxlQXr22k+uE2Fi73Mv28PffpENiebM6eUqVPLcLutriz51Ca8RwKTDcNYD3iA1+q3JEk0DZo2\n4PGt7Xl+7DbSG6dy/wON6NUrnd27NZ3QTv6zq4Q7ztjKpOdO5ZeOH/jrq3vp0ydodVlJyxEOR/+l\nTmFhsS1fT9lxjKyuot3mH35wMGpUKitWuGnQIMyUKWX07Rv46X7OXu9PjmeLFl3nqoXDsGSJiwlj\nnew76OE8PuQ1riZ1+cuRxVo2YcfrnJOTddQejibiSkw1bRpm4UIfc+eWkpICI0ak0bdvOt9/X+l3\ntGInuuy8i8ju3hmdv2adXbsc9O6dztCh6fiCLmY0eZj36UST1o0if1jFMgpviTmHA669NsgHHxyi\nU6cg777r4oILGrB0qYtwOLITnasgHwBXQb5ucFogGIQFC9x06tSA9993cdFFQdauLWHAJzdycPk7\ncTdjKBkpvMUyzZuHWby4lGnTfPj9MHhwOjffnMb3TU4j2LoNAMHWbdTDi7Ft21LIy8tg4sQ00tPD\nLFhQyssvl9KiRfioB1ZL7Cm8xVIOBwwcGODvfz9Ebm6Qt95yc0FeDotGrqdo+Wr18GKotBSmTPHQ\nrVsGW7c66d07wIcfltCrV1BnTMYhhbfEhVatwrz+eimTJ/soLnYw8PZfcNvzHfmhJHFWxMWdiuPJ\n8HpZu9ZJp04NmDs3lebNw/z5zyXMm+fjuONsOdcgKSi8JW44nTB4cIDVq0s466xylixxc+65DZgy\nxcO+fVZXl2AqbgqH865mzJnr6NUrg927HQwe7Of99w9x4YXlVlco1VB4S9xp0ybE3/5WwqOP+mjY\nMMzcuam0b5/Jo496OHjQ6uoSQ9mWL3ihoCO/4QteOHg1vz25mBUrSpg8uYwGDayuTmpC4S1xyeWC\nG28MsGHDIaZM8ZGWFmbmzEiIP/aYR7MHa8nrjcwiOfna9tzCsxygEQ8fN4MVy72ceaaWvdqJFulU\nwY6T+usqXtt86BA895yHJ57wUFTkoEmTEHfe6WfgwADpddzELl7bXJ+KiuDZZz08+2zk59egAQzo\n42Vo58/I+cMpSXFT2I7XuapFOgrvKtjxYtdVvLe5uBieesrDggUeiosdNGsWYvhwP/36BUhNrd1j\nxnub6+L77x0sWOBh4UI3JSUOsrPDDBrk5557UgmFErPNR2PH66wVlpIwsrJg1Cg/mzZ5GTasjOJi\nB/fdl0aHDg1YtMhNIGB1hRaqNHvk668djByZSvv2DViwwEOjRmEeeMDH5s1eRo/2c9xxVhcrdaWe\ndxXs+Je6ruzW5sJCB3PnenjhBTc+n4MWLUL06hXgyiuDtG1bszFcu7X5Z1XMHtlekMbUzIdYUnIp\noZCDVq0iw0vXXHPkK5OEaPMxsmObNWxSS3a82HVl1zZ//72Dxx7z8Oc/uzl0KPL73rZtOVddFeTK\nKwO0bHn0X0G7tvmw/fvh78/s5i/Td7GcSwA4/eRi7rrXxWWXBXH+zCFWdm9zbdixzQrvWrLjxa4r\nu7e5pATeecfFsmUuVq924fdHfvfPOaecq64KcPnlQXJyjvx1tGObv/nGwYoVLlascLF+vZNgMNLO\njnzAmONIQctuAAAFqUlEQVRf4ty1D+DIOvpNSDu2ua7s2GaFdy3Z8WLXVSK1+cAB+NvfXCxd6mbd\nOiehkAOnM0zHjpEgv/TSIFkOLzk/7Kawacu4nnERDsPnn6ewfHkksD///Mfu9FlnldOjR5AeFxzk\nt2yr0Ta6iXSda8qObVZ415IdL3ZdJWqb9+xx8Ne/unj9dTebN0eCL9UT5lLPSnp4l3J6iyKar5xL\nepMaBHgs9hr3eglv/5K1+3/H8jVZrFzp4ttvI/MLPJ4w558fCezu3YP86lfH/vRK1OtcFTu2WeFd\nS3a82HVl6zbXMFR37nTwl7+4WbYogLn7x69LSQlzyikhTj89xGmnhTjttHJOPz1E06bhHzdmqrgx\n6CrIJ9i6Tb1snBUOR/64fP11SuTfl0G+WbSe9w+1Zz/ZADRsGObii4Pk5QXp0iVIVh23fLH1da4l\nO7ZZ4V1LdrzYdWXbNtciVMPFXr678BY2727Gp40682mb3mz/MjJ/vLImTUL85jeRUP9dgwJ+P70v\nDTmImwCHFr2Co90ZuN3899/P3SAMhyMzY77+OoWdOysF9dcp7NyZQknJT5+jLdlFT/5Kl4c7cu4N\np9brOZG2vc51YMc2K7xryY4Xu67s2mbX5o1k51303/eLlq+u2RFd3iPHvMNh2L3bwfbtTj7/PIXt\n21PYvt3J7t01XxLhcIRxuyNL/COBHsbnc+D1/vR5mJERplWrECef/OO/Vr88xNn3Xs7xO9dTXk+9\n+/9l1+tcF3Zsc1Xh7YplISLREjTaEmzd5r897xof4JCZCa1+DxVPaocDTjwxzIknBrnkkh+/7OBB\n+OKLSKDnby+n7IeD+DMaE8SF3w/BoINAIHICTSAAgcCR77vdlUM6/N+gbtYs/DN7ZafC6qXsj9EZ\nnmJPCm9JDJmZFK1cE7UbiQ0bQm5uObm5h7dKzQSCFf+i4PCJNSJHoeXxkjji9YiuSsvWReqLwlsk\nmipupGbnXUR2984KcKk3Cm+RKHKZO3AV5EfeLsjHZe6wuCJJFApvkSg6fCMVOLYbqSLV0A1LkWiK\n8o1USV4Kb5Fo08wRiQINm4iI2JDCW0TEhhTeIiI2pPAWEbEhhbeIiA0pvEVEbCgmW8KKiEj9Us9b\nRMSGFN4iIjak8BYRsSGFt4iIDSm8RURsSOEtImJDCm8RERvSlrA1YBjGr4ENQDPTNH1W1xNNhmE0\nAv4f0BDwACNM01xvbVX1zzCMFGA+cAZQBgwyTfMra6uKLsMw3MDzwElAKvCgaZpvWFpUjBiG0RTY\nDHQ1TfNLq+upD+p5V8MwjIbATCJP8GQwAlhtmmYnYCDwhLXlRM0VQJppmh2Ae4lc40TXD9hrmmZH\noAcwz+J6YqLij9ZTQKnVtdQnhXcVDMNwAE8DY4ESi8uJldlEftEh8sosUV9pnA+sADBN82OgvbXl\nxMQSYHzF2w4gaGEtsTQDeBL4zupC6pOGTSoYhnEzcPf/fHgX8KppmlsNw7Cgqug6SptvNE1zo2EY\nvyQyfDI89pXFREPgQKX3yw3DcJmmmbCBZpqmF8AwjCzgNWCctRVFn2EYA4FC0zRXGoZxn9X11Cft\nbVIFwzC+Ar6teDcX+MQ0zQssLCkmDMP4LfAqMMo0zeVW1xMNhmHMAj42TXNxxfvfmqZ5gsVlRZ1h\nGC2A14H5pmk+b3U90WYYxgdAuOLfmUA+cLlpmt9bWlg9UM+7CqZpnnr4bcMw/gl0s6yYGDEM4zdE\nXl5fa5rmVqvriaJ1wGXAYsMwcoFtFtcTdYZhNANWAUNN01xtdT2xULmzZRjGGuD2RAhuUHjLT00F\n0oA5FUNFB0zT7GltSVHxOtDVMIyPiIz/3mhxPbEwFsgGxhuGcXjsO880zYS6kZcsNGwiImJDmm0i\nImJDCm8RERtSeIuI2JDCW0TEhhTeIiI2pPAWEbEhhbeIiA39f4kmj2ec/YErAAAAAElFTkSuQmCC\n",
    "text/plain": [
    "<matplotlib.figure.Figure at 0x10e809390>"
    ]
    },
    "metadata": {},
    "output_type": "display_data"
    }
    ],
    "source": [
    "yr = np.polyval(reg, x)\n",
    "plt.plot(x, y, 'r.')\n",
    "plt.plot(x, yr, 'b')"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": null,
    "metadata": {
    "collapsed": true
    },
    "outputs": [],
    "source": []
    },
    {
    "cell_type": "code",
    "execution_count": null,
    "metadata": {
    "collapsed": true
    },
    "outputs": [],
    "source": []
    },
    {
    "cell_type": "code",
    "execution_count": null,
    "metadata": {
    "collapsed": true
    },
    "outputs": [],
    "source": []
    },
    {
    "cell_type": "code",
    "execution_count": null,
    "metadata": {
    "collapsed": true
    },
    "outputs": [],
    "source": []
    },
    {
    "cell_type": "code",
    "execution_count": null,
    "metadata": {
    "collapsed": true
    },
    "outputs": [],
    "source": []
    },
    {
    "cell_type": "code",
    "execution_count": null,
    "metadata": {
    "collapsed": true
    },
    "outputs": [],
    "source": []
    },
    {
    "cell_type": "code",
    "execution_count": null,
    "metadata": {
    "collapsed": true
    },
    "outputs": [],
    "source": []
    },
    {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
    "<img src=\"http://hilpisch.com/tpq_logo.png\" width=350px align=right>"
    ]
    }
    ],
    "metadata": {
    "kernelspec": {
    "display_name": "Python 3",
    "language": "python",
    "name": "python3"
    },
    "language_info": {
    "codemirror_mode": {
    "name": "ipython",
    "version": 3
    },
    "file_extension": ".py",
    "mimetype": "text/x-python",
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
    "version": "3.6.1"
    }
    },
    "nbformat": 4,
    "nbformat_minor": 2
    }
  7. @yhilpisch yhilpisch revised this gist May 23, 2017. 1 changed file with 2 additions and 0 deletions.
    2 changes: 2 additions & 0 deletions 00_cqf_ml_elective.md
    View file
    Original file line number Diff line number Diff line change
    @@ -44,6 +44,8 @@ Download and install **Miniconda 3.6** from https://conda.io/miniconda.html

    If you have either Miniconda or Anaconda installed there is not need to install anything new.

    conda create -n elective python=3.6
    (source) activate elective
    conda install numpy pandas=0.19 scikit-learn matplotlib
    conda install pandas-datareader pytables
    conda install ipython jupyter
  8. @yhilpisch yhilpisch revised this gist May 23, 2017. 1 changed file with 4 additions and 0 deletions.
    4 changes: 4 additions & 0 deletions 00_cqf_ml_elective.md
    View file
    Original file line number Diff line number Diff line change
    @@ -44,6 +44,10 @@ Download and install **Miniconda 3.6** from https://conda.io/miniconda.html

    If you have either Miniconda or Anaconda installed there is not need to install anything new.

    conda install numpy pandas=0.19 scikit-learn matplotlib
    conda install pandas-datareader pytables
    conda install ipython jupyter


    Finance
    -------
  9. @yhilpisch yhilpisch revised this gist May 23, 2017. 1 changed file with 8 additions and 0 deletions.
    8 changes: 8 additions & 0 deletions 00_cqf_ml_elective.md
    View file
    Original file line number Diff line number Diff line change
    @@ -10,6 +10,14 @@ General resources:
    * http://tpq.io
    * http://hilpisch.com

    Wifi
    ----

    You can use the following wifi in this venue:

    SSID Fitch Guest
    PW Ft1ch#2016#!


    Abstract
    --------
  10. @yhilpisch yhilpisch revised this gist May 22, 2017. No changes.
  11. @yhilpisch yhilpisch created this gist May 22, 2017.
    94 changes: 94 additions & 0 deletions 00_cqf_ml_elective.md
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,94 @@
    Machine Learning for Finance
    ============================

    <img src="http://hilpisch.com/images/tpq_globe.png" width=300px>

    A CQF elective with Dr. Yves J. Hilpisch, The Python Quants GmbH

    General resources:

    * http://tpq.io
    * http://hilpisch.com


    Abstract
    --------
    This CQF elective is about machine learning and deep learning with Python applied to finance. It starts with techniques to retrieve financial data from open data sources and covers Python packages like NumPy, pandas, scikit-learn and TensorFlow. It provides the basis to further explore these recent developments in data science to improve traditional financial tasks such as the pricing of American options or the prediction of future stock market movements.

    Among other, the elective covers:

    * Getting and working with financial time series data in Python
    * Using linear OLS regression to predict financial prices & returns
    * Using scikit-learn for machine learning with Python
    * Application to the pricing of American options by Monte Carlo simulation
    * Applying logistic regression to classification problems
    * Predicting stock market returns as a classification problem
    * Using TensorFlow for deep learning with Python
    * Using deep learning for predicting stock market returns

    Overview **slides** under http://hilpisch.com/cqf_ml_elective.pdf


    Python
    ------

    Download and install **Miniconda 3.6** from https://conda.io/miniconda.html

    If you have either Miniconda or Anaconda installed there is not need to install anything new.


    Finance
    -------

    Download the paper by Longstaff and Schwartz (2001) about the Least-Squares Monte Carlo algorithm to price American options from [Paper about LSM algorithm](https://people.math.ethz.ch/~hjfurrer/teaching/LongstaffSchwartzAmericanOptionsLeastSquareMonteCarlo.pdf)


    Basic Definitions
    -----------------

    > "You can think of **deep learning, machine learning and artificial intelligence** [AI] as a set of Russian dolls nested within each other, beginning with the smallest and working out. Deep learning is a subset of machine learning, which is a subset of AI. ... That is, all machine learning counts as AI, but not all AI counts as machine learning. For example, symbolic logic (rules engines, expert systems and knowledge graphs) as well as evolutionary algorithms and Baysian statistics could all be described as AI, and none of them are machine learning."
    > "**Neural networks** are a set of algorithms, modeled loosely after the human brain, that are designed to recognize patterns. They interpret sensory data through a kind of machine perception, labeling or clustering raw input. The patterns they recognize are numerical, contained in vectors, into which all real-world data, be it images, sound, text or time series, must be translated."
    > "**Deep learning** maps inputs to outputs. It finds correlations. It is known as a 'universal approximator', because it can learn to approximate the function f(x) = y between any input x and any output y, assuming they are related through correlation or causation at all. In the process of learning, a neural network finds the right f, or the correct manner of transforming x into y, whether that be f(x) = 3x + 12 or f(x) = 9x - 0.1."
    > "All **classification tasks** depend upon labeled datasets; that is, humans must transfer their knowledge to the dataset in order for a neural to learn the correlation between labels and data. This is known as supervised learning. ... Any labels that humans can generate, any outcomes you care about and which correlate to data, can be used to train a neural network."
    Source: [https://deeplearning4j.org](https://deeplearning4j.org/)


    Machine Learning
    ----------------

    Here some resources to get a first overview of basic **machine learning** concepts and **logistic regression** for classification:

    * [Visual Guide to Basics of Neural Networks](http://jalammar.github.io/visual-interactive-guide-basics-neural-networks/)
    * [Logistic Regression Basics & Algorithm](https://rasbt.github.io/mlxtend/user_guide/classifier/LogisticRegression/)
    * [Generalized Linear Models in `scikit-learn`](http://scikit-learn.org/stable/modules/linear_model.html)


    Neural Networks
    ---------------

    Here some resources to get a first overview of **neural networks**

    * [A Neural Network in 11 Lines of Python](https://iamtrask.github.io/2015/07/12/basic-python-network/)
    * [Introduction to Deep Neural Networks](https://deeplearning4j.org/neuralnet-overview)
    * [A Guide to Deep Learning](http://yerevann.com/a-guide-to-deep-learning/)
    * [Neural Networks and Deep Learning (free e-book)](http://neuralnetworksanddeeplearning.com/)

    TensorFlow
    ----------

    Some resources to get started with Google's `TensorFlow` package for deep learning:

    * [Hello, TensorFlow!](https://www.oreilly.com/learning/hello-tensorflow)
    * [TensorFlow Tutorials](https://www.tensorflow.org/tutorials/)


    Data
    ----

    Download a HDF5 database file, containing a single `pandas DataFrame` object with hitorical equites data, from http://hilpisch.com/equities.h5

    <img src="http://hilpisch.com/tpq_logo.png" width=250px>
    962 changes: 962 additions & 0 deletions 01_neural_network_python.ipynb
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,962 @@
    {
    "cells": [
    {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
    "<img src=\"http://hilpisch.com/tpq_logo.png\" width=350px align=\"right\">"
    ]
    },
    {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
    "# Neural Network in Python"
    ]
    },
    {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
    "&copy; Dr. Yves J. Hilpisch\n",
    "\n",
    "The Python Quants GmbH"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 1,
    "metadata": {
    "collapsed": true
    },
    "outputs": [],
    "source": [
    "import numpy as np\n",
    "from pylab import plt\n",
    "plt.style.use('seaborn')\n",
    "%matplotlib inline"
    ]
    },
    {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
    "## Neural Network for Regression"
    ]
    },
    {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
    "[Regression & Neural Networks](https://www.cs.cmu.edu/afs/cs.cmu.edu/academic/class/15381-s06/www/nn.pdf)"
    ]
    },
    {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
    "### Linear OLS Regression"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 2,
    "metadata": {
    "collapsed": true
    },
    "outputs": [],
    "source": [
    "x = np.linspace(0, 10, 5)\n",
    "y = 3 * x + 2.5 + np.random.standard_normal(len(x)) * 1.5"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 3,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "[<matplotlib.lines.Line2D at 0x118cf36d8>]"
    ]
    },
    "execution_count": 3,
    "metadata": {},
    "output_type": "execute_result"
    },
    {
    "data": {
    "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW8AAAD3CAYAAADSftWOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADwRJREFUeJzt3V2MXPV5gPFn8RgonYVu1YE0FQmRYr+4UtVGLtjlI6zk\nEMetUpo0zUUFbYNaRMUFtEFQwFalCC6QgiOFCFoZHAgqEuLDUYJKzUXDR1qFUIeIoi7vbqKmQW1R\nVnRpdjBpGHt6MYMwjndnPDuzs3+f53ezZ87MnH3/GD8+Pjsznmi320iSynLSuAeQJB0/4y1JBTLe\nklQg4y1JBTLeklSg2mp8k/n5xRW9pGVq6jQWFg4Oa5wiVG3NVVsvuOaqWMmaG43JiaXuK+LMu1Zb\nN+4RVl3V1ly19YJrropRrbmIeEuS3s14S1KBjLckFch4S1KBjLckFch4S1KBjLckjUqzCc891/k6\nZMZbkkah2WRq+zRs3dr5OuSAG29JGoFazlCbm+1sz81Sy5mhHt94S9IItGITrQ0bO9sbNtKKTUM9\nfs/PNomIdcAeIIA2cDWwHngcmOs+7O7MfGiok0lSyep1FvY/ReNHP2ThzPdBvT7Uw/fzwVQfB8jM\nCyNiGrgN+DqwOzPvGOo0knQiqdfhA1tgfnHoh+552SQzvwpc1b35fuB1YDPwOxHxTETcGxGTQ59M\nkrSkiX7/AeKIuB/4BPAp4FeAFzPzQETcAkxl5vVLPbfVOtSu4qeJSdIKLfmRsH3HGyAi3gM8B1yQ\nmf/Z3ferwJ2ZuW2p563087wbjUnmR/DXjrWsamuu2nrBNVfFSta8os/zjogrIuKm7s2DwGHgsYg4\nv7tvG3BgoMkkSQPp5weWjwFfjohn6LzK5DrgFeDOiHgLeJV3rolLklZBz3hn5hvAp49x14XDH0eS\n1A/fpCNJBTLeklQg4y1JBTLeklQg4y1JBTLeklQg4y1JBTLeklQg4y1JBTLeklQg4y1JBTLeklQg\n4y1JBTLeklQg4y1JBTLeklQg4y1JBTLeklQg4y1JBTLeklQg4y1JBTLeklQg4y1JBar1ekBErAP2\nAAG0gauBnwD3dW+/BFyTmYdHN6Yk6Uj9nHl/HCAzLwR2ArcBu4GdmXkxMAFcNrIJJUk/o2e8M/Or\nwFXdm+8HXgc2A0939z0BfGQk00mSjqnnZROAzGxFxP3AJ4BPAZdmZrt79yJwxnLPn5o6jVpt3YoG\nbTQmV/T8ElVtzVVbL7jmqhjFmvuKN0Bm/nFE3Ag8B/zcEXdN0jkbX9LCwsHBputqNCaZn19c0TFK\nU7U1V2294JqrYiVrXi76PS+bRMQVEXFT9+ZB4DDwLxEx3d23A3h2oMkkSQPp58z7MeDLEfEMsB64\nDpgB9kTEyd3tR0Y3oiTpaD3jnZlvAJ8+xl2XDH8cSVI/fJOOJBXIeEtSgYy3JBXIeEtSgYy3JBXI\neEtSgYy3JBXIeEtSgYy3pP41m9QOPA/N5rgnqTzjLak/zSZT26eZ2rGNqe3TBnzMjLekvtRyhtrc\nbGd7bpZazox5omoz3pL60opNtDZs7Gxv2EgrNo15omrr+/O8JVVcvc7C/qeo5Uwn3PX6uCeqNOMt\nqX/1Oq3N5417CuFlE0kqkvGWpAIZb0kqkPGWpAIZb0kqkPGWpAIZb0kqkPGWpAIZb0kq0LLvsIyI\n9cBe4BzgFOBW4BXgcWCu+7C7M/OhEc4oSTpKr7fHXw68lplXRMQvAt8FPgfszsw7Rj6dJOmYesX7\nYeCR7vYE0AI2AxERl9E5+74uMxdHN6Ik6WgT7Xa754MiYhL4GrCHzuWTFzPzQETcAkxl5vXLPb/V\nOtSu1dYNY15JqpKJpe7o+amCEXE2sA+4KzMfjIhfyMzXu3fvA+7sdYyFhYP9DnpMjcYk8/PVOrmv\n2pqrtl5wzVWxkjU3GpNL3rfsq00i4izgSeDGzNzb3b0/Is7vbm8DDgw0lSRpYL3OvG8GpoBdEbGr\nu+8vgS9ExFvAq8BVI5xPknQMy8Y7M68Frj3GXReOZhxJUj98k44kFch4S1KBjLckFch4S1KBjLck\nFch4S1KBjLckFch4S1KBjLckFch4S1KBjLckFch4S1KBjLckFch4S1KBjLckFch4S1KBjLckFch4\nS1KBjLckFch4S1KBjLckFch4S1KBjLckFai23J0RsR7YC5wDnALcCvwbcB/QBl4CrsnMwyOdUpL0\nLr3OvC8HXsvMi4GPAV8CdgM7u/smgMtGO6Ik6WjLnnkDDwOPdLcngBawGXi6u+8J4KPAvuUOMjV1\nGrXauhWMCY3G5IqeX6Kqrblq6wXXXBWjWPOy8c7MJkBETNKJ+E7g85nZ7j5kETij1zdZWDi4oiEb\njUnm5xdXdIzSVG3NVVsvuOaqWMmal4t+zx9YRsTZwDeABzLzQeDI69uTwOsDTSVJGtiy8Y6Is4An\ngRszc2939wsRMd3d3gE8O7rxJEnH0uua983AFLArInZ1910LfDEiTgZmeOeauCRplfS65n0tnVgf\n7ZLRjCNJ6odv0lG1NZvUDjwPzea4J5GOi/FWdTWbTG2fZmrHNqa2TxtwFcV4q7JqOUNtbrazPTdL\nLWfGPJHUP+OtymrFJlobNna2N2ykFZvGPJHUv16vNpFOXPU6C/ufopYznXDX6+OeSOqb8Va11eu0\nNp837imk4+ZlE0kqkPGWpAIZb0kqkPGWpAIZb0kqkPGWpAIZb0kqkPGWpAIZb0kqkPGWpAIZb0kq\nkPGWpAIZb0kqkPGWpAIZb0kqkPGWpAIZb0kqUF//kk5EbAFuz8zpiPgQ8Dgw17377sx8aFQDSpJ+\nVs94R8QNwBXAG91dm4HdmXnHKAeTJC1tot1uL/uAiPh94EXggczcGhF3A0En/HPAdZm5uNwxWq1D\n7Vpt3ZBGlqTKmFjqjp5n3pn5aEScc8SubwP3ZOaBiLgF+Gvg+uWOsbBwsM85j63RmGR+ftk/H044\nVVtz1dYLrrkqVrLmRmNyyfsG+YHlvsw88PY28KFBhpIkDW6QeO+PiPO729uAA8s9WJI0fH292uQo\nfw7cGRFvAa8CVw13JElSL33FOzN/AGztbn8HuHCEM0mSevBNOpJUIOMtSQUy3pJUIOMtSQUy3pJU\nIOMtSQUy3pJUIOMtSQUy3pJUIOMtSQUy3pJUIOMtSQUy3pJUIOMtSQUy3pJUIOMtSQUy3pJUIOMt\nSQUy3pJUIOMtSQUy3pJUIOMtSQUy3ieaZpPageeh2Rz3JJJGqNbPgyJiC3B7Zk5HxAeB+4A28BJw\nTWYeHt2I6luzydT2aWpzs7Q2bGRh/1NQr497Kkkj0PPMOyJuAO4BTu3u2g3szMyLgQngstGNp+NR\nyxlqc7Od7blZajkz5okkjUo/Z97fBz4JPNC9vRl4urv9BPBRYN9yB5iaOo1abd2gMwLQaEyu6Pkl\nOu41X3Q+nHsuvPwynHsuUxedX9SZt7/G1eCah6NnvDPz0Yg454hdE5nZ7m4vAmf0OsbCwsHBputq\nNCaZn19c0TFKM/Ca//4fqeUMrdgEb7bhzTL+u/lrXA2u+fifu5S+rnkf5cjr25PA6wMcQ6NSr9Pa\nfN64p5A0YoO82uSFiJjubu8Anh3eOJKkfgxy5v1ZYE9EnAzMAI8MdyRJUi99xTszfwBs7W7PApeM\ncCZJUg++SUeSCmS8JalAxluSCmS8JalAxluSCmS8JalAxluSCmS8JalAxluSCmS8JalAxluSCmS8\nJalAxluSCmS8JalAxluSCmS8JalAxluSCmS8JalAxluSCmS8JalAxluSCmS8JalAxluSClQb9IkR\n8R3gx92b/56ZnxnOSJKkXgaKd0ScCkxk5vRwx5Ek9WOi3W4f95MiYgvwFeA/6PwBcHNmfmupx7da\nh9q12rqBh5SkippY8o4B4/1rwFbgHmAD8AQQmdk61uPn5xeP/5scodGYZH5+cSWHKE7V1ly19YJr\nroqVrLnRmFwy3oNe854FvpeZbWA2Il4Dfhl4ZcDjSZKOw6CvNrkSuAMgIt4LnA7897CGkiQtb9Az\n73uB+yLim0AbuHKpSyaSpOEbKN6Z+VPgD4c8iySpT2v/TTrNJjz3XOerJAlY6/FuNpnaPg1bt3a+\nGnBJAtZ4vGs5Q21utrM9N0stZ8Y8kSStDWs63q3YRGvDxs72ho20YtOYJ5KktWHgzzZZFfU6C/uf\novGjH7Jw5vugXh/3RJK0JqzteEMn2B/YAhV7V5YkLWdNXzaRJB2b8ZakAhlvSSqQ8ZakAhlvSSqQ\n8ZakAg30jzFIksbLM29JKpDxlqQCGW9JKpDxlqQCGW9JKpDxlqQCGW9JKtCa/UjYiDgJuAv4deD/\ngD/NzO+Nd6rRioj1wF7gHOAU4NbM/NpYh1olEXEmcAC4NDNfHvc8oxYRNwG/C5wM3JWZ9455pJHq\n/r99P53/tw8Bf3Yi/zpHxBbg9sycjogPAvcBbeAl4JrMPLzS77GWz7x/Dzg1M38L+CvgjjHPsxou\nB17LzIuBjwFfGvM8q6L7G/tvgTfHPctqiIhp4ALgQuAS4OyxDrQ6fhuoZeYFwOeA28Y8z8hExA3A\nPcCp3V27gZ3d39cTwGXD+D5rOd4XAf8AkJnfAn5zvOOsioeBXd3tCaA1xllW0+eBvwH+a9yDrJLt\nwL8C+4CvA4+Pd5xVMQvUun+jPh14a8zzjNL3gU8ecXsz8HR3+wngI8P4Jms53qcD/3vE7UMRsWYv\n8wxDZjYzczEiJoFHgJ3jnmnUIuJPgPnM3D/uWVbRL9E5GfkD4Grg7yJiYrwjjVyTziWTl4E9wBfH\nOs0IZeajvPsPp4nMfPtzSBaBM4bxfdZyvH8MTB5x+6TMPOHPRCPibOAbwAOZ+eC451kFVwKXRsRT\nwG8AX4mI94x3pJF7DdifmT/NzAR+AjTGPNOo/QWdNW+k83Os+yPi1B7POVEceX17Enh9GAddy/H+\nJzrXyYiIrXT+mnlCi4izgCeBGzNz77jnWQ2Z+eHMvCQzp4HvAn+Uma+OeaxR+ybwsYiYiIj3Aj9P\nJ+gnsgXe+Zv0/wDrgXXjG2dVvdD9OQfADuDZYRx0LV+G2EfnjOyf6Vz//cyY51kNNwNTwK6IePva\n947MrMQP8qoiMx+PiA8D36ZzAnVNZh4a81ij9gVgb0Q8S+cVNjdn5htjnmm1fBbYExEnAzN0Lomu\nmB8JK0kFWsuXTSRJSzDeklQg4y1JBTLeklQg4y1JBTLeklQg4y1JBfp/eQ22ll+VCW4AAAAASUVO\nRK5CYII=\n",
    "text/plain": [
    "<matplotlib.figure.Figure at 0x10f213ef0>"
    ]
    },
    "metadata": {},
    "output_type": "display_data"
    }
    ],
    "source": [
    "plt.plot(x, y, 'r.')"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 4,
    "metadata": {
    "collapsed": true
    },
    "outputs": [],
    "source": [
    "reg = np.polyfit(x, y, deg=1)"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 5,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "array([ 3.24746398, 2.9487911 ])"
    ]
    },
    "execution_count": 5,
    "metadata": {},
    "output_type": "execute_result"
    }
    ],
    "source": [
    "reg"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 6,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "[<matplotlib.lines.Line2D at 0x118d2e438>]"
    ]
    },
    "execution_count": 6,
    "metadata": {},
    "output_type": "execute_result"
    },
    {
    "data": {
    "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW8AAAD3CAYAAADSftWOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlgVOW9//H3mZ1ARJBoERfceNQq1xZFXFCkdftZ97Ut\ntmqpiliFEFbDJmBJIAmgVhFFqVrrlWKt9lq64AI/r0vpVVzwAUpxwS1gULLMljn3j4m3tAIJyUxm\n+7z+4cyZzJnvk8CHJ+c83zmO67qIiEhu8WS6ABER2XMKbxGRHKTwFhHJQQpvEZEcpPAWEclBvs54\nk9ra7R1a0tKjRxF1dY2pKicnFNqYC228oDEXio6MuaSk2NnVczkx8/b5vJkuodMV2pgLbbygMReK\ndI05J8JbRET+lcJbRCQHKbxFRHKQwltEJAcpvEVEcpDCW0QkBym8RUTSpb4eXnkl+WeKKbxFRNKh\nvp4eZw+BQYOSf6Y4wBXeIiJp4LNr8a1fl9xevw6fXZvS4yu8RUTSYHPPb/Kj4t8wlkriR/Qjbo5K\n6fFb/WwTY4wXWAQYwAVuBPzAM8D6li+7x1r7eEorExHJQa4Ljz3mY+rUffli+yV8/6wt1N07DLp1\nS+n7tOWDqc4HsNaeYowZAswCngaqrbVVKa1GRCSH/eMfDmVlIVau9NG1q8vs2WHGju3F1q3bU/5e\nrYa3tfa3xphnWh4eDGwDBgDGGHMhydn3KGtt6qsTEckB8TgsXOinsjJIU5PDmWfGqawM06ePi8cT\nSst7Om29AbExZglwMXAZ0AdYY61dbYy5DehhrS3b1Wvj8Wa3ED9NTETy3xtvwPDh8Ne/QkkJLFgA\nV14Jzi4/zHWP7PIobQ5vAGPMN4BXgJOttZtb9h0N3Gmt/c6uXtfRz/MuKSmmtrawJvaFNuZCGy9o\nzLkuHIbq6gB33RUgHne4/PIYM2aE6dnzX7+uI2Pu0Od5G2OuNsZMbHnYCCSAZcaYgS37vgOsbldl\nIiI56L//28sZZ3Rl3rwgvXu7/PrXjdx999eDO53acsFyGfCgMeZFkqtMRgEfAHcaY2LAJ8D16StR\nRCQ7bN8Ot98eZMmSAI7jcv31USZMiKR6IUmbtOWCZQNwxU6eOiX15YiIZKfly72MGxfi4489HHlk\nM9XVYY4/PpGxejrlHpYiIrnqs88cbrstyFNP+fH7XcaNi3DLLVECgczWpfAWEdkJ14XHH/cxZUqI\nbdscjj++mZqaMMZkbra9I4W3iMi/ee+9ZLPNCy/4KCpyueOOMNdeG8ObRSueFd4iIi2am2HRIj+z\nZwdpbHT4znfizJkT5oADOrTaOS0U3iIiwDvveCgtDfG3v3np2TPB3LlhLr00nqpmm5RTeItIQYtE\noKYmwIIFyWabSy+NMWNGhF69sm+2vSOFt4gUrFde8VJaGmT9ei99+iSYM6eJ7363OdNltYnCW0QK\nTn09zJwZ5MEH/QAMHx5l0qTMNNu0l8JbRArKn/7kZezYEB995KFfv2SzzcCB2bH8b08ovEWkIGzZ\n4lBeHmTZsmSzTVlZhFtvjRIMZrqy9lF4i0hec1144gkfU6YE+fxzDwMGJGfbRx2Ve7PtHSm8RSRv\nffCBw9ixIVasSDbbzJwZ5ic/ya5mm/ZSeItI3mluhsWL/cyalWy2GTIkzty5YQ46KLuX/+0JhbeI\n5JV33/UwenSI1au99OjhUlnZxOWXZ2+zTXspvEUkL0QiMH9+gPnzA8RiDhdfHGPmzAglJfkz296R\nwltEct5rryVb2631sv/+CSormzjrrNxotmkvhbeI5Kz6erjjjiAPPODHdR2uvTZKeXmE4uJMV5Z+\nCm8RyUkrVngpKwvx4YceDj+8merqCIMG5fdse0cKbxHJKVu3OkyeHGTpUj8+n0tpaYRRo6KEQpmu\nrHMpvEUkJ7guLFvmo7w8yNatHr71rWSzzTe/mdvNNu2l8BaRrPfhhw7jxoX48599dOniMn16mOuv\nz49mm/ZSeItI1kok4MEH/cycGaShweG005LNNn375ufyvz3RangbY7zAIsAALnAjEAYeann8FjDS\nWluYv7uISFqsW5dstnntNS977+2yYEETV16Zf8027eVpw9ecD2CtPQUoB2YB1UC5tXYw4AAXpq1C\nESko0ShUVQUYOrSI117zcsEFMVatauCqqxTcO2o1vK21vwWub3l4MLANGAC80LLvWeC7aalORArK\n6tUezjyziIqKID17uixZ0sT994fZd1+dJvl3bTrnba2NG2OWABcDlwFnWmu/+m5uB7rv7vU9ehTh\n83XsykJJSQGsuv83hTbmQhsvaMxfaWiA8nKYPz+5quSGG6CiwkP37l0yUGHqpePn3OYLltbaHxtj\nxgOvADt+R4tJzsZ3qa6usX3VtSgpKaa2dnuHjpFrCm3MhTZe0Ji/8txzyTvbvP++h8MOS1BdHeak\nk5qJRqG2NkOFplBHfs67C/1WT5sYY642xkxsedgIJIC/GmOGtOw7F1jZrspEpGB9/jncfHOIK68s\nYvNmh1tvjfDccw2cdFLhdEl2RFtm3suAB40xLwJ+YBSwFlhkjAm0bC9NX4kikk9cF556ysekSUG2\nbPHQv38zNTVhjj1WC9b2RKvhba1tAK7YyVOnp74cEclnH34Iw4d3YfnyZLPN1Klhbrghhk8dJ3tM\n3zIRSbtEApYs8TNzJmzf7mPw4GSzzSGHaBVJeym8RSStNmxwKC0N8fLLPvbeG+bNa+L739ea7Y5S\neItIWsRicPfdAaqqAkQiDt/7Xoz77vPj88UzXVpeUHiLSMq9/rqHUaNCvPOOl/32SzB7dpjzzotT\nUuLPi+V/2UDhLSIp09AAlZVBFi70k0g4XH11lClTInTfbRuftIfCW0RS4sUXvZSWJpttDjkkQVVV\nE6eeqjXb6aLwFpEO2bYNpk4N8dhjfrxel5/9LEJZWZQu+dHZnrUU3iLSLq4LzzzjY8KEILW1Ho49\nNtls07+/mm06g8JbRNquvh6fXcsH3b/J+Nt78oc/+AmFXMrLI4wYEcXvz3SBhUPhLSJtU19P97PO\n4KENpzHWU8WXCT+nnBKnqirMoYeq2aazKbxFpE3eW7GJsg338gJD6J7YRs0t7/L9SX3wtOWWLpJy\n+raLyG7FYrBgQYBTbxrECwzhIp5kzSHn88NR3RXcGaSZt4js0po1yWabt97yUlKSYPa0bVx8aHfi\n5j+hW7dMl1fQFN4i8jWNjTBnTpB77/XT3Ozwgx9EmTYtwt57+4hzQqbLExTeIvJvVq1KNtts2uTh\n4IOTzTannaZmm2yj8BYRAL74AqZPD/LIIwE8HpebbooyblyEoqJMVyY7o/AWkf9rtvnsMw9HH93M\nvHlhjjtOzTbZTOEtUsA+/dRhwoQgv/+9n2DQ5bbbItx0k5ptcoHCW6QAuS48+qifadOCfPmlw6BB\ncaqrwxx+uJptcoXCW6TAbNzoUFYWYtUqH8XFLnPmhLn66pjWbOcYhbdIgYjH4d57/VRWBgmHHc45\nJ0ZFRYTevTXbzkUKb5EC8OabHkaPDrFmjZdevRLceWeYCy7QfSRz2W7D2xjjBxYDfYEgMBP4AHgG\nWN/yZfdYax9PY40i0k5NTVBVFeDuuwM0NztcdVWMadPC9OyZ6cqko1qbeQ8DtlprrzbG9AReB24H\nqq21VWmvTkTa7aWXks02Gzd6OOigBHPnNjFkiJpt8kVr4f0EsLRl2wHiwADAGGMuJDn7HmWt3Z6+\nEkVkT3z5ZbLZ5uGHk802N9wQZcKECF27ZroySSXHdVu/WGGMKQZ+BywiefpkjbV2tTHmNqCHtbZs\nd6+Px5tdn8+binpFZDeeegpuugk++giOOQYeeAAGDsx0VdIBu7wq0eoFS2PMgcCTwC+stb8yxuxt\nrd3W8vSTwJ2tHaOurrGthe5USUkxtbWFNbkvtDEX2nghtWP+9FOHSZOCPP20n0DAZcKEKDffHCUQ\ngNralLxFSujnvOev3ZXdruw0xuwH/BEYb61d3LJ7uTHmq//LvwOsbldVItJhrguPPeZj8OCuPP20\nn4ED46xY0UhpaTK4JX+1NvOeBPQAJhtjJrfsKwVqjDEx4BPg+jTWJyK7sGmTw5gxIVau9NG1q8vs\n2WGuuUbNNoVit+Ftrb0VuHUnT52SnnJEpDXxONx3n5+KiiBNTQ5nnhmnsjJMnz5qtikkatIRySFv\nv51stnn9dS/77JOgpibMxRer2aYQKbxFckA4DNXVAe66K0A87nD55TFuvz3CPvtotl2oFN4iWe7l\nl72UlgbZsMHLAQckm22GDlWzTaFTeItkqe3b4fbbgyxZEsBxXH760ygTJ0Z0318BFN4iWWn5ci/j\nxoX4+GMPRx7ZTHV1mOOP151t5J8U3iJZpLbW4bbbgvz2t378fpexYyPceqvWbMvXKbxFsoDrwuOP\n+5g6NURdncOAAc3U1IQ58kjNtmXnFN4iGfbeew5jx4Z4/nkfRUUud9wR5tprY3j1cUCyGwpvkQxp\nbk7e2Wb27CCNjQ5Dh8aZMyfMgQdq+Z+0TuEtkgHvvONh3Dh49dUQPXsmmDs3zKWXqtlG2k7hLdKJ\nIhGoqQmwYEGAeBwuuSTGzJkRevXSbFv2jMJbpJO88kqy2Wb9ei99+iRYuNBh4MBwpsuSHKXPHxNJ\ns/p6mDAhyAUXdGHDBg8/+UmUlSsbOO+8TFcmuUwzb5E0+tOfvIwdG+Kjjzz065dsthk4UMv/pOMU\n3iJpsGWLQ3l5kGXL/Ph8LqWlEUaPjhIMZroyyRcKb5EUcl1YutTH5MlBPv/cw7e/nZxtH320ZtuS\nWgpvkRT54INks82KFclmmxkzwgwfrmYbSQ+Ft0gHNTfD4sV+Zs1KNtsMGRJn7twwBx2k5X+SPgpv\nkQ54993knW1Wr/bSo4dLRUUTV1yhZhtJP4W3SDtEIjB/foD58wPEYg4XXZRsttl3X822pXMovEX2\n0GuveSgtDWGtl969E1RWNnH22bqzjXQuhbdIG9XXwx13BHngAT+u63DNNVEmT45QXJzpyqQQ7Ta8\njTF+YDHQFwgCM4F3gIcAF3gLGGmt1TooyWsrVngpKwvx4YceDjssQU1NE4MGabYtmdNae/wwYKu1\ndjBwDnAXUA2Ut+xzgAvTW6JI5mzd6jBiRIirririk08cRo+O8NxzDQpuybjWTps8ASxt2XaAODAA\neKFl37PAWcCTaalOJENcF5Yt81FeHmTrVg/HHZdstjnmGP2SKdnBcd3Wr44bY4qB3wGLgLnW2v1b\n9g8FrrPWDtvd6+PxZtfnU6eC5Ib334cRI+C//gu6dIGZM+GWW8CnK0TS+Xa56LTVv47GmANJzqx/\nYa39lTGmcoeni4FtrR2jrq6xLUXuUklJMbW12zt0jFxTaGPOhvEmEvDgg35mzgzS0OAweHCcqqow\nffu61NWl/v2yYcydTWPe89fuym7PeRtj9gP+CIy31i5u2f0/xpghLdvnAivbVZVIFlm3zsP55xcx\ncWIInw/mz29i6dIm+vbVum3JTq3NvCcBPYDJxpjJLftuBRYYYwLAWv55Tlwk50SjsGBBgHnzAkSj\nDhdcEGPWrAj77afQluy22/C21t5KMqz/3enpKUek86xenWy2WbvWyze+kaCiIsy558YzXZZIm+gS\njBSchgaYPTvIffclm21+fO5mJv/cy177d810aSJtptugSUF57jkvp5/elYULAxxycDN/6TOMh549\ngIMvPz3ZQimSIxTeUhA+/xxuvjnElVcWsXmzwy23RFg5bxVDNz8KgG/9Onx2bYarFGk7nTaRvOa6\n8NRTPiZNCrJli4f+/ZupqQlz7LEJqDfEj+iHb/064kf0I26OynS5Im2m8Ja89dFHDuPHh1i+3Eco\n5DJlSpgbb4z9s9mmWzfqlj+Pz65NBne3bhmtV2RPKLwl7yQSsGSJnxkzgtTXO5x6avLONoceupPl\nf926ER9wQucXKdJBCm/JK+vXeygtDfLKKz722sulujrMD38Y051tJO8ovCUvxGJw110BqqqSzTbn\nnRdj9mw120j+UnhLznv9dQ+jRoV45x0v++6bYPbsMN/7npptJL8pvCVnNTRAZWWQhQv9JBIOw4ZF\nmTo1Qvfuma5MJP0U3pKTXnjBy5gxId5/30Pfvgmqq5s49VTdIEEKh8JbckpdHUybFuKxx/x4vS43\n3xyhrCxKUVGmKxPpXApvyQmuC08/7WPixCC1tR6OOaaZefPC9O+vO9tIYVJ4S9b7+GOH8eOD/OEP\nfkIhl/LyCCNGRPH7M12ZSOYovCVrJRLwyCN+pk8Psn27w8knx6mu3kWzjUiBUXhLVvr73x3GjAnx\n0ks+iotd5s4NM2xYDI8+Sk0EUHhLlonF4J57AsyZEyAScTjnnBgVFRF699ZsW2RHCm/JGmvWJJtt\n3nrLS0nJP5tt1Nou8nX6JVQyrrERxo2Ds88u4q23vPzgB1FWrWrg/PMV3CK7opm3ZNTKlclmm02b\n4OCDXaqqmjjtNDXbiLRG4S0Z8cUXMG1akEcfDeDxuIwZAzff3EBX3UZSpE0U3tLpnnnGx4QJQT77\nzMPRRyfvbHPWWV2prc10ZSK5Q+EtnebTTx0mTAjy+9/7CQZdJk2KMHKkmm1E2qNN4W2MORGosNYO\nMcZ8C3gGWN/y9D3W2sfTVaDkPteFRx/1M21akC+/dBg0KNlsc/jhWv4n0l6thrcxZhxwNdDQsmsA\nUG2trUpnYZIfNm50KCsLsWqVj27dXCorw/zoR2q2Eekox3V3P/sxxlwKrAEettYOMsbcAxiSwb8e\nGGWt3b67Y8Tjza7P501RyZIL4nGoqYEpUyAchvPPh1/8Ag44INOVieSUXS6WbXXmba39jTGm7w67\nXgXut9auNsbcBkwFynZ3jLq6xjbWuXMlJcXU1u72/4e8k8tjfvNND6NHh1izxkuvXgkWLIhw4YXJ\nNdu7uiiZy+NtL425MHRkzCUlxbt8rj0XLJ+01m77ahu4sz1FSf5paoKqqgB33x2gudnhyitjTJ8e\npmfPTFcmkn/ac+ZxuTFmYMv2d4DVKaxHctRLL3k544yuLFgQpE8fl8cfb+TOOxXcIunSnpn3COBO\nY0wM+AS4PrUlSS758kuYPj3Iww8HcByXG26IMn58hG7dMl2ZSH5rU3hbazcBg1q2/wacksaaJEc8\n+6yP8eODfPKJh6OOaqa6OsyAAbqzjUhnUJOO7LFPP3WYNCnI00/7CQRcxo+P8LOfRQkEMl2ZSOFQ\neEubuS78+tc+pk4NsW2bw8CBcaqrI/Trp9m2SGdTeEubbNqUbLZ58UUfXbu6/PznYa69Vs02Ipmi\n8Jbdisfhvvv8VFQEaWpy+O5341RWhjngALW2i2SSwlt26e23k802r7/uZZ99EtTUhLn4Yt0gQSQb\nKLzla8JhqK4OcNddAeJxh8suizFjRoR99tFsWyRbKLzlX7z8spfS0iAbNng54IAEc+c2MXSo7mwj\nkm0U3gLA9u0wY0aQhx5KNtv89KdRJk5Us41ItlJ4C8uXexk3LsTHH3swJtlsc8IJWv4nks0U3gWs\nttbhttuC/Pa3fvx+l7FjI9xyS5RgMNOViUhrFN4FyHXhP//Tx5QpIerqHAYMSN5H8sgjNdsWyRUK\n7wLz/vvJZpvnn/dRVOQya1aY666L4dW9MkRyisK7QDQ3w/33+/n5z4M0NjoMHRpnzpwwBx6o5X8i\nuUjhXQDWrvVQWhpi9WovPXsmmDMnzGWXqdlGJJcpvPNYJALz5gVYsCBALOZwySUxZs6M0KuXZtsi\nuU7hnadefTU52163zkufPgkqK5s480w124jkC4V3nqmvh1mzgixe7AfguuuilJer2UYk3yi888if\n/+xl7NgQmzd7OOKIZqqrI5x4ombbIvlI4Z0HtmxxKC8PsmyZH5/PpbQ0wujRarYRyWcK7xzmurB0\nqY/Jk4N8/rmHb3872dp+9NFqthHJdwrvHPXBBw7jxoX4y1+SzTYzZoQZPlzNNiKFQjexyjHNzbBo\nkZ/Bg7vyl7/4OP30OC+80MANN7QEd309vtWvJa9cikjeatPM2xhzIlBhrR1ijDkceAhwgbeAkdZa\n/Z7eCaz1MGpUstmmRw+Xioomrrhih2ab+np6nD0E3/p1xI/oR93y59EyE5H81OrM2xgzDrgfCLXs\nqgbKrbWDAQe4MH3lCUA0CnPmBBg6tIjVq71cdFGMlSsbuPLKf+2S9Nm1+NavS26vX4fPrs1QxSKS\nbm2Zef8duAR4uOXxAOCFlu1ngbOAJ3d3gB49ivD5OnYytqSkuEOvz0UlJcW8/DIMHw5vvw19+sA9\n98D55/sB/9dfcOpAOPJIePddOPJIepw6MKdm3oX6My40GnNqtBre1trfGGP67rDLsdZ+1V+9Heje\n2jHq6hrbV12LkpJiamu3d+gYuaZLl2JKS6Pcf78f13W45pookydHKC6G2trdvPC/VuCza4mbo6DJ\nhabc+L4V4s9YYy4MHRnz7kK/PatNdjy/XQxsa8cxZDdWrPAyfjy8916Aww5LUFPTxKBBbWy26daN\n+IAT0lugiGRce1ab/I8xZkjL9rnAytSVU9i2bnUYOTLEVVcVsXkzjB4d4bnnGtoe3CJSMNoz8x4D\nLDLGBIC1wNLUllR4XBeefNJHeXmQLVs8HHdcMw895GX//aOZLk1EslSbwttauwkY1LK9Djg9jTUV\nlM2bk802f/qTjy5dXKZNC3P99TF69y7e/bltESlo6rDMkEQCHnzQz8yZQRoaHAYPjjN3bphDDtFn\nbYtI6xTeGbB+vYfRo4O8+qqP7t1d5s9v4qqrdGcbEWk7hXcnikbhrrsCVFcHiEYdLrggxqxZEfbb\nT7NtEdkzCu9O8re/eRg9OsTatV6+8Y0EFRVhzj03numyRCRHKbzTrKEBZs8OsmiRn0TC4eqro0yd\nGmGvvTJdmYjkMoV3Gj3/vJeyshDvv+/h0EMTVFc3cfLJWrMtIh2n8E6DujqYMiXE44/78Xpdbrkl\nwpgxUbp0yXRlIpIvFN4p5Lrwu9/5mDgx2WzTv38zNTVhjj1Wn5grIqml8E6Rjz5yGD8+xPLlPkIh\nlylTwtx4YwyfvsMikgaKlg5KJOCXv/Rz++1B6usdTj012Wxz6KFa/ici6aPw7oANGxxKS0O8/LKP\nvfZyqa4O88MfxtRsIyJpp/Buh1gM7r47QFVVgEjE4bzzYsyerWYbEek8Cu899PrryWabt9/2su++\nCWbPDvO976nZRkQ6l8K7jRoboaIiyMKFyWabYcOiTJkSYe+9M12ZiBQihXcbvPiilzFjQrz3noe+\nfRNUVTUxeLCabUQkcxTeu7FtG0ydGuKxx5LNNjffHKGsLEpRUaYrE5FCp/DeCdeFZ57xMWFCkNpa\nD8cc08y8eWH691ezjYhkB4X3v/nkE4fx44M8+6yfYNClvDzCiBFR/P5MVyYi8k8K7xaJBDzyiJ/p\n04Ns3+5w0klxqqvDHHaYlv+JSPZReAMbNyabbV56yUdxscvcuWGGDYvh8WS6MhGRnSvo8I7F4J57\nAsydGyAcdjjnnBgVFRF699ZsW0SyW8GG95o1yWabN9/00qtXgrvvTjbbqLVdRHJBu8PbGPM34MuW\nh/+w1l6bmpLSq7ER5s4NcM89AZqbHb7//RjTpoXp0SPTlYmItF27wtsYEwIca+2Q1JaTXqtWeSkt\nDbFpk4eDDko225x+upptRCT3tHfm/R9AkTHmjy3HmGStfTl1ZaXWF1/A9OlBHnkkgMfjMmJElHHj\nInTtmunKRETax3HdPb84Z4w5FhgE3A8cATwLGGvtTj+hKR5vdn0+b0fqbLcnn4SRI+Hjj6F/f7j/\nfjjhhIyUIiKyp3Z5Fa69M+91wAZrrQusM8ZsBXoDH+zsi+vqGtv5NkklJcXU1m7fo9d8+qnDhAlB\nfv97P4GAy6RJUUaOTDbb1NZ2qJxO0Z4x57JCGy9ozIWiI2MuKSne5XPtDe/rgGOBm4wx+wN7AR+3\n81gp5brwq1/5mTYtyBdfOJx4Ypzq6ghHHKHWdhHJH+0N7weAh4wxqwAXuG5Xp0w608aNDmVlIVat\n8tGtm0tFRZgf/1jNNiKSf9oV3tbaKPCDFNfSbvE43Huvn8rKIOGww9lnx6moCLP//mq2EZH8lP1N\nOvX1sPEd2Pcg6Nbta0+/+aaH0tIQb7yRbLZZsCDMhReq2UZE8lt2n1Cor6fH2UNg0KDkn/X1//dU\nUxPMnBngrLOKeOMNL1dcEWPVqgYuukjBLSL5L6tn3j67Ft/6dcnt9evw2bXEB5zASy8lm202bkw2\n28yZ08QZZ6jZRkQKR1bPvOPmKOJH9EtuH9GPz3sfTVlZkIsuKuIf/3C44YYozz/foOAWkYKT1TNv\nunWjbvnzlHz2Pr964wjGn7M3n3zi4aijmqmuDjNggJb/iUhhyu7wBj5rLGZk1Yk88QQEAi7jx0f4\n2c+iBAKZrkxEJHOyOrzr6mDIkCK2bIETTkjOto3RbFtEJKvDOxCAk05q5uyzPVx2WaOabUREWmR1\neHftCg88EKakxJ8Tn0ciItJZNJcVEclBCm8RkRyk8BYRyUEKbxGRHKTwFhHJQQpvEZEcpPAWEclB\nCm8RkRzUrrvHi4hIZmnmLSKSgxTeIiI5SOEtIpKDFN4iIjlI4S0ikoMU3iIiOUjhLSKSg7L2ZgzG\nGA/wC+A/gAgw3Fq7IbNVpZcxxg8sBvoCQWCmtfZ3GS2qkxhj9gVWA2daa9/NdD3pZoyZCFwABIBf\nWGsfyHBJadXyd3sJyb/bzcBP8/nnbIw5Eaiw1g4xxhwOPAS4wFvASGtth+/nmM0z74uAkLX2JGAC\nUJXhejrDMGCrtXYwcA5wV4br6RQt/7AXAk2ZrqUzGGOGACcDpwCnAwdmtKDO8f8An7X2ZOB2YFaG\n60kbY8w44H4g1LKrGihv+XftABem4n2yObxPBf4AYK19GTg+s+V0iieAyS3bDhDPYC2daS5wL/BR\npgvpJGcDbwJPAk8Dz2S2nE6xDvC1/Ea9FxDLcD3p9Hfgkh0eDwBeaNl+FvhuKt4km8N7L+CLHR43\nG2Oy9jRWLp2MAAABjklEQVRPKlhr6621240xxcBSoDzTNaWbMeYaoNZauzzTtXSiXiQnI5cDNwKP\nGmOczJaUdvUkT5m8CywCFmS0mjSy1v6Gf/3PybHWfvU5JNuB7ql4n2wO7y+B4h0ee6y1eT8TNcYc\nCDwHPGyt/VWm6+kE1wFnGmOeB44DfmmM+UZmS0q7rcBya23UWmuBMFCS4ZrSbTTJMfcjeR1riTEm\n1Mpr8sWO57eLgW2pOGg2h/f/J3meDGPMIJK/ZuY1Y8x+wB+B8dbaxZmupzNYa0+z1p5urR0CvA78\nyFr7SYbLSrdVwDnGGMcYsz/QlWSg57M6/vmb9OeAH/BmrpxO9T8t1zkAzgVWpuKg2Xwa4kmSM7KX\nSJ7/vTbD9XSGSUAPYLIx5qtz3+daawviQl6hsNY+Y4w5DXiV5ARqpLW2OcNlpVsNsNgYs5LkCptJ\n1tqGDNfUWcYAi4wxAWAtyVOiHaaPhBURyUHZfNpERER2QeEtIpKDFN4iIjlI4S0ikoMU3iIiOUjh\nLSKSgxTeIiI56H8BP+v/SV4BDoIAAAAASUVORK5CYII=\n",
    "text/plain": [
    "<matplotlib.figure.Figure at 0x118d2e518>"
    ]
    },
    "metadata": {},
    "output_type": "display_data"
    }
    ],
    "source": [
    "yr = np.polyval(reg, x)\n",
    "plt.plot(x, y, 'r.')\n",
    "plt.plot(x, yr, 'b')"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 7,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "0.33264115729215088"
    ]
    },
    "execution_count": 7,
    "metadata": {},
    "output_type": "execute_result"
    }
    ],
    "source": [
    "((y - yr) ** 2).mean()"
    ]
    },
    {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
    "### Network Training &mdash; Single Step"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 8,
    "metadata": {
    "collapsed": true
    },
    "outputs": [],
    "source": [
    "l0 = np.array((x, len(x) * [1])).T"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 9,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "array([[ 0. , 1. ],\n",
    " [ 2.5, 1. ],\n",
    " [ 5. , 1. ],\n",
    " [ 7.5, 1. ],\n",
    " [ 10. , 1. ]])"
    ]
    },
    "execution_count": 9,
    "metadata": {},
    "output_type": "execute_result"
    }
    ],
    "source": [
    "l0"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 10,
    "metadata": {
    "collapsed": true
    },
    "outputs": [],
    "source": [
    "weights = np.array(((2., 2.)))"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 11,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "array([ 2., 7., 12., 17., 22.])"
    ]
    },
    "execution_count": 11,
    "metadata": {},
    "output_type": "execute_result"
    }
    ],
    "source": [
    "l1 = np.dot(l0, weights)\n",
    "l1"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 12,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "array([ 3.7550671 , 10.19476625, 18.8091978 , 27.45154775, 35.71997609])"
    ]
    },
    "execution_count": 12,
    "metadata": {},
    "output_type": "execute_result"
    }
    ],
    "source": [
    "y"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 13,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "array([ 1.7550671 , 3.19476625, 6.8091978 , 10.45154775, 13.71997609])"
    ]
    },
    "execution_count": 13,
    "metadata": {},
    "output_type": "execute_result"
    }
    ],
    "source": [
    "d = y - l1\n",
    "d"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 14,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "71.424912179632841"
    ]
    },
    "execution_count": 14,
    "metadata": {},
    "output_type": "execute_result"
    }
    ],
    "source": [
    "(d ** 2).mean() # MSE"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 15,
    "metadata": {
    "collapsed": true
    },
    "outputs": [],
    "source": [
    "alpha = 0.01 # learning rate"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 16,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "array([ 2.57619274, 0.35930555])"
    ]
    },
    "execution_count": 16,
    "metadata": {},
    "output_type": "execute_result"
    }
    ],
    "source": [
    "update = alpha * np.dot(d, l0)\n",
    "update"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 17,
    "metadata": {},
    "outputs": [],
    "source": [
    "weights += update # updating weights"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 18,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "array([ 4.57619274, 2.35930555])"
    ]
    },
    "execution_count": 18,
    "metadata": {},
    "output_type": "execute_result"
    }
    ],
    "source": [
    "weights"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 19,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "59.054474469073625"
    ]
    },
    "execution_count": 19,
    "metadata": {},
    "output_type": "execute_result"
    }
    ],
    "source": [
    "l1 = np.dot(l0, weights)\n",
    "d = y - l1\n",
    "(d ** 2).mean() # new MSE"
    ]
    },
    {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
    "### Network Training &mdash; Multi Step"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 20,
    "metadata": {
    "collapsed": true
    },
    "outputs": [],
    "source": [
    "weights = np.array(((1., 100.)))"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 21,
    "metadata": {},
    "outputs": [
    {
    "name": "stdout",
    "output_type": "stream",
    "text": [
    "MSE after 0 iterations: 7427.49\n",
    "MSE after 5 iterations: 4264.45\n",
    "MSE after 10 iterations: 2835.80\n",
    "MSE after 15 iterations: 2105.21\n",
    "MSE after 20 iterations: 1670.17\n"
    ]
    }
    ],
    "source": [
    "for _ in range(25):\n",
    " # layer 1\n",
    " l1 = np.dot(l0, weights)\n",
    "\n",
    " # deltas of layer 1\n",
    " d = y - l1\n",
    " \n",
    " # print MSE\n",
    " if _ % 5 == 0:\n",
    " print('MSE after %4d iterations: %6.2f' % (_, (d ** 2).mean()))\n",
    "\n",
    " # update weights based on deltas\n",
    " weights += alpha * np.dot(d, l0)"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 22,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "[<matplotlib.lines.Line2D at 0x118ee5f28>]"
    ]
    },
    "execution_count": 22,
    "metadata": {},
    "output_type": "execute_result"
    },
    {
    "data": {
    "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW8AAAD5CAYAAADodLT+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd8XNWd9/HPzNwZ1VEfyb3bJzbFgDGY4oIx2MYSHRII\nmFBTSDY9G4iz2eyT7G4SNsnuk2UpCYH0gh82SGAbMLaxTQ0YsMEcuXerjnqdmfv8cUYaG7AlSxpN\n+71fL15M0dw5x7K+vjrzu+fnsG0bIYQQicUZ6wEIIYQ4dRLeQgiRgCS8hRAiAUl4CyFEApLwFkKI\nBCThLYQQCcjq6wuUUp8BPhO+mw6cBVwM/BywgW3AvVrrUHSGKIQQ4sMcp1LnrZT6b+AdoBT4qdZ6\nvVLqIWCN1vqpE72upqZ5UMXk+fmZ+P1tgzlEwkm1OafafEHmnCoGM2efz+s40XP9XjZRSp0LnKa1\nfgSYBWwIP7UKWDSgkfWTZbmiefi4lGpzTrX5gsw5VURrzn0umxzjfuD74dsOrXXP2XQzkHuyF+bn\nZw56Aj6fd1CvT0SpNudUmy/InFNFNObcr/BWSuUBSmu9LvzQsevbXqDhZK8f7K9JPp+XmprmQR0j\n0aTanFNtviBzThWDmfPJQr+/yybzgLXH3N+ilFoQvr0U2DigkQkhhBiQ/i6bKGD3Mfe/DjyqlPIA\n24Enh3pgQgghTqxf4a21/smH7lcC86MyIiGEEH2Si3SEECIBSXgLIUQCivvwDjYFqfpDFaEuuYBT\nCCF6xH14+39fy/ZPb2fHuduo+fkRAnWBWA9JCCFiLu7DO2dJHqO/PJpQc5Dqfz1M5dnvcvjr++j4\noD3WQxNCiJiJ+/D2TExj6s+nMu2dMxnxf8ZgFbvx/7aWI9/eH+uhCSFEzJzK5fEx5cpxUfjZEgru\nKqZ5TSOugsjl9lX/fgh3iYe8GwtwZqXe3glCiNSTMOHdw+FykHNFXu/9YEuQ+oerCbWGqPrXQ+Tf\nWkTBHcV4xnhiOEohhIiuuF826Ysr28XU107H982RONwO6n5RxY7ZWzlw1246d3fEenhCCBEVCR/e\nAFaxm+JvjmLaljMY/V8TSP9EBk3lfhxOsxWubdvY3YPaUlwIIeJKUoR3D2eak7xPFTLpxelMXj8D\nz4Q0AFo3NlM5a6uUGgohkkZShXcPh8NB+vSM3vvdB7sItUipoRAieSRleH9Y/s1FptTwB2OwSkyp\n4a5573PwC3tiPTQhhBiQlAhvAJfXReE9JUx99XTGPjGZzIuy8UxM632+fWsbwZZgDEcohBD9l3Cl\ngoPlcDnIWZpHztI87JD5ENMO2Bz4zC6CTUHybymi4E4pNRRCxLeUOfP+OL3VKF02eZ8qNKWG/x0p\nNWx7vQXblioVIUT8Senw7uHMdB5fajg9g6an/ewp1bRuTK1+e0KIxJByyyYn01NqmPvJAtpebaFx\nZT1ZF5kGoN1Hu2j4Yx35y31YhfLHJoSILTnz/hgOh4OsC7yMemA8DpdZWvE/UUv1v5lSw0Nf20fH\ndik1FELEjoR3PxV+oYQRPxyLVeKm4Xe17Jr/Pnuvr6R5bWOshyaESEHy+38/ubwuCu8upuAOH83P\nN1L/SDWtLzXjzHbhvTQ31sMTQqQYCe9T5HA5yFmSR86SPDq2tYHl6H3u4Of3YJW4KbjTh2ds2kmO\nIoQQgyPhPQjpp2f23g42B2nd2ESgOkDdQ1XkLMuj8J4SMs7LwuFwnOQoQghx6voV3kqp+4ArAQ/w\nILABeBywgW3AvVrrlO4Q7PK6mPrmGTT9r5+6h6toKm+gqbyB9LMyGf2z8aSfltn3QYQQop/6/MBS\nKbUAuBC4CJgPjAV+CqzQWs8FHMBVURxjwnCmOcn7ZCGT1k5nwt+m4b0ij07djlXsBsyVnLKroRBi\nKPSn2mQxsBV4CigHKoBZmLNvgFXAoqiMLkH1lBqOe3wy094+E8tnwrtpVYOUGgohhkR/lk2KgPFA\nKTAReBpwaq17rhtvBk5abpGfn4llDa63pM/nHdTrY8Z3zO28DmpHp9Hwu1oafldL/qJ8xnxlDAVL\nC3ov1T/upYk65wFKtfmCzDlVRGPO/QnvOuADrXUXoJVSHZilkx5eoOFkB/D72wY+QszEa2qS4DL1\nuelM2jSDlhcaqXukGv8Lfvwv+MlemMP4P0097kuTZs79lGrzBZlzqhjMnE8W+v1ZNtkELFFKOZRS\no4AsYG14LRxgKbBxQCNLQQ6XA+/iPCasnMbkddPJu7kQ79JIQ+XmFxvp2t8ZwxEKIRJBn2feWusK\npdQ84HVM2N8L7AEeVUp5gO3Ak1EdZZJKPy2T0T+f0Hs/1B7i0Bf2EGwI4r+miOzbCsk8X0oNhRAf\n1a9SQa31tz7m4flDPBbhghHfH0vdI1XUrqyldmUt6TMzKbynmJyr8nF6ZDcDIYQhaRBHnJ5wqeEL\n0zlrw1l4r8ij4902Dt27l/a3WmM9PCFEHJErLOOQw+Egb14e46ZPpmtfJ03lfjLPzwagc2cHtQ9W\nUXh38XFNloUQqUXOvOOcZ3waRV8c0bvu3fDXusiuhtdV0vxcQ287NyFE6pAz7wRT/K1RZJ6TRd0j\n1bRubKZ1YzOeSWkUfqGEguW+vg8ghEgKcuadYD6u1LD7UNdxa+J2t5yJC5Hs5Mw7gfWUGpasGH1c\nYO+9rhKryKLgnhIpNRQiSUl4JwGryN17O9gUJNQapOnVFpoqGqTUUIgkJT/NScaV42LSC9OZ8PQ0\nvMvy6NhqSg13nLOV9i1SbihEspAz7yTkcDjImuMla46Xrn2d1P+qmqaKBjxT0wFzJWfXnk7SZ0ip\noRCJSs68k5xnfBoj/mUsU984HVe22dmx4ck6di2QUkMhEpmceacIhyvyoWXa5HSy5nojpYYT0yi4\np5i8Txb2BrwQIr7JmXcKyrrQGyk1/HQh3Ye7OHrfAfbfujPWQxNC9JOceaew9NMyGf2zCZR8ZzT+\n39SS9onIGrj/97V4JqeReX62lBoKEYckvAVWkRvf10b23g82BTnynQPYbSHSzwyXGl4tpYZCxBP5\naRQf4fQ6Gf+nqeSU5tGxrY1DXzSlhtX/cYRAvTRQFiIeSHiLjzClhtmMfWwyU18/ncLPlxBqD1Hz\no8MEjnbHenhCCCS8RR8849IY8f0xTHvnTMb+alJvbXjbW63svbaS5jVSaihELMiat+gXV7aLnLL8\n3vutLzXRuqmZ1k3hUsO7i8n7lJQaCjFc5MxbDIjvKyOZvH5GpNTw/gNUznyX6p8cjvXQhEgJEt5i\nwNJnZDD6ZxOYtuUMir89CmeGk1BLqPf5QG03ti1LKkJEgyybiEHrKTUs/GIJdqcJa9u22Xt1JY50\nZ2RXwzQ5VxBiqMhPkxgyTo8Tl9eseYeaQ6Sp9Eip4aytVD9wmECNVKsIMRQkvEVUuHJcjP3VMaWG\nHTY1Pz5C5TlbaXujJdbDEyLh9WvZRCn1FtAUvrsH+CHwOGAD24B7tdahj3+1SGU9pYa+b46k4c91\nND3tJ31mJgABf4C211rwXpYb41EKkXj6DG+lVDrg0FovOOaxp4EVWuv1SqmHgKuAp6I2SpHwXNku\nCu8spvDO4t7H/L+tpfoHh/BMSKPrq2OxyrKl1FCIfurPmfdMIFMp9Vz46+8HZgEbws+vAi5Hwluc\nIu/luXTt6aDxyXp2fnknzhVO8m8uouCuYjzj02I9PCHiWn/Cuw14APglMBUT1g6tdU8NWDNw0t97\n8/MzsazBnVH5fN5BvT4RJf2cfV7Gzi2m62ddHH74MIcfPEzdw9WEdncz87mZsR7dsEj67/HHkDkP\njf6EdyWwMxzWlUqpOsyZdw8v0HCyA/j9bQMfIWbiNTXNgzpGokm1OU9YMYGM2/NpetqPe4ynd+7V\nPzqMZ0Ka2dUwyUoNU+17DDLngbz2RPrz03AH8B8ASqlRQA7wnFJqQfj5pcDGAY1MiGM4PU7yri8k\na475CxtsCFD7i6Mc+lJ4V0MpNRSiV3/C+1dAnlJqE/BnTJh/Gfi+UuoVwAM8Gb0hilTlyrOY8vJp\nFH6hhFBnuNTw7K0c+vJeug50xnp4QsRUn8smWusu4OaPeWr+0A9HiON5xqYx4p/H4PuGKTWs/2U1\nDX+uw/cN0zzCtm0IHd+jU4hUkFyLiCJp9ZQaTtl8GpPWfALPWFON0rquiR1ztlH3SBXB5mCMRynE\n8JHwFgnF4XSQMTOr937n7k4CR7s5uuIglTPf5ch3D9C1V5ZURPKT8BYJrfCuYrOr4X2jcGa5qH+4\nmh3nb+PwN/fFemhCRJWEt0h4VpEb31dHMvXN0xn94ATSZ2biHu3pfb7jg3ZCnbJ7g0gusiWsSBo9\npYa51xVAePnb7rbZ96kd0G2Tf7uPgtt8WD53bAcqxBCQM2+RdBwOBw7LVJ+EOkPkXlPwkVLDjm2D\nu3BMiFiT8BZJzZXtYsT3xjDt7TMY+e9jcY/10PDHOnYt3E7ba7I1rUhcEt4iJbiyXRTcYUoNx/1h\nCnk3FZIx21StdB/qklJDkXAkvEVKcTgdeBflMvo/J+BwmqWV+l/XSKmhSDgS3iLlFX6+hOL7jy81\n3H/bLlpfSa0NlERikWoTkfKsQgvfV0ZS+IUSmsobqHu4iuZVDTjcDrIuSL3tS0VikPAWIszpcZJ3\nXQG51+bT/kYrzpzIHvQHv7gHz/g0U2pYLKWGIvYkvIX4EIfDQeZ52b33A/4ALc81EmwIUvufR8m9\ntoCCu4vJOCMzhqMUqU7WvIXog5VvMfWtY0oN/1TH7ku3s+caTefOjlgPT6QoCW8h+uG4UsM/TiFr\nQQ7tb7XiKjC/vNrdtpQaimElyyZCnAKH04H30ly8l+YSqO7GCod3U4Wfw1/fR97NRRTeWYxnojRQ\nFtElZ95CDNCxH1zaARtntov6R6rZMWcb+5fvpHVzs2kWIUQUSHgLMQTybihk2ptnMOahiWSclUnz\n6kb2XlPJwbt2x3poIknJsokQQ8ThdpB7bQE51+TT/vdW6h6pJmtupE68ZX0TadMzcJdIqaEYPAlv\nIYaYw+Egc3Y2mbMj5Yah1iAH7t6N3W52OSy4pxgWygVAYuBk2USI4eByUHL/aNzjPDT82ZQablmw\nhaZnG7CDsi4uTp2EtxDDwJnupOB2H1M2RUoNGzc0cuAzu2h/W/YWF6euX8smSqli4E3gMiAAPA7Y\nwDbgXq219JgSoh+OLTXMrHay7/eHyDjHXKnZodvx/7ZWSg1Fv/R55q2UcgMPA+3hh34KrNBazwUc\nwFXRG54QySvrtCx8Xx2Jw2G2pm34Q52UGop+68+yyQPAQ8Dh8P1ZwIbw7VXAoiiMS4iUU7Ji9EdK\nDXcv3E7DyvpYD03EoZMumyilPgPUaK3XKKXuCz/s0Fr3nA40A7l9vUl+fiaW5erry07K50u9T+ZT\nbc6pNl/46JyLP5vD5HvG0fRqEwd/fpCalTWEtnTg+5z5Ojto43A5YjHUISPf56HR15r3HYCtlFoE\nnAX8Big+5nkv0NDXm/j9g/tAxufzUlOTWhvjp9qcU22+0Mecpzgp/sU48r49AocTamrMEsreKytx\nj/dQeE8JGWcm3q6G8n0+9deeyEnDW2s9r+e2Umo98DngJ0qpBVrr9cBSYN2ARiWE6JNnjKf3drA+\nSKCum7bXWmj8Sz2ZF2RTeE8x3iV5CX82Lk7dQC7S+TrwqFLKA2wHnhzaIQkhPo5VaDFl02m0rGui\n7uFqWtc30fZKC+5xHsY9MZn00xLvTFwMXL/DW2u94Ji784d+KEKIvhxbatih26l/tJrm5xvxTDCl\nhaHWIIHqgJQapgC5PF6IBJWuMhj1wHhCXSGcHlM41vDnOo7cdwDv4lwK7ykh86Ls3lJEkVwkvIVI\ncD3BDeAen0bG2abUsHl1I2kzMij8bDG51xTgTJcLqpOJfDeFSCLeS3OZtHo6E59V5FydT6du5/CX\n93Hgjl2xHpoYYnLmLUQSyjw3m8xzs+k+1EX9r2vIOCer9zn/H2tJPy0zIUsNRYSEtxBJzD3aQ8mK\n0b33A/UBjvzjfuwOW0oNE5wsmwiRQlx5Lsb+ejLZl+TQ9koLB27fzY7zt1H7P1XSQDnBSHgLkUJ6\nSg3H/3kqkzfOIH95EYGabqq+d5DAka5YDy/5tLTAa6+Z/w8xCW8hUlRPqeG0LWcy5qGJpE3LAKDt\n9Razq+Em2dVwUFpayF+8AObMMf8f4gCX8BYixVkFFrnXFvTeb37elBnuvbaSXZdsx//HWkIdsmX/\nqWp6Ywe/3zGHF7kEa0cllt4+pMeX8BZCHKfkO6NNqeE1kVLDynO2UvtgVayHFvdqahw88YSbG27I\nQN08l8/wBN/kJwSmTiOgpg/pe0m1iRDiI3pLDf/JlBr6f1NDsD7Q+3ywMYArV+ID4OhRB888Y1FR\nYfHKKy5CIVO5c/bZQUovb+VzFzvwn7YesrNPfqBTJH/6QogT6ik19H11BHbArH/bIZvdSz7A8rlN\nqeHS1Cs1PHjQBHZ5ucUbb7iwbTP/2bODlJV1s2xZgLFjbcCBz3dOVLbBlfAWQvTJmRVpphL0B/GM\nT6PlxSbaXjW7GhbcWUz+p4tw5Qyu6Uo827vXQUWFRUWFm7feMvN0OGwuuCBIWVmAK64IMHLk8H3A\nK+EthDglVqHF+D9NpbOynbpHq2n4Sx1V3ztIzY8PM+FvKqmu3Ny1y0F5uZvycoutW01gu1w28+YF\nKCsLsHRpgOLi2FTkSHgLIQYkbVoGo34ynuL7R+P/bS3NqxtIn27KDQN1ATreayNrrjehdjW0bdDa\nSXm5WcPevt0Ettttc+mlAUpLAyxZEqCwMPYllBLeQohBsfItfP8wAt8/jOh9zP+bGqr/7TBp08O7\nGl4bv7sa2jZs2+bsXcPescMEdlqazeLFAUpLu1m8OEBeXowH+iES3kKIIZe9MIeOD9ppetrP4a/s\no+oHhyhY7iPnGxPiInVsG955x5xhl5e72bvX/MOSkWGzbFk3ZWUBLrssgDeOeyXHwR+jECLZZMzM\nYuzDk+j+Xhf1j5lSw5qfHiGoOxn564kxGVMoBG++6aS83M0zz1gcOGACOzPT5uqrTWAvXBggK6uP\nA8UJCW8hRNS4R4VLDb82koa/1lEyp4Du8HPVDxwmXWXgvSJ6pYbBILz+uovycotnnrE4csQEttdr\nc/31JrAXLAiQkRGVt48qCW8hRNQ5M50U3OYjz+elpqaZQE03NT89AgFMqeEdxeR/unBILvwJBODl\nl01gP/usRU2NCey8PJubbuqmtLSbefOCpCV4m08JbyHEsLN8bqZsOM2UGv65lqp/NqWGeTcVUvSV\nkbhL3Kd0vK4u2LTJRUWFCez6ehPYhYUhbr21i9LSABdfHMR9aoeNaxLeQoiYSJuazqgfj6P4vlH4\nf1tL/WPV1D9eQ9GXTNVKz46GJyo17OyEDRtclJe7Wb3aorHRfF1xcYjbb++irCzAnDlBrCRNuSSd\nlhAiUfSUGhZ9voT2La24R3kAaHm+kaofHqLwnhJTapjhpL0dXnzRlPQ995xFS4sJ7FGjQnzyk92U\nlgaYPTuIK3kv9OzVZ3grpVzAo4ACbOBzQAfwePj+NuBerbXsGSmEGDCH20HmeZHNmzo+6KCzsoPD\nX93Hge8e4o1RI/jFgTEcak8HYNy4EMuXd1NW1s3ZZ4dwxmcZedT058y7DEBrfZFSagHwQ8ABrNBa\nr1dKPQRcBTwVtVEKIVJKUxNsGDWG9fNGUvBSFUtajjCn8iDncog9s0Yy8d9Hc+aZIRLo4s0h12d4\na63/VylVEb47HmgAFgEbwo+tAi5HwlsIMQgNDbB6tdn4af16F11dJpmVysC7eCRXWFVkVlRx8RUO\nimaaX/Q7d3bgmZCGw0q9FHf0t82RUuoJ4BrgeuBxrfWo8OMLgTu01rec6LWBQNC2rBRYhBJCnJKa\nGvjb3+DJJ2HtWlPmBzBzJlx3nflvxozI19u2jd1t4/Q4CXWFeHX8qzjSHIz50hhG3DkCd14SlZMY\nJ/xXqd/hDaCUGgG8BuRorfPDj10FXKa1/uKJXldT0zyoXVx84drQVJJqc061+ULqznnbthaefdZs\n/LR5c6R5wcyZZmvV0tJuJk3qOzKCDQGq/vUwDX+pw24L4cx0kndTIQV3F5M2KT3aU+m3wXyffT7v\nCcO7Px9Y3gqM0Vr/G9AGhIC/K6UWaK3XA0uBdQMamRAiJRw+bJoXrF4NmzZl9TYvmDUr0rxg/PhT\nO8dz5Vm9pYYNvwuXGv6qhvrHapj03CfImJkg17kPUH8+sPx/wK+VUi8BbuArwHbgUaWUJ3z7yegN\nUQiRiPbvjzQv+Pvfe5oXwPnnmzPsZcsCjBo1+K1VrXyLoi+NoPDzJTQ922C2pg3vKd51oJPWDc3k\nXmdKDZNJfz6wbAVu/Jin5g/9cIQQiWz3bgcVFW4qKizeftsEttNpM3eu2Qt7+fJ0XK72qLy3w3KQ\ne2U+uVfm9z5W/8sa6v6niqofHCT/Nh8Ft/twj/BE5f2Hm1ykI4QYlMpKJxUV5sKZ994zgW1ZNpdc\nYgJ76dIARUXmDNvnS6emZvjGVvjZYhweB/7f1FD7s6PU/t+j5F5VQOFni8k4K7GXVSS8hRD919KC\n64PtbOUMytfmUFFhobUJbI/H5vLLI80L8vP7ONYwcI/yUPKd0fi+OpKGJ+uof6SaxpX1YNuMeWhS\nrIc3KBLeQog+2TZsfa2D52/7G0/5L6ESHwDp6TZLl5qtVS+/PEBOTowHegLOTCcFy33k31pE64Zm\nrBGRksJD/7CXNJVO/i1FQ7Kr4XBJnJEKIYaVbcNbb5nmBRUVFvv3e4HPkUkrN/AXltw3g0vuHk92\ndp+HihsOh4PsBZF/YQLV3TQ97SfUFqLmJ0fI+1S41HBy/JQanoiEtxCiVyhkmhc884ypwz50yFRo\nZGfbXHtlOze98Q2uOPJrPFPH4r97PSRQcH8cq9jNtLfPwP+7Wup/VU39Y6bUMPuyXEb+cCyeCfG7\n6beEtxApLhiEV1+NdJupqjKBnZtr88lPmo2f5s0Lkp4OtKygU19Hq5pOQp1yn4Qrz6LoiyMo/Jwp\nNax7uIrWzc248sxavt1tYwfsuCs1lPAWIgV1d8PmzSawV62yqK01wVRQEOKWWyLNCzwfrqrLziYw\na/bwD3gYHFtq2H2oC1eeicfGp+o5+k8H4q7UUMJbiBTR1QUvvRRpXuD3m6scfb4Qt91mmhdceGHy\nNi84Fe7RkYAOtYfAJu5KDeXbJEQSa2+H9evN+vWaNRZNTSawR44McdddpkrkvPNSo3nBQBXc5iPv\nhkIaVkZKDRtX1pN3UyGj/3NCzMYl4S1EkmltNd1mKipMt5nWVhPYY8aEuPlms4Y9a1bqNS8YDGem\nk4JbfeTfYkoN6x6pOq5xROvmZtJPzxjWUkMJbyGSQEsLPPecCey1ay3a201gT5gQ4o47zJLIzJmp\n3bxgKPSUGmYvyOntsRlsCbJ/+U4IMqylhhLeQiSoxkZYs8YE9rp1Fp2dJpmnTAly5ZVm46fTT5fA\njpZjGyP7vjayd0fDnlLDwnuKyZrnjdr7S3gLkUDq6023mfJyNy+95KK72wTI9OlBSksDlJUFUEoC\nezi5sl0U3TuCws+W0LSqgfqHq2h5vpGW5xuZtHY6LIzOZacS3kLEuepqB6tWmY2fNm92EQyaZD7z\nTBPYpaXdTJky+K1VxeA4LAe5ZfnkluXTvqWV5hcayTgjM2rvJ+EtRBw6etTRe5XjK69Eus2cc06Q\n0tJuSksDTJgggR2vMs7OIuPs6JYSSngLEScOHjSBXV5u8cYbrt5uM+edF+htXjBmjAS2MCS8hYih\nvXsj3WbeeivSvODCC82SyLJlAUaMkMAWHyXhLcQw27nTdJtZtQq2bDG1wi6Xzfz55gx76dIAPp8E\ntjg5CW8hosy2QWsn5eVmDXv7dnOG7XbDokUByspM84KCghgPVCQUCW8hosC2Yds2Z3hJxGLHDhPY\naWk2S5aYDxw//ekMuruj089RJD8JbyGGiG3D22/3nGG72bvXXH+ekWFTWmr2EbnsskDvTqp5eQxr\nP0eRXCS8hRiEUAj+/ndnb8f0gwdNYGdl2VxzjTnDXrgwQFZi97oVceik4a2UcgOPAROANOAHwPvA\n44ANbAPu1VqHojpKIeJIMGi6zfSsYR89agI7J8fmhhvMxk8LFoSbFwgRJX2ded8C1Gmtb1VKFQBv\nh/9bobVer5R6CLgKeCrK4xQipgIBePnlSLeZnuYF+fk2N99smhfMnRskLX67Zokk01d4/xV4Mnzb\nAQSAWcCG8GOrgMuR8BZJqKsLNm2KdJuprzeBXVQUYvlyE9gXXRTE7e7jQEJEwUnDW2vdAqCU8mJC\nfAXwgNa6pwi1GciN6giFGEYdHbBhg+k2s2aNRWOjucqxpCTEnXeawJ4zR5oXiNjr8wNLpdRYzJn1\ng1rrPyilfnzM016goa9j5OdnYlmD+9vu80Vva8V4lWpzjtV829pg9WpYuRLKy6G52Tw+dizcfjtc\nfz1ccIETp9MDDG3/wlT7HoPMeaj09YFlCfAc8EWt9drww1uUUgu01uuBpcC6vt7E728b1CB9Pi81\nNc2DOkaiSbU5D/d8W1rghRcsKv4XXljroa3TnFyMGxdi+XJz4czZZ0e2Vq2rG/oxpNr3GGTOA3nt\nifR15n0/kA98Vyn13fBjXwb+SynlAbYTWRMXIq41NZluM+XlpnlBR4dJ5qlUcl3+Wi77zfWcfl66\n7IUtEkJfa95fxoT1h82PznCEGFp+v+k2U17uZsMGF11dJpk/8YkgV87az62/L+M03sPhB7+lCDhm\nx3jEQvSPXKQjkk5tbaR5waZNLgIBE9innx6krCxAaWmAqVND0JJO/uvdOHZAYOo0Amp6jEcuRP9J\neIukUFVl9sJ+5hnTbaanecHZZwdZtsx0m5k06UM79WVn41+zHktvN8Gdnf0xRxYiPkl4i4R16FCk\n28xrr0WaF8yebbrNLFsWYNy4PrZWzc4mMEuWSkTikfAWCWX/ftO8oLzczZtvmgoRh8Nmzpxgb7eZ\nkSNlL2wMsHPoAAAKqElEQVSR/CS8Rdzbvds0Lygvt3jnHRPYLpfN3LmR5gUlJRLYIrVIeIu4VFlp\ntlYtL7d4/30T2JZls3ChCewlSwIUFkpgi9Ql4S3igm3De+85ezd+0toEtsdjs3ix+cBx8eIAeXkx\nHqgQcULCW8SMbcO77zrDGz/Bjh1m0+v0dJtly8xe2JdfHsCbeldTC9EnCW8xrEIheOutSPOC/ft7\nmhfAVVeZbjMLFwakak+IPkh4i6gLhUzzgp5+jocPm8D2em2uu84E9o03ZtDS0hHjkQqROCS8RVQE\nAvDqq5HmBdXVJrBzc20+9SnTbWbevEjzgowMs1mUEKJ/JLzFkOnuNs0LKiosnn3Woq7OBHZhYYhb\nb+1i2bIAF18cxDO0u6oKkZIkvMWgdHbCSy+Z5gWrV1s0NJirHIuLQ9x+u2lecMEFQSz5mybEkJIf\nKXHK2tth3Tqzfr1mjUVzswnskSND3HijqRKZPVu6zQgRTRLeol9aW2HtWhPYzz1n0dZmAnvs2BC3\n3GLWsM85J4TTGeOBCpEiJLzFCTU3w/PPm6scX3zRor3dBPbEiSHKyrooKwtw5pkhaV4gRAxIeIvj\nNDSY5gUVFW7WrYs0L5g2LUhpqbk0fcYMCWwhYk3CW1BX52D1anOGvXGji+5uk8wzZkSaFygVivEo\nhRDHkvBOUdXVDp591gT2yy+7CAZNYM+cac6wS0u7mTxZNn4SIl5JeKeQI0cizQteeSXSvGDWLNO8\noLQ0wPjxEthCJAIJ7yR34IAJ7PJyN2+8EWlecN55keYFo0dLYAuRaCS8k9CePY7ejZ+2bDGB7XTa\nXHyxWb9etkyaFwiR6CS8k8TOnQ7Ky023mW3bIt1mFiyINC/w+SSwhUgW/QpvpdT5wI+01guUUlOA\nxwEb2Abcq7WWUoRhZtvwwQdmL+yKCosPPjCB7XbbLFoUoKzMNC8oKIjxQIUQUdFneCulvgXcCrSG\nH/opsEJrvV4p9RBwFfBU9IYoetg2bNsWCeydO01gp6XZLF1qPnBcfFEjBUfeJ6CmI5tiC5G8+nPm\nvQu4Fvht+P4sYEP49irgciS8o8a2YcsWJ+XlZg173z5z/Xlmpk1ZmdkLe9GicPOClhbyFy/A2lFJ\nYOo0/GvWS4ALkaT6DG+t9Uql1IRjHnJorXsWT5uB3L6OkZ+fiWUNbpciny91emGFQrB5M6xc6WXl\nSti/3zzu9cJNN8H118OSJQ4yM92AO/LC3e/DjkoArB2V+Kr3w8Tzh38CA5RK3+MeMufUEI05D+QD\ny2PXt71AQ18v8PvbBvA2ET6fl5qa5kEdI94Fg/Daa5HmBUePmjPsnBybG280a9jz5wdJTzdf39pq\n/jtO8Tjyp06LnHkXj4ME+XNLhe/xh8mcU8Ng5nyy0B9IeG9RSi3QWq8HlgLrBjQqQXc3vPyyCexn\nn7WorTWBnZ9vc+edsGhRG3PnnkLzguxs/GvWY+ntsuYtRJIbSHh/HXhUKeUBtgNPDu2QkltXF2zc\n6Ap3THfj95urHIuKQtx2m2lecOGFQUaN8lJTEzz1N8jOJjBr9hCPWggRb/oV3lrrvcCc8O1KYH4U\nx5R0Ojpg/XrTbWbNGoumJhPYI0aEuOsuUyVy/vnSvEAI0X9ykU6UtLWZ5gXPPGO6zbS2msAeMybE\nTTeZ5gXnnivNC4QQAyPhPYRaWkzzgooKi7VrI91mxo83/RzLygKcdZbshS2EGDwJ70FqbOxpXmCx\nbp1FZ6dJ5ilTInthn366BLYQYmhJeA+A30+4eYGbDRsizQumT490m1FKAlsIET0S3v1UU+Ng1SrT\nvGDzZheBgEnmM87oaV4QYOpU2eJFCDE8JLxP4ujR45sXhEImsM85J8iyZabbzMSJslOfEGL4SXh/\nyKFDDioqzBn2G29Eus3Mnh2krKybZcsCjB0rgS2EiC0Jb2DfPhPYFRVu3nwz0m3mggvMh45XXBFg\n5EgJbCFE/EjZ8N61y3SbKS+3ePfdSPOCefPMB45LlwYoLpbAFkLEp5QKb63NXtjl5Rbbt0eaF1x6\naU/zgiCFhRLYQoj4l9Thbdvw3nvO3jXsHTsizQuWLDHr14sXB8jLi/FAhRDiFCVdeNs2vPNOT7cZ\nN3v2mOvPMzJsli0zzQsuuyyAN/W2FBZCJJH4D++WFtNkoHjcCbc4DYXgzTdNt5lnnrE4cCDSbebq\nq01gL1wYICtrOAcuhBDRE9/hHW7rxY5K8j/U1isYhNdfd4WrRCyOHDGB7fXaXH+9CewFCwJkZMRu\n+EIIES1xHd6W3o51TFsv3v+AjZ3n93abqakxgZ2XZ3PTTd2UlnYzb16QtLRYjloIIaIvrsM7oKbT\nPmUGL+0cw19y7uRvy+dRV28+dCwqCnHrraZ5wcUXB3G7+ziYEEIkkbgOb393NjNb3uUILmiC4uIQ\nd9xhAnvOnCBWXI9eCCGiJ67jz+2GWeeGmDzZxaWXtnHeeUFpXiCEEMR5eGdnw2OPdeDzuQfWz1EI\nIZKUnMcKIUQCkvAWQogEJOEthBAJSMJbCCES0IA+sFRKOYEHgZlAJ3CX1nrnUA5MCCHEiQ30zPtq\nIF1rfQHwbeA/hm5IQggh+jLQ8L4YWA2gtX4VOHfIRiSEEKJPA63zzgEaj7kfVEpZWuvAx31xfn4m\nluUa4FsZPl/q7eGaanNOtfmCzDlVRGPOAw3vJuDY0ThPFNwAluVyDPB9hBBCfIyBLptsBq4AUErN\nAbYO2YiEEEL0aaBn3k8BlymlXgYcwO1DNyQhhBB9cdi2NNwVQohEIxfpCCFEApLwFkKIBCThLYQQ\nCShu9/NOxUvwlVJu4DFgApAG/EBr/XRMBzVMlFLFwJvAZVrrD2I9nmhTSt0HXAl4gAe11r+K8ZCi\nKvx3+wnM3+0gcHcyf5+VUucDP9JaL1BKTQEeB2xgG3Cv1jo02PeI5zPvVLwE/xagTms9F1gC/CLG\n4xkW4R/sh4H2WI9lOCilFgAXAhcB84GxMR3Q8LgCsLTWFwL/AvwwxuOJGqXUt4BfAunhh34KrAj/\nXDuAq4bifeI5vFPxEvy/At8N33YAJ7zwKck8ADwEHI71QIbJYsy1EU8B5UBFbIczLCoBK/wbdQ7Q\nHePxRNMu4Npj7s8CNoRvrwIWDcWbxHN4f+wl+LEazHDQWrdorZuVUl7gSWBFrMcUbUqpzwA1Wus1\nsR7LMCrCnIzcAHwO+L1SKtmvQm7BLJl8ADwK/FdMRxNFWuuVHP+Pk0Nr3VOT3QzkDsX7xHN4n9Il\n+MlCKTUWWAf8Vmv9h1iPZxjcgbngaz1wFvAbpdSI2A4p6uqANVrrLq21BjoAX4zHFG1fxcx5GuZz\nrCeUUul9vCZZHLu+7QUahuKg8RzeKXcJvlKqBHgO+Eet9WOxHs9w0FrP01rP11ovAN4Glmutj8Z4\nWNG2CViilHIopUYBWZhAT2Z+Ir9J1wNuYHC71SWOLeHPOQCWAhuH4qDxvAyRipfg3w/kA99VSvWs\nfS/VWqfEB3mpQmtdoZSaB7yOOYG6V2sdjPGwou1nwGNKqY2YCpv7tdatMR7TcPk68KhSygNsxyyJ\nDppcHi+EEAkonpdNhBBCnICEtxBCJCAJbyGESEAS3kIIkYAkvIUQIgFJeAshRAKS8BZCiAQk4S2E\nEAno/wMWE91GkIqUywAAAABJRU5ErkJggg==\n",
    "text/plain": [
    "<matplotlib.figure.Figure at 0x118e92898>"
    ]
    },
    "metadata": {},
    "output_type": "display_data"
    }
    ],
    "source": [
    "yr = np.polyval(reg, x)\n",
    "plt.plot(x, y, 'r.')\n",
    "plt.plot(x, yr, 'b')\n",
    "plt.plot(x, l1, 'm--')"
    ]
    },
    {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
    "## Neural Network for Classification"
    ]
    },
    {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
    "https://iamtrask.github.io/2015/07/12/basic-python-network/"
    ]
    },
    {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
    "### Sigmoid Function"
    ]
    },
    {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
    "https://en.wikipedia.org/wiki/Sigmoid_function"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 23,
    "metadata": {
    "collapsed": true
    },
    "outputs": [],
    "source": [
    "# sigmoid function\n",
    "def sigmoid(x, deriv=False):\n",
    " if deriv == True:\n",
    " return x * (1 - x)\n",
    " return 1 / (1 + np.exp(-x))"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 24,
    "metadata": {
    "collapsed": true
    },
    "outputs": [],
    "source": [
    "x = np.linspace(-10, 10)\n",
    "y = sigmoid(x)\n",
    "d = sigmoid(x, deriv=True)"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 25,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "image/png": "iVBORw0KGgoAAAANSUhEUgAAAloAAAHRCAYAAACyxBdsAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd8FHX+x/HXtjQSSiB0AUEcEBERVLpKUZAOgohdOcF2\n9gPkTn+WUw9BRU9RBMspiCAdpIggKkhVEZSMAtKLoQdSd3d+fywlIj27mU3m/Xw89rE7M7v7/XwI\nLO/Md3bGZVkWIiIiIhJ+brsLEBERESmqFLREREREIkRBS0RERCRCFLREREREIkRBS0RERCRCFLRE\nREREIsRrdwEnkpaWHvFzTpQqlcDevRmRHiZqObl/J/cOzu5fvTuzd3B2/07uHQqm/5SUJNfJtjl2\nj5bX67G7BFs5uX8n9w7O7l+9O5eT+3dy72B//44NWiIiIiKRpqAlIiIiEiEKWiIiIiIRoqAlIiIi\nEiFhC1qGYVxpGMZXJ1jf0TCMZYZhfGcYxt/CNZ6IiIhItAtL0DIM4x/ASCDuuPU+4FXgWuAq4B7D\nMMqFY0wRERGRaBeu82itA7oBHx23vjaw1jTNvQCGYXwLtADGh2lcERGRcxIMQmYmZGdDVpaLrCzI\nzAzdZ2W5yMmB3FzIyXEdvofc3L+uz82FQAD8fvD7XXkeH1nvOvr4yM2yIBAIPTcY5Oj9sccuLCv0\nvCPrg8Fjy3++P/bcvLcjPbpcEAgUO7ruRM87+b3rtM85/vHptp3L8071ulOJjYVJk6Bu3TN/73AL\nS9AyTXOCYRjVTrCpOLA/z3I6UOJ071eqVEKBnPciJSUp4mNEMyf37+Tewdn9q/fCLxCAvXshLQ12\n7Qrd9u2DAwcgPT10O/L4z+uSOHgQMjJCASsnx+5OTsztBo8nFJDc7mO3Ey0fWXfkcd5b3vfxet1H\n18Nfn3v8uiPLx9+fbNvxj0+17Uyfdzpn8tz4eEhJsffvfqTPDH8AyNtdErDvdC8qiDPYpqQkkZaW\nHvFxopWT+3dy7+Ds/tV79PZuWaFQtGWLm61bXWzZ4mbbNhe7d4due/Ycu+3d68KyzuJ/ZCAhAYoV\nC1KsGJQsaREfD3FxFnFxx+7j448sh9bFxoLPZxETAz4fxMRYh+//vOz1HrlZeL2hcHNk+djjY8HH\n47GOPj7RfbhF+88+0gqi/1MFuUgHrTVATcMwkoGDhKYNh0R4TBERiUIHD8Kvv7pZu9bN1q1utmxx\nsXVrKFht3erm4MGThyePx6JUKYuUFAvDCJKcbJGcbFG6dOi+ZEmLYsUgKckiMdEiMfHY42LFoEKF\nJNLSDhVgtyIhEQlahmH0BhJN0xxhGMajwGxCB96/Z5rm1kiMKSIi0SEnB9audZOaGrqtWeNhzRo3\nmzadeHdNiRIWVaoEqVzZolKlIJUqWVSuHKRiRYuyZUOhqnjxyOztEYm0sAUt0zQ3AI0OPx6TZ/00\nYFq4xhERkejh98PKlW4WLvSyalUoWK1b58bv//PeqTJlgjRv7qdWrSA1awapUiUUpCpVCpJUNA4d\nEzmhSE8diohIEeL3w6pVbhYu9LBwoZfFiz0cOnQsVCUlWdSvH6RWrQC1awepVSuIYQRJSTmLr5SJ\nFCEKWiIiclKBAPz8s5tvvz0WrNLTjwWrmjUDNGkSoFmzAA0aBKhUyTqrb46JFHUKWiIi8ieWBUuW\neBg/3su0aT727TuWnKpXD9KlSy5NmwZo2jRAuXLaUyVyKgpaIiICwO+/uxg/3sf48T42bgwdeV6+\nfJCbbz4WrCpUULASORsKWiIiDrZvH0yZ4mPcOB/LloVOFJ2QYNGjRy49e+bSrFkAT+TPHy1SZClo\niYg4TG4ufPmlh3HjfMyZ4yUnx4XLZXHVVX569Mjl+uv9JCbaXaVI0aCgJSLiEIEATJjg5eWXY49O\nDdaqFaBnz1y6d/drWlAkAhS0RESKOMuCCRNg0KAETNNDTIzFHXfkcOutuVx8cVDfEhSJIAUtEZEi\nyrJg/nwPL74Yy8qV4Ha76d07h8cey+G887T3SqQgKGiJiBRBixd7eOGFGBYvDn3M9+oFDz10iBo1\nFLBECpKClohIEbJypZsXXohl/vzQx/t11/np3z+ba64pRlqaQpZIQVPQEhEpAnbvdjFwYCyTJ/sA\naN7cz8CB2TRsGLS5MhFnU9ASESnkvv3Ww333xbFjh5vLLgswaFA2zZsH7C5LRFDQEhEptHJzYfDg\nGF5/PQa3GwYNyuaBB3J0glGRKKKgJSJSCG3Y4OLee+NZscJDlSpB3nknkwYNNE0oEm3cdhcgIiJn\n57PPvLRsWYwVKzx0757L/PmHFLJEopT2aImIFBIHD8KAAXGMG+ejWDGL//43k549/XaXJSKnoKAl\nIlII/PCDm75949mwwU39+gGGD8+kenWdrkEk2mnqUEQkigWD8MYbMbRvn8CGDW4eeCCbadMyFLJE\nCgnt0RIRiVKBADz6aByffOKjbNkgb76ZyVVX6bQNIoWJgpaISBTy++GBB+KYONFHvXoBxozJJCVF\ne7FEChsFLRGRKJObC/36xTFtmo+GDQOMHZtB8eJ2VyUi50JBS0QkimRnw9/+FsesWT4aN/YzenQm\niYl2VyUi50pBS0QkSmRmwp13xjNvnpcWLfz873+ZJCTYXZWI5Ie+dSgiEgUOHYJbbgmFrNat/Xz8\nsUKWSFGgPVoiIjZLT4feveNZssRLu3a5jBiRRWys3VWJSDhoj5aIiI3274eePRNYssRL5865jByp\nkCVSlChoiYjYZM8e6N49gRUrPPTokcvw4Vn4fHZXJSLhlO+pQ8Mw3MBbQD0gG+hjmubaPNtvBh4D\nAsB7pmkOz++YIiKFXVqaixtuiGfNGg+33JLDyy9n4/HYXZWIhFs49mh1AeJM02wMDACGHrd9CNAa\naAo8ZhhGqTCMKSJSaGVmho7JWrPGw1135TBkiEKWSFEVjqDVDJgFYJrmYqDhcdt/AkoAcYAL0KmN\nRcSxLAseeyyOlSs99OqVy4svZuPWQRwiRVY4vnVYHNifZzlgGIbXNE3/4eXVwArgEDDRNM19p3vD\nUqUS8Hoj/+tdSkpSxMeIZk7u38m9g7P7t7v3116Dzz6DK6+E99/3ERdXcAdl2d273Zzcv5N7B3v7\nD0fQOgDk7cB9JGQZhnEJ0B44HzgIfGwYRg/TNMef6g337s0IQ1mnlpKSRFpaesTHiVZO7t/JvYOz\n+7e796+/9vD44/GULWsxYkQG6ekW6QVUjt29283J/Tu5dyiY/k8V5MKxw3ohcD2AYRiNgFV5tu0H\nMoFM0zQDwB+AjtESEcfZuNHFPffE4XbDe+9lUqGCjqIQcYJw7NGaBLQxDGMRoWOw7jQMozeQaJrm\nCMMw3gG+NQwjB1gHfBCGMUVECo1Dh+COO+LZs8fNkCFZXHFF0O6SRKSA5DtomaYZBPodtzo1z/a3\ngbfzO46ISGFkWfDII3H8/LOH227L4bbbcu0uSUQKkL7rIiISQW++6WPyZB9XXOHnhRey7S5HRAqY\ngpaISITMm+fh+edjKV8+yKhRWcTE2F2RiBQ0BS0RkQhYv95F377xeL3wwQeZlCung99FnCgcB8OL\niEgeBw+GDn7fv9/FsGGZXHaZDn4XcSrt0RIRCSPLggcfjCM11cPdd+dw003+079IRIosBS0RkTB6\n7bUYZszw0bixn2ef1cHvIk6noCUiEiZLl7p56aUYKlUKMnJkFr6Cu7qOiEQpBS0RkTDIzISHHooH\nYPjwLFJSdPC7iChoiYiExX/+E8u6dW7uuSeXRo0CdpcjIlFCQUtEJJ+WLXMzfLiPatWCDByo47JE\n5BgFLRGRfAhNGcYB8PrrWSQk2FyQiEQVBS0RkXwYPDiWtWs99OmjKUMR+SsFLRGRc7R8+bEpwyef\n1JShiPyVgpaIyDnIygpNGQaDLoYNy6JYMbsrEpFopKAlInIOBg+O4bffPPTpk0PjxpoyFJETU9AS\nETlLK1a4eeutGKpWDTJokKYMReTkFLRERM6CpgxF5GwoaImInIUhQ2L49dfQBaObNNGUoYicmoKW\niMgZ+v57N//9b2jK8J//1JShiJyegpaIyBnIO2X42muaMhSRM6OgJSJyBoYOjcE0Pdx1Vw5Nm2rK\nUETOjIKWiMhprFzp5o03YqhSRVOGInJ2FLRERE4hGIT+/UNThq++mkViot0ViUhhoqAlInIKn37q\n5fvvPXTtmkvz5poyFJGzo6AlInISBw7Ac8/FkpBg8fTTmjIUkbOnoCUichIvvxzLrl1uHnkkh4oV\nLbvLEZFCSEFLROQE1qxxM3Kkj/PPD9KvX47d5YhIIaWgJSJyHMuCQYNiCQRcPP98FrGxdlckIoWV\nN79vYBiGG3gLqAdkA31M01ybZ/vlwCuAC9gB3GKaZlZ+xxURiZRp07x8+62XNm38tGmjA+BF5NyF\nY49WFyDONM3GwABg6JENhmG4gHeBO03TbAbMAqqGYUwRkYg4dAiefjqWmBiL557T74Qikj/hCFpH\nAhSmaS4GGubZdiGwG3jEMIwFQLJpmmYYxhQRiYg33ohh61Y3996bQ/XqOgBeRPLHZVn5+yAxDGMk\nMME0zZmHlzcB1U3T9BuG0RSYC1wGrAWmA/8xTXPeqd7T7w9YXq8nX3WJiJyt9evhoougTBlITUUn\nJxWRM+U62YZ8H6MFHACS8iy7TdP0H368G1hrmuYaAMMwZhHa43XKoLV3b0YYyjq1lJQk0tLSIz5O\ntHJy/07uHZzd/+l6v//+OLKzfTz1VCaZmX4yMwuwuAhz8s8dnN2/k3uHguk/JSXppNvCMXW4ELge\nwDCMRsCqPNvWA4mGYVxweLk58HMYxhQRCasvv/Qwa5aPJk38dOniP/0LRETOQDj2aE0C2hiGsYjQ\nrrM7DcPoDSSapjnCMIy7gTGHD4xfZJrmjDCMKSISNtnZMGhQHG63xb//nY3rpJMAIiJnJ99ByzTN\nINDvuNWpebbPA67I7zgiIpHyzjsxrF/v5u67c6hTJ2h3OSJShOiEpSLiaNu3u3jllRhKlw7Sv7+u\nZygi4aWgJSKO9swzsWRkuBg0KIeSJe2uRkSKGgUtEXGsxYs9TJzo49JLA/TunWt3OSJSBCloiYgj\nBQLw5JOhixi++GIWbn0aikgE6KNFRBzp00+9rF7toUePXBo00AHwIhIZCloi4jgHD8ILL8QSH28x\naJAOgBeRyFHQEhHHeeONGP74w8399+dQsaKuZygikaOgJSKOsmWLi+HDYyhfPsj99+fYXY6IFHEK\nWiLiKM8/H0tWlotBg7IpVszuakSkqFPQEhHHWLyYo6dz6NFD1zMUkchT0BIRR7AseOSR0ONnn83W\n6RxEpEDoo0ZEHGHSJC+LF0PHjrk0ahSwuxwRcQgFLREp8jIz4bnnYomJgX/9S6dzEJGCo6AlIkXe\n22/HsHWrm4cfhmrVdDoHESk4CloiUqTt3Oli2LAYypQJMmiQ3dWIiNMoaIlIkfbiizFkZLjo3z+H\n4sXtrkZEnEZBS0SKrFWr3HzyiY/atQPcfHOu3eWIiAMpaIlIkWRZ8NRTsViWi2eeycbrtbsiEXEi\nBS0RKZJmzvSycKGXNm38XH21TucgIvZQ0BKRIicnB555Jhav1+L//k+ncxAR+yhoiUiRM2qUj99/\nd3PHHbnUrBm0uxwRcTAFLREpUnbtcjF0aCwlS1o8/rj2ZomIvRS0RKRIeeGFGA4ccPH449kkJ9td\njYg4nYKWiBQZP/7oZvRoH7VqBbjzTp3OQUTsp6AlIkVCMAgDB8ZhWS5eeCEbn8/uikREFLREpIgY\nN87LihUeOnfOpVkznc5BRKKDgpaIFHoHDsCzz8aSkKDTOYhIdFHQEpFC7+WXY9m1y81DD+VQqZJl\ndzkiIkfl+6IUhmG4gbeAekA20Mc0zbUneN4IYI9pmgPyO6aIyBGm6WbUKB9Vqwa5994cu8sREfmT\ncOzR6gLEmabZGBgADD3+CYZh9AXqhmEsEZGjLAuefDIWv9/F889nERdnd0UiIn8WjqDVDJgFYJrm\nYqBh3o2GYTQBrgTeCcNYIiJHTZ/u5ZtvvLRq5efaa3UAvIhEH5dl5e94BsMwRgITTNOceXh5E1Dd\nNE2/YRgVgA+ArkBPoNaZTB36/QHL6/Xkqy4RKdoyMqB2bdi+HVavhgsvtLsiEXEw18k25PsYLeAA\nkJRn2W2apv/w4x5AGeBzoDyQYBhGqmmaH5zqDffuzQhDWaeWkpJEWlp6xMeJVk7u38m9Q9Hp/z//\niWHTplgefDCbUqVySEs7/WuKSu/nwsm9g7P7d3LvUDD9p6QknXRbOILWQqAjMM4wjEbAqiMbTNN8\nHXgdwDCMOwjt0fogDGOKiINt3Ojiv/+NoXz5II88ogPgRSR6hSNoTQLaGIaxiNCuszsNw+gNJJqm\nOSIM7y8i8idPPRVLdraLp5/OIjHR7mpERE4u30HLNM0g0O+41akneN4H+R1LRGT+fA8zZ/po1MhP\nt27+079ARMRGOmGpiBQaOTkwaFAsbrfFCy9k4zrp4aciItFBQUtECo0RI3ysXevh9ttzufjioN3l\niIicloKWiBQKO3e6GDo0luTkIAMG6HqGIlI4KGiJSNSzLBgwIJZDh1wMHJhDqVJ2VyQicmYUtEQk\n6k2Z4mXGjNAB8Lfemmt3OSIiZ0xBS0SiWlqaiwEDYomPt3jttSzc+tQSkUJEH1kiEtUGDIhlzx43\ngwZlU716/i4ZJiJS0BS0RCRqTZniZdo0H1de6adPH00Zikjho6AlIlEp75ThsGGaMhSRwkkfXSIS\nlQYOjGX3bjdPPqkpQxEpvBS0RCTqTJ3qZepUH1dcoSlDESncFLREJKrs2uWif/9Y4uJCU4Yej90V\niYicOwUtEYkqeacMa9TQlKGIFG4KWiISNaZN8zJlio/LLw/wt79pylBECj8FLRGJCnmnDF9/PVNT\nhiJSJChoiUhUePLJWHbtcjNwoKYMRaToUNASEdtNm+Zl8mQfDRsGuOceTRmKSNGhoCUittq9W1OG\nIlJ0KWiJiG0sC/7xj9CU4YAB2VxwgaYMRaRoUdASEdu8+abv6LUM+/bVlKGIFD0KWiJii3nzPDz/\nfCwVKgQZOVInJhWRoklBS0QK3Pr1Lvr2jcfng/ffz6RcOU0ZikjR5LW7ABFxloMH4Y474tm/38Ww\nYZlcdlnQ7pJERCJGe7REpMBYFjz4YBypqR769Mnhppv8dpckIhJRCloiUmBeey2GGTN8NGni55ln\nsu0uR0Qk4hS0RKRAzJnj4aWXYqhcOXTwu89nd0UiIpGnoCUiEbd2rYt7740nNhY++CCTMmV08LuI\nOEO+D4Y3DMMNvAXUA7KBPqZprs2z/SbgYcAPrALuM01TR7+KOMSBA3DbbfGkp7t4661MLrlE//xF\nxDnCsUerCxBnmmZjYAAw9MgGwzDigeeBa0zTbAqUADqEYUwRKQSCQbj//njWrvXQr18ON9ygg99F\nxFnCEbSaAbMATNNcDDTMsy0baGKaZsbhZS+QFYYxRaQQePnlGGbP9tK8uZ+nntLB7yLiPOEIWsWB\n/XmWA4ZheAFM0wyaprkTwDCMB4FE4IswjCkiUW7GDC9Dh8ZSpUqQd9/NxKuz9omIA7ksK38HpRqG\n8Qqw2DTNcYeXt5imWTnPdjcwGLgQ6JVn79ZJ+f0By+vV9ThECqvvvoNrrw1NHS5aBPXq2V2RiEhE\nuU62IRy/Yy4EOgLjDMNoROiA97zeITSF2OVMD4Lfu/e0WSzfUlKSSEtLj/g40crJ/Tu5d4h8/4sW\neejdO57sbHj33SwqVvSTlhax4c6Kk3/2Tu4dnN2/k3uHguk/JSXppNvCEbQmAW0Mw1hEKNHdaRhG\nb0LThMuBu4FvgHmGYQAMM01zUhjGFZEos2CBh9tui8fvh5Ejs2jfXge/i4iz5TtoHd5L1e+41al5\nHutcXSIOMHeuhzvvjAdC58pq0yZgc0UiIvZTCBKRfPv8cy+33x6P2w3/+59ClojIEQpaIpIvU6Z4\n6dMnDp8PxozJ5JprFLJERI5Q0BKRczZ+vJe+feOIj4dx4zJo2lQhS0QkLwUtETkno0f7eOCBOJKS\n4LPPMrjiCl1aR0TkeApaInLW3nvPxyOPxFGqlMXEiRnUr6+QJSJyIgpaInJW3n7bx4ABcZQpE2TS\npEzq1lXIEhE5GV0UQ0TOSE4OPP98LG+/HUP58kEmTMikZk2FLBGRU1HQEpHTWrfORd++8fz0k4fq\n1YOMGZNB9er5u3yXiIgTaOpQRE7KsmDsWC+tWhXjp5883HRTLnPnHlLIEhE5Q9qjJSIndOAAPPFE\nHJMm+UhKsnjnnUy6dtUldUREzoaCloj8xbJlbu69N55Nm9w0bBhg+PBMqlbVXiwRkbOlqUMROSoQ\ngFdeiaFTpwS2bHHx6KPZTJ2aoZAlInKOtEdLRADYts3FfffFsWiRl4oVgwwfnkXjxjrTu4hIfmiP\nlojDWRZMmuTl6quLsWiRl/btc5k//5BClohIGGiPlohDWRbMm+fhxRdj+eknD/HxFi+/nMVtt+Xi\nctldnYhI0aCgJeJA333n4YUXYliyJPQR0LVrLv37Z+u0DSIiYaagJeIgP/7oZsgQmDMnAYC2bXPp\n3z+HOnV0hncRkUhQ0BJxgNRUNy+9FMPnn/sAaN7cz8CB2TRsqIAlIhJJCloiRdjvv7t4+eVYJkzw\nYlkuGjYMMHiwh4svzrS7NBERR1DQEiliDh2CmTO9jBvn4+uvPQSDLurUCTBwYDZt2gQoWzaJtDS7\nqxQRcQYFLZEiIBiEhQs9jBvnY/p0L4cOhb422KBBgL59c+jUyY9bJ3MRESlwCloihZhpuhk/3stn\nn/nYti2UpKpUCdK3bw49euRSo4a+RSgiYicFLZFCZtMmF7Nnh6YGV670AJCUZHHLLTn06OHnyisD\n2nslIhIlFLREotyWLS6+/dbDokVeFi70sHlzKEV5PBZt2vjp0SOX667zEx9vc6EiIvIXCloiUWbb\nNhcLF3oO37xs3Hhs91TJkhbt2uXSokWAjh39lC2rqUERkWimoCViE8uCHTtcrFnjJjXVzZo1HpYu\n9fD778eCVfHiFm3b5tK0aYAmTQLUqRPUtKCISCGioCVSAPbsgdRUz9FQFbp52L//zxcVTEwMTQc2\nbeqnWbNQsPJ4bCpaRETyTUFLJJ8sC3btcrF1q4stW9xs2xa637rVxdatbjZvdpGW9ufdUG63RY0a\nQVq0CFKr1rHb+ecH8epfpYhIkZHvj3TDMNzAW0A9IBvoY5rm2jzbOwJPAX7gPdM0383vmCKRlpMD\ne/e62L3bxZ49odvxj9PSQkFq61YX2dmuE75PTIxFxYoW9ev7qVUrcDRQXXBBkLi4Am5KREQKXDh+\nd+4CxJmm2dgwjEbAUKAzgGEYPuBV4HLgELDQMIyppmnuDMO4IlgWZGUdubnIzITsbNeflrOyQssH\nD7pITw+9ZufOWA4ePLLOxcGDkJ7u4tAhF3v3htadiTJlgtSuHaRSpSCVK1tUqhSkUqVj9ykplo6p\nEhFxsHAErWbALADTNBcbhtEwz7bawFrTNPcCGIbxLdACGB+Gcc9ZTg589RWkpf354BfrBF/gOtG6\ns3Gq98y77USPj927Tvu6vPfH3060PikJ9u/35lnn+tP2YPDYPYTuj9yOrA8GXXkeQyBw/GPXX9b5\n/UfuXXke/3VdTg7k5roO30NOjuvw/V/Xn5uYPy253RZJSaFjpM47L0jp0halS1skJ4dueR/nXdZe\nKREROZVwBK3iwP48ywHDMLymafpPsC0dKBGGMfNlxAgfzz4LkGB3KTaL3hMvuVwWMTHg84Wm30L3\nkJgIMTHBo8sxMaGwExd35P7Y4/j4P29LSrJITLSoXDmBQOAQiYkWiYmhcBUfD65zzWwiIiInEY6g\ndQBIyrPsPhyyTrQtCdh3ujcsVSoBrzdyX7Xq2xcSEkJ7Uo53ov9sz+Y/4DN9/ZF1ebed6PHp7o9f\nd7Lbiba73Sd+7pH1ee+P3E627PGEbqd77PWe/hZ6jStPj5FIQMUi8J6FR0pK0umfVESpd+dycv9O\n7h3s7T8cQWsh0BEYd/gYrVV5tq0BahqGkQwcJDRtOOR0b7h3b0YYyjo5nw/6908iLS09ouNEs5SU\n6Ok/GAxNCebkFMx40dS7HZzcv3p3Zu/g7P6d3DsUTP+nCnLhCFqTgDaGYSwitOvhTsMwegOJpmmO\nMAzjUWA24Cb0rcOtYRhTREREJOrlO2iZphkE+h23OjXP9mnAtPyOIyIiIlLY6IvnIiIiIhGioCUi\nIiISIQpaIiIiIhGioCUiIiISIQpaIiIiIhHisvJ7jRkREREROSHt0RIRERGJEAUtERERkQhR0BIR\nERGJEAUtERERkQhR0BIRERGJEAUtERERkQhR0BIRERGJEAUtERERkQhR0BIRERGJEAUtERERkQhR\n0BIRERGJEAUtERERkQhR0BIRERGJEAUtERERkQhR0BIRERGJEAUtERERkQhR0BIRERGJEAUtERER\nkQhR0BIRERGJEAUtERERkQhR0BIRERGJEAUtERERkQhR0BIRERGJEAUtERERkQhR0BIRERGJEAUt\nERERkQhR0BIRERGJEAUtERERkQhR0BIRERGJEAUtERERkQhR0BIRERGJEAUtERERkQhR0BIRERGJ\nEAUtERERkQhR0BIRERGJEAUtERERkQhR0BIRERGJEAUtERERkQjx2l3AiaSlpVuRHqNUqQT27s2I\n9DBRy8n9O7l3cHb/6t2ZvYOz+3dy71Aw/aekJLlOts2xe7S8Xo/dJdjKyf07uXdwdv/q3bmc3L+T\newf7+3ds0BIRERGJNAUtERERkQhR0BIRERGJEAUtERERkQgpkG8dGobhBt4C6gHZQB/TNNcWxNgi\nIiIidimoPVpdgDjTNBsDA4ChBTSuiIiIiG0K6jxazYBZAKZpLjYMo2EBjSsiDpUdyGZ/9n72Z+87\nerN25LBp13YOZO9nX/Y+DmTvJ8N/iEAwSMAKELACBK0AgWDg8HIwtGwFsCyLOG8c8d4EErwJJPgS\niPfGk+ATUBOOAAAgAElEQVQtRrw3nnjfscfJccmUTShH2YRylI4vg9cdlacsFJECUFD/+osD+/Ms\nBwzD8Jqm6T/Rk0uVSiiQ816kpCRFfIxo5uT+ndw7FP7+s/3ZbE3fyqb9m05423xgMwdzDtpdJgAu\nXJRJKEP5xPJ/uVVKqsQFyRdQs3RNiscWj3gthf3nnl9O7t/JvYO9/RdU0DoA5O3SfbKQBRTIGWxT\nUpJIS0uP+DjRysn9O7l3KDz9W5bFloObMfesIXVPKuaeNazd9yub0zfzR8bOk76uZGxJqiRVo0x8\nCiViS1AytiTFY0L3lUqXw5MbR4nYEpSILUmJmJIU8xXD7fbgcXnwuNx4XJ48y6Gb2xU6yiInmENG\n7iEy/Zlk5GaQ6c8gw595dF2mP4NDuYfYm7WHPzJ28kfGH/yRuZO0jD/YuG8Tq/5YddK6U+LLUr1k\nDaqXOHwreQHVS9Tg/BLVSfAl5PvPs7D83CPFyf07uXcomP5PFeQKKmgtBDoC4wzDaASc/NNGRBzF\nsix2HNpO6p41mHvXkLo7dG/uMTmY++cPR5/bR8XESjSt2JxKSZWpnFiZSknnUSmxEpUSQ/eJMSf/\nwMvvB26sJ5ZYTyylzvH1mf5M0jL+OBrCth7czPr961i/bx3r969j2Y4lLNn+3V9eV7FYJS4uU5e6\nKfWol1KfeimXUr5YBVyuk171Q0SiREEFrUlAG8MwFgEu4M4CGldEosyh3EP8+Mf3LN+xlBU7l7F8\n5zJ2Zab96Tlet5cLStbEKFUbI7kWtZIvolZybaqVOL9QH+8U742nSvGqVCle9YTbswPZbDqw8Wj4\nWrdvLb/vX8dv+35lzsZZzNk46+hzy8SnUC/lUuqlXErdw/eVEisrfIlEmQL5xDJNMwj0K4ixRCR6\nWJbF7wfWs3zH0tBt5zLW7P6ZgBU4+pxKiZVpX70TtZJrUyu5NkZybaqXqEGMJ8bGyu0R64mlZqkL\nqVnqwr9s+yPjD1al/cjKw7dVaSv5ctMXfLnpi6PPKR1XmgblLqdJpeY0q9ScOqXr4nE7+zp3InYr\nvL8aikjUsSyL9fvX8tXmeSzYPJ9lO5awO2v30e2xnlgalLucBuUup2H5K2hY7nIqJFa0seLCo2xC\nWVpVvZZWVa89um5X5i5+Ohy6QgHshz/t+SoeU4LGFZvQpGJzOl7clgru8xW8RAqYgpaI5Mu+rL18\ns/Vrvto8j682f8nm9E1Ht52XVIUWla8+GqwuLnOJI/dURUqZ+DK0rNKallVaH123NX0Li7Z9y6Jt\n37Jw6zfM3jCT2Rtm8vSiJykRW5LGFZrQpFIzmlW6ijqlL9ZUo0iEKWiJyFnxB/18v3MFX23+kvmb\nv+SHP1YQtIIAlIgtSccaXbj6vJZcfV5LzkuqYnO1zlMpqTI9jF70MHoBsCV9M4u2fcv3u5fw5fp5\nzNrwObM2fB56bmJlrq3WluuqtaNppRbEemLtLF2kSFLQEpHTysjNYN6muUxfP5m5G7/gQE7otHge\nl4eG5a7g6vNack2VVlyacpmmpqJM5aTz6GncxP0p95CWln40eM3f9CVfbprD+6tH8v7qkRTzJXL1\neS25rlo7Wle9jjLxZewuXaRIUNASkRM6mJPOFxtnM339VL7cOIcMf+j8duclVaFrzRu45rxWNKvU\nnOKxJWyuVM7GkeDV07gJf9DP0u2LmbXhc2Zv+JwZ66cyY/1UXLi4vPyVXFutHW2rXc+FyYbdZYsU\nWgpaInLU/ux9zN4wk+nrpjB/85dkB7IBqFHyAjpW70KHGp2oW6aejuspIrxuL00qNaNJpWY80+Tf\nrN33G7M2fM6cDTNZtmMJS3cs5vnFT1M7+SK6X9iTbjV7UDnpPLvLFilUFLREHO5g7kGmr5vClLUT\n+XrLV+QGcwGolVybDtU707FGF2ol11a4KuJcLtfRU0s8WP9hdmfuZu7G2cz4fRrzNn7B84v/j+cX\n/x+NKjSh+4U96VSjC6Xiku0uWyTqKWiJOJBlWSzZsZhP1nzE1HWTOZQbui5g3TL16FC9Ex1qdD7h\nuZzEOUrHl+bGWr25sVZv9mXtZfr6qUz4dRyLtn3L4u2LePKbJ2hVpQ3dL+zJtdXaEe+Nt7tkkaik\noCXiINsObuXd1ImMWvEe6/evA0LHXPWrdz89jF5UL1HD5golGpWMK8UtF93OLRfdztb0LUxaO4EJ\nv447+g3GRF8S7at35MZavWlasbn2forkoaAlUsRl+bOY9fsMPkn9mAVb5hO0gsR54uhesyc31b6F\nZpVaHL1ossjpVEqqzAP1H+KB+g+RumcNE34dx8TfxvOpOYZPzTFcULImt9e5i57GTZpaFAFclmXZ\nXcNfpKWlR7woXc3cuf07pfdf95i8//O7TPh1HPuy9wHQoNzl3HN5H1qVu96R3xZ0ys/+RCLZe9AK\nsnT7Yv73y/tMWzeZ7EA2cZ44Ol/QjTsuvpvLyja0fS+XfvbO7B0Kpv+UlKST/gXXHi2RIiRoBfly\n4xxG/DScBVvmA1A2oRwP1H+YXsbNXJhsOP5DV8LP7XLTqGITGlVswnNNX2Js6mg+/HnU0b1cdcvU\n4/Y6d9Htwh4k+hLtLlekQGmPlkM5uf+i2Ht6zgE+WfMxo1aP4Pf96wFoUrEZfer2o+351+N1H/ud\nqij2f6bUe8H1HrSCfL3lKz78+T1m/T6DgBUg0ZdED+NGbq9zNxeVrlNgtYB+9k7tHbRHS0TyYf2+\ntYxc9Q6fpI7mUO5BYj2x9K51K30u6cfFZeraXZ44mNvlPnopph2HtvPxLx/y0S8fHD0T/dXnteSB\n+g/TvNJVtk8rikSSgpZIIWNZFl9tnse7Pw1n7qY5AFQoVpGHLnuUWy+6k9LxpW2uUOTPyherwOOX\nD+DhBo/zxcbZjFj51uGLkM/jkpRLuf/Sv9OxRpc/7XkVKSr0t1qkkAgEA0xfP4VXVwzhl92rAbi8\n/JX8rW4/2lfvhM/js7lCkVPzur20O7897c5vzw87V/Dmj68zff0U+n5xF/9e/Az3XvoAvWrdQjFf\nMbtLFQkbBS2RKJcbyGXib+MZ9v1Q1u77DbfLTbeaN9D3kvupX66B3eWJnJP65Row8roPWb9/HW//\n+F/Gpo5m4DdP8PKyF7nr4nu4q+49urC1FAk6eY5IlMoOZPPhz+/ReMxlPDivHxsO/M7NtW9jUe8V\nvN3mPYUsKRKql6jB4KteZcWtP/Now39gWRZDlr9Eg4/q0P/rR9mcvsnuEkXyRUFLJMpk5GYwYuVb\nXPFxPZ5Y8DA7M3Zw18V/Y8nNP/LqNf/V2dulSEpJSGHAFf/k+9t+4YVmgykTn8L7q0fSaHR9+n/9\nKNsPbrO7RJFzoqlDkShxMCed91aP5O2Vb7ArcxcJ3mLcW+9B7rv0QcoVK293eSIFopivGH0u6ccd\nF/dh4m/jGbLsJd5fPZIxaz7ijjp38+Blj1I2oazdZYqcMQUtEZtl+bP44OeRvLZiCHuy9pAUU5xH\nGzzB3y65T98gFMfyur30NG6i6wU3MM78hKHL/8M7P73FR798wN11+3J//b+THKd/HxL9NHUoYpNA\nMMDY1NE0GdOApxY+SW7QT/8rBvH9rasZcOW/FLJEAJ/Hx80X3cZ3N3/PSy2GUjy2BG/88CoNP7qE\nl5Y+z/7Dl5cSiVYKWiIFzLIs5myYSctxTfn7vHtJy/yDe+s9yLJbVvJYw/6UiC1pd4kiUSfWE3v0\nWMXnmr5InDeOV5YPpuHHl/Dq8pc5mOPcM59LdFPQEilAS7cvodPkttzy+Y2Ye1PpVetmvuv9Pc80\n/bemQUTOQLw3nr717mfZLT/xz0bP4MbFi0uf44rRl/K/n98nEAzYXaLInyhoiRQAc08qt828iQ6T\n2rBk+3e0rXY9X934Ha+3HE7lpPPsLk+k0CnmK8bfL3uE5beu4onLB5KRm8HjCx6i5bhmLNg83+7y\nRI4K28HwhmGUAD4GigMxwKOmaX5nGEYjYBjgB+aYpvlMuMYUiXY7D+3gxSXPMdYcTdAKckX5Rvyz\n8TM0qtDY7tJEioSkmOI8cflAbr3ojtC/tdTR9JjWmWurtuXpJs9Ts9SFdpcoDhfOPVqPAl+apnkV\ncAfw5uH1bwO9gWbAlYZh1A/jmCJRKSeQw5s/vE7jMQ0Yk/oRNUteyP/ajWVa19kKWSIRUL5YBYa1\nfIu5Pb6macXmzNk4i6s+bcST3zzBnqzddpcnDhbOoPUq8M7hx14gyzCM4kCsaZrrTNO0gNlA6zCO\nKRJ15m2ay9WfNuaZ7/6Jz+1lcItX+erG72h7/vW4XC67yxMp0uqm1GNi5+l80HYM5yVVYeSqd7hy\ndH1e/e5VcgI5dpcnDuSyLOusX2QYxt3AI8etvtM0zWWGYZQHZgIPA+uACaZpXnn4dXcB1U3T/Oep\n3t/vD1her+es6xKx0/q963l09qNMMafgdrnp16Afz7V8juT4ZLtLE3GknEAOby59k2e/fpZ9Wfuo\nmVyTl9u8TCejk37pkXA76V+ocwpaJ2MYRl1gLPC4aZozD+/RWmya5kWHtz8E+EzTHHKq90lLSw9f\nUSeRkpJEWppzvw7s5P7D3XtGbgav//AKb/4wjOxANo0qNOHfzQdTt8wlYRsjnPSzV+9OsztzN2+u\nHsrw5cMJWAFaV7mWF1sMoWrxanaXViCc/LOHguk/JSXppEErbFOHhmFcBIwHepumORPANM0DQI5h\nGDUMw3AB1wHfhGtMETtZlsXUtZNo+klDXlk+mFJxybzdZhRTusyM2pAl4kSl40vzxvVvsODGxTSv\nfDVzN82hxdgrGbZiqKYTJeLCeYzWi0AcMMwwjK8Mw5hyeH0/YDSwFPjBNM0lYRxTxBbmnlS6T+1I\nnzm3k5bxB3+v/yiLeq+gW80empIQiVIXJht81nEKw1uPpJgvkX8veYZW45qxeNsiu0uTIixsp3cw\nTbPzSdYvBhqFaxwRO2UHsnn9+1d4bcUQcoO5tKl6Hc81fZHqJS+wuzQROQMul4vuF/akVZU2/HvJ\ns/zv5/foNLktN9W6hacaP6dLX0nY6YSlImdo2Y4ltB7XnJeXvUiZ+BQ+bPcJo9uPV8gSKYRKxpXi\n5ateZUa3L6hTui6fpH5MkzGXMWbNRwStoN3lSRGioCVyGgdz0nnymyfoMPFazL2p3FHnbr7ptYR2\n57e3uzQRyaeG5a/gix4LeK7pi+QEc3l4/v10mXw9qXvW2F2aFBEKWiKnMHfjbJqPvZKRq96hRskL\nmNp1NoOvepXisSXsLk1EwsTr9tK33v0svGkZ7at3YvH2RbQc15SXljyng+Ul3xS0RE5gV+Yu+n1x\nF71n9GBnxg4ebfAE83ou1FndRYqwiomVeL/tx3x8/aeUT6jAKyte5trPrmZV2kq7S5NCTEFLJA/L\nshhnfkKzTxoy8bfPuKxsA+b2+IYBV/6LOG+c3eWJSAG4tlo7FvT6jlsvupNfdq/mugnXMHjpC9q7\nJedEQUvksO0Ht3HTjO488GVfsvxZPN/0JWZ0m8tFpevYXZqIFLCkmOIMvXoYn3aYRLmE8gxZ/hJt\nJ7Rk9a5VdpcmhYyClggw+bcJXPVpo9B1Cs9ryde9lnBPvfvwuHUpKBEnu6ZKKxbc+B231L6d1bt+\n4trPrmLo8v+QG8i1uzQpJBS0xNH2Ze2l3xd3c88Xd5ITyGFwi1f5tMMkqhSvandpIhIliseW4JVr\n3mBshwmkxJflP0v/TbuJrfhl9892lyaFgIKWONaCzfO56tPGTPxtPA3KNWRez2+54+K7dWZ3ETmh\nllXa8HWvxdxU6xZ+SvuRNuNb8Oryl/EH/XaXJlFMQUscJzM3k39+258e0zqTlvkH/a8YxLSuc3Ti\nURE5rRKxJRnW8i3GtB9P6fgyvLj0OdpPbM36/evsLk2ilIKWOMrKP36gwYgGjPhpODVLXsjn3eby\nWMP+eN1huxqViDhA66rX8fWNi+lxYS9++ON7Wo1rzqepY7Asy+7SJMooaIkj+IN+Xlk+mHYTW7Fm\n1xr+Vrcfc3t+w6VlL7O7NBEppErGleLN1iN4u80o3C43D87rx71z+5Cec8Du0iSK6Nd4KfI27P+d\ne+f2YcXOZVQoVpEPu37ApcV1nXMRCY9uNXvQoNzl9Pvibib+Np7lO5fxduuRNCx/hd2lSRTQHi0p\n0qatm0Kr8c1ZsXMZXS/ozoIbv6NNjTZ2lyUiRUzV4tWY2mUWjzR4nM0HNtJx0nW8tmIIgWDA7tLE\nZgpaUiRlB7IZ+M3j3D37VgJBP6+3HM47175PybhSdpcmIkWUz+Nj4JVPMbHzdMomlOOFJc9yw9RO\nbD+4ze7SxEYKWlLkrN+/jvYT2zBq1QhqJddm9g1f0avWzXaXJSIO0bRSc+bfuJB253dg4bZvuPrT\nxny+frrdZYlNFLSkSJmydiKtx7Xgp7Qf6V3rVmZ1n4+RXMvuskTEYZLjSvNB29EMbvEqmf5M7pjV\nmycWPEKmP9Pu0qSAKWhJkZDlz+KJBY/wtzl3ELSCvNlqBK+1fJMEX4LdpYmIQ7lcLu64+G7m9FhA\n7eQ6fPjzKDpMvJYN+3+3uzQpQApaUuit2/cb7Sa04sOfR1E7uQ5f9FhAD6OX3WWJiABQK7k2s26Y\nxy21b2fVrpW0+ewqvtgwy+6ypIAoaEmhNuHXcbQefxU/717FrRfdyawb5lGz1IV2lyUi8ifx3nhe\nueYNXrvmTbL8mdz8eU9eWvq8vpXoAApaUihlB7J5YsEj3Du3DwDvtHmPoVcPI94bb3NlIiIn17v2\nrXzebS5VilfjleWDuWlGd3Zn7ra7LIkgBS0pdHYe2kHXye358OdR1Cldly97fE3XmjfYXZaIyBmp\nm1KPuTcsoE3V6/hq8zxaj2/O9zuX212WRIiClhQqS7cvofX4FizfuZRuNW9gRrcvdDFoESl0SsaV\n4qPrP2XgFf9i28GtdJx0He+vHqlrJRZBClpSaPzv5/fpOuV60jL/4JkmLzC89Sh9q1BECi23y80j\nDZ/g046TSIpJov/Xj/LAl33JyM2wuzQJo7Bf69AwjFrAEqCcaZpZhmE0AoYBfmCOaZrPhHtMKdqy\nA9k8+c0TfPTLByTHJfPutR/SvPJVdpclIhIWV5/Xkrk9vqHP7NsY/+tYVu9axfvtPqZ6iRp2lyZh\nENY9WoZhFAeGAtl5Vr8N9AaaAVcahlE/nGNK0bbj0Ha6TL6ej375gIvLXMKcGxYoZIlIkVM56Tym\ndJ3FHXXuZs2en2n72TUs2Dzf7rIkDMIWtAzDcAEjgCeBjMPrigOxpmmuM03TAmYDrcM1phRtS7Yv\npvX4FqzYuYzuNXsyvescqhSvandZIiIREeuJZfBVrzLsmrc4lHuIXtO7MWrVCLvLknw6p6lDwzDu\nBh45bvVGYKxpmisNwziyrjhwIM9z0oHq5zKmOIdlWXz483sM+vYfBK0gzzV9kXsuuQ+Xy2V3aSIi\nEXdT7VuoXvIC7px1MwO/eZzUPWt4odlgfB6f3aXJOXCF6xsOhmGsBbYcXmwELAU6AItN07zo8HMe\nAnymaQ451Xv5/QHL6/WEpS4pXHICOdw/435G/jCSMgllGHfDOK45/xq7yxIRKXAb922k89jOrNy5\nkqurXc1nPT6jdEJpu8uSEzvpnoCwBa28DMPYANQ6fDD8j0B3YD0wA3jGNM0lp3p9Wlp6xL/fmpKS\nRFpaeqSHiVrR2P/erD3cOesWFm37lrpl6vFBu9Gcl1Ql7ONEY+8Fycn9q3dn9g6Ft/+DuQd5YG5f\nPv99GlWLV+Pj68dhJNc6q/corL2HS0H0n5KSdNKgVRCnd+gHjCa0h+uH04Uscab1+9Zy/cTWLNr2\nLe2rd2Ja19kRCVkiIoVJoi+R99p+xKMNnmDjgQ20m9CKuRtn212WnIWwn94BwDTNankeLyY0lShy\nQou3LeL2mTexN3svD9Z/hEGNnsbt0ineREQgdL6tAVf+CyO5Ng/Nu4+bZ/TkqcbPcd+lD+rY1UJA\n/5uJrcaZn9B9akfSc9N59er/8q/GzyhkiYicQNeaNzCly0zKJpTjme/+yd/n3Ut2IPv0LxRb6X80\nsUXQCvLSkud44Mu+JPiK8WmHSdx80W12lyUiEtXql2vAFz0WUL/sZXxqjqHblA66KHWUU9CSApfl\nz6LfF3fxyoqXqVb8fD7vNlcnIRUROUPli1VgcpeZdL2gO8t2LKH9xNb8vn+93WXJSShoSYFKy0ij\n25QOTF47kSsrNGZm93nULHWh3WWJiBQq8d54hrcZxd/rP8r6/etoP7E1P+xcYXdZcgIKWlJgUves\nod2ElizfuZQbLryRzzpNpXS8zgkjInIu3C43/2z8f/ynxSvsydpD1yntmbNhpt1lyXEUtKRAfLNl\nAe0ntmFT+kb6XzGIN1uNINYTa3dZIiKF3p0X9+GDtmOwsLht5k18sHqU3SVJHgpaEnFT1k6k1/Ru\nZPuzeLvNKB5r2F9fSRYRCaO251/PpM4zSI5L5h9fP8Lz3/0fQStod1mCgpZE2Mif3uaeOXcS541n\nbMeJdKvZw+6SRESKpMvKNWRGt7lUL1GD1394hfvn3kNOIMfushxPQUsiwrIsXlj8LE9++w9SEsoy\nucvnNKvUwu6yRESKtPNLVGdGt7k0KHc5E34bR6/p3diXtc/ushxNQUvCzh/088j8B3jt+yFUL1GD\nGd2+oG6ZS+wuS0TEEUrHl2ZCp2m0O78D3279mubvN2dr+ha7y3IsBS0Jq4zcDO6Y2ZsxqR9xaUp9\npnf7gqrFq9ldloiIoyT4Enjvuo/oU7cvq/9YTbuJrViz+xe7y3IkBS0Jmz1Zu7lhaifmbJzF1ee1\nZGKXGZSJL2N3WSIijuRxe/h3s8EMaTOEHYe203lyW1bsXGZ3WY6joCVhsTV9C50mtWX5zqV0r9mT\nj68fR6Iv0e6yREQczeVy8ViTx3i95XDSc9LpPqUTCzbPt7ssR1HQknxL3bOG6ye25te9Jv3qPcCb\nrUcQ44mxuywRETmsV62bGXXdR/iDudw8owcz1k+zuyTHUNCSfFmyfTEdJ13H9kPbeLrx8zzb9AXc\nLv21EhGJNtdX78AnHSbgdfu4e/atjE0dbXdJjqD/EeWcfbV5Hj2ndeZQ7kH+2+od7q//d7tLEhGR\nU2he+Somdp5GiZgS/H3evbyz8k27SyryFLTknMzeMJNbZvQkaAX5oO1oeho32V2SiIicgcvKNWRK\n11mUSyjPvxYO5KWlz2NZlt1lFVkKWnLWJv82gTtn3YzX7WV0+/FcW62d3SWJiMhZqJVcm+nd5lC1\neDVeWT6YQd/+Q5fsiRAFLTkrY1NH02/u3cR7ExjbcRItKl9td0kiInIOqhavxvSuc6idXIeRq97h\ngS/7khvItbusIkdBS87Ye6vf5e/z7qVETAkmdJpKowqN7S5JRETyoVyx8kzuMoMG5S7ns18/5e7Z\nt5Llz7K7rCJFQUvOyH9/GMaArx8jJb4sk7p8zqVlL7O7JBERCYNSccmM7zSFqypfw6wNn3PzjB4c\nyj1kd1lFhoKWnJJlWfxn6b959rt/UbFYJaZ2nclFpevYXZaIiIRRoi+Rj9uPo935Hfhm6wJumdGT\ng7kH7S6rSFDQkpOyLIv/W/RPhi7/D1WLV2Nq11nUKFnT7rJERCQCYj2xjLz2QzpU78zCbd/Qe/oN\nHMxJt7usQk9BS04oaAX5x9ePMnzlG9QseSFTu8yiSvGqdpclIiIR5PP4eKfNe3Su0Y3F2xfRa3p3\n0nMO2F1WoaagJX8RCAZ4aN59fPjzKOqUrsvkLjOpkFjR7rJERKQA+Dw+hrcZSbeaN7B0x2J6TuvK\ngez9dpdVaHnD9UaGYXiAV4CGQCzwf6ZpTjcMoxEwDPADc0zTfCZcY0r4BYIB/j7vXsb/OpbLyjbg\nkw4TKBWXbHdZIiJSgLxuL2+2ehePy8v4X8fSY1pnxnWcTInYknaXVuiEc4/WrYDPNM2mQGfggsPr\n3wZ6A82AKw3DqB/GMSWMAsEAD8+/n/G/jqVBuYaM6zhZIUtExKE8bg+vtxxOr1o388Mf33PD1M7s\nzdpjd1mFTjiD1nXAVsMwZgDvAtMMwygOxJqmuc40TQuYDbQO45gSJkEryKNfPcin5hguK9uATztM\nonhsCbvLEhERG3ncHl675k1uqX07K9N+oPvUTuzJ2m13WYXKOU0dGoZxN/DIcavTgCygA9ACeJ/Q\nnqy8R9GlA9VP9/6lSiXg9XrOpbSzkpKSFPExotmR/oNWkHum3cMnqR/TsGJDvrj1C0rGFe3dw/rZ\nO7d/9e5cTu4/v71/2OM9is2I450V79BzRmfm3jqXlGIpYaou8uz82Z9T0DJNcxQwKu86wzDGAtMP\n77laYBjGhYRCVt7ukoB9p3v/vXszzqWss5KSkkRamnO/tnqk/6AV5IkFD/PRLx9QL6U+Y9pOIDfd\nQ1p60f2z0c/euf2rd2f2Ds7uP1y9P3vFYHKyA7y/eiRXvXc1n/1/e/ceJ2Pd/3H8NbuLZe06ZJ0q\nHeT+SEWlokKKiEgnqaWQitJR3aK7k87cqVQqRBI5pYOcWscipw48qrv6KioStc5yWGt3fn/M8Ju2\ntcTMXjsz7+dfc32va+Z6f/e7e81nruu711z2Iellin+xVRRjX1ghF85LhwuA1gBmVg9Y7ZzbBuwx\ns5pm5iNweXF+GPcpRyDPn0fvj3vx1rcjOa1SPSa0fY/yyRW8jiUiIsWQz+fjmcYDufm0Hny36Vuu\n/OBSsnZmeR2r2AtnoTUM8JnZYmAo0CPY3gMYAywFljnnloRxn3KY/H4/fT65l1HfjuDUSnV557IP\nNPFdREQK5fP5eKJRf7rXvQ23+Xvaf6gJ8gcTtts7OOeygRsLaF8MNAzXfuTI+f1+7ph+ByOD98lS\nkSUiIofK5/Px2PlPsydvD2988zodPryCdy6brH+gOgDdsDTO+P1+HlxwP4M/G8zJFU/hncsmUzH5\nKGmQhe8AABUfSURBVK9jiYhIFPH5fDzd+Fmuq92J5VnLyJjaXt+NeAAqtOKI3+/noU/7MOzr1zi1\n8qlMavchR5VWkSUiIv9cgi+B55q+tP8O8jdMu5Zde3d5HavYUaEVJ/x+P48vfoShX72KVajN7Btm\nU6l0Ja9jiYhIFEtMSOSli4bQ+oS2LFj7CV1ndCQ7N9vrWMWKCq04MejLgby87AVOKl+LSe2mUDml\nsteRREQkBpRILMHQFm/QvEYL5qyexc2ZXcjJzfE6VrGhQisODP96CE8teYxjyh7LxLYfULmMiiwR\nEQmfkoklGXHJaBof05QZP02l5+ybyc3L9TpWsaBCK8aN+34Mfef/m8plqvBOu8kcnXqM15FERCQG\nJSclM6rVWBpUO5f3f3yXu+f2JM+f53Usz6nQimFTVk7m7rk9KV+qPBPavs+J5Wp6HUlERGJYSokU\n3r50YuA7c93b3P/Jvfj9fq9jeUqFVoyau3o23Wd2pXRSGca1eZc6R53idSQREYkDqSXTGNfmXU6t\nVJc3/zechz/tG9fFlgqtGLR43SK6zMgg0ZfI6NbjObPKWV5HEhGROFI+uQIT2r6PVajNkK9eYcBn\nT3kdyTMqtGLMV1nL6Ti1PTl5OQxvOYrzj27sdSQREYlDlUpX4p3LJnNc2vEM/Lw/r3/1mteRPKFC\nK4as2OTo8OEV/LlnO680G8bFx1/idSQREYljVVKqMqHt+6SXrswDC3rz7g8TvY5U5FRoxYjV236h\n/Yft2Lh7IwObvsjlta7yOpKIiAgnlDuR8W3fI61kOW6f3Z05q2d6HalIqdCKAet3rOOqyW1Zt+M3\n+p33FJ3qdPY6koiIyH6nVjqN0a3Hk+RL4sYZ1/PZ+iVeRyoyKrSi3NbsLXT48Ap+2fYz9551P7ee\nfrvXkURERP6mYfXzGNbyTbJzs+k4tT3fb/rO60hFQoVWFNu9dzc3TL+O7zZ9y02ndaf32Q94HUlE\nROSAWh7fiucvfJktwZMEa7av9jpSxKnQilK5ebncNutmFv32Ke1qXskTjfrj8/m8jiUiIlKoa2t3\n5NHznmTdjt+45sPL2bBrg9eRIkqFVhTy+/38Z0Fvpqz6gPOrN+bl5kNI8GkoRUQkOtx2+h3cccY9\nrNzyIxlTruLPPdu9jhQxeneOQoO+HMiIb4ZR56hTebPV25RKLOV1JBERkX/kwYaPklH7epZnLaPz\n9Ayyc7O9jhQRKrSizNjvRvPUksc4puyxjGszibRS5byOJCIi8o/5fD6ebTqIVie0Yf7aj7l15k3k\n5uV6HSvsVGhFkZk/z6DXvDuoUKoC49u+R9WUal5HEhEROWxJCUkMuXgE51VvxJRVH9Bn/n0x972I\nKrSixBe/f8ZNmZ0pmViS0ZdOoFaFf3kdSURE5IglJyUzqtXY/V9C/dKyF7yOFFYqtKLAj5t/oOPU\n9mTnZjO0xUjOrtrA60giIiJhk1aqHG9fOpGjyx7DE4sfiamv6lGhVcyt37GODlOuYNPuTTx7wSBa\nHt/K60giIiJhVzWlGmPbTCKtZDnunH0rC9cu8DpSWKjQKsa2ZW/luilXs2b7au4/5z/6ah0REYlp\ntSuezBuXjMaPn84zMnCbvvc60hFToVVM7cndQ9cZnfjfxq/pfEo3etXv7XUkERGRiGt8zAW8cOFg\ntmZvIWPq1fy+Y73XkY5IUrheyMzKAeOAskA20Mk5t97MGgKDgL1ApnOuX7j2Gav8fj+95t3B/LUf\n0+qENjzT+Fnd9V1EROJGe7uWX7ev4emlj9Nx2jW8f/k0ypYo63WswxLOM1pdgK+dc42B8cC/g+2v\nARlAI6CBmZ0Rxn3GpOe+GMAEN5YzK9fn1eavk5iQ6HUkERGRInV3/fvodHJnvspazi0fdWFv3l6v\nIx2WcBZaXwOpwcdpQI6ZpQGlnHMrnXN+4COgeRj3GXMmrZhA/6VPcmxqDUa1Hk+ZEmW8jiQiIlLk\nfD4f/Zs8x0U1mjNrdSb3f3JvVN5jy3c4oc2sG3BPvuaewAgClw0rAo2BXcAk51yD4PNuBE50zj1Y\n2Ovv3ZvrT0qKv7M4C1YvoNmoZiQnJbPwxoWcUvkUryOJiIh4anv2dpqMbMLy9ct5utnT9GnUx+tI\nBTng/J7DmqPlnBsODA9tM7N3gQHOuSFmVheYROByYWrIZqnAloO9/ubNOw8n1j+Snp5KVlbx+RLL\nVVtX0m5SO3Lzcnm9xSgq+2pENF9x639Riue+Q3z3X32Pz75DfPc/Fvo+quV4Wk1qRt/ZfSnvS+eq\nf11zyM8tiv6np6cecF04Lx1uBrYGH/8BpDnntgF7zKymmfmAlsD8MO4zJmzavZGMKVezafcmBlzw\nPE2PvcjrSCIiIsVG6D227ppzW1TdYyuchdZDwA1m9gnwHnBzsL0HMAZYCixzzi0J4z6jXnZuNl1n\ndGLV1pXcfsbdXF+ni9eRREREip3aFU9mZKsx+++x9cPmFV5HOiRhu72Dc+43oHUB7YuBhuHaTyzx\n+/30mnsHi377lDYntuPBho96HUlERKTYanR0E56/8GVun92dTtOuYcZVc6iQXNHrWIXSDUs9NPDz\n/kxcMY76Vc5icPOhJPg0HCIiIoW5xq7j7jPv46etq+j20Q3k5OZ4HalQemf3yDsrxjPgs6eokXoc\nb7YaR+mk0l5HEhERiQp9GjxI6xPasmDtJ/SZX7xv+6BCywOLf1vI3XN6klayHGMunUjlMpW9jiQi\nIhI1EnwJDG4+lFMr1eWtb0cy7KtXvY50QCq0itiqLT/Sefp15JHH8JajsIq1vY4kIiISdVJKpDC6\n9Xgql6nCwwsfYPYvmV5HKpAKrSK0LXsr10+7ls3ZmxnQ5HkuOPZCryOJiIhEreplj2ZUq7GUTCjJ\nzZld+X7Td15H+hsVWkUkNy+XHjO78cOWFfSodzud6nT2OpKIiEjUO7PKWbx40av8mbOdTtM6sGHX\nBq8j/YUKrSLy5JJ+zFqdyYXHNuPhcx/zOo6IiEjMuLzWVdx3Vh9Wb/uZrjM6kp2b7XWk/VRoFYEJ\nbiwvL3uBmuVPYmiLN0hKCNvty0RERAS47+w+tKt5JUvWLaL3x/cUm/9EVKEVYV/8/hn3zruTtJLl\neKvVeMqVKu91JBERkZiT4Etg0EWvcHr6GYz9fjSDl7/odSRAhVZErfvzN7pM70hOXg5DW7zBSRVq\neR1JREQkZpUpUYZRrcdRLaU6jy96mBk/TfM6kgqtSNm1dxddZmTw+871PHLuE1xUo7nXkURERGJe\n1ZRqvNV6HMlJyfSY2Y0VG739TkQVWhGw7zsMl/3xJR0sgx71enodSUREJG7UTT+dwc2G4SeP7zd8\n72kWzcqOgJeXD2LSDxOoX+Vs/nvBC/h8Pq8jiYiIxJU2NS+j5fGtqF61IllZ2z3LoTNaYTbz5xk8\nsegRqqVUZ+QlY0hOSvY6koiISFwqkVjC6wgqtMJpxSZH95ndKJVYijdbvU2VlKpeRxIREREP6dJh\nmGzevYnrp3fgz5ztvHbxcE6vfKbXkURERMRjOqMVBrl5udyS2ZWftq7irjPv5cpa7b2OJCIiIsWA\nCq0weHrJ43z861xaHHcJfRs85HUcERERKSZUaB2hKSsn8+Ky5zih3IkMbj6UBJ9+pCIiIhKgquAI\n/LB5BXfM6UGZpDKMvORtfb2OiIiI/IUmwx+mP/dsp8v0DHbk/MmQi0dw8lF1vI4kIiIixYzOaB0G\nv9/PnXNu44ctK+herydX1Lra60giIiJSDKnQOgwvLx/ElFUfcG7183m44WNexxEREZFiSoXWP/TJ\nr/N4cvGjVE2pxrAWbxaLu86KiIhI8XREc7TM7AqgvXMuI7jcEBgE7AUynXP9gu2PAJcG2+92zi09\notQe+XX7GrpndiXRl8iIlm9RuUxlryOJiIhIMXbYhZaZDQJaAstDml8DrgJWAVPN7AzAB1wANACO\nBSYBZx/ufr2ye+9ubpzRiY27N9K/yXOcVfUcryOJiIhIMXcklw4XArfuWzCzNKCUc26lc84PfAQ0\nBxoROLvld86tBpLMLP1IQnuh7/z7WJ61jGtrd6TLKd28jiMiIiJR4KBntMysG3BPvuauzrnxZtY0\npC0N2BayvB04EdgNbMzXXg7IOpzAXnjr25GM+W4UddNPp3+T5/D5fF5HEhERkShw0ELLOTccGH4I\nr7UNSA1ZTgW2AHsO0H5AFSqUISkp8RB2eWTS01MPus3StUvpO/8+KpauyAcZ71GjfOzMyzqU/seq\neO47xHf/1ff4Fc/9j+e+g7f9D9sNS51z28xsj5nVJDBHqyXQj8AE+AFm9ixwDJDgnNtQ2Gtt3rwz\nXLEOKD09lays7YVus2HXBq6YcCU5uTm82mw4KTlHHfQ50eJQ+h+r4rnvEN/9V9/js+8Q3/2P575D\n0fS/sEIu3HeG7wGMARIJzMtaAmBm84FFBOaE9QzzPiMiz59Hz1k389uOtfQ95yEurNHM60giIiIS\nZY6o0HLOzQPmhSwvBhoWsN2jwKNHsq+iNuiLgcxdM5vmNVpwV/17vY4jIiIiUUg3LC3AgrWf0P+z\nJ6mecjQvNRtCgk8/JhEREfnnVEHk88fOP+gxsxsJvgSGthjJUaWP8jqSiIiIRCkVWiFy83K5dWY3\n/tj5Ow827Mc51Rp4HUlERESimAqtEM9+/gzz137MJSdcyq31bvc6joiIiEQ5FVpB89bM4bnPB1Aj\n9ThevPAV3ZRUREREjpgKLWD9jnXcNusmkhKSGNZiJOWTK3gdSURERGJAuO+jFXX25u3llsyubNi1\ngacaDeCMKvW9jiQiIiIxIu7PaPVf+iSL1y2kbc3L6XZad6/jiIiISAyJ60Jr1i8fMejLgRyfdgLP\nN31J87JEREQkrOK20FqzdQ09Z91CqcRSDG85irRS5byOJCIiIjEmLudo5eTm0OGdDmzO3sx/L3iB\n09LreR1JREREYlBcntEa9e0bLPp1EVfWupob6nT1Oo6IiIjEqLg8o3V21XO4q8Fd3HVab83LEhER\nkYiJy0KrbvrpNKvTmKys7V5HERERkRgWl5cORURERIqCCi0RERGRCFGhJSIiIhIhKrREREREIkSF\nloiIiEiEqNASERERiRAVWiIiIiIR4vP7/V5nEBEREYlJOqMlIiIiEiEqtEREREQiRIWWiIiISISo\n0BIRERGJEBVaIiIiIhGiQktEREQkQpK8DlAUzOwKoL1zLiO43BAYBOwFMp1z/fJtXxoYDVQGtgOd\nnXNZRZs6fMysD3BJcLE8UNU5VzXfNoOARgT6C9DOObe16FJGjpn5gF+BH4JNi5xzffNtczPQncDv\nxBPOuSlFmzIyzKwcgd/lNKAk0Ms5tyjfNjE19maWALwC1AOygZuccz+GrG8LPExgrEc454Z5EjRC\nzKwEMAI4HihF4Pd5csj6e4CbgH3HtO7OOVfUOSPFzL4EtgUXf3LOdQ1ZF+tj3wXoElxMBk4ncLzf\nElwfk2NvZg2A/s65pmZ2EjAS8APfAD2dc3kh2xZ6fIiEmC+0gm8iLYHlIc2vAVcBq4CpZnaGc25Z\nyPpbga+dc4+a2bXAg8BdRZU53JxzzwDPAJjZFKB3AZvVB1o65zYUZbYiUhP40jnXtqCVZlYVuBM4\ni8DBaYGZzXTOZRdhxkjpBcx2zr1gZgaMBc7Mt02sjf3lQLJz7tzgh6qBQDvYX4Q8D5wN7AA+NbPJ\nzrnfPUsbfp2Ajc65682sIoFj3+SQ9fWBG5xzX3iSLoLMLBnwOeeaFrAu5sfeOTeSQJGBmQ0mUExu\nCdkk5sbezHoD1xMYU4DngAedc/PM7DUCf/vvhTzlgMeHSImHS4cLCRROAJhZGlDKObfSOecHPgKa\n53tOI2BG8PH0AtZHJTO7EtjsnMvM154A1AKGmtmnZnajJwEjpz5wtJnNNbNpwYIj1DnAp8657OCZ\nnB+BukWeMjKeB4YEHycBu0NXxujY7//7dc4tJlBA73My8KNzbrNzbg+wAGhS9BEjaiLwUPCxj8DZ\nm1D1gb5mtsDM+hJb6gFlzCzTzOYE30j3iYexB8DMzgJOcc4NzbcqFsd+JXBlyHJ94OPg44Levws7\nPkREzJzRMrNuwD35mrs658abWdOQtjT+/7QyBC6XnJjveWnA1pD15cIYNaIK+Tl8BvQFrivgaSnA\nSwQ+CSQCc83sc+fcVxENGwEH6H9P4Gnn3EQza0TgUtrZIetDxxuibMz3KWzsg2ftRgN351sfM2Mf\nIv945ppZknNubwHronKsC+Oc+xPAzFKBdwickQ81DhhM4Dj4npm1iZVL5cBO4FngdQIfIKabmcXL\n2Id4AOhXQHvMjb1zbpKZHR/S5AueRIGCx7iw40NExEyh5ZwbDgw/hE23Aakhy6nAlkK2KWh9sXWg\nn4OZ1QG2HOBa9E5gkHNuZ3DbOQQ+GUbdm21B/TezMgQ/1TvnFphZdTML/WM8lN+JYq+QsT+NwAH2\nPufcx/lWx8zYh8g/ngkhB9GYGOuDMbNjCVwuecU593ZIuw94Yd8cPDObCpwBRPWbbYgVBM5a+YEV\nZrYRqAasIX7Gvjxgzrm5+dpjfez3yQt5fLD3d/jr8SEi4uHS4V8457YBe8ysZvAXryUwP99mnwKt\ng49bFbA+GjUncBq1IP8iMF8hMTiPoRHwZZEli7xHCJ7JMbN6wJqQIgtgKdDYzJKDk8dPJjCJMuoF\nC+yJQIZzrqDxj8Wx3//3G7x09HXIuu+AWmZW0cxKErh0tOjvLxG9zKwKkAnc75wbkW91GvCNmZUN\nHv8uAmJmvg5wI4E5N5hZdQL9XRdcF/NjH9QEmF1Ae6yP/T7LQq5iFfT+XdjxISJi5ozWP9QDGEPg\nUkmmc24JgJllAm2AV4E3zWwBsAfI8CpoGBkw8y8NZr0IfPqbbGZvAYuBHGCUc+5/HmSMlGeA0WZ2\nKYEzW13gb/1/kcAfZALwH+fc7gO9WJR5msAE/0HBqWlbnXPtYnzs3wMuNrOFBOYodTWzDKCsc25o\nsO8fERjrEc65tR5mjYQHgArAQ2a2b67WMCAl2P8HgLkE/uNqtnNumkc5I2E4MDJ47PYTKLyuMbN4\nGXsIHOtX7V/46+9+LI/9PvcCw4LF9HcELp9jZqMIXEb/2/Eh0oF8fr//4FuJiIiIyD8Wd5cORURE\nRIqKCi0RERGRCFGhJSIiIhIhKrREREREIkSFloiIiEiEqNASERERiRAVWiIiIiIRokJLREREJEL+\nD2s04O7N06jmAAAAAElFTkSuQmCC\n",
    "text/plain": [
    "<matplotlib.figure.Figure at 0x118fcab38>"
    ]
    },
    "metadata": {},
    "output_type": "display_data"
    }
    ],
    "source": [
    "fig, ax = plt.subplots(2, sharex=True, figsize=(10, 8))\n",
    "ax[0].plot(x, y, 'b')\n",
    "ax[1].plot(x, d, 'g');"
    ]
    },
    {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
    "### The Data"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 26,
    "metadata": {
    "collapsed": true
    },
    "outputs": [],
    "source": [
    "# input dataset (features)\n",
    "# layer 0\n",
    "l0 = np.array([[0, 0, 1],\n",
    " [0, 1, 1],\n",
    " [1, 0, 1],\n",
    " [1, 1, 1] ])"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 27,
    "metadata": {
    "collapsed": true
    },
    "outputs": [],
    "source": [
    "# output dataset (labels) \n",
    "y = np.array([[0,\n",
    " 0,\n",
    " 1,\n",
    " 1]]).T"
    ]
    },
    {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
    "### Single Step"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 28,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "array([[-0.16595599],\n",
    " [ 0.44064899],\n",
    " [-0.99977125]])"
    ]
    },
    "execution_count": 28,
    "metadata": {},
    "output_type": "execute_result"
    }
    ],
    "source": [
    "# initialize weights randomly with mean 0\n",
    "np.random.seed(1)\n",
    "weights = 2 * np.random.random((3, 1)) - 1\n",
    "weights"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 29,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "array([[-0.99977125],\n",
    " [-0.55912226],\n",
    " [-1.16572724],\n",
    " [-0.72507825]])"
    ]
    },
    "execution_count": 29,
    "metadata": {},
    "output_type": "execute_result"
    }
    ],
    "source": [
    "np.dot(l0, weights)"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 30,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "array([[ 0.2689864 ],\n",
    " [ 0.36375058],\n",
    " [ 0.23762817],\n",
    " [ 0.3262757 ]])"
    ]
    },
    "execution_count": 30,
    "metadata": {},
    "output_type": "execute_result"
    }
    ],
    "source": [
    "l1 = sigmoid(np.dot(l0, weights))\n",
    "l1"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 31,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "array([[-0.2689864 ],\n",
    " [-0.36375058],\n",
    " [ 0.76237183],\n",
    " [ 0.6737243 ]])"
    ]
    },
    "execution_count": 31,
    "metadata": {},
    "output_type": "execute_result"
    }
    ],
    "source": [
    "e = y - l1\n",
    "e"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 32,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "0.30994584990928159"
    ]
    },
    "execution_count": 32,
    "metadata": {},
    "output_type": "execute_result"
    }
    ],
    "source": [
    "(e ** 2).mean()"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 33,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "array([[ 0.19663272],\n",
    " [ 0.23143609],\n",
    " [ 0.18116102],\n",
    " [ 0.21981987]])"
    ]
    },
    "execution_count": 33,
    "metadata": {},
    "output_type": "execute_result"
    }
    ],
    "source": [
    "sigmoid(l1, True)"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 34,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "array([[-0.05289153],\n",
    " [-0.08418501],\n",
    " [ 0.13811206],\n",
    " [ 0.14809799]])"
    ]
    },
    "execution_count": 34,
    "metadata": {},
    "output_type": "execute_result"
    }
    ],
    "source": [
    "d = e * sigmoid(l1, True)\n",
    "d"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 35,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "array([[ 0.28621005],\n",
    " [ 0.06391297],\n",
    " [ 0.14913351]])"
    ]
    },
    "execution_count": 35,
    "metadata": {},
    "output_type": "execute_result"
    }
    ],
    "source": [
    "u = np.dot(l0.T, d)\n",
    "u"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 36,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "array([[ 0.12025406],\n",
    " [ 0.50456196],\n",
    " [-0.85063774]])"
    ]
    },
    "execution_count": 36,
    "metadata": {},
    "output_type": "execute_result"
    }
    ],
    "source": [
    "weights += u\n",
    "weights"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 37,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "0.25652643745572179"
    ]
    },
    "execution_count": 37,
    "metadata": {},
    "output_type": "execute_result"
    }
    ],
    "source": [
    "l1 = sigmoid(np.dot(l0, weights))\n",
    "e = y - l1\n",
    "(e ** 2).mean()"
    ]
    },
    {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
    "### Multiple Steps"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 38,
    "metadata": {},
    "outputs": [
    {
    "data": {
    "text/plain": [
    "array([[-0.16595599],\n",
    " [ 0.44064899],\n",
    " [-0.99977125]])"
    ]
    },
    "execution_count": 38,
    "metadata": {},
    "output_type": "execute_result"
    }
    ],
    "source": [
    "# initialize weights randomly with mean 0\n",
    "np.random.seed(1)\n",
    "weights = 2 * np.random.random((3, 1)) - 1\n",
    "weights"
    ]
    },
    {
    "cell_type": "code",
    "execution_count": 39,
    "metadata": {},
    "outputs": [
    {
    "name": "stdout",
    "output_type": "stream",
    "text": [
    "\n",
    "after 0 iterations\n",
    "layer 1: [[ 0.2689864 0.36375058 0.23762817 0.3262757 ]]\n",
    "errors: [[-0.2689864 -0.36375058 0.76237183 0.6737243 ]]\n",
    "MSE: 0.309945849909\n",
    "\n",
    "after 200 iterations\n",
    "layer 1: [[ 0.07532702 0.06151182 0.95022027 0.93886839]]\n",
    "errors: [[-0.07532702 -0.06151182 0.04977973 0.06113161]]\n",
    "MSE: 0.00391823955246\n",
    "\n",
    "after 400 iterations\n",
    "layer 1: [[ 0.05175173 0.04198947 0.96596069 0.9579657 ]]\n",
    "errors: [[-0.05175173 -0.04198947 0.03403931 0.0420343 ]]\n",
    "MSE: 0.00184172847165\n",
    "\n",
    "after 600 iterations\n",
    "layer 1: [[ 0.0416479 0.03375824 0.97260506 0.96615046]]\n",
    "errors: [[-0.0416479 -0.03375824 0.02739494 0.03384954]]\n",
    "MSE: 0.00119261002357\n",
    "\n",
    "after 800 iterations\n",
    "layer 1: [[ 0.03574306 0.02896968 0.97647479 0.97093628]]\n",
    "errors: [[-0.03574306 -0.02896968 0.02352521 0.02906372]]\n",
    "MSE: 0.000878735940664\n"
    ]
    }
    ],
    "source": [
    "for _ in range(1000):\n",
    " # forward propagation\n",
    " # layer 1\n",
    " l1 = sigmoid(np.dot(l0, weights))\n",
    "\n",
    " # errors of layer 1\n",
    " e = y - l1\n",
    " if _ % 200 == 0:\n",
    " print('\\nafter %d iterations' % _)\n",
    " print('layer 1:', l1.T)\n",
    " print('errors: ', e.T)\n",
    " print('MSE: ', (e ** 2).mean())\n",
    "\n",
    " # multiply errors by the slope of the \n",
    " # sigmoid at the values in l1\n",
    " d = e * sigmoid(l1, True)\n",
    "\n",
    " # update weights\n",
    " weights += np.dot(l0.T, d)"
    ]
    },
    {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
    "<img src=\"http://hilpisch.com/tpq_logo.png\" width=350px align=\"right\">"
    ]
    }
    ],
    "metadata": {
    "kernelspec": {
    "display_name": "Python 3",
    "language": "python",
    "name": "python3"
    },
    "language_info": {
    "codemirror_mode": {
    "name": "ipython",
    "version": 3
    },
    "file_extension": ".py",
    "mimetype": "text/x-python",
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
    "version": "3.6.1"
    }
    },
    "nbformat": 4,
    "nbformat_minor": 2
    }