{
    "cells": [
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "<a href=\"https://www.bigdatauniversity.com\"><img src=\"https://ibm.box.com/shared/static/cw2c7r3o20w9zn8gkecaeyjhgw3xdgbj.png\" width=\"400\" align=\"center\"></a>\n\n<h1 align=\"center\"><font size=\"5\">Classification with Python</font></h1>"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "In this notebook we try to practice all the classification algorithms that we learned in this course.\n\nWe load a dataset using Pandas library, and apply the following algorithms, and find the best one for this specific dataset by accuracy evaluation methods.\n\nLets first load required libraries:"
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [],
            "source": "import itertools\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom matplotlib.ticker import NullFormatter\nimport pandas as pd\nimport numpy as np\nimport matplotlib.ticker as ticker\nfrom sklearn import preprocessing\n%matplotlib inline"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### About dataset"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "This dataset is about past loans. The __Loan_train.csv__ data set includes details of 346 customers whose loan are already paid off or defaulted. It includes following fields:\n\n| Field          | Description                                                                           |\n|----------------|---------------------------------------------------------------------------------------|\n| Loan_status    | Whether a loan is paid off on in collection                                           |\n| Principal      | Basic principal loan amount at the                                                    |\n| Terms          | Origination terms which can be weekly (7 days), biweekly, and monthly payoff schedule |\n| Effective_date | When the loan got originated and took effects                                         |\n| Due_date       | Since it\u2019s one-time payoff schedule, each loan has one single due date                |\n| Age            | Age of applicant                                                                      |\n| Education      | Education of applicant                                                                |\n| Gender         | The gender of applicant                                                               |"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Lets download the dataset"
        },
        {
            "cell_type": "code",
            "execution_count": 2,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "--2020-11-08 12:29:01--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv\nResolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196\nConnecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.\nHTTP request sent, awaiting response... 200 OK\nLength: 23101 (23K) [text/csv]\nSaving to: \u2018loan_train.csv\u2019\n\nloan_train.csv      100%[===================>]  22.56K  --.-KB/s    in 0.07s   \n\n2020-11-08 12:29:01 (304 KB/s) - \u2018loan_train.csv\u2019 saved [23101/23101]\n\n"
                }
            ],
            "source": "!wget -O loan_train.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### Load Data From CSV File  "
        },
        {
            "cell_type": "code",
            "execution_count": 3,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>9/8/2016</td>\n      <td>9/22/2016</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/9/2016</td>\n      <td>10/8/2016</td>\n      <td>28</td>\n      <td>college</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/9/2016</td>\n      <td>10/8/2016</td>\n      <td>29</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30       9/8/2016   \n1           2             2     PAIDOFF       1000     30       9/8/2016   \n2           3             3     PAIDOFF       1000     15       9/8/2016   \n3           4             4     PAIDOFF       1000     30       9/9/2016   \n4           6             6     PAIDOFF       1000     30       9/9/2016   \n\n    due_date  age             education  Gender  \n0  10/7/2016   45  High School or Below    male  \n1  10/7/2016   33              Bechalor  female  \n2  9/22/2016   27               college    male  \n3  10/8/2016   28               college  female  \n4  10/8/2016   29               college    male  "
                    },
                    "execution_count": 3,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df = pd.read_csv('loan_train.csv')\ndf.head()"
        },
        {
            "cell_type": "code",
            "execution_count": 4,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "(346, 10)"
                    },
                    "execution_count": 4,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df.shape"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### Convert to date time object "
        },
        {
            "cell_type": "code",
            "execution_count": 5,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>2016-09-08</td>\n      <td>2016-09-22</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>28</td>\n      <td>college</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>29</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30     2016-09-08   \n1           2             2     PAIDOFF       1000     30     2016-09-08   \n2           3             3     PAIDOFF       1000     15     2016-09-08   \n3           4             4     PAIDOFF       1000     30     2016-09-09   \n4           6             6     PAIDOFF       1000     30     2016-09-09   \n\n    due_date  age             education  Gender  \n0 2016-10-07   45  High School or Below    male  \n1 2016-10-07   33              Bechalor  female  \n2 2016-09-22   27               college    male  \n3 2016-10-08   28               college  female  \n4 2016-10-08   29               college    male  "
                    },
                    "execution_count": 5,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df['due_date'] = pd.to_datetime(df['due_date'])\ndf['effective_date'] = pd.to_datetime(df['effective_date'])\ndf.head()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "# Data visualization and pre-processing\n\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Let\u2019s see how many of each class is in our data set "
        },
        {
            "cell_type": "code",
            "execution_count": 6,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "PAIDOFF       260\nCOLLECTION     86\nName: loan_status, dtype: int64"
                    },
                    "execution_count": 6,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df['loan_status'].value_counts()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "260 people have paid off the loan on time while 86 have gone into collection \n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Lets plot some columns to underestand data better:"
        },
        {
            "cell_type": "code",
            "execution_count": 7,
            "metadata": {},
            "outputs": [],
            "source": "# notice: installing seaborn might takes a few minutes\n# !conda install -c anaconda seaborn -y"
        },
        {
            "cell_type": "code",
            "execution_count": 8,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAUXklEQVR4nO3dfZBldX3n8ffHAUFABKQlLaMwGnRLs+6gLeqyklEMDlbKCZsnKHVxYzJqSfYh666QbKnR0uwaDVt5Io6RhUp4EDVGRCMQlMUYZWhgHGZkJoIgtAxME1MKxiAM3/3jni5uJrene/re7j597/tVdeqe87vn4Xun73e+9/zuueeXqkKSpLZ50nIHIElSLxYoSVIrWaAkSa1kgZIktZIFSpLUShYoSVIrWaAWSZJjklya5NtJbk7ytSRnDGjf65JcNYh9LYUk1yeZWO44tDyGKReSjCW5McmtSV65iMd5eLH2vZJYoBZBkgB/CdxQVc+pqpcAZwKrlymeA5bjuNIQ5sKpwI6qOrGqvjKImDQ7C9TieDXw46r6k5mGqvpOVf0BQJJVSX43yU1JtiZ5a9O+rjnb+FSSHUkuaRKcJOubtr8B/v3MfpMcmuTCZl+3JtnQtL85ySeTfA64pp8Xk+SiJBck+XLzKfinm2PenuSirvUuSDKZZHuS355lX6c1n6BvaeI7rJ/Y1HpDkwtJ1gIfAl6XZEuSp8z2fk5yd5IPNs9NJnlxkquT3Jnkbc06hyW5rtn2tpl4exz3v3f9+/TMq6FVVU4DnoD/BJy/j+c3Av+zmT8ImATWAOuA79P5dPkk4GvAvwMOBu4FTgACXAFc1Wz/QeCNzfwRwN8BhwJvBqaAo2aJ4SvAlh7Ta3qsexFweXPsDcAPgH/dxHgzsLZZ76jmcRVwPfCiZvl6YAI4GrgBOLRpfxfw7uX+ezkt3jSEufBm4A+b+Vnfz8DdwNub+fOBrcBTgTFgd9N+AHB4177uANIsP9w8ngZsal7rk4CrgFOW+++6VJNdP0sgyR/RSa4fV9VL6bzpXpTkF5pVnkYn4X4MbK6qqWa7LcDxwMPAXVX1rab9z+kkNs2+Xp/knc3ywcCzm/lrq+p7vWKqqv3tP/9cVVWS24AHquq2JpbtTYxbgF9KspFO4o0DL6CTmDNe3rR9tfkw/GQ6//FoRAxJLsyY6/18ZfN4G3BYVT0EPJTkn5IcAfwQ+GCSU4DHgWOBY4D7u/ZxWjPd2iwfRuff54YFxryiWKAWx3bg52cWquodSY6m8+kQOp+Gfr2qru7eKMk64JGupj088Tea7aaJAX6+qnbuta+X0UmA3hslX6HziW5v76yqv+7RPhPX43vF+DhwQJI1wDuBl1bVPzRdfwf3iPXaqjprtrg0dIYxF7qPt6/38z5zBngDnTOql1TVo0nupnfO/E5VfXQfcQwtv4NaHF8CDk7y9q62Q7rmrwbenuRAgCTPS3LoPva3A1iT5LnNcndCXA38elf//InzCbCqXllVa3tM+0rIfTmczn8C309yDHB6j3W+Dpyc5CebWA9J8rwFHk8rwzDnQr/v56fR6e57NMmrgON6rHM18Ctd320dm+QZ+3GMFc0CtQiq03n8c8BPJ7kryWbgYjp91AB/CnwTuCXJNuCj7ONstqr+iU43xuebL4a/0/X0+4EDga3Nvt4/6NczH1X1DTrdENuBC4Gv9lhnmk4f/mVJttJJ8H+1hGFqiQ1zLgzg/XwJMJFkks7Z1I4ex7gGuBT4WtO9/il6n+0NpZkv5CRJahXPoCRJrWSBkiS1kgVKktRKFihJUiu1okCtX7++6Py2wclpWKaBMT+chmyat1YUqAcffHC5Q5Bay/zQqGpFgZIkaW8WKElSK1mgJEmtZIGSJLWSBUqS1EoWKElSK1mgNDKOGx8nSd/TcePjy/1SpJHggIUaGffcfz9Tz1zd935W3zc1gGgkzcUzKElSK1mgJEmtNGeBSnJhkt3NCJUzbe9N8t0kW5rpdV3PnZfkjiQ7k7x2sQKXJA23+ZxBXQSs79F+flWtbaYvACR5AXAm8MJmmz9OsmpQwUqSRsecBaqqbgC+N8/9bQAur6pHquou4A7gpD7ikySNqH6+gzonydamC/DIpu1Y4N6udaaatn8hycYkk0kmp6en+whDGj7mh7TwAnUB8FxgLbAL+EjTnh7r9hz/o6o2VdVEVU2MjY0tMAxpOJkf0gILVFU9UFV7qupx4GM80Y03BTyra9XVwH39hShJGkULKlBJun9KfwYwc4XflcCZSQ5KsgY4AdjcX4iSpFE0550kklwGrAOOTjIFvAdYl2Qtne67u4G3AlTV9iRXAN8EHgPeUVV7Fid0SdIwm7NAVdVZPZo/vo/1PwB8oJ+gJEnyThKSpFayQEmSWskCJUlqJQuUJKmVLFCSpFayQEmSWskCJUlqJQuUJKmVLFCSpFayQEmSWskCJUlqJQuUJKmVLFCSpFaas0A1Q7rvTrKtq+13k+xohnz/TJIjmvbjk/woyZZm+pPFDF6SNLzmcwZ1EbB+r7ZrgZ+qqhcBfwec1/XcnVW1tpneNpgwJUmjZs4CVVU3AN/bq+2aqnqsWfw6naHdJUkamEF8B/UrwF91La9JcmuS/5fklbNtlGRjkskkk9PT0wMIQxoe5ofUZ4FK8lt0hna/pGnaBTy7qk4EfgO4NMnhvbatqk1VNVFVE2NjY/2EIQ0d80Pqo0AlORv4WeANVVUAVfVIVf19M38zcCfwvEEEKkkaLQsqUEnWA+8CXl9V/9jVPpZkVTP/HOAE4NuDCFSSNFoOmGuFJJcB64Cjk0wB76Fz1d5BwLVJAL7eXLF3CvC+JI8Be4C3VdX3eu5YkqR9mLNAVdVZPZo/Psu6nwY+3W9QkiR5JwlJUitZoCRJrWSBkiS1kgVKktRKFihJUitZoCRJrWSBkiS1kgVKktRKFihJUitZoCRJrWSBkiS1kgVKktRKFihJUitZoCRJrTRngUpyYZLdSbZ1tR2V5Nok32oej+x67rwkdyTZmeS1ixW4JGm4zecM6iJg/V5t5wLXVdUJwHXNMkleAJwJvLDZ5o9nRtiVJGl/zFmgquoGYO9RcTcAFzfzFwM/19V+eVU9UlV3AXcAJw0oVknSCFnod1DHVNUugObxGU37scC9XetNNW3/QpKNSSaTTE5PTy8wDGk4mR/S4C+SSI+26rViVW2qqomqmhgbGxtwGNLKZn5ICy9QDyQZB2gedzftU8CzutZbDdy38PAkSaNqoQXqSuDsZv5s4LNd7WcmOSjJGuAEYHN/IUqSRtEBc62Q5DJgHXB0kingPcD/Aq5I8hbgHuAXAapqe5IrgG8CjwHvqKo9ixS7JGmIzVmgquqsWZ46dZb1PwB8oJ+gJEnyThKSpFayQEmSWskCJUlqJQuUJKmVLFCSpFayQEmSWskCJUlqJQuUJKmVLFCSpFayQEmSWskCJUlqJQuUJKmVLFCSpFaa827ms0nyfOATXU3PAd4NHAH8GjAzTvVvVtUXFhyhJGkkLbhAVdVOYC1AklXAd4HPAP8ROL+qPjyQCCVJI2lQXXynAndW1XcGtD9J0ogbVIE6E7isa/mcJFuTXJjkyF4bJNmYZDLJ5PT0dK9VpJFlfkgDKFBJngy8Hvhk03QB8Fw63X+7gI/02q6qNlXVRFVNjI2N9RuGNFTMD2kwZ1CnA7dU1QMAVfVAVe2pqseBjwEnDeAYkqQRM4gCdRZd3XtJxrueOwPYNoBjSJJGzIKv4gNIcgjwM8Bbu5o/lGQtUMDdez0nSdK89FWgquofgafv1famviKSJAnvJCFJaikLlCSplSxQkqRWskBJklrJAiVJaiULlCSplfq6zFxaSbLqQFbfNzWQ/UhafBYojYza8ygve/cX+97Pje9bP4BoJM3FLj5JUitZoCRJrWSBkiS1kgVKktRKFihJUitZoCRJrdTveFB3Aw8Be4DHqmoiyVHAJ4Dj6YwH9UtV9Q/9hSlJGjWDOIN6VVWtraqJZvlc4LqqOgG4rlnWCDpufJwkfU/HjY/PfTBJQ2cxfqi7AVjXzF8MXA+8axGOo5a75/77mXrm6r73M4i7P0haefo9gyrgmiQ3J9nYtB1TVbsAmsdn9NowycYkk0kmp6en+wxDGi7mh9R/gTq5ql4MnA68I8kp892wqjZV1URVTYyNjfUZhjRczA+pzwJVVfc1j7uBzwAnAQ8kGQdoHnf3G6QkafQsuEAlOTTJU2fmgdOAbcCVwNnNamcDn+03SEnS6OnnIoljgM8kmdnPpVX1xSQ3AVckeQtwD/CL/YcpSRo1Cy5QVfVt4N/0aP974NR+gpIkyTtJSJJayQIlSWolC5QkqZUsUJKkVrJASZJayQIlSWolC5QkqZUsUJKkVrJASZJayQIlSWolC5QkjbhBjX496BGwF2NEXUnSCjKo0a9hsCNgewYlSWqlfsaDelaSLye5Pcn2JP+5aX9vku8m2dJMrxtcuJKkUdFPF99jwH+rqluagQtvTnJt89z5VfXh/sOTJI2qfsaD2gXsauYfSnI7cOygApMkjbaBfAeV5HjgRODGpumcJFuTXJjkyFm22ZhkMsnk9PT0IMKQhob5IQ2gQCU5DPg08F+q6gfABcBzgbV0zrA+0mu7qtpUVRNVNTE2NtZvGNJQMT+kPgtUkgPpFKdLquovAKrqgaraU1WPAx8DTuo/TEnSqOnnKr4AHwdur6rf62rv/pXWGcC2hYcnSRpV/VzFdzLwJuC2JFuatt8EzkqyFijgbuCtfUUoSRpJ/VzF9zdAejz1hYWHI0lSh3eSkCS1kvfi06LJqgMHcl+urDpwANFIWmksUFo0tedRXvbuL/a9nxvft34A0UhaaezikyS1kgVKktRKFihJUitZoCRJrWSBkqQlNqgh1gc5vHobeRWfJC2xQQ2xPsjh1dvIMyhJUitZoCRJrWQXnySNuEHd9WVmX4NigZKkETeou77AYO/8YhefJKmVFq1AJVmfZGeSO5Kc2+/+vCxTkkbLonTxJVkF/BHwM8AUcFOSK6vqmwvdp5dlStJoWazvoE4C7qiqbwMkuRzYACy4QLXNcePj3HP//X3v59k/8RN8Z9euAUQ03JJeY2OqjcyNuQ3qooQnrTpwqHMjVTX4nSa/AKyvql9tlt8EvKyqzulaZyOwsVl8PrBz4IHM39HAg8t4/H4Y+9KbT9wPVtWCvy1uUX6s1L8RGPtymSv2eefGYp1B9Srp/6wSVtUmYNMiHX+/JJmsqonljmMhjH3pLUXcbcmPlfo3AmNfLoOMfbEukpgCntW1vBq4b5GOJUkaQotVoG4CTkiyJsmTgTOBKxfpWJKkIbQoXXxV9ViSc4CrgVXAhVW1fTGONSDL3pXSB2Nfeis17oVYya/V2JfHwGJflIskJEnql3eSkCS1kgVKktRKI1OgkqxKcmuSq5rlo5Jcm+RbzeORXeue19yiaWeS1y5f1JDkiCSfSrIjye1JXrGCYv+vSbYn2ZbksiQHtzX2JBcm2Z1kW1fbfsea5CVJbmue+/2sgF9RmhvLEru5MZ/cqKqRmIDfAC4FrmqWPwSc28yfC/zvZv4FwDeAg4A1wJ3AqmWM+2LgV5v5JwNHrITYgWOBu4CnNMtXAG9ua+zAKcCLgW1dbfsdK7AZeAWd3wL+FXD6cr139uO1mxtLG7e5Mc/cWPbkWKJ/4NXAdcCru5JwJzDezI8DO5v584Dzura9GnjFMsV9ePNGzl7tKyH2Y4F7gaPoXC16FXBam2MHjt8rCfcr1madHV3tZwEfXY5///14zebG0sdubswzN0ali+//AP8DeLyr7Ziq2gXQPD6jaZ9588yYatqWw3OAaeD/Nl0wf5rkUFZA7FX1XeDDwD3ALuD7VXUNKyD2Lvsb67HN/N7tbWZuLDFz45+179PQF6gkPwvsrqqb57tJj7bluhb/ADqn1hdU1YnAD+mcTs+mNbE3fdIb6JzmPxM4NMkb97VJj7a2/gZitlhX0mswN8yNxTDQ3Bj6AgWcDLw+yd3A5cCrk/w58ECScYDmcXezfptu0zQFTFXVjc3yp+gk5UqI/TXAXVU1XVWPAn8B/FtWRuwz9jfWqWZ+7/a2MjeWh7kxz9cw9AWqqs6rqtVVdTydWy59qareSOfWS2c3q50NfLaZvxI4M8lBSdYAJ9D5cm/JVdX9wL1Jnt80nUpnyJLWx06n++LlSQ5prtY5FbidlRH7jP2KtenqeCjJy5vX/B+6tmkdc8Pc6MPS5MZyfEm4XBOwjie+CH46nS+Hv9U8HtW13m/RufpkJ8t8FRawFpgEtgJ/CRy5gmL/bWAHsA34MzpX9rQyduAyOt8HPErn095bFhIrMNG83juBP2SvL/HbOpkbSx67uTGP3PBWR5KkVhr6Lj5J0spkgZIktZIFSpLUShYoSVIrWaAkSa1kgWqxJHuSbGnuePzJJIfMst7fLnD/E0l+v4/4Hl7otlI/zI3R4GXmLZbk4ao6rJm/BLi5qn6v6/lVVbWnDfFJS8ncGA2eQa0cXwF+Msm6JF9OcilwGzzxaa157vo8MUbOJTNjriR5aZK/TfKNJJuTPLVZf2YMoPcm+bMkX2rGePm1pv2wJNcluaUZy2XD8rx8aVbmxpA6YLkD0NySHACcDnyxaToJ+KmquqvH6icCL6Rzn6uvAicn2Qx8AvjlqropyeHAj3ps+yLg5cChwK1JPk/nHltnVNUPkhwNfD3JleWpt1rA3BhunkG121OSbKFzO5d7gI837ZtnScCZ56aq6nFgC51xXJ4P7KqqmwCq6gdV9ViPbT9bVT+qqgeBL9NJ9gAfTLIV+Gs6t8g/ZjAvT1owc2MEeAbVbj+qqrXdDU2vxA/3sc0jXfN76PyNw/xuz7/3OgW8ARgDXlJVj6Zz5+uD57EvaTGZGyPAM6jRsAN4ZpKXAjR97L0+nGxIcnCSp9O5eehNwNPojBn0aJJXAcctVdDSEjA3WswzqBFQVT9O8svAHyR5Cp0+9tf0WHUz8Hng2cD7q+q+5gqpzyWZpNMtsmOp4pYWm7nRbl5mLqBzpRLwcFV9eLljkdrE3Fg+dvFJklrJMyhJUit5BiVJaiULlCSplSxQkqRWskBJklrJAiVJaqX/D2+HYNn0yXqOAAAAAElFTkSuQmCC\n",
                        "text/plain": "<Figure size 432x216 with 2 Axes>"
                    },
                    "metadata": {
                        "needs_background": "light"
                    },
                    "output_type": "display_data"
                }
            ],
            "source": "import seaborn as sns\n\nbins = np.linspace(df.Principal.min(), df.Principal.max(), 10)\ng = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\ng.map(plt.hist, 'Principal', bins=bins, ec=\"k\")\n\nplt.show()"
        },
        {
            "cell_type": "code",
            "execution_count": 9,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAZBklEQVR4nO3de5QU5bnv8e9PmDgiGEFGGR1hRsULChl1djTBJIjKYXtDj5dojIF1POFo8MKKxqi5rJPtWoREl5psbyHRwEoCyt5RcJMVFQkcg1EjIl4QIx4d2bPlrolyBALynD+6ZjLAwPQM1dPVPb/PWrW66+3qt56X6Zen663qehURmJmZZc1exQ7AzMysLU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QKZN0kKTpkt6W9KKkZyWdn1LdIyTNSaOuriBpgaSGYsdhxVdO/UJSlaTnJb0k6QsF3M+GQtVdKpygUiRJwCzg6Yg4LCJOBC4BaooUT89i7NestTLsF6cBb0TE8RHxxzRisrY5QaVrJPD3iLi/uSAi3o2IfwWQ1EPSbZJekPSKpP+VlI9Ijjb+XdIbkn6TdGokjU7KFgL/vbleSftKejCp6yVJY5LycZL+TdJ/AE/uSWMkTZV0n6T5yTffLyX7XCZpaqvt7pO0SNJSST/YRV2jkm/Ni5P4eu9JbFZSyqZfSKoHfgycKWmJpH129dmW1ChpUvLaIkknSHpC0v+VdGWyTW9J85L3vtocbxv7/Varf582+1hZiggvKS3AtcCdu3l9PPDd5PnewCKgDhgB/I3cN8q9gGeBU4BK4D+BwYCAmcCc5P2TgK8mz/cH3gT2BcYBTUC/XcTwR2BJG8vpbWw7FXgo2fcY4ENgaBLji0B9sl2/5LEHsAAYlqwvABqA/sDTwL5J+beB7xf77+Wla5Yy7BfjgLuT57v8bAONwFXJ8zuBV4A+QBWwJinvCezXqq63ACXrG5LHUcCUpK17AXOALxb779oVi4eACkjSPeQ61N8j4p/IfdCGSbow2eTT5DrZ34E/R0RT8r4lQC2wAXgnIpYn5b8m15lJ6jpX0g3JeiUwMHk+NyLebyumiOjomPl/RERIehVYHRGvJrEsTWJcAlwsaTy5zlYNDCHXGZudnJQ9k3wB/hS5/2ysGyqTftGsvc/2Y8njq0DviPgI+EjSJkn7A/8PmCTpi8A24BDgIGBVqzpGJctLyXpvcv8+T3cy5pLhBJWupcAFzSsRMUFSf3LfCCH3DeiaiHii9ZskjQA2tyr6hH/8bXZ1s0QBF0TEX3ao6yRyH/q23yT9kdy3uB3dEBFPtVHeHNe2HWLcBvSUVAfcAPxTRHyQDP1VthHr3Ii4dFdxWVkrx37Ren+7+2zvtv8Al5E7ojoxIrZIaqTt/vPDiPjZbuIoSz4Hla4/AJWSrmpV1qvV8yeAqyRVAEg6UtK+u6nvDaBO0uHJeutO8ARwTasx+ePzCTAivhAR9W0su+uEu7MfuY7/N0kHAf/cxjbPAcMlHZHE2kvSkZ3cn5Wecu4Xe/rZ/jS54b4tkk4FBrWxzRPA/2h1busQSQd2YB8lywkqRZEbMD4P+JKkdyT9GZhGblwa4BfA68BiSa8BP2M3R7ERsYnc0MXvkpPB77Z6+VagAnglqevWtNuTj4h4mdzQw1LgQeCZNrZZS27cfoakV8h16qO7MEwronLuFyl8tn8DNEhaRO5o6o029vEkMB14Nhlq/3faPtorO80n48zMzDLFR1BmZpZJTlBmZpZJTlBmZpZJTlBmZpZJXZqgRo8eHeR+v+DFS3dYOsX9xEs3XNrUpQlq3bp1Xbk7s5LkfmKW4yE+MzPLJCcoMzPLJCcoMzPLJN8s1szK3pYtW2hqamLTpk3FDqVbq6yspKamhoqKiry2d4Iys7LX1NREnz59qK2tJbmPrHWxiGD9+vU0NTVRV1eX13s8xGdmZW/Tpk0ccMABTk5FJIkDDjigQ0exTlBFMKi6GkmpLIOqq4vdHLOS4ORUfB39G3iIrwhWrFpF08E1qdRV815TKvWYmWWNj6DMrNtJcxQj35GMHj16UF9fz3HHHcdFF13Exx9/DMDWrVvp378/N99883bbjxgxgkWLcpMO19bWMnToUIYOHcqQIUP47ne/y+bN/5igd+nSpYwcOZIjjzySwYMHc+utt9I8ldLUqVOpqqqivr6e+vp6vva1rwEwbtw46urqWsp/+tOfpvJvmyYfQZlZt5PmKAbkN5Kxzz77sGTJEgAuu+wy7r//fr75zW/y5JNPctRRRzFz5kwmTZq0y2Gw+fPn079/fzZs2MD48eMZP34806ZNY+PGjZx77rncd999jBo1io8//pgLLriAe++9lwkTJgDw5S9/mbvvvnunOm+77TYuvPDCPWh5YfkIysysi33hC1/grbfeAmDGjBlcd911DBw4kOeee67d9/bu3Zv777+fWbNm8f777zN9+nSGDx/OqFGjAOjVqxd33303kydPLmgbuoITlJlZF9q6dSu///3vGTp0KBs3bmTevHmcffbZXHrppcyYMSOvOvbbbz/q6upYvnw5S5cu5cQTT9zu9cMPP5wNGzbw4YcfAvDwww+3DOX98pe/bNnuW9/6Vkv5q6++ml4jU+IhPjOzLrBx40bq6+uB3BHUFVdcwezZszn11FPp1asXF1xwAbfeeit33nknPXr0aLe+5nNMEbHLYcHm8lId4ssrQUlqBD4CPgG2RkSDpH7Aw0At0AhcHBEfFCZMM7PS1vocVLMZM2bwzDPPUFtbC8D69euZP38+p59++m7r+uijj2hsbOTII4/k2GOP5emnn97u9bfffpvevXvTp0+fVNvQ1ToyxHdqRNRHREOyfhMwLyIGA/OSdTMzy8OHH37IwoULWbFiBY2NjTQ2NnLPPfe0O8y3YcMGvvGNb3DeeefRt29fLrvsMhYuXMhTTz0F5I7Urr32Wm688cauaEZB7ckQ3xhgRPJ8GrAA+PYexmNmVnADBwxI9TeEAwcM6PB7HnnkEUaOHMnee+/dUjZmzBhuvPHG7S4hb3bqqacSEWzbto3zzz+f733ve0DuyGz27Nlcc801TJgwgU8++YTLL7+cq6++uvMNygg1j2PudiPpHeADcjMf/iwipkj6a0Ts32qbDyKibxvvHQ+MBxg4cOCJ7777bmrBlypJqf5QN5+/oRVF3j+bdz8prGXLlnHMMccUOwxjl3+LNvtKvkN8wyPiBOCfgQmSvphvMBExJSIaIqKhqqoq37eZdSvuJ2Y7yytBRcR7yeMa4FHgs8BqSdUAyeOaQgVpZmbdT7sJStK+kvo0PwdGAa8BjwFjk83GArMLFaSZmXU/+VwkcRDwaHI9fU9gekQ8LukFYKakK4AVwEWFC9PMzLqbdhNURLwNfKaN8vXAaYUIyszMzLc6MjOzTHKCMrNu5+CagalOt3FwzcB297lq1SouueQSDj/8cIYMGcKZZ57Jm2++2e5UGW39nqm2tpZ169ZtV7bjtBr19fW8/vrrALz55puceeaZHHHEERxzzDFcfPHF292fr3fv3hx11FEt03EsWLCAs88+u6XuWbNmMWzYMI4++miGDh3KrFmzWl4bN24chxxySMtvt9atW9dyZ4w95Xvx5WlQdTUrVq0qdhhmloKV//WfnPT9x1Or7/l/Gb3b1yOC888/n7Fjx/LQQw8BsGTJElavXs24ceN2O1VGR7R1z71NmzZx1llncccdd3DOOecAuak7qqqqWm69NGLECG6//XYaGnI3ClqwYEHL+19++WVuuOEG5s6dS11dHe+88w5nnHEGhx12GMOGDQNyc109+OCDXHXVVR2OeXecoPLkWXDNrLPmz59PRUUFV155ZUtZfX09DzzwQJtTZYwYMaJTCaot06dP53Of+1xLcoLcXSnydfvtt3PLLbdQV1cHQF1dHTfffDO33XYbv/rVrwCYOHEid955J1//+tdTibmZh/jMzArstdde22lKDCCvqTI6ovWwXX19PRs3btzlvvPVVowNDQ0sXbq0ZX3gwIGccsopLQkrLT6CMjMrknymyuiIXU2rsSfairGtsltuuYVzzz2Xs846K7V9+wjKzKzAjj32WF588cU2yxctWrRdWdpTZexq3x15/44xLl68mCFDhmxXdsQRR1BfX8/MmTM7va8dOUGZmRXYyJEj2bx5Mz//+c9byl544QUGDx5c8KkyvvKVr/CnP/2J3/3udy1ljz/+eN4z6N5www388Ic/pLGxEYDGxkYmTZrE9ddfv9O23/nOd7j99ttTiRs8xGdm3VD1IYe2e+VdR+vbHUk8+uijTJw4kcmTJ1NZWUltbS133XVXu1NlTJ06dbvLup977jkAhg0bxl575Y4xLr74YoYNG8bDDz/MwoULW7a99957+fznP8+cOXOYOHEiEydOpKKigmHDhvGTn/wkr7bV19fzox/9iHPOOYctW7ZQUVHBj3/845bZgVs79thjOeGEE1i8eHFedbcnr+k20tLQ0BA7HiqWirSnyPB0G91Cx08iUNr9JKs83UZ2FGK6DTMzsy7lBGVmZpnkBGVm3YKHwouvo38DJygzK3uVlZWsX7/eSaqIIoL169dTWVmZ93t8FZ+Zlb2amhqamppYu3ZtsUPp1iorK6mpyf8CMSeoErc3nfvFeVsGDhjAuytXplKXWZZUVFS03EvOSocTVInbDL6JrZmVpbzPQUnqIeklSXOS9X6S5kpanjz2LVyYZmbW3XTkIonrgGWt1m8C5kXEYGBesm5mZpaKvBKUpBrgLOAXrYrHANOS59OA89INzczMurN8j6DuAm4EtrUqOygiVgIkjwe29UZJ4yUtkrTIV9CYtc39xGxn7SYoSWcDayKiU/drj4gpEdEQEQ1VVVWdqcKs7LmfmO0sn6v4hgPnSjoTqAT2k/RrYLWk6ohYKakaWFPIQM3MrHtp9wgqIm6OiJqIqAUuAf4QEV8FHgPGJpuNBWYXLEozM+t29uRWR5OBMyQtB85I1s3MzFLRoR/qRsQCYEHyfD1wWvohmZmZ+WaxZmaWUU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSe0mKEmVkv4s6WVJSyX9ICnvJ2mupOXJY9/Ch2tmZt1FPkdQm4GREfEZoB4YLelk4CZgXkQMBuYl62ZmZqloN0FFzoZktSJZAhgDTEvKpwHnFSRCMzPrlvI6ByWph6QlwBpgbkQ8DxwUESsBkscDd/He8ZIWSVq0du3atOI2KyvuJ2Y7yytBRcQnEVEP1ACflXRcvjuIiCkR0RARDVVVVZ2N06ysuZ+Y7axDV/FFxF+BBcBoYLWkaoDkcU3q0ZmZWbeVz1V8VZL2T57vA5wOvAE8BoxNNhsLzC5UkGZm1v30zGObamCapB7kEtrMiJgj6VlgpqQrgBXARQWM08zMupl2E1REvAIc30b5euC0QgRlZmbmO0mYmVkmOUGZmVkmOUGZmVkmOUGZmVkmlXWCGlRdjaRUFjMz61r5XGZeslasWkXTwTWp1FXzXlMq9ZiZWX7K+gjKzMxKlxOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllUrsJStKhkuZLWiZpqaTrkvJ+kuZKWp489i18uGZm1l3kcwS1Fbg+Io4BTgYmSBoC3ATMi4jBwLxk3czMLBXtJqiIWBkRi5PnHwHLgEOAMcC0ZLNpwHmFCtLMzLqfDp2DklQLHA88DxwUESshl8SAA3fxnvGSFklatHbt2j2L1qxMuZ+Y7SzvBCWpN/BbYGJEfJjv+yJiSkQ0RERDVVVVZ2I0K3vuJ2Y7yytBSaogl5x+ExGPJMWrJVUnr1cDawoTopmZdUf5XMUn4AFgWUTc0eqlx4CxyfOxwOz0w7OutDfsdtr7jiyDqquL3RwzK3H5TPk+HLgceFXSkqTsFmAyMFPSFcAK4KLChGhdZTPQdHBNKnXVvNeUSj1m1n21m6AiYiGgXbx8WrrhZJd6VKT2n656fiq9unpUpFKPmVnW5HMEZUB8soWTvv94KnU9/y+jU63LzKwc+VZHZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSWV9J4k0b09kZmZdq6wTVNq3JzIzs67jIT4zM8skJygzM8skJygzM8uksj4H1R2kOk+V55ayDBlUXc2KVatSqWufvXqwcdsnqdQ1cMAA3l25MpW6bPecoEqcLwSxcrVi1apUZ3j2bNGlp90hPkkPSloj6bVWZf0kzZW0PHnsW9gwzcysu8nnHNRUYMev1jcB8yJiMDAvWTdrsTcgKZVlUHV1sZtjZkXQ7hBfRDwtqXaH4jHAiOT5NGAB8O0U47IStxk8pGJme6SzV/EdFBErAZLHA3e1oaTxkhZJWrR27dpO7s6svJVLPxlUXZ3akbNZwS+SiIgpwBSAhoaGKPT+zEpRufSTtC9ssO6ts0dQqyVVAySPa9ILyczMrPMJ6jFgbPJ8LDA7nXDMzMxy8rnMfAbwLHCUpCZJVwCTgTMkLQfOSNbNzMxSk89VfJfu4qXTUo7FzMysRebuxeergMzMDDJ4qyNfBWRmZpDBBGXF4xvPmlmWOEFZC9941syyJHPnoMzMzMAJyszMMsoJyszMMskJyszMMskJyjLPc0sVln97aFnlq/gs8zy3VGH5t4eWVU5QVhD+TZWZ7SknKCsI/6bKzPaUz0GZmVkm+QjKMi/N4cK9elSkdjJ/4IABvLtyZSp1lYtUh3Z7fsrDxB0wqLqaFatWpVJXVj7bTlCWeWkPF/qCgMJJ+2/lYeL8lePFLh7iMzOzTMrcEVSaQwRmZla6MpegfPWXmZnBHiYoSaOBnwA9gF9ExORUojIrkHL5fVaaJ8StY9K80GavnhVs27ollbrKUacTlKQewD3AGUAT8IKkxyLi9bSCM0tbuRyhl+MJ8VKxzRftdJk9uUjis8BbEfF2RPwdeAgYk05YZmbW3SkiOvdG6UJgdET8z2T9cuCkiLh6h+3GA+OT1aOAv3Q+3O30B9alVFcWuD3Z1dm2rIuIvA6z3E/y5vZkW6p9ZU/OQbU1CLtTtouIKcCUPdhP2zuXFkVEQ9r1Fovbk11d0Rb3k/y4PdmWdnv2ZIivCTi01XoN8N6ehWNmZpazJwnqBWCwpDpJnwIuAR5LJywzM+vuOj3EFxFbJV0NPEHuMvMHI2JpapG1L/XhkCJze7KrlNtSyrG3xe3JtlTb0+mLJMzMzArJ9+IzM7NMcoIyM7NMynyCknSopPmSlklaKum6pLyfpLmSliePfYsdaz4kVUr6s6SXk/b8ICkvyfY0k9RD0kuS5iTrJdseSY2SXpW0RNKipCzz7XFfyT73k47JfIICtgLXR8QxwMnABElDgJuAeRExGJiXrJeCzcDIiPgMUA+MlnQypdueZtcBy1qtl3p7To2I+la/6SiF9rivZJ/7SUdEREktwGxy9//7C1CdlFUDfyl2bJ1oSy9gMXBSKbeH3G/g5gEjgTlJWSm3pxHov0NZybXHfSVbi/tJx5dSOIJqIakWOB54HjgoIlYCJI8HFi+yjkkO85cAa4C5EVHS7QHuAm4EtrUqK+X2BPCkpBeTWxBBibXHfSWT3E86KHPzQe2KpN7Ab4GJEfFhWre7L4aI+ASol7Q/8Kik44odU2dJOhtYExEvShpR7HhSMjwi3pN0IDBX0hvFDqgj3Feyx/2kc0riCEpSBbkO95uIeCQpXi2pOnm9mtw3rJISEX8FFgCjKd32DAfOldRI7o72IyX9mtJtDxHxXvK4BniU3J37S6I97iuZ5X7SCZlPUMp9/XsAWBYRd7R66TFgbPJ8LLnx9syTVJV8G0TSPsDpwBuUaHsi4uaIqImIWnK3u/pDRHyVEm2PpH0l9Wl+DowCXqME2uO+kl3uJ51U7BNteZyIO4XcWOcrwJJkORM4gNwJx+XJY79ix5pne4YBLyXteQ34flJeku3ZoW0j+MfJ35JsD3AY8HKyLAW+UyrtcV8pjcX9JP/FtzoyM7NMyvwQn5mZdU9OUGZmlklOUGZmlklOUGZmlklOUGZmlklOUGZmlklOUGZmlklOUGVA0qzkho1Lm2/aKOkKSW9KWiDp55LuTsqrJP1W0gvJMry40Zt1HfeV0uIf6pYBSf0i4v3kdjAvAP8NeAY4AfgI+APwckRcLWk6cG9ELJQ0EHgicvMHmZU995XSUjJ3M7fdulbS+cnzQ4HLgf8TEe8DSPo34Mjk9dOBIa3ucL2fpD4R8VFXBmxWJO4rJcQJqsQlt+4/HfhcRHwsaQG5ScN29U1vr2TbjV0ToVk2uK+UHp+DKn2fBj5IOtzR5Kb67gV8SVJfST2BC1pt/yRwdfOKpPoujdaseNxXSowTVOl7HOgp6RXgVuA54L+ASeRmU30KeB34W7L9tUCDpFckvQ5c2fUhmxWF+0qJ8UUSZUpS74jYkHwrfBR4MCIeLXZcZlnjvpJdPoIqX/9b0hJy8+i8A8wqcjxmWeW+klE+gjIzs0zyEZSZmWWSE5SZmWWSE5SZmWWSE5SZmWWSE5SZmWXS/wfDQWC+dUWUHQAAAABJRU5ErkJggg==\n",
                        "text/plain": "<Figure size 432x216 with 2 Axes>"
                    },
                    "metadata": {
                        "needs_background": "light"
                    },
                    "output_type": "display_data"
                }
            ],
            "source": "bins = np.linspace(df.age.min(), df.age.max(), 10)\ng = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\ng.map(plt.hist, 'age', bins=bins, ec=\"k\")\n\ng.axes[-1].legend()\nplt.show()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "# Pre-processing:  Feature selection/extraction"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### Lets look at the day of the week people get the loan "
        },
        {
            "cell_type": "code",
            "execution_count": 10,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>2016-09-08</td>\n      <td>2016-09-22</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>28</td>\n      <td>college</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>29</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30     2016-09-08   \n1           2             2     PAIDOFF       1000     30     2016-09-08   \n2           3             3     PAIDOFF       1000     15     2016-09-08   \n3           4             4     PAIDOFF       1000     30     2016-09-09   \n4           6             6     PAIDOFF       1000     30     2016-09-09   \n\n    due_date  age             education  Gender  \n0 2016-10-07   45  High School or Below    male  \n1 2016-10-07   33              Bechalor  female  \n2 2016-09-22   27               college    male  \n3 2016-10-08   28               college  female  \n4 2016-10-08   29               college    male  "
                    },
                    "execution_count": 10,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df.head()"
        },
        {
            "cell_type": "code",
            "execution_count": 11,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAZsklEQVR4nO3de3hU9b3v8fdHSI0I1BtqJIVExQsIO2p6rEVbxMtDvaHbe9GCx25OrTeOpW61tj27nsdS4fHS7a3WqrRV1FpvpacqUtiKFStiFBGLbk0xFRSwrVJBQb/nj1lJAwQySdZkFjOf1/PMMzNr1vqt7wr58p3fbya/nyICMzOzrNmq2AGYmZm1xQXKzMwyyQXKzMwyyQXKzMwyyQXKzMwyyQXKzMwyyQUqZZJ2kXS3pDckPS/pGUknptT2CEnT02irO0iaLam+2HFY8ZVSXkjqJ+lZSS9IOrSA51lVqLa3FC5QKZIk4CHgyYjYPSIOBE4HqosUT89inNestRLMi8OBVyNi/4h4Ko2YrG0uUOkaCXwcEbc0b4iIP0fEfwJI6iFpsqTnJL0k6X8l20ckvY37Jb0q6a4kqZE0Ktk2B/jX5nYlbSvp9qStFySNTraPk/QrSb8BHu/KxUi6U9LNkmYl73y/nJxzkaQ7W+13s6R5khZK+o9NtHVU8q55fhJf767EZluUkskLSXXA1cDRkhokbbOp321JjZKuSl6bJ+kASY9J+m9J30j26S1pZnLsguZ42zjvt1v9fNrMsZIUEb6ldAMuBK7dzOvjgSuSx1sD84BaYATwd3LvKLcCngEOASqBt4BBgID7gOnJ8VcBZyaPtwMWA9sC44AmYIdNxPAU0NDG7Yg29r0TuCc592jgfWBoEuPzQF2y3w7JfQ9gNjAseT4bqAd2Ap4Etk22/zvwvWL/e/nWPbcSzItxwA3J403+bgONwLnJ42uBl4A+QD/g3WR7T6Bvq7ZeB5Q8X5XcHwXcmlzrVsB04EvF/nftjpuHgApI0o3kEurjiPg8uV+0YZJOTnb5LLkk+xj4Y0Q0Jcc1ADXAKuDNiHgt2f5LcslM0tbxkiYmzyuBAcnjGRHxXlsxRURHx8x/ExEhaQHwTkQsSGJZmMTYAJwqaTy5ZKsCBpNLxmZfSLY9nbwB/gy5/2ysDJVIXjRr73f7keR+AdA7Ij4APpC0RtJ2wD+AqyR9CfgU6A/sAixr1cZRye2F5Hlvcj+fJzsZ8xbDBSpdC4GTmp9ExHmSdiL3jhBy74AuiIjHWh8kaQTwUatNn/DPf5tNTZYo4KSI+NMGbR1E7pe+7YOkp8i9i9vQxIh4oo3tzXF9ukGMnwI9JdUCE4HPR8Rfk6G/yjZinRERZ2wqLitppZgXrc+3ud/tzeYPMIZcj+rAiFgrqZG28+eHEfGTzcRRkvwZVLp+D1RKOrfVtl6tHj8GnCupAkDSXpK23Ux7rwK1kvZInrdOgseAC1qNye+fT4ARcWhE1LVx21wSbk5fcon/d0m7AF9pY5+5wHBJeyax9pK0VyfPZ1ueUs6Lrv5uf5bccN9aSYcBA9vY5zHgf7b6bKu/pJ07cI4tlgtUiiI3YHwC8GVJb0r6IzCV3Lg0wG3AK8B8SS8DP2EzvdiIWENu6OK3yYfBf2718pVABfBS0taVaV9PPiLiRXJDDwuB24Gn29hnOblx+2mSXiKX1Pt0Y5hWRKWcFyn8bt8F1EuaR6439Wob53gcuBt4Jhlqv5+2e3slp/nDODMzs0xxD8rMzDLJBcrMzDLJBcrMzDLJBcrMzDKpWwvUqFGjgtzfL/jmWzncOsV54lsZ3trUrQVqxYoV3Xk6sy2S88Qsx0N8ZmaWSS5QZmaWSS5QZmaWSZ4s1sxK3tq1a2lqamLNmjXFDqWsVVZWUl1dTUVFRV77u0CZWclramqiT58+1NTUkMwja90sIli5ciVNTU3U1tbmdYyH+Mys5K1Zs4Ydd9zRxamIJLHjjjt2qBfrAmVlZWBVFZJSuQ2sqir25VgHuDgVX0f/DTzEZ2VlybJlNO1WnUpb1W83pdKOmbXNPSgzKztp9qTz7U336NGDuro69ttvP0455RQ+/PBDANatW8dOO+3EZZddtt7+I0aMYN683KLDNTU1DB06lKFDhzJ48GCuuOIKPvronwv0Lly4kJEjR7LXXnsxaNAgrrzySpqXUrrzzjvp168fdXV11NXV8bWvfQ2AcePGUVtb27L9xz/+cSo/2zTl1YOS9L+Br5ObkmIBcDa5FTHvBWqARuDUiPhrQaI0M0tRmj1pyK83vc0229DQ0ADAmDFjuOWWW7j44ot5/PHH2Xvvvbnvvvu46qqrNjkMNmvWLHbaaSdWrVrF+PHjGT9+PFOnTmX16tUcf/zx3HzzzRx11FF8+OGHnHTSSdx0002cd955AJx22mnccMMNG7U5efJkTj755C5ceWG124OS1B+4EKiPiP2AHsDpwKXAzIgYBMxMnpuZWTsOPfRQXn/9dQCmTZvGRRddxIABA5g7d267x/bu3ZtbbrmFhx56iPfee4+7776b4cOHc9RRRwHQq1cvbrjhBiZNmlTQa+gO+Q7x9QS2kdSTXM/pbWA0uWWbSe5PSD88M7PSsm7dOn73u98xdOhQVq9ezcyZMzn22GM544wzmDZtWl5t9O3bl9raWl577TUWLlzIgQceuN7re+yxB6tWreL9998H4N57720Zyrvjjjta9vv2t7/dsn3BggXpXWRK2i1QEfEXYAqwBFgK/D0iHgd2iYilyT5LgZ3bOl7SeEnzJM1bvnx5epGblRDnSelbvXo1dXV11NfXM2DAAM455xymT5/OYYcdRq9evTjppJN48MEH+eSTT/Jqr/kzpojY5LBg8/bTTjuNhoYGGhoaOPvss1tenzx5csv2oUOHdvEK09fuZ1CStifXW6oF/gb8StKZ+Z4gIm4FbgWor6/f5LTqZuXMeVL6Wn8G1WzatGk8/fTT1NTUALBy5UpmzZrFEUccsdm2PvjgAxobG9lrr70YMmQITz755Hqvv/HGG/Tu3Zs+ffqkeg3dLZ8hviOANyNieUSsBR4Avgi8I6kKILl/t3BhmpmVlvfff585c+awZMkSGhsbaWxs5MYbb2x3mG/VqlV885vf5IQTTmD77bdnzJgxzJkzhyeeeALI9dQuvPBCLrnkku64jILK51t8S4AvSOoFrAYOB+YB/wDGApOS+4cLFaSZWZoG7Lprqn/HNmDXXTt8zAMPPMDIkSPZeuutW7aNHj2aSy65ZL2vkDc77LDDiAg+/fRTTjzxRL773e8CuZ7Zww8/zAUXXMB5553HJ598wllnncX555/f+QvKCDWPY252J+k/gNOAdcAL5L5y3hu4DxhAroidEhHvba6d+vr6aP5ev1kxSEr1D3XbyZ9OTV3gPEnfokWL2HfffYsdhrHJf4s2cyWvv4OKiO8D399g80fkelNmZmap80wSZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZlZ2dqsekOpyG7tVD2j3nMuWLeP0009njz32YPDgwRx99NEsXry43aUy2vp7ppqaGlasWLHetg2X1airq+OVV14BYPHixRx99NHsueee7Lvvvpx66qnrzc/Xu3dv9t5775blOGbPns2xxx7b0vZDDz3EsGHD2GeffRg6dCgPPfRQy2vjxo2jf//+LX+7tWLFipaZMbrKCxaaWdlZ+pe3OOh7j6bW3rM/GLXZ1yOCE088kbFjx3LPPfcA0NDQwDvvvMO4ceM2u1RGR7S1rMaaNWs45phjuOaaazjuuOOA3NId/fr1a5l6acSIEUyZMoX6+noAZs+e3XL8iy++yMSJE5kxYwa1tbW8+eabHHnkkey+++4MGzYMyK11dfvtt3Puued2OObNcQ/KzKzAZs2aRUVFBd/4xjdattXV1bF48eKCL5Vx9913c/DBB7cUJ8jNSrHffvvldfyUKVO4/PLLqa2tBaC2tpbLLruMyZMnt+wzYcIErr32WtatW5da3OACZWZWcC+//PJGS2IAeS2V0RGth+3q6upYvXr1Js+dr7ZirK+vZ+HChS3PBwwYwCGHHMIvfvGLTp+nLR7iMzMrknyWyuiITa2c2xVtxdjWtssvv5zjjz+eY445JrVzuwdlZlZgQ4YM4fnnn29z+4bzLqa9VMamzt2R4zeMcf78+QwePHi9bXvuuSd1dXXcd999nT7XhlygzMwKbOTIkXz00Uf89Kc/bdn23HPPMWjQoIIvlfHVr36VP/zhD/z2t79t2fboo4/mvYLuxIkT+eEPf0hjYyMAjY2NXHXVVXzrW9/aaN/vfOc7TJkyJZW4wUN8ZlaGqvp/rt1v3nW0vc2RxIMPPsiECROYNGkSlZWV1NTUcN1117W7VMadd9653te6586dC8CwYcPYaqtcH+PUU09l2LBh3HvvvcyZM6dl35tuuokvfvGLTJ8+nQkTJjBhwgQqKioYNmwY119/fV7XVldXx49+9COOO+441q5dS0VFBVdffTV1dXUb7TtkyBAOOOAA5s+fn1fb7clruY20eBkBKzYvt1GevNxGdnRkuQ0P8ZmZWSZlrkANrKpK7a+7B1ZVFftyzMyskzL3GdSSZctSHYIxM4PNf6XbukdHP1LKXA/KzCxtlZWVrFy5ssP/QVp6IoKVK1dSWVmZ9zGZ60GZmaWturqapqYmli9fXuxQylplZSXV1fmPkLlAmVnJq6ioaJlLzrYcHuIzM7NMcoEyM7NMcoEyM7NMcoEyM7NMcoEyM7NMyqtASdpO0v2SXpW0SNLBknaQNEPSa8n99oUO1szMyke+PajrgUcjYh/gX4BFwKXAzIgYBMxMnpuZmaWi3QIlqS/wJeBnABHxcUT8DRgNTE12mwqcUKggzcys/OTTg9odWA7cIekFSbdJ2hbYJSKWAiT3O7d1sKTxkuZJmue/4jZrm/PEbGP5FKiewAHAzRGxP/APOjCcFxG3RkR9RNT369evk2GalTbnidnG8ilQTUBTRDybPL+fXMF6R1IVQHL/bmFCNDOzctRugYqIZcBbkvZONh0OvAI8AoxNto0FHi5IhGZmVpbynSz2AuAuSZ8B3gDOJlfc7pN0DrAEOKUwIZqlRz0qUlsnTD0qUmnHzNqWV4GKiAagvo2XDk83HLPCik/WctD3Hk2lrWd/MCqVdsysbZ5JwszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMinvAiWph6QXJE1Pnu8gaYak15L77QsXppmZlZuO9KAuAha1en4pMDMiBgEzk+dmZmapyKtASaoGjgFua7V5NDA1eTwVOCHd0MzMrJzl24O6DrgE+LTVtl0iYilAcr9zWwdKGi9pnqR5y5cv71KwZqXKeWK2sXYLlKRjgXcj4vnOnCAibo2I+oio79evX2eaMCt5zhOzjfXMY5/hwPGSjgYqgb6Sfgm8I6kqIpZKqgLeLWSgZmZWXtrtQUXEZRFRHRE1wOnA7yPiTOARYGyy21jg4YJFaWZmZacrfwc1CThS0mvAkclzMzOzVOQzxNciImYDs5PHK4HD0w/JzMzMM0mYmVlGuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUAVwcCqKiSlchtYVVXsyzEzK4gOrQdl6ViybBlNu1Wn0lb1202ptGNmljXuQZmZWSa5QJmZWSa5QJmZWSa5QJmZWSa5QJmZWSa5QJmZWSa5QJmZWSa5QJmZWSa5QJmZWSa1W6AkfU7SLEmLJC2UdFGyfQdJMyS9ltxvX/hwzcysXOTTg1oHfCsi9gW+AJwnaTBwKTAzIgYBM5PnZmZmqWi3QEXE0oiYnzz+AFgE9AdGA1OT3aYCJxQqSDMzKz8d+gxKUg2wP/AssEtELIVcEQN23sQx4yXNkzRv+fLlXYvWrEQ5T8w2lneBktQb+DUwISLez/e4iLg1Iuojor5fv36didGs5DlPzDaWV4GSVEGuON0VEQ8km9+RVJW8XgW8W5gQzcysHOXzLT4BPwMWRcQ1rV56BBibPB4LPJx+eGZmVq7yWbBwOHAWsEBSQ7LtcmAScJ+kc4AlwCmFCdHMzMpRuwUqIuYA2sTLh6cbjpmZFdvAqiqWLFuWSlsDdt2VPy9d2qljveS7mZmtZ8myZTTtVp1KW9VvN3X6WE91ZJk3sKoKSancSkWaP5OBVVXFvhyzNrkHZZmXlXdzWeKfiZUD96DMzCyTSroHtTWkNqzTlQ/6rGvUo8Lv8s3KUEkXqI/AwyAlID5Zy0HfezSVtp79wahU2jGzwvMQn5mZZZILlJmZZZILlJmZZZILlJmZZZILlJmZZZILlJmZZZILlJmZZZILlJmZZZILlJmZZZILlJmZZVJJT3VkZmYdl+b8l+pR0eljXaDMzGw9WZn/0kN8ZmWuedZ/L35oWeMelFmZ86z/llXuQZmZWSa5QFlB7FY9ILVhIzMrTx7is4JY+pe3MvEhq5ltuTJXoLLy9UYzK66BVVUsWbYslbYG7Lorf166NJW2rPtkrkBl5euNW4rmb2ClwUlsWbJk2TJ/eaPMdalASRoFXA/0AG6LiEmpRGV58zewzKxUdfpLEpJ6ADcCXwEGA2dIGpxWYGZmacnq33oNrKpKLa5ePXqW3BeTutKD+h/A6xHxBoCke4DRwCtpBGZmlpasjjSkPYyZxWvsCkVE5w6UTgZGRcTXk+dnAQdFxPkb7DceGJ883Rv4UztN7wSs6FRQWw5fY2lo7xpXREReH4Q6T9rkaywN+Vxjm7nSlR5UW/3AjapdRNwK3Jp3o9K8iKjvQlyZ52ssDWleo/NkY77G0tCVa+zKH+o2AZ9r9bwaeLsL7ZmZmbXoSoF6DhgkqVbSZ4DTgUfSCcvMzMpdp4f4ImKdpPOBx8h9zfz2iFiYQkx5D3NswXyNpaGY1+ifb2nwNW5Gp78kYWZmVkieLNbMzDLJBcrMzDIpMwVK0ihJf5L0uqRLix1P2iR9TtIsSYskLZR0UbFjKhRJPSS9IGl6sWMpBEnbSbpf0qvJv+fB3Xjuks4TKJ9cKfU8ga7nSiY+g0qmTVoMHEnu6+vPAWdERMnMSiGpCqiKiPmS+gDPAyeU0jU2k3QxUA/0jYhjix1P2iRNBZ6KiNuSb7D2ioi/dcN5Sz5PoHxypdTzBLqeK1npQbVMmxQRHwPN0yaVjIhYGhHzk8cfAIuA/sWNKn2SqoFjgNuKHUshSOoLfAn4GUBEfNwdxSlR8nkC5ZErpZ4nkE6uZKVA9QfeavW8iRL7hWxNUg2wP/BscSMpiOuAS4BPix1IgewOLAfuSIZnbpO0bTedu6zyBEo6V0o9TyCFXMlKgcpr2qRSIKk38GtgQkS8X+x40iTpWODdiHi+2LEUUE/gAODmiNgf+AfQXZ8FlU2eQOnmSpnkCaSQK1kpUGUxbZKkCnIJd1dEPFDseApgOHC8pEZyw08jJf2yuCGlrgloiojmd/T3k0vC7jp3yecJlHyulEOeQAq5kpUCVfLTJim3yMrPgEURcU2x4ymEiLgsIqojoobcv+HvI+LMIoeVqohYBrwlae9k0+F03xIzJZ8nUPq5Ug55AunkSiaWfC/gtElZMhw4C1ggqSHZdnlE/L8ixmSdcwFwV1Ik3gDO7o6TlkmegHOllHQpVzLxNXMzM7MNZWWIz8zMbD0uUGZmlkkuUGZmlkkuUGZmlkkuUGZmlkkuUBki6f9Imphie/tIakimGdkjrXZbtd8oaae02zXbHOdJ+XCBKm0nAA9HxP4R8d/FDsYso5wnGeUCVWSSvpOs7/MEsHey7d8kPSfpRUm/ltRLUh9JbyZTwCCpb/LOrEJSnaS5kl6S9KCk7SUdDUwAvp6srXOTpOOTYx+UdHvy+BxJ/zd5fKakPybvJn+SLO+ApKMkPSNpvqRfJXOktb6GbSQ9Kunfuu0HZ2XFeVKeXKCKSNKB5KY62R/4V+DzyUsPRMTnI+JfyC01cE6y7MBsclP0kxz364hYC/wc+PeIGAYsAL6f/NX9LcC1EXEY8CRwaHJsf2Bw8vgQ4ClJ+wKnAcMjog74BBiTDE1cARwREQcA84CLW11Gb+A3wN0R8dOUfjRmLZwn5csFqrgOBR6MiA+T2Zqb51XbT9JTkhYAY4Ahyfbb+OdUIWeTm8b+s8B2EfFfyfap5NZg2dBTwKGSBpObD+sd5RaGOxj4A7l5sg4Enkumlzmc3HT5XyCXpE8n28cCA1u1+zBwR0T8vCs/CLPNcJ6UqUzMxVfm2ppr6k5yK4i+KGkcMAIgIp6WVCPpy0CPiHg5Sbz2TxLxF0nbA6PIvUvcATgVWBURH0gSMDUiLmt9nKTjgBkRccYmmn4a+Iqku8PzZlnhOE/KkHtQxfUkcGIyNt0HOC7Z3gdYmoyjj9ngmJ8D04A7ACLi78BfJTUPS5wF/Bdte4bcePuT5N4pTkzuAWYCJ0vaGUDSDpIGAnOB4ZL2TLb3krRXqza/B6wEburoxZvlyXlSplygiihZ1vpeoIHc2jfNSfBdciuIzgBe3eCwu4DtySVfs7HAZEkvAXXADzZxyqeAnhHxOjCf3LvDp5JYXiE3hv540s4MoCoilgPjgGnJ9rnAPhu0OwGolHR13hdvlifnSfnybOZbGEknA6Mj4qxix2KWVc6T0uDPoLYgkv4T+ApwdLFjMcsq50npcA/KzMwyyZ9BmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJv1/wof9+c/Zg1kAAAAASUVORK5CYII=\n",
                        "text/plain": "<Figure size 432x216 with 2 Axes>"
                    },
                    "metadata": {
                        "needs_background": "light"
                    },
                    "output_type": "display_data"
                }
            ],
            "source": "df['dayofweek'] = df['effective_date'].dt.dayofweek\nbins = np.linspace(df.dayofweek.min(), df.dayofweek.max(), 10)\ng = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\ng.map(plt.hist, 'dayofweek', bins=bins, ec=\"k\")\ng.axes[-1].legend()\nplt.show()\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "We see that people who get the loan at the end of the week dont pay it off, so lets use Feature binarization to set a threshold values less then day 4 "
        },
        {
            "cell_type": "code",
            "execution_count": 12,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n      <th>dayofweek</th>\n      <th>weekend</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>2016-09-08</td>\n      <td>2016-09-22</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>28</td>\n      <td>college</td>\n      <td>female</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>29</td>\n      <td>college</td>\n      <td>male</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30     2016-09-08   \n1           2             2     PAIDOFF       1000     30     2016-09-08   \n2           3             3     PAIDOFF       1000     15     2016-09-08   \n3           4             4     PAIDOFF       1000     30     2016-09-09   \n4           6             6     PAIDOFF       1000     30     2016-09-09   \n\n    due_date  age             education  Gender  dayofweek  weekend  \n0 2016-10-07   45  High School or Below    male          3        0  \n1 2016-10-07   33              Bechalor  female          3        0  \n2 2016-09-22   27               college    male          3        0  \n3 2016-10-08   28               college  female          4        1  \n4 2016-10-08   29               college    male          4        1  "
                    },
                    "execution_count": 12,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df['weekend'] = df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\ndf.head()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "## Convert Categorical features to numerical values"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Lets look at gender:"
        },
        {
            "cell_type": "code",
            "execution_count": 13,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "Gender  loan_status\nfemale  PAIDOFF        0.865385\n        COLLECTION     0.134615\nmale    PAIDOFF        0.731293\n        COLLECTION     0.268707\nName: loan_status, dtype: float64"
                    },
                    "execution_count": 13,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df.groupby(['Gender'])['loan_status'].value_counts(normalize=True)"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "86 % of female pay there loans while only 73 % of males pay there loan\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Lets convert male to 0 and female to 1:\n"
        },
        {
            "cell_type": "code",
            "execution_count": 14,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n      <th>dayofweek</th>\n      <th>weekend</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>0</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>1</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>2016-09-08</td>\n      <td>2016-09-22</td>\n      <td>27</td>\n      <td>college</td>\n      <td>0</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>28</td>\n      <td>college</td>\n      <td>1</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>29</td>\n      <td>college</td>\n      <td>0</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30     2016-09-08   \n1           2             2     PAIDOFF       1000     30     2016-09-08   \n2           3             3     PAIDOFF       1000     15     2016-09-08   \n3           4             4     PAIDOFF       1000     30     2016-09-09   \n4           6             6     PAIDOFF       1000     30     2016-09-09   \n\n    due_date  age             education  Gender  dayofweek  weekend  \n0 2016-10-07   45  High School or Below       0          3        0  \n1 2016-10-07   33              Bechalor       1          3        0  \n2 2016-09-22   27               college       0          3        0  \n3 2016-10-08   28               college       1          4        1  \n4 2016-10-08   29               college       0          4        1  "
                    },
                    "execution_count": 14,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\ndf.head()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "## One Hot Encoding  \n#### How about education?"
        },
        {
            "cell_type": "code",
            "execution_count": 15,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "education             loan_status\nBechalor              PAIDOFF        0.750000\n                      COLLECTION     0.250000\nHigh School or Below  PAIDOFF        0.741722\n                      COLLECTION     0.258278\nMaster or Above       COLLECTION     0.500000\n                      PAIDOFF        0.500000\ncollege               PAIDOFF        0.765101\n                      COLLECTION     0.234899\nName: loan_status, dtype: float64"
                    },
                    "execution_count": 15,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df.groupby(['education'])['loan_status'].value_counts(normalize=True)"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "#### Feature befor One Hot Encoding"
        },
        {
            "cell_type": "code",
            "execution_count": 16,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>education</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>45</td>\n      <td>0</td>\n      <td>High School or Below</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>33</td>\n      <td>1</td>\n      <td>Bechalor</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>15</td>\n      <td>27</td>\n      <td>0</td>\n      <td>college</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>28</td>\n      <td>1</td>\n      <td>college</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>29</td>\n      <td>0</td>\n      <td>college</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Principal  terms  age  Gender             education\n0       1000     30   45       0  High School or Below\n1       1000     30   33       1              Bechalor\n2       1000     15   27       0               college\n3       1000     30   28       1               college\n4       1000     30   29       0               college"
                    },
                    "execution_count": 16,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df[['Principal','terms','age','Gender','education']].head()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "#### Use one hot encoding technique to conver categorical varables to binary variables and append them to the feature Data Frame "
        },
        {
            "cell_type": "code",
            "execution_count": 138,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>weekend</th>\n      <th>Bechalor</th>\n      <th>High School or Below</th>\n      <th>college</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>45</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>33</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>15</td>\n      <td>27</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>28</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>29</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n0       1000     30   45       0        0         0                     1   \n1       1000     30   33       1        0         1                     0   \n2       1000     15   27       0        0         0                     0   \n3       1000     30   28       1        1         0                     0   \n4       1000     30   29       0        1         0                     0   \n\n   college  \n0        0  \n1        0  \n2        1  \n3        1  \n4        1  "
                    },
                    "execution_count": 138,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "Feature = df[['Principal','terms','age','Gender','weekend']]\nFeature = pd.concat([Feature,pd.get_dummies(df['education'])], axis=1)\nFeature.drop(['Master or Above'], axis = 1,inplace=True)\nFeature.head()\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### Feature selection"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Lets defind feature sets, X:"
        },
        {
            "cell_type": "code",
            "execution_count": 131,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "(346, 8)"
                    },
                    "execution_count": 131,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "X = Feature\nX[0:5]\nX.shape"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "What are our lables?"
        },
        {
            "cell_type": "code",
            "execution_count": 19,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "array(['PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF'],\n      dtype=object)"
                    },
                    "execution_count": 19,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "y = df['loan_status'].values\ny[0:5]\n"
        },
        {
            "cell_type": "code",
            "execution_count": 20,
            "metadata": {},
            "outputs": [],
            "source": "from sklearn import preprocessing\nle_y = preprocessing.LabelEncoder()\nle_y.fit(y)\nle_y = le_y.transform(y)\n\n"
        },
        {
            "cell_type": "code",
            "execution_count": 132,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "(346,)"
                    },
                    "execution_count": 132,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "le_y.shape"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "## Normalize Data "
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Data Standardization give data zero mean and unit variance (technically should be done after train test split )"
        },
        {
            "cell_type": "code",
            "execution_count": 133,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "(346, 8)"
                    },
                    "execution_count": 133,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "X= preprocessing.StandardScaler().fit(X).transform(X)\nX.shape"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "# Classification "
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Now, it is your turn, use the training set to build an accurate model. Then use the test set to report the accuracy of the model\nYou should use the following algorithm:\n- K Nearest Neighbor(KNN)\n- Decision Tree\n- Support Vector Machine\n- Logistic Regression\n\n\n\n__ Notice:__ \n- You can go above and change the pre-processing, feature selection, feature-extraction, and so on, to make a better model.\n- You should use either scikit-learn, Scipy or Numpy libraries for developing the classification algorithms.\n- You should include the code of the algorithm in the following cells."
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# K Nearest Neighbor(KNN)\nNotice: You should find the best k to build the model with the best accuracy.  \n**warning:** You should not use the __loan_test.csv__ for finding the best k, however, you can split your train_loan.csv into train and test to find the best __k__."
        },
        {
            "cell_type": "code",
            "execution_count": 23,
            "metadata": {},
            "outputs": [],
            "source": "def hyperparameter(X_len, accuracy_list, hyperparameter = \"Hyperparameter\"):\n    x_value = np.linspace(1.0, X_len, X_len)\n    plt.plot(x_value, accuracy_list)\n    plt.xlabel(f\"{hyperparameter}\")\n    plt.ylabel(\"accuracy_score\")\n    print(f\"Max accuracy:{np.max(accuracy_list)}\")\n    print(f\"Best {hyperparameter}: {np.argmax(accuracy_list)}\")\n    return np.argmax(accuracy_list), np.max(accuracy_list)"
        },
        {
            "cell_type": "code",
            "execution_count": 64,
            "metadata": {},
            "outputs": [],
            "source": "from sklearn.neighbors import KNeighborsClassifier\nfrom sklearn import metrics\nfrom sklearn.model_selection import train_test_split\naccuracy = []\nlength = int(np.round(np.sqrt(le_y.shape[0])))+1\n# for i in range(length):\nfor i in range(length):\n    X_train, X_test, y_train, y_test = train_test_split(X, le_y, test_size = 0.2, random_state = i)\n    KNN = KNeighborsClassifier(n_neighbors = i+1, metric = \"euclidean\").fit(X_train, y_train)\n    score = metrics.accuracy_score(y_test, KNN.predict(X_test))\n    accuracy.append(score)"
        },
        {
            "cell_type": "code",
            "execution_count": 65,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "Max accuracy:0.8428571428571429\nBest k_value: 16\n"
                },
                {
                    "data": {
                        "text/plain": "(16, 0.8428571428571429)"
                    },
                    "execution_count": 65,
                    "metadata": {},
                    "output_type": "execute_result"
                },
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEICAYAAABF82P+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXicV3X48e8Z7ctI8siWZEvW4k3e49iOHLYsJGQjiRNKmxBaaCiEtKFAKZBQspL8ytKFtkALaWlZmhICIXH2lZAQIJLtLJa8yJska7FkWaNltEsz9/fHzNhjeSSNpHln0/k8jx5LM+8779V4NGfuPfeeK8YYlFJKqYls0W6AUkqp2KQBQimlVFAaIJRSSgWlAUIppVRQGiCUUkoFpQFCKaVUUMlWPriIXAH8K5AE/Jcx5hsT7s8F/hco9bXlH40x/+O7rxFwAW5g3BizdbrrLVy40JSXl4fzV1BKqYS2e/fuk8aYRcHusyxAiEgS8D3gA0ALsFNEnjDG7As47DZgnzHmGhFZBNSLyEPGmFHf/RcbY06Ges3y8nJ27doVrl9BKaUSnog0TXaflUNMVcBhY8xR3xv+w8D2CccYwC4iAmQDTmDcwjYppZQKkZUBohhoDvi5xXdboO8Ca4A2oBb4nDHG47vPAC+IyG4RucXCdiqllArCygAhQW6bWNfjcuBtYAmwCfiuiOT47nuPMWYzcCVwm4hcEPQiIreIyC4R2dXZ2RmmpiullLIyQLQASwN+LsHbUwh0M/Ar43UYaABWAxhj2nz/ngAewztkdRZjzIPGmK3GmK2LFgXNsyillJoFKwPETmCliFSISCpwI/DEhGOOAZcAiEghUAkcFZEsEbH7bs8CLgPqLGyrUkqpCSybxWSMGReRzwDP453m+t/GmL0icqvv/u8D9wM/EpFavENStxtjTorIMuAxb+6aZOD/jDHPWdVWpZRSZ5NEKve9detWo9NclVIqdCKye7J1ZrqSWik1rzU7B3mnuSfazYhJGiCUUvPa/U/t48YH36ClezDaTYk5GiCUUvPa/vY+hsbcfO3JfdMfPM9ogFBKzVsDI+M0O4cozsvghX0d/PpAR7SbFFM0QCil5q2DHS4A/u6qNawoyOaeJ/YyPOaOcqtihwYIpdS85Q8Q64tz+Nr2dTQ7h/j3Vw5HuVWxQwOEUmreOtDuIiMliaULMnn38oVs37SE7796lIaTA9FuWkzQAKGUmrcOdrhYVZiNzeYtHffVq9aQlmzj7h11JNIasdnSAKGUmrfq212sKrSf+rkgJ50vXLaK3x46ybN17VFsWWzQAKGUmpdO9o9wsn+UyiL7Gbf/2fllrF2cw9ee3Ef/yPzenkYDhFJqXjrY7k1QTwwQyUk2Hrh+Pe19w/zby4ei0bSYoQFCKTUv1XcEDxAAm0sXcON5S/nh6w3U+wLJfKQBQik1L9W3u1iQmcKi7LSg999+xWpy0pO56/H5m7DWAKGUmpfqO1xUFtnxbStwlgVZqdx+xWpqGp386s3WCLcuNmiAUErNO8YYDra7qCw8e3gp0J9sXcq5pXn8/TP76R0ci1DrYocGCKXUvNPSPcTAqJvKopwpj7PZhAeuW0/34Cj/+EJ9hFoXOzRAKKXmnYOnEtTZ0x67bkkuH3tXOf9b3cSelvm1b4QGCKXUvHPANzNp1TRDTH5fuGwVC7PTuOvxOtye+ZOw1gChlJp3Dna4KM7LwJ6eEtLxOekp3PnBNbzT0svPao5Z3LrYoQFCKTXveEtsTD+8FOjac5bwrmX5fOu5A5zsH7GoZbFFA4RSal4Zc3s40tk/bYJ6IhHh/uvWMTTm5hvPHrCodbFFA4RSal5pODnAmNuElKCeaEWBnU++bxm/3N3CzkanBa2LLRoglFLzir90RmXhzHoQfn/9/hUU52Vw52N1jLk94WxazNEAoZSaVw52uEiyCcsLsmZ1fmZqMndfs5b6Dhc//n1jeBsXYzRAKKXmlQPtLioWZpGWnDTrx7hsbSHvX13At188SHvvcBhbF1s0QCil5pWDHdOX2JiOiHDvNesY9xjuf3pfmFoWezRAKKXmjcHRcY45B4OW+J6p0vxMbrt4BU/vOc5rBzvD0LrYowFCKTVvHOrox5jQV1BP55YLllGen8m3nk/Maa8aIJRS80b9JLvIzVZ6ShKXry+ivt2VkCU4NEAopeaN+g4X6Sk2Sh2ZYXvMMkcWY27D8d6hsD1mrNAAoZSaN+rbXawssJNkC75J0Gz4g80x52DYHjNWaIBQSs0b/l3kwqks3xcgujRAKKVUXHIOjNLpGpnzFNeJFuemk2wT7UEopVS8CneC2i85yUbxggyaNEAopVR8Or2LXHgDBHjzEDrEpJRScepAu4u8zBQK7Glhf+xSR6YOMSmlVLw62OFiVaEdkfDNYPIry8+kd2iM3sGxsD92NFkaIETkChGpF5HDInJHkPtzReRJEXlHRPaKyM2hnquUUqEyxnCw3cVqC4aXIHGnuloWIEQkCfgecCWwFviIiKydcNhtwD5jzDnARcA/iUhqiOcqpVRI2nqHcY2Mh63ExkSlDm/p8CbngCWPHy1W9iCqgMPGmKPGmFHgYWD7hGMMYBdvny8bcALjIZ6rlFIhqW/vA6xJUIO3cB8kXg8i2cLHLgaaA35uAbZNOOa7wBNAG2AHbjDGeEQklHOVSghff3Y/T71zfE6Pcf25xXzx8sowtSjx1Lf3A+Er0jdRdloy+VmpCTeTycoAESwTNLGa1eXA28D7geXAiyLy2xDP9V5E5BbgFoDS0tJZN1apaHB7DP9XfYwluRlsKMmd1WO82dTN42+3aoCYQn17H4tz08nNSLHsGqX5mTRpgAhZC7A04OcSvD2FQDcD3zDGGOCwiDQAq0M8FwBjzIPAgwBbt25NvHKKKqEdaO/DNTzOrduXcf25JbN6jO/++hD/+MJB+kfGyU6z8k86ftV39Fs2vORX6shkV2O3pdeINCtzEDuBlSJSISKpwI14h5MCHQMuARCRQqASOBriuUrFvZoGJwBVFfmzfozKohzg9EIwdaZxt4cjJ/rDXmJjojJHJsd7hxgd91h6nUiyLEAYY8aBzwDPA/uBR4wxe0XkVhG51XfY/cC7RaQWeBm43RhzcrJzrWqrUtFS0+CkZEEGxXkZs34M/xufv5SEOlNj1wCjbo/1PYj8LDwGWnsSp+y3pf1RY8wzwDMTbvt+wPdtwGWhnqtUIjHGUNPg5MLKRXN6nJIFGWSmJmmAmITVCWo//1qIpq4BKhZmWXqtSNGV1EpFyZHOfroGRtlW4ZjT49hswspCuwaISdS392ETWFGQbel1/GW/mxNoqqsGCKWipDoM+Qe/1YV2zUFMor7DRfnCLNJTkiy9zqLsNNKSbQk1k0kDhFJRUtPgZJE9jfL8uW9/uarITpdvvwN1pvp2l+UJavD25EodmQlV9lsDhFJRYIyh+qiTbRWOsBSP89cY0l7EmYZG3TQ5By1PUPuVOjJ1iEkpNTfNziHa+4bnnH/w8ydgD2ge4gyHTrgwhoj0IMC7WO6YcxDv0q74pwFCqSiobugCwpN/AFhkTyM/K5WDGiDOYNUucpMpc2QyOOrmZP9oRK5nNQ0QSkVBTYOTvMwUVoZxZs2qQjv1OsR0hoMdLtKSbZTlR2ba6emifYlR1VUDhFJRUNPopKrcgc0Wvs1rKou8M5k8nsQY3giHA+0uVhZmkxTG53kq/rLfiVLVVQOEUhHW3jtMU9cgVWHKP/hVFtkZHHUn1EreufLvIhcpJQsyECFhprpqgFAqwvz5h21hyj/4aaL6TD2Do3T0jVi2i1ww6SlJFOWkJ0zZbw0QSkVYTYOT7LRk1iwO7xvXqkJvPkOnunr5E9SR7EGAd6qrDjEppWalpsHJ1vIFJCeF98/Pnp5CcV6G9iB8/An71b5qt5GSSIvlNEAoFUFd/SMcOtEf9vyD3+oiu0519alvd5GTnkxhTlpEr1uWn0mna4ShUXdEr2sFDRBKRdDORm/9pXAtkJtoVZGdI539CbUnwWzVt7uoLLKHZaX6TCx1JM7+1BoglIqg6gYnack2NhTnWfL4q4vsjHsMDScTYx7+bBljqO9wRWyBXCD/mgsNEEqpGalpcLK5dAGpydb86Z2eydRnyePHi+O9w7iGxyNWYiNQ4L4Q8U4DhFIR0jc8xr7jfWxbZs3wEsDyRdkk22Tez2TyJ6grI5ygBliQmYI9LVl7EEqp0O1qdGIMliWoAVKTbVQszDq1i9p85U/UR6MHISKnivbFu5AChIiUicilvu8zRCTyz7pSca66wUlKknDu0gWWXqeyyE59x/weYqpvd1GUk05uZkpUrl/qyEyIxXLTBggR+RTwS+AHvptKgMetbJRSiaimwcnGkjwyUq3d2ayy0E6zc4iBkXFLrxPL6jtcrIpCgtqvND+Tlu4h3HFeFyuUHsRtwHuAPgBjzCGgwMpGKZVoBkfHqW3ptWx6a6BV83zzoHG3h0Mn+iNaYmOiUkcmo24P7X3DUWtDOIQSIEaMMaeKm4tIMhDfYVGpCHuzqYdxj7E0/+A333eXa3IOMjruiXiJjUBlvqqu8T6TKZQA8aqI/B2QISIfAH4BPGlts5RKLDUNXdgEtpRZm38AWLogk4yUpHlbcqM+iglqP/9U13jffjSUAHE70AnUAp8GngHutLJRSiWa6gYn65bkYk+3PmlqswmrCrPnbQ+ivt2FCKwsDN9mTDO1JC+dZJvEfdnv5KnuFBEbsMcYsx74z8g0SanEMjLu5q3mHj52flnErrmq0M4r9Scidr1YUt/uojw/i/QUaycDTCU5yUbxgoy4n+o6ZQ/CGOMB3hGR0gi1R6mEs6ell9FxT0TyD36VRXZO9o9ysn8kYteMFQc7XFEdXvJLhLLfoQwxLQb2isjLIvKE/8vqhimVKKqPejcIOq88sgECmHeVXYfH3DR2DUR1iqtfIgSIKYeYfO6zvBVKJbDqBieVhXYWZKVG7Jr+AFHf4eLdKxZG7LrRdvhEPx5DVKe4+pU6MukZHKN3aIzcjOgs2JuraXsQxphXgQOA3fe133ebUmoa424Pu5u6Izq8BLAoO40FmSnzLlEdrV3kginL95X9juNEdSgrqf8EqAH+GPgToFpEPmx1w5RKBHvb+hgcdVtaoC8YEaGyyD7vprrWd7hITbZR7ntzjqZSR/yX/Q5liOmrwHnGmBMAIrIIeAlv+Q2l1BSqG7z5h6oI5h/8Kgvt/HJ3C8aYiG+aEy317S5WLMoO+3aus1HqC1JNzvhdLBfKs2jzBwefrhDPU2req2lwUrEwi4Kc9Ihfu7Ioh4FRNy3dQxG/drTUt7tiIv8AkJ2WTH5WalwvlgulB/GciDwP/Mz38w3As9Y1SanE4PEYahqcXLl+cVSuX1nkXSh2sMN1ahvMRNY7OEZ733BMzGDyW+rIjOvFcqEkqb+Et5LrRuAc4EFjzJetbpiKnCOd/XTNw/nyVqvvcNE3PB7x/IPf6d3l5kce4tQmQTGQoPYri/N9IUJJUlcAzxhjvmCM+Ru8PYpyqxumIsPjMdzwgzd44On90W5KwqlpcALWbhA0FXt6CsV5GfNmJtPpXeRiJ0CUOjJp6xlidNwT7abMSii5hF8Agb+d23ebSgCHTvRzsn+Et451R7spCae6oYvivAxKFkRveGdVYfapqZ+J7mC7C3t6MotzI5/vmUypIxOPgdae+MwDhRIgkgPLffu+j9yKH2WpGt8sm8auQXoHx6LcmsRhjDf/EK3eg19lUQ5HOvsZc8fnJ9iZqG/3ltiIpRlbZfnxPdU1lADRKSLX+n8Qke3AyVAeXESuEJF6ETksIncEuf9LIvK276tORNwi4vDd1ygitb77doX6C6mZqfYNgwDUtfVGsSWJ5ejJAU72j0Zkg6CpVBZlM+Y2NJyM36mWoTDGRH0XuWD8Zb+Pxem+EKEEiFuBvxORYyLSjLf896enO0lEkoDvAVcCa4GPiMjawGOMMf9gjNlkjNkEfAV41RjjDDjkYt/9W0P8fdQM+D/lXrBqEeAtKqfCI9r5B7/KwhyAhB9m6ugboXdoLGamuPoV2NNIS7bFbQ9i2mmuxpgjwPkikg2IMSbUV1oVcNgYcxRARB4GtgP7Jjn+I5yeSqsioLFrkBOuET53aSGNJweoa9UAES7VR7tYmJ1GxcKsqLZjeUEWSTZJ+ES1P0EdCyU2AtlsEtdTXUOZxfQ5EckBBoBvi8ibInJZCI9dDDQH/Nziuy3YNTKBK4BHA242wAsisltEbgnhemqG/PmHbRUONpTksqe1J8otSgzGGKobnGyrcER9PDwtOYmKhVkJP9W1vr0PiK0prn5lcVzVNZQhpk8YY/qAy4AC4GbgGyGcF+wvY7K9rK8BfjdheOk9xpjNeIeobhORC4JeROQWEdklIrs6OztDaJbyq25wkp+VyvJF2WwozqXZOUT3wOj0J6optXQPcbx3OGrrHyaqLLQnfg+ivZ8Ce1pEK+aGaqkvQBgz2dtf7AolQPjf6K8C/scY8w7B3/wnagGWBvxcArRNcuyNTBheMsa0+f49ATyGd8jqLMaYB40xW40xWxctWhRCs5Sff5aNiLCxOBeAWh1mmrNYyT/4VRbZOeYcZHB0PNpNsUx9R19MrX8IVJafyeCom5P98ffhK5QAsVtEXsAbIJ4XETtnrouYzE5gpYhUiEgq3iBw1kZDIpILXAjsCLgty3cdRCQLb++lLoRrqhC19gzR0j106k1snQaIsKlpcJKbkcKqgth4w1pVaMcYONTRH+2mWMLtMRzq6I/J4SUIKPsdh8NMoQSIvwDuwFvRdRDvGoib/XeKyLpgJxljxoHPAM8D+4FHjDF7ReRWEbk14NDrgReMMYHzwAqB10XkHbylxp82xjw3g99LTcOff/AHiNyMFCoWZrGnRfMQc1Xd0MV55Q5sttiYj++f2ZOoM5maugYYGffE3BRXv1NTXeOwqmsos5g8wJsBP3fhrejq91Ng8yTnPgM8M+G270/4+UfAjybcdhRv3SdlkZoGJ/b0ZFYX5Zy6bUNxLrsanVOcpabT0TdMY9cgH91WFu2mnLLUkUl6iu3UTJ9E48+vxNoUVz//SvpjXfG3mjocZbtj42OSmpHqBidV5Q6SAj7lbizJpa13OOIb3Q+OjjMy7o7oNa3izz/ESoIaIMkmrCq0W9aDGB5zRzW/Ud/ejwisjJEhvYnSU5IoykmPy30hwhEg4i81P891ukY42jlwVhJ1Q5TyEDf84A3uejwxUkw1DU6yUpNYuzhn+oMjaFWh3bIexK3/u5vt3/1dVIK8MYbfHuqkPD+LjNSkiF8/VKX5mXG5L4Ru/DMPTTbLZl1xLiJQG8EV1Sdcw9S29vJKfWdcTgOcqLqhiy3ljpjY0SxQZaGdTtcIzjBPY244OcBv6js5dKKf//ptQ1gfOxQ73m5jV1M3n3xfRcSvPRNlcbpYLhyv4vibuzXP1TR0kZmaxHpfj8EvOy2ZZQuzIlpyY2eDt4psp2uExjj8AwrkHBjlYEd/1OsvBVNpUaL6ZzXHSLYJ716ez3d+fSiin5L7hsd44On9nLM0jxvPK43YdWej1JHJCdcIQ6PxNZQaykrqR0XkgyIS9FhjzPnhb5ayUnWDky1lC0gJ8il3Y0ketRFcUV3T0IU/DeKfWRWvdvoS/LEcIMK5YG54zM0vdjVz2bpC/vGPz8Emwn1PTlZJJ/z++YWDdA2M8MD29Wfk0mKRf3/q5u74+hAUSg/iP4CbgEMi8g0RWW1xm5SFegZHqe9wUVUe/E1sQ3EuHX0jdPQNR6Q91Q1O3rNiIflZqWdUlo1HNQ1O0pJtbCjJnf7gCCuwp5GXmRLWkhvP1bXTPTjGR7eVsSQvg89espKX9nfw0r6OsF1jMnWtvfzkD4382fllMfl8T+Sf6hpvw0yhbDn6kjHmo3insjYCL4rI70XkZhFJsbqBKrx2NnZjzOSrfDf6/tgikYfwB6ttFQ6qKhxUH43vAFHd0MW5pXmkJcdeslTEO5MpnD2Ih6qbKM/P5F3L8gH4xHsqWFmQzb1P7rV0KMXjMdz5eB2OrFT+9rJKy64TTvG6L0RIOQgRyQf+HPgk8Bbwr3gDxouWtUxZoqahi9RkG+cszQt6/9olOdgE9kRgJtPpYJVPVYXDt7o7vv6A/PqGx9jX1kdVRX60mzKp1UV2Dra7wjIZ4GCHi52N3dy0rfTUgsDUZBv3X7eelu4hvvfK4TlfYzI/39XM2809/N1Va8jNiI/PqAsyU8hOS467fSFCyUH8CvgtkAlcY4y51hjzc2PMXwPZVjdQhVdNg5NNS/NITwn+KTczNZmVBfaIlP72B6uNJbmnejQ743Sh3u6mbjwGzo/B/IPfqkI7rpFx2nrnPnz4f9XHSE2y8eEtS8+4/fxl+Vx/bjEPvnaUI53hL+3hHBjlm88doKrCwfXnBi0OHZNEhNI4rOoaSg/iu8aYtcaYrxtjjgfeoRv5xJf+kXHq2vqmTaKuL85lT0uv5dNOA4PV6qIcctKTT03BjTc1DU6SbcK5pQui3ZRJnS650TenxxkcHefRN1u4akMRjiDVU79y1WrSUmzcs2Nv2F9D33z2AP3D4zxw3fqol1KfqbL8TJoSMECsEZFT4xEiskBE/srCNimL7G7qxu0x01YZ3ViSy8n+EdotTFT7g5X/E3eSTTivPH7zEDUNTjaW5Mb0Yq2Vhf4AMbdP9k+9cxzX8Dg3TVJOpMCezhcvq+T1wyd5as/xoMfMxu4mJz/f1cxfvLci5jYGCkWpI5MW5xBuT/ys9wklQHzKGHNq3qMxphv4lHVNUlapaegiySZsnuZTrn9WiJXrIU4Hq9Nj9lUVDo6eHOCEKzIzqMJlaNTNnpaemM4/gLcg45Lc9Dn3IB6qOcbKgmzOK5/8dfSn55exbkkO9z+1D9fw2JyuBzDu9nDn43tZnJvOZy9ZOefHi4bS/ExG3Z6IzRAMh1AChE0C+nK+vaZjb1cONa2aBicbinPJSpu6RuPaxTkk2cTSmUw1DV0k24TNZaeT5afyEL7Fc/HirWPdjLlNTK5/mGhVkZ36OZT9rmvt5Z3mHj66rXTKIZ4km/DAdevp7B/hX146NOvr+f3kD03sP97H3Vevnfb1G6vicaprKAHieeAREblERN6Pd2MfLb0dZ4bH3LzT3BvSm1h6ShKrCu2WzmSqaXCyvjiXzNTTf+zen5PibsFcdYMTm8CWKT5Rx4rKIjtHTvQz5g5lS5ezPVR9jPQUG9dvLpn22HNLF3DjeaX86PeN7D8++17Lib5h/vnFg1y4ahFXrC+a9eNEW5nDO9U1nmoyhRIgbgd+DfwlcBvwMvBlKxulwu/t5h5G3Z6QdznbWJxLbUuPJYnqU8FqQsXTlCQbW8oWxNWCOWMMr9SfYM3iHHLSY3/KZWWhnVG3h6ZZTLd0DY+x4+1Wrtm4JOTppV++vJLcjBTufLwOzyzH3h94ej+jbg/3Xbsu7hLTgZbkpZNkk7iq6hrKQjmPMeY/jDEfNsb8kTHmB8aY+Coooqg+6kQEtk6ygnqiDSW5dA+O0dId/hr2bx3zBqtgvZmqcgf1HS56BuOjxNdjb7Wyp6WXPz0/dvZ/mMqqOSSqd7zdxuCom4/O4HddkJXKHVeuZndTN798s2XG1/zd4ZM88U4bf3nhcsoXZs34/FiSnGSjOC+DY8742RcilHUQK0XklyKyT0SO+r8i0TgVPjWNXawpygn5k5+/9LcV6yFqGrzBaktZkABR4cAY7yK6WNc7NMbfP7OfTUvzuGHr0ulPiAErCrKxycynuhpjeKj6GOuW5HDODEtbfHhzCVvKFvCNZw/MKPCPjnu4a0cdpY5M/vKi5TO6Zqwqy8+Mq8VyoQwx/Q/eekzjwMXAT/DuIqfixOi4h91N3SEPLwGsXmwnJUksyUNMFazOWZpHarItLvIQ//RCPc6BUR64bn3MbC86nfSUJMoXZs14b4i3mnvYf7yPm6ZJTgdj8yWse4fG+Nbz9SGf95+/PcrRzgHu275u0oWd8WapI77WQoQSIDKMMS8DYoxpMsbcC7zf2mapcKpr62V4LPiQzmTSkpOoLLKHfSaTP1hNtuNaekoSm5bmxfyCudqWXv73jSY+9q7ys8qmx7rVRTPfXe6hN46RlZrE9k2zW728ZnEOH39XOT+rOcbbzdNXC27pHuQ7vz7EFeuKuLiyYFbXjEVljkx6BsfoHZr71N9ICCVADPtKfR8Skc+IyPVA4vyPzQP+xWfnzXAa5obiPPaEOVFd2zp9sNpW4aCurY/+kehtYzkVt8dw5+O1OLLS+MJlq6LdnBlbVWinyTkYckG93sExntrTxnXnFpM9hymmf/OBlSzKTuPOx2unXSx235P7EIS7r1k76+vFIv9U13iZyRRKgPg83jpMnwW2AH8KfNzKRqnwqmnoYkVBNguz02Z03saSXPqGx8NaP8bfMzhvimR5VYUDt8ewuyk28xAP7zzGOy293PnBNXExc2mi1UV2jIFDJ0LrRTz6Zgsj4x5u2ja3TXns6SncdfVa6lr7eKi6adLjXt7fwYv7OvjcpStZkpcxp2vGGv++EPFSk2nKAOFbFPcnxph+Y0yLMeZm30ymNyLUPjVHbo9hV+PM8g9+/kR1OFdU+4NV/hTBakvZApJtEpN5iK7+Eb71XD3nL3OwfdOSaDdnVk7PZJo+QHiT001sWprHuiVzH0q7euNi3rtiIf/wfD2drpGz7h8adXPPE3tZUZDNJ94T29uIzka8LZabMkD4prNukXiefDzP7T/eh2tkfFarfFcV2klNslEbpkS1P1hN15bM1GTWF+fGZB7iG88eYGAkPovF+ZXlZ5GWbAspQNQ0ODnSOcBH59h78BMR7tu+juExN19/Zv9Z9//7bw7T0j3E/dvXk5ocW/t6h4M9PQVHVmpi9CB83gJ2iMificiH/F9WN0yFh3/R2Wx6EKnJNtYstrOnJTxbkPqDVSht2Vbh4J3mXobHYmfJza5GJ7/Y3cIn37eMFQXxVyzOL8kmrCzMDmkm00PVx8hJT+bqjeHrLS1flM0tFyzjV2+18sbR073Eo539/ODVo1x/bjHvWh7bda3mwlv2O96nkVoAAB8iSURBVD6muoYSIBxAF96ZS9f4vq62slEqfGoauih1ZLI4d3ZjuRtKctnb2jfrVbCBZhKsqiocjLo9vHUscvtjT8VbLK6OJbnpfPaSFdFuzpxVFuZM24M42T/Cs3XH+dDmkrBXqf3MxSspzsvgrsfrGHN7MMZw9469pCXb+MpVib2rcakjMzGGmAB8eYeJX5+IROPU3BhjqGlwzqr34LexOA/XyDiNYVjcM5NgtbXcgQgxM8z0o983cqDdxd3XrDujflS8qizK5oRrhO6ByReu/XJ3C2NuE7bhpUAZqUncd+06Dp3o579fb+Dp2uO8fvgkX7y8kgJ7etivF0vK8jNp6xmadT2sSJr2lS4i/wOc9fFRg0TsO3yin+7BsTkFCH/p79rWXpYtmv0Ggv5gdcmawpCOz81IYU1RDjWNXUB0yzu39w7z7RcPcnHlIi5fF1r7Y11lUQ7g3Tp027Kzh3M8HsPPao5RVeE4tY9EuF26tpBL1xTwry8fIjstmXVLcuKmZMlcLHVk4jHQ2j0UlvIhbo/BJliSEwtliOkp4Gnf18tADhD+vQTnsbt31PH4W61hf9w3fJ++51KGemVBNmnJtjnPZPIHq5m0parCwe6mbkbHo/tJ64Gn9zHuMdx3bfwmpieq9M9kmiQP8bsjJ2nqGrSk9xDonmvW4TGGzv4RHrhuPUlxsiJ9Lsoc4Z3q+sPXj/JnP6xhcDT864am7UEYYx4N/FlEfga8FPaWzFOu4TF+8ocmHt3dwvnL8inKDV/3uqbBSVFO+qmpdbORnGRj3ZKcOa+oPh2sQk8+bqtw8KPfN1Lb2suWsuiU0n79kHdXtL+5dNWpOeyJoDAnjdyMlEnzEA+9cQxHVqrl5bWXOjL5lxs2cbJ/NKa3aw2nsnxvryEcJTfaeob4l5cO8e7l+ZYMfc5mHtlKwNqPFfNIXau3aNrAqJsHnt4Xtsf1Dul0UVXhmPOn3g3FudS19c5pq0R/sFrqCD1Z7h8ai1YeYmTczd076ijPz+TTFy6LShusIiJUFgYvudHRN8yL+zv44y0lpCVbXwPpivWL58XQkl+BPY3UZFtYVlPf/9Q+PMZwzzXrwtCys4VSzdUlIn3+L+BJvHtEqDCobfXO0vn4u8p4as9xXj90MiyPe8w5SEffyJzyD34bSvIYHHVztHN2I4uzDVb52WmsKMiO2oK5/3ztKEdPDnDf9vUJUywu0Koi71TXiaVUHtnZjNtj+EiVfg60gs0mvplMc5v48Zv6Ezxb185nLl7B0jmMEkwllFlMdmNMTsDXqonDTmr29rT0UpyXwVeuWkN5fiZ376hjZHzuc/+rw5B/8NsYkKieDX+wmqxA31SqKhzsauyO+Ebvzc5BvvPrw1y1oYgLVy2K6LUjpbIoB9fwOMd7T++R7PYlp9+7YmHc778Qy+Y61XV4zLvifNnCLD51gXW921B6ENeLSG7Az3kicp1lLZpnalt72ViSS3pKEvdtX8/RkwP852tz326j+qgTR1YqKwpmP/PIb/mibDJSkmadqPYXC5xNsNpW4cA1Mj6nLStn474n95JkE+66OrGKxQUKlqj+Tf0J2nqHLU9Oz3eljkyanYOzLoT5g1eP0tQ1yNe2r7d0GDCUHMQ9xphT7wzGmB7gHstaNI/0Do7R1DV4airphasWcdWGIr7z68NzHp+saeyiqnzu+QfwrrxdX5wz6x5EdYM3WC2fxTRZ/xBZJLchfXFfBy/tP8HnL1056wWG8aAySE2mh6qPsciexqVrE2M6b6wqy89kYNRN1xTrUCbT1DXA935z2FvXauVCC1p3WigBItgx8b9SKAbUtXnfcDcE7Cdw19VrSbIJ9z25d9aP29YzRLNzKCz5B78NxXnsbetlfBaLe+YSrBbnZlDqyIxYHmJo1M29T+xlVWE2NydgsbhAuZkpFOWkc9AXIFq6B3ml/gQ3bF1KSlLi1UGKJaWznOpqjOGeJ/aSmmSLSO82lFfBLhH5ZxFZLiLLROTbwG6rGzYf+IdsAgPE4twMPn/pSl7af4IX93XM6nF3Ns6+/tJkNpbkMjzm4fAME9X+YDWb/INfVYWDmgZnWPelmMx3XzlEa4+3WNx8eJOsLLJzwBcgfr6zGYAbq+Jj+9R4VuYv+z3DPMTzezv4TX0nn790JYU51q84D+Uv4K+BUeDnwCPAEHCblY2aL2pbeyh1ZJKXmXrG7Te/p4JVhdnc+8TekDd1CfTGUSf29GTWLM4JV1NP7Zo20zxEzRyKBfpVVTjoHhzj0Alr12ce6eznwdeO8qHNxUFXFyeiyiI7hzv7GR5z8/DOZi6uLKBkQeKs94hV/ud4JonqwdFxvvbkXlYX2fnzd5db1LIzhTKLacAYc4cxZqvv6++MMfFRijDG7WnpPZV/CJSSZOP+7etp7Rniu68cmvHj1jR0cV65I6yrUpctzCIrNWnGC+aqG7zBanXR7IPVtgjkIbzF4urISEniK1eusew6saay0M7ouIcfvt5Ap2tEk9MRkp6SRFFO+oyGmP7t5cO09Q7zwHXrSY5Q7zaUWUwvikhewM8LROT5UB5cRK4QkXoROSwidwS5/0si8rbvq05E3CLiCOXceOccGKWle4iNk+xnvG1ZPh/aXMyDrx3lyAyGdU72j3CkcyCsw0vgnbu9vjiXPTNMVIcjWJU6MinKSbd0wdyTe47zu8NdfOnyShbZZ7bzXjyrLPImqr/3ymGW5KZzUQLt/xzrZlL2+1CHi//67VH+eEsJW6fYjTHcQglDC30zlwAwxnQTwp7Uvt3ovgdcCawFPiIiZ2RVjDH/YIzZZIzZBHwFeNUY4wzl3HjnnxEUrAfh95Ur15CeksTdO+pCHn/fGYYhnclsLMll//G+kKtQhitYiYgvD9FlSR7CNTzGA0/tY0NxLjdtmz8regFWFGRjExgcdXNjVem8qIUUK0rzM0PqQRhjuPPxOrLSkrnjysiWQg8lQHhE5FS/U0TKCVLdNYgq4LAx5qgxZhR4GNg+xfEfAX42y3PjTq1vE571k/QgABbZ0/jy5ZX87nAXT+05HtLjVjc4yUhJOiPxHS4bSvIYHfdwMISNZuB0sArHYr2qCgcdfSOW1NH/9ouH5lWxuEDpKUmU52eRZBNuOE+T05FU5siko29k2k2xdrzdRnWDk9uvWD3lVr1WCCVAfBV4XUR+KiI/BV7F+2l/OsVAc8DPLb7bziIimcAVgH+F9kzOvUVEdonIrs7OzhCaFRv2tPSybGHWtJve37StjA3Fudz/1D5cw2PTPm51g5MtZQssmYHjHw4LNQ/hD1ZTBcFQbbOoLtO+tj5+/IdGPlJVyjlL86Y9PhF9eGsJn3rfsojMilGn+Ys/TrXmqXdojAee3s85S/O4MQoBPJQk9XPAVqAe70ymv8U7k2k6wT6KTdbzuAb4nTHG/9cf8rnGmAf9CfRFi+KnJEJda29Ib5xJNuH+69bT2T/Ct1+cOmHdOzjGgfY+S4aXwDs1z56eHHIeIpzBakVBNo6s1LAmqj0ew1076sjLSOH2yxN7F7Op/NVFKyI+dKFOr4WYqlf87RcP4hwY4YHt67FFoXcbSpL6k3j3gfhb39dPgXtDeOwWIDDklQBtkxx7I6eHl2Z6btzpdI3Q1jt8qsbRdDYtzeOmqlJ+/IdG9rVNXnJiV5MTY6zJP4A3F7ChODekHkS4g5WIUFXu8G0gFB6/3N3C7qZu7rhyNbmZU/fklAq3UwFikh5EXWsvP/lDI396ftmUuUorhfLR7nPAeUCTMeZi4FwglLGcncBKEakQkVS8QeCJiQf56jxdCOyY6bnxqq717AVy0/nS5ZXkZqRw1466SfeHrmlwkppkY5OFQyUbSnI50N43bUFBf7AKR/7Br6rCQbNziLaeUDqwU+seGOXrz+7nvPIF/NHmkjC0TqmZcWSlkp2WHHSIyePxJqYdWan87WWVUWidVygBYtgYMwwgImnGmAPAtC02xowDnwGeB/YDjxhj9orIrSJya8Ch1wMvBK6tmOzcUH+pWLenpRcRWDeDAJGXmcpXrlzN7qZufvlmS9Bj3mhwsmlpnqWlqTcW5zHmNtNueF/tC1bhHNcP5/4Q33q+nr7hce6/Ljpdd6VEhKWTlP3++a5m3m7u4asfXENuRvR6t6EEiBbfOojHgRdFZAchDvcYY57xlQdfboz5f77bvm+M+X7AMT8yxtwYyrmJora1h+WLsslOm1lJqz/aXMLWsgV8/Zn9Z202PzAyTl1rr2XDS36hlv6utiBYrVmcgz09ec55iLeOdfPwzmPc/O7yOS3gU2quyhxnT3V1DozyzecOsK3CwXWbgs7NiZhQktTXG2N6jDH3AncBPwS03Pcc7GnpnXSB3FRsvoR13/A433q+/oz73jzm3TPB6gBRsiCDvMyUKfMQVgWrJJtwXrljToX73L6ue4E9jc9/YFUYW6fUzJXlZ9LcPXTGsPE3nz1Av693G+090Gc0vcQY86ox5gnf2gQ1Cx19w5xwjcw66bRmcQ43v7uch3ce461j3adur2lwkmQTy/du9ieqp6rJ5A9WcynQN5mqCgdHOgc42T8yq/P/940m9rb1cdfVa2fcg1Mq3JY6Mhkd99Dh8m7atLvJyc93NfMX761gla8cezQlfrnKGON/Yw11BlMwn//AKgrsady1o+7UTmvVR52sL84lKwJvehtLcjnY4Zp0gU/1UW+w2mzBJvRzyUOccA3zjy/U876VC/nghsXhbppSM+av6trUNci428NXH6tjcW46n71kZZRb5qUBIsJqW3uxCaxdPPsAkZ2WzF1Xr6WutY+HqpsYHnPzdnNPWGcMTWVDcS7jHjPpLm81DdYFqw3FuWSkJM0qQHz9mQOMjHm479p1Ue+6KwUB+0J0DfKTPzRxoN3FPdesjcgHvVBogIiw2pYeVhbYyUidW/L2gxsW894VC/mH5+t5aX8Ho24PVREq4rWhxDszKVii2upglZJkY0vZghknqv9wpIvH3mrl0xcuY9ksdrZTygpL8jJIsgk7G53884sHuXDVIi5fVxTtZp2iASKCjDHUtgYv8T1TIsLXtq9jZMzDHY/WIgLnRShALMlNJz8rNWge4p3mHsuDVVWFgwPtffQOTl96BGB03MPdO+pY6sjgtotXWNYupWYqJclGcV4Gv9jdwqg79nq3GiAi6HjvMCf7R+eUfwi0bFE2n75wGf0j46wuyonYamARYUNJ8BXVNQ1Oy4NVVYUDY07vnDed//5dA4dO9HPvNessXSOi1Gz4h5n+6qLllC/MinJrzqQBIoKCbTE6V3910QpWFmRz+brIbjK/sTiXQydcZ+14V93gtDxYbVqaR2qSjZoQAkRbzxD/+tIhPrC2kEvWRPY5UioU55U7WF1k59YLl0e7KWeJjUzIPFHb2kOyTcK6FWhGahIv/M0FEe+WbijJw2Ng3/FetpR5ewtjbg+7m7otLxudnpLEpqV5IeUhvvbkPgyGe65JqO1EVAL53KUr+ewlK2JqaMlPexARtKell1WF9rAPc0TjheUfJgvMQ9S19jI05rZ8sR54h5nqWnsZGBmf9JhX6k/w3N52/vr9K3WfZRXTYjE4gAaIiDHGUNfaG7b8Q7QV5qSzyJ52Rh6ixsLd7CaqqnDg9hh2N3UHvX94zM09O/ayfFEWn3rfMsvbo1Qi0gARIS3dQ3QPjoVl85xYsXHCHtXVDU6WL8piYQR2vdpStoAkm0y6HuI/fnOEY85B7t++ntRkfZkrNRv6lxMh/jUDidKDAG/p7yOd/fSPjOP2GHY2OqmqyI/ItbPSkllfnBs0QDSeHOA/Xj3Ctecs4d0rFkakPUolIg0QEbKnpZeUJKGyKPr1VcJlY0kuxsDe1l4OtPfhGh6P2Gpu8O418XZzzxklP4wx3P3EXlKTbNz5wTURa4tSiUgDRITUtvawuiiHtOTEmYfvHy6rbe2NaP7Br6rcwajbwzvNPadue66undcOdvKFD6yiQPdYVmpONEBEgDGGPS3hWUEdSwrs6SzOTae2tZfqo06WOjJYkpcRseufV+5AhFPTXQdGxvnaU/tYsziHj72rLGLtUCpR6TqICGjqGsQ1PD6rPSBinb/0d+/QGBdXFkT02rmZKawuyjnVe/m3lw9xvHeY7960meQk/eyj1FzpX1EE+Gf6JFoPArwBouHkAM6B0YjmH/y2VTjY3dTN3rZefvh6AzdsXWr5nhhKzRcaICKgrrWX1GRbTGwAEm6BQS+S+YfAaw6NubnlJ7vJTk/m9itXR7wNSiUqDRARsKelhzWLc0hJwGEPf12pwpy0U5ufRJI/KLX2DHH7FatxZKVGvA1KJSrNQVjM4zHUtfZx/bnR3XzcKvnZaawoyGbT0ryolAtYmJ3GmsU5ZKTYuGGrtTWglJpvNEBYrKFrgP6R8YTMP/g98ul3kZ4Svd7RQ5/cRmqyDZstNuvZKBWvNEBYrDYMe1DHumgP60T7+kolqsQbFI8xe1p6SU+xsUK3uVRKxRkNEBarbe1h3ZJcnZevlIo7+q5lIbcvQR3OHeSUUipSNEBY6GhnP0Njbg0QSqm4pAHCQnvmQYJaKZW4NEBYqLa1l8zUJJZpglopFYc0QFhoT0sP65fkkqTz85VScUgDhEXG3R72tvUl9AI5pVRi0wBhkUMn+hkZ92j+QSkVtzRAAO29w/QNj4X1Mf0rqHUGk1IqXs37ANE7OMZl336Vf3q+PqyPu6e1B3taMuX5WWF9XKWUipR5HyByM1O4/txifvpGE3W+jX3Coba1j3XFOVpATikVt+Z9gAD4wmWVOLLS+OrjdXg8Zs6PNzruYf/xPjaW5IWhdUopFR0aIIDcjBS++sHVvNPcw8M7m+f8eAc7XIyOezT/oJSKa5YGCBG5QkTqReSwiNwxyTEXicjbIrJXRF4NuL1RRGp99+2ysp0A120qZluFg28+d4Cu/pE5PVZtq66gVkrFP8sChIgkAd8DrgTWAh8RkbUTjskD/h241hizDvjjCQ9zsTFmkzFmq1XtDGgLD1y3noGRcb753IE5Pdaell5y0pMpdUR+C06llAoXK3sQVcBhY8xRY8wo8DCwfcIxNwG/MsYcAzDGnLCwPdNaWWjnL95XwSO7Wtjd5Jz149S29rCxJDpbcCqlVLhYGSCKgcAB/RbfbYFWAQtE5DcisltEPhZwnwFe8N1+y2QXEZFbRGSXiOzq7Oycc6M/+/6VLMlN56uP1THu9sz4/OExN/XtLl1BrZSKe1YGiGAfnydOEUoGtgAfBC4H7hKRVb773mOM2Yx3iOo2Ebkg2EWMMQ8aY7YaY7YuWrRozo3OSkvm7mvWcqDdxY//0DTj8+vbXYy5DRs1Qa2UinNWBogWYGnAzyVAW5BjnjPGDBhjTgKvAecAGGPafP+eAB7DO2QVEZevK+KiykV8+8WDdPQNz+hcf4J6vQYIpVScszJA7ARWikiFiKQCNwJPTDhmB/A+EUkWkUxgG7BfRLJExA4gIlnAZUCdhW09g4hw37XrGHV7eODp/TM6t7allwWZKZQsyLCodUopFRmWBQhjzDjwGeB5YD/wiDFmr4jcKiK3+o7ZDzwH7AFqgP8yxtQBhcDrIvKO7/anjTHPWdXWYMrys/iri5bz5Dtt/O7wyZDP29PaywZNUCulEoAYM/eVw7Fi69atZteu8C2ZGB5zc/m/vEaSTXj2c+8jLTlp2uPX3fM8f3nhcr54eWXY2qGUUlYRkd2TLSXQldRTSE9J4r5r13G0c4D/+m3DtMfvO96H22N0BpNSKiFogJjGRZUFXLm+iO/8+hDNzsEpj63VPaiVUglEA0QI7rp6LTYR7nty35TH7WnpZWF2GkU56RFqmVJKWUcDRAiW5GXwuUtW8tL+Dl7a1zHpcXWtvWwoztEEtVIqIWiACNEn3lvByoJs7n1yL0Oj7rPuHxwd59AJFxu0xLdSKkFogAhRSpKNB65bT0v3EN975fBZ9+9r68Nj0BXUSqmEoQFiBrYty+dD5xbz4GtHOdLZf8Z9e/x7UGuCWimVIDRAzNBXrlpDWoqNe3bsJXANSW1rL4U5aRRqgloplSA0QMzQInsaX7q8ktcPn+SpPcdP3b6npYcNxZp/UEolDg0Qs/DRbWWsL87h/qf24RoewzU8xtGTA7r+QSmVUDRAzEKSTXjgug109o/wLy8dYm9bH8Zo/kEplViSo92AeLVpaR4fqSrlR79vpGdwDIANOoNJKZVAtAcxB1++vJLcjBQefbOFJbnpLMxOi3aTlFIqbDRAzEFeZip3XLka0OElpVTi0SGmOfrw5hL2tfVxUeXctztVSqlYogFijmw24d5r10W7GUopFXY6xKSUUiooDRBKKaWC0gChlFIqKA0QSimlgtIAoZRSKigNEEoppYLSAKGUUiooDRBKKaWCksBNb+KdiHQCTdFuxyQWAiej3YgpaPvmRts3N9q+uZlL+8qMMUFLQSRUgIhlIrLLGLM12u2YjLZvbrR9c6Ptmxur2qdDTEoppYLSAKGUUiooDRCR82C0GzANbd/caPvmRts3N5a0T3MQSimlgtIehFJKqaA0QISRiCwVkVdEZL+I7BWRzwU55iIR6RWRt31fd0e4jY0iUuu79q4g94uI/JuIHBaRPSKyOYJtqwx4Xt4WkT4R+fyEYyL6/InIf4vICRGpC7jNISIvisgh378LJjn3ChGp9z2Xd0Swff8gIgd8/3+PiUjeJOdO+VqwsH33ikhrwP/hVZOcG63n7+cBbWsUkbcnOTcSz1/Q95SIvQaNMfoVpi9gMbDZ970dOAisnXDMRcBTUWxjI7BwivuvAp4FBDgfqI5SO5OAdrxztKP2/AEXAJuBuoDbvgXc4fv+DuCbk7T/CLAMSAXemfhasLB9lwHJvu+/Gax9obwWLGzfvcAXQ/j/j8rzN+H+fwLujuLzF/Q9JVKvQe1BhJEx5rgx5k3f9y5gP1Ac3VbN2HbgJ8brDSBPRBZHoR2XAEeMMVFd+GiMeQ1wTrh5O/Bj3/c/Bq4LcmoVcNgYc9QYMwo87DvP8vYZY14wxoz7fnwDKAn3dUM1yfMXiqg9f34iIsCfAD8L93VDNcV7SkRegxogLCIi5cC5QHWQu98lIu+IyLMiEun9Sg3wgojsFpFbgtxfDDQH/NxCdILcjUz+hxnN5w+g0BhzHLx/wEBBkGNi5Xn8BN4eYTDTvRas9BnfENh/TzI8EgvP3/uADmPMoUnuj+jzN+E9JSKvQQ0QFhCRbOBR4PPGmL4Jd7+Jd9jkHOA7wOMRbt57jDGbgSuB20Tkggn3S5BzIjrVTURSgWuBXwS5O9rPX6hi4Xn8KjAOPDTJIdO9FqzyH8ByYBNwHO8wzkRRf/6AjzB17yFiz9807ymTnhbkthk9hxogwkxEUvD+Rz5kjPnVxPuNMX3GmH7f988AKSKyMFLtM8a0+f49ATyGtxsaqAVYGvBzCdAWmdadciXwpjGmY+Id0X7+fDr8w26+f08EOSaqz6OIfBy4Gvio8Q1ITxTCa8ESxpgOY4zbGOMB/nOS60b7+UsGPgT8fLJjIvX8TfKeEpHXoAaIMPKNWf4Q2G+M+edJjinyHYeIVOH9P+iKUPuyRMTu/x5vMrNuwmFPAB/zzWY6H+j1d2UjaNJPbtF8/gI8AXzc9/3HgR1BjtkJrBSRCl+P6EbfeZYTkSuA24FrjTGDkxwTymvBqvYF5rSun+S6UXv+fC4FDhhjWoLdGannb4r3lMi8Bq3MwM+3L+C9eLtwe4C3fV9XAbcCt/qO+QywF++MgjeAd0ewfct8133H14av+m4PbJ8A38M7+6EW2Brh5zAT7xt+bsBtUXv+8Aaq48AY3k9kfwHkAy8Dh3z/OnzHLgGeCTj3KryzTo74n+sIte8w3rFn/2vw+xPbN9lrIULt+6nvtbUH7xvW4lh6/ny3/8j/mgs4NhrP32TvKRF5DepKaqWUUkHpEJNSSqmgNEAopZQKSgOEUkqpoDRAKKWUCkoDhFJKqaA0QCillApKA4RSUxCR8sBS0GF83ItE5KlwP65S4aQBQimlVFAaIJQKkYgsE5G3ROS8IPdVB1aWFZHfiMgWEakSkd/7zvu9iFQGOfdeEfliwM91vsqdiMifikiNb1OaH4hIkjW/nVJn0wChVAh8b+yPAjcbY3YGOeRhvHsH+GsNLTHG7AYOABcYY84F7gb+fgbXXAPcgLdq6CbADXx0Tr+IUjOQHO0GKBUHFuEthvZHxpi9kxzzCPAicA/eQOEvVZ4L/FhEVuKtqZMyg+teAmwBdvrqE2YQvGqnUpbQAKHU9HrxFr97D97CbGcxxrSKSJeIbMT7qf/TvrvuB14xxlzvGzb6TZDTxzmzN5/u+1eAHxtjvjLXX0Cp2dAhJqWmN4p3S8ePichNUxz3MPBlvJVoa3235QKtvu//fJLzGvHui4yIbAYqfLe/DHxYRAp89zlEpGyWv4NSM6YBQqkQGGMG8G7A8zciMtm+vr/EW3P/kYDbvgV8XUR+h3cT+WAeBRwi8jbwl3jLM2OM2QfciXdbyz14h7CisT+4mqe03LdSSqmgtAehlFIqKE1SKzUDInI58M0JNzcYY66PRnuUspIOMSmllApKh5iUUkoFpQFCKaVUUBoglFJKBaUBQimlVFAaIJRSSgX1/wFhYs4YCaMVfgAAAABJRU5ErkJggg==\n",
                        "text/plain": "<Figure size 432x288 with 1 Axes>"
                    },
                    "metadata": {
                        "needs_background": "light"
                    },
                    "output_type": "display_data"
                }
            ],
            "source": "hyperparameter(length, accuracy, \"k_value\")"
        },
        {
            "cell_type": "code",
            "execution_count": 66,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "Max accuracy:0.8\nBest random state: 9\n"
                },
                {
                    "data": {
                        "text/plain": "(9, 0.8)"
                    },
                    "execution_count": 66,
                    "metadata": {},
                    "output_type": "execute_result"
                },
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO29eXhb53Wg/x4C3EGC4ioJkKiFlGVJpDdZcr1lcZrY2RxPs3hLMuk0rn+JO+lMJ42TtpN0ZtppJ+10mXHiOGmaJvWSxHZiJ3Fip8nElpxYtiTbpGTZJrWQIrVxEcF9AXl+fwCgYQgUQRIXC3He58Ej4N7vu/fo8uIefGcVVcUwDMMwYslLtwCGYRhGZmIKwjAMw4iLKQjDMAwjLqYgDMMwjLiYgjAMwzDi4k63AMmkurpa161bl24xDMMwsoZ9+/b1qmpNvH3LSkGsW7eOvXv3plsMwzCMrEFEOubaZyYmwzAMIy6mIAzDMIy4mIIwDMMw4mIKwjAMw4iLKQjDMAwjLo4qCBG5XkReE5F2Ebk7zn6viPxIRF4WkYMi8olE5xqGYRjO4piCEBEXcA9wA7AFuEVEtsQM+zTwiqpeBLwV+FsRKUhwrmEYhuEgTuZB7ADaVfUIgIg8BNwIvBI1RoEyERHAA/QDQWBnAnMNIyP4zeE+fnO4d0nHeMsFNVxWX5kkiYyF8NqpIfqGJ7iyoTrdomQcTioIH3A86nMXoQd/NP8XeBw4AZQBH1HVGRFJZC4AInIHcAfA2rVrkyO5YSyALz1+kNdODyGyuPmq8HRbL499+qrkCmYkxF88cYgD3QH2/ek7kMX+EZcpTiqIeFc6tjvRu4CXgLcDG4Gfi8iuBOeGNqreB9wHsH37dut+ZKQUVaWjf4Tfu3o9f/rexVlBv/jYAR7Z342q2gMqxagqrV0DnB2dontgDP+KknSLlFE46aTuAtZEffYTWilE8wngUQ3RDhwFNic41zDSTs/QBONTM6ytWvyDpaGujOGJIKcGx5MomZEIXWfHODs6BcCB7kCapck8nFQQLwCNIrJeRAqAmwmZk6LpBK4DEJE64ALgSIJzDSPtdPSPArC2cgkKosYDQNvp4aTIZCROa5RSaOkyBRGLYwpCVYPAXcCTwCHge6p6UETuFJE7w8P+O3CliLQCvwA+p6q9c811SlbDWCydfUtXEI11IQXRfsYURKpp6QqQ7xIaaz1vUhZGCEeruarqE8ATMdvujXp/AnhnonMNI9Po6B8lT1iS7bqqtICKknzaTEGknNbuATavLGfr6nJ+euCU+YFisExqw1gCnX0jrPIWU+Be/FdJJPQL9rApiJSiqrR0BWjye2nyewmMTXG8fyzdYmUUpiAMYwl09o8uybwUoaHWQ9uZoSRIZCRKR98oQ+NBmn1emn0VALR0D6RZqszCFIRhLIHO/lHqlxDBFKGhtoyzo1P0DU8kQSojESI+hya/l00rPRS48mg1R/WbMAVhGItkZCJI7/Aka5K0ggDMD5FCWrsDFLjz2FRXRqHbxeZVZeaojsEUhGEsks5wiGsyVhCNtRbJlGpauga4cFU5+a7QY7DJ56W1O8DMjOXbRjAFYRiLpDMJORARVnmLKC1wmYJIETMzyoHuQZp93tltTT4vQ+PB2dwWwxSEYSyaSA5EfWXpko8lIjTUekxBpIijfSMMTwRp8kcpiPD7li5zVEcwBWEYi6SjfwRvcT7ekvykHG+jRTKljIgzujlKQWyqK6PAbY7qaExBGMYi6ewfS4p5KUJjbRmnBycYHJ9K2jGN+LR0BSjKz5stcwKQ78pjy6pyWsxRPYspCMNYJJ19I0sq0hdLgzmqU8aB7gBbV3txu978CGz2ezlojupZTEEYxiKYnlG6ziZ7BWEKIhVMzygHTgRoinJQR2jyeRmZnOZI70gaJMs8TEEYxiI4MTBGcEapT6KCWFNZQoE7zxSEwxzpGWZ0cjqugmj2hzKqWy2jGjAFYRiLIpkhrhFcecKG6lJTEA7TEsdBHWFjTSnF+S4r/R0m5xVEcHqG//CtF3hgT2e6RTGyiFkFkUQfBFhNplTQ2h2gpMDFhigHdQS3K48tq8stkilMzisItyuPgycG2XusP92iGFlER98o+S5hlbc4qcdtqPXQdXaMscnppB7XeIOWrgG2rfbiyotf1rvJ5+XgiUGC0zMplizzyHkFAaGGLe09tqw3Eud4/yj+FSVzPmQWS2NtGapw2O5HRwhOz/DKycE3JcjF0uz3MjY1zeEec1SbggA21oQyWC20zUiUjv6RpPofIkRCXU1BOEN7zzDjUzNx/Q8Rmi2jehZTEIRWEKOT05y0pvFGgnT2JacPRCzrqkOrEutP7QwR53O8CKYI66s9lBa4rLIrpiCA6Kbx5hw05mdgdJLB8WBSqrjGUuh2UV9ZYpFMDtHaFcBT6GZd1dz1s1x5wtZwZddcx1EFISLXi8hrItIuInfH2f9ZEXkp/DogItMiUhne959E5GB4+4MiUuSUnI11ZYAlKBmJ0REu0peMPhDxsEgm52jpDrDNV07ePL6jZp+XV04MMpXjjmrHFISIuIB7gBuALcAtIrIleoyqfllVL1bVi4HPA0+rar+I+ID/CGxX1W2AC7jZKVkrSwuoLC0wBWEkRDL7QMSjodZDR98ok8Hcfjglm8ngDIdODs4mw52PJr+XieBMzpv6nFxB7ADaVfWIqk4CDwE3nmf8LcCDUZ/dQLGIuIES4IRjkoKVWjYSxokkuWga6zwEZ5SOPouiSSavnx5iMjhzXv9DhMiYXM+odlJB+IDjUZ+7wtvOQURKgOuBRwBUtRv4G6ATOAkEVPWpOebeISJ7RWRvT0/PooUNLeuHUbVIJuP8dPaNUu0ppKTA7cjxG2rM5OkEEZ/C+SKYIqyrKqWs0J3zGdVOKoh4Rr65nr7vA55V1X4AEVlBaLWxHlgNlIrI7fEmqup9qrpdVbfX1NQsWtjGWg+BsSl6hycXfQwjN+joH3HMvASwsTbkQDUFkVxauwOUF7kTWvnl5QnbzFHtqILoAtZEffYzt5noZt5sXnoHcFRVe1R1CngUuNIRKcO80TTenIPG+XEqxDVCSYEbX0UxbaYgkkprV4AmvxeRxJIbm/1eXj05lNO+ICcVxAtAo4isF5ECQkrg8dhBIuIF3gI8FrW5E7hCREok9Ne8DjjkoKw01oaW9YftS2mch4lgKF/GSQUB4ex+uxeTxkRwmldPDdLkm99BHaHJ72VyeobXczj83TEFoapB4C7gSUIP9++p6kERuVNE7owaehPwlKqORM3dAzwM7Adaw3Le55SsAHXlhXgK3farzTgvXWfHUHXOQR2hocbD4Z5hpi27Pym8dmqIqWlNyP8QoTmsTHLZD+GMly2Mqj4BPBGz7d6Yz98CvhVn7heBLzoo3puwpvFGIjgd4hqhsc7DRHCG7rNjSa8Ym4skkkEdy5rKYrzF+eFIprUOSZbZWCZ1FJFIJsOYi84+Z8p8x2I+seTS2hVgRUk+/hWJV98VEZp83pxeQZiCiKKx1kPP0ASBUWsab8Sns3+U4nwXNZ5CR89joa7JpaU7QJO/ImEHdYQmv5fXTg0xPpWb5ddNQUQx2zS+x361GfHpCEcwLfRBs1C8JfnUlBXaijYJjE9N03Z6iOYFmJciNPu8BGeUV0/l5jPBFEQUkUgm+9VmzEVn/4hjNZhiaTSfWFI4dHKQ4IyybREKItI3ojVHS3+bgojCt6KYQndeztdfMeKjqnT2jzruoI4QCZqw7P6lsZAM6lh8FcVUlhbkrB/CFEQUrjwJNQ+yZi1GHHqGJhifmkmZgmis9TA8EeT04ERKzrdcaekKUO0pYJV34QWhI47qXM2oNgURQ0Otx1YQRlwiIa6pMjFttEimpNDaFaDJl3gGdSzNfi9tZ4Zzsk+4KYgYGms9dA+MMToZTLcoRoYR6QNRnzIfhPnElsroZJC2M0M0JVDiey62+bxMzyivnBxMomTZgSmIGGZ7Ap+xUsvGm+noH0Uk5KtKBdWeArzF+RbJtAReOTHIjLKoCKYIzTnsqDYFEUNjnYW6GvE53j/Kam8xhW5XSs4nIhbJtEQivoOmRTioI6wsL6LaU0hLDvohTEHEUF9Vituaxhtx6OgbYU1lalYPEaz8y9Jo7QpQW1ZIXfniOxaLCM1+L605GMlkCiKGfFce66pL7UtpnENn/xj1lXM3u3eChloP/SOT9A1bJNNiaOkOLCq8NZYmn5f2nmFGJnLLN2kKIg4NFupqxDAyEaR3eCLlhfNms/vtB8uCGZ4IcrhneEElvuei2e9FlZxzVJuCiENjnTWNN97M8bPO9qGei8a6cCST/WBZMAe7A6guLkEulkgV2FxLmDMFEYeGWg/TM8oxaxpvhJkNcU3xCmK1t4iSApf5xBZBxEG9mBIbsdSWF7GyvCjnIplMQcRhY004Qcm+lEaY2TLfKV5BRPqUHLYVxIJp6Qqw2ltETVlyKu9u83lzLpLJFEQcNtZ4EDG7r/EGnf2jlBe5qSgpSPm5G2osu38xHOgOLCm8NZZmv5cjPSMMjedOOwBTEHEoLnDhX1FsJQ6MWTr6R9PW2W1jrYdTg+M59WBaKoPjUxzpHVlQB7n5iCibA92546h2VEGIyPUi8pqItIvI3XH2f1ZEXgq/DojItIhUhvdViMjDIvKqiBwSkd9yUtZYGmvLbAVhzHK8fzTlIa4RGi2SacEcmE2QW3oEU4SIsgm1IM0NHFMQIuIC7gFuALYAt4jIlugxqvplVb1YVS8GPg88rar94d3/APxMVTcDFwGHnJI1Hg21Ho70jljTeIPpGaXrbPpWEBbqunBaF9GDej6qPYX4KopptRVEUtgBtKvqEVWdBB4CbjzP+FuABwFEpBy4FvgnAFWdVNWUqu2GWg+TwRmOhyt4GrnLycAYU9Oacgd1hLWVJRS48kxBLICW7gD+FaFeDsmkyefNqUgmJxWEDzge9bkrvO0cRKQEuB54JLxpA9AD/LOIvCgi3xCRlK7v32gab1/KXKczxVVcY3G78lhv2f0LorUrORnUsTT5vRzrG82ZvvVOKoh4xdfnste8D3g2yrzkBi4FvqqqlwAjwDk+DAARuUNE9orI3p6enqXKPIst640IHSnuAxGPhjqP/VhJkIHRSTr7R5OSQR1LxGR14ERuhLs6qSC6gDVRn/3AiTnG3kzYvBQ1t0tV94Q/P0xIYZyDqt6nqttVdXtNTc0SRX6D8qJ86soLLZLJoLN/FHeesLoitYX6ommo8XD87CjjU7nXtGahRKKMkul/iJBrGdVOKogXgEYRWS8iBYSUwOOxg0TEC7wFeCyyTVVPAcdF5ILwpuuAVxyUNS6NtWUctl9tOU9n3yj+FcW48hbXkSwZNNZ5UMUS5hKgJRxl5ISCWFFawJrK4pyJZHJMQahqELgLeJJQBNL3VPWgiNwpIndGDb0JeEpVY+ta/AFwv4i0ABcDf+mUrHNhTeMNCK0g1lalJ8Q1gpk8E6e1K0B9VQneknxHjt/sq8iZFYTbyYOr6hPAEzHb7o35/C3gW3HmvgRsd1C8eWmo9TAyOc3JwHhazQtGeunoG+HiNcm3Zy+E9dWl5Fl2f0K0dAW4ZK1zf68mv5eftJ7k7MgkK5IcJZVpWCb1ebBIJmNgdJLB8WDaQlwjFLpd1FdZJNN89A1P0D0w5kgEU4Tm2YS55b+KMAVxHiyD1egMRzClK0kumoZai2Saj9kWow5EMEXYagrizYhIvYi8I/y+WETKnBUrM6jyFLKiJJ92i2TKWTrSVMU1Hg21Ho71jjA1bX1K5iKSQb3NV+7YObzF+ayvLqUlBxLm5lUQIvJJQmGmXwtv8gM/dFKoTMJqMuU2syuIDFAQjbUegjNKh/UpmZPW7gAbqkspK3LGQR1hmy83elQnsoL4NHAVMAigqm1ArZNCZRIbw8t6i2TKTTr7Rqn2FFBa6Gg8R0JYJNP8tCa5xPdcNPu8nAiM0zO0vHuFJ6IgJsK1lAAQETdzZ0QvOxprPQyMTtE3Mjn/YGPZ0dk/mhGrB7BGVvNxZmick4FxR/IfYnmj9PfyXkUkoiCeFpEvAMUi8tvA94EfOStW5jAbyWRfypyks3+U+jTnQEQoLXTjqyi2/tRzEHlYNyexxPdcbF1djsjyz6hOREF8jlDhvFbg9wnlNfypk0JlEo114WW9fSlzjongNCcCY2mtwRRLQ611l5uLlq4AIqGHt9OUFeWzobp02UcyndewKiJ5QIuqbgO+nhqRMouV5UV4Ct20n7ZIplyj++wYqumr4hqPhloPzx3pY3pG01r6IxNp7QrQUONJmb+o2V/Brw/3puRc6eK8KwhVnQFeFpG1KZIn4xARNtZ6bAWRg3RkUA5EhMZaDxPBGbrPjqVblIyjJUUO6ghNPi+nByc4PTiesnOmmkRMTKuAgyLyCxF5PPJyWrBMwprG5yaRZlGZtoIAaO+xFW00pwdDEUWpcFBHiGRrL+dw10TWYn/uuBQZTmOdh0f2dxEYm8Jb7Gx8tZE5dPSNUpSfR01ZYbpFmSU61PXtm+vSLE3mEHEWO1liI5Ytq8vJk9DK5R1bluffYt4VhKo+DbwKlIVfh8LbcoaGGos/z0U6+kIhriKZY+uvKCmg2lNoK9oYWrsGyBPYsip1CqKkwE1DrWdZtyBNJJP6w8DzwIeADwN7ROSDTguWSUQimaw3RG5xvH+UtZWZEeIaTaP5xM6hpTvAproyigtcKT1vk6+C1u7Ask2kTcQH8SfA5ar6cVX9GLAD+DNnxcos/CtKKHDnWXe5HEJVMypJLpqGWg/tpy27P4Kq0toVSKn/IUKz30vv8CQnA8vTUZ2IgshT1TNRn/sSnLdscOUJG2s8ZmLKIXqGJxibmqY+gyKYIjTWeRiaCHJmmZd5SJQTgXH6RiZT6n+IEImaWq75EIk86H8mIk+KyL8XkX8P/AT4qbNiZR5Wajm36MygKq6xNFjJjTcRiSJqSkEGdSxbVpXjypNlG8mUiJP6s4QquTYDFwH3qeofOy1YptFY66F7YIzRyWC6RTFSQCb1gYilIZLdbyZPAFq7B3DnCZtXpr4LQVG+i011ZbQs0xXEvGGuIrIeeEJVHw1/LhaRdap6zGnhMomG2lDT+CM9I2xLg63TSC0dfaOIgH9F5rWarfEUUl7kthVtmL3HzrJ5VRlF+al1UEdo8pXz81dOo6oZFfGWDBIxMX0fiO5QMh3eNi8icr2IvCYi7SJyd5z9nxWRl8KvAyIyLSKVUftdIvKiiPw4kfM5iXWXyy06+0dZVV5EoTs9D53zISI01lmfEoCRiSD7O89y1cbqtMnQ5K/g7OgUXcswuz0RBeGOLvcdfj9vp24RcQH3ADcAW4BbRGRL9BhV/bKqXqyqFwOfB55W1f6oIZ8BDiUgo+PUV5XiyhOLZMoROvtHM9K8FKHBgiYAeP5oP1PTytWN6VMQy7lHdSIKokdE3h/5ICI3AolUqNoBtKvqkbBSeQi48TzjbwEejDqPH3gP8I0EzuU4Be481lWV2JcyR4gkyWUqDbUe+kYm6c/xPiW72nopdOdx+brK+Qc7xOZVZeS7ZFmW/k5EQdwJfEFEOkXkOKHy37+fwDwfcDzqc1d42zmISAlwPfBI1Oa/B/6YN5u34s29Q0T2isjenp6eBMRaPBbJlBuMTgbpHZ7ImD4Q8XjDUZ3b9+Pu9h52rK9Mm/8BoNDt4oKVZcuyeVAiUUyHVfUKQmaiLap6paq2J3DseN6auTJ73gc8GzEvich7gTOqui8B+e5T1e2qur2mpiYBsRZPY20ZHX2jTAatafxyJhLBlEl9IGKx8i9wKjDO66eHubohfealCE2+Clq6BpZd8mIipTY+IyLlwAjwdyKyX0TemcCxu4A1UZ/9wIk5xt5MlHmJUA/s94vIMUKmqbeLyL8mcE5Haaj1MD2jHLOm8cuajr7Mq+Iai6+imOJ8V077xHa3hyzd6fQ/RGj2exkcD87+uFguJGJi+l1VHQTeCdQCnwD+KoF5LwCNIrJeRAoIKYFzyoSLiBd4C/BYZJuqfl5V/aq6Ljzvl6p6ewLndBRrGp8bRMp8Z7IPIi9P2FhbmtP34u62HqpKC7hwpfMd5OYjUuZjufkhElEQEVPRu4F/VtWXiW8+ehOqGgTuAp4kFIn0PVU9KCJ3isidUUNvAp5S1Yz/Wb6xxoOIZbAudzr6RikrclNRktml3RtrczfUVVXZ3d7HVQ3V5GVAZ71NdWUUuPKWXSRTIv0g9onIU8B64PMiUsY8juMIqvoEoR7W0dvujfn8LeBb5znGr4BfJXI+pykucOFfYU3jlzud/aPUV2VWme94NNR6+MGL3QyNT1FWlNnKLNm8emqI3uEJrskA8xKEohwvXFVGyzIr/Z3ICuI/AHcTqug6SigH4hORnSKy1SHZMpJQd7nctfvmAplaxTWWiMnzcE/GL76Tzu62kP/hmkZnA1MWQpPfy4HuQWZmlo+jOpEophlV3a+qA+HPfaraEjXkO45Jl4E01pVxpHeE6WV0ExhvMD2jdJ3NzD4QseSyT+yZth4aaj2s9BalW5RZmn0VDE8EObqMgliSUbY7s9fhSaahxsNkcGbWkWksL04Gxpia1qxYQdRXlpDvyr3s/vGpaZ4/2p8R4a3RREp/L6d8iGQoiJz6KW0JSsubSJnvTOwDEYvblcf66tKc63S4r+MsE8GZjPE/RGis9VDozltWkUw51fgnGUSW9ZZRvTzpzIIQ12gaa8ty7l7c1daLO0/YuaEq3aK8Cbcrj62ry5dVb4hkKIicKgZTXpRPXXmhrSCWKR39o7jzhFUZZNs+HxtrPRzvH2V8ajrdoqSM3e09XFq/Ak9hIkGYqaXJ5+XAicCy8VEmkkn9iIi8R0Tijg2X4cgpGmo91qxlmdLZP4p/RTFuV3YsrhtrPcyE+5TkAn3DExw8Mcg1GeZ/iNDkr2B0cpojyyQUPpFvwVeBW4E2EfkrEdnssEwZTyRBabnVXTFCPohMrsEUy2wk0zJ5IM3Hs4f7UM2M8hrxiPTFXi5+iETCXP9NVW8DLgWOAT8XkV+LyCdEJLeyc8JsrPUwMjnNycB4ukUxkkxH30hWOKgjrK8uJU+gPUdyc3a39VBe5KY5Df2nE2FjjYfifNeyyahOyIgnIlXA7cBHgReB+4GrgY8Db3VKuEwlurvc6orMa0m5UL7zm2NsqPFwVZqW7f/y62M8d6RvScd430WreXfTqiUdIzA6xeB4MGsc1BDqiby2siStjuonWk8yPBHkw9vXzD94Cagqu9t6uXJjNa4MKK8RD1eesM1XnjsKQkQeBTYTSoh7n6qeDO/6rojsdVK4TCU6kunaTZmTybkYpqZn+O8/OcRFfm9aFMT41DT/86eH8BS6qSydt1FhXAbHgjz1ymkqSvK5cgmtJzv6Q3b8bEiSi+bS+hX8/OBpRieDlBSk1nE7EZzmT394gNHJIO/auhJvsXNGhSO9I5wIjPOpt2WmeSlCk6+CB57vIDg9kzW+rLlI5G76v6r6y3g7VHV7kuXJCqpKC6goyV8WkUyvnx5iMjjDi50DDE8EUx4Zsr/jLONTM9xzazPXXVi3qGMMjU9x01d+zafv38/jd129aB9CtoW4Rrhlx1oe3d/Nj14+wUcuX5vScz958PRsV7sf7O/i31+13rFzRcprXJtB5TXi0ez38s1nZ2jvGWZzBlSaXQqJqLcLRWTW4CciK0TkUw7KlPGICI3LJJIpErMdnFGeO7w0M89i2NW+9Jj2sqJ8vv6x7UzPKHd8Zx+jk8FFHSfSByKTe1HHY3v9CjbVebh/T2fKz33/cx2sqSym2e/l/j2djgZu7GrrZW1lScb/fbYto9LfiSiIT0bqMAGo6lngk86JlB1E2o9meyRTS3eAskI3Rfl5sw1YUsnutl4uXbv0mPb11aX84y2X8NqpQT77/ZZF/V2O949S7SnIyPj68yEi3LaznpauQEqTtNrPDLHnaD+37qjn9p31tJ0Z5oVjZx0519T0DM8d6cvY6KVoNlSXUlrgWhYJc4koiDyJqnssIi5CFV1zmobaMgZGp+jL8qbxrV0BmvxedqyvYlebsz29Yzk7MsmBE4GkfenfekEtn7t+Mz9pPclXfnV4wfM7sizENZqbLvVRnO/igec7UnbOB/YcJ98lfGi7n/detIqyIjcP7HHm/C8dD5lAMzX/IZq8PGGbz0vLMnBUJ6IgngS+JyLXicjbCbUG/ZmzYmU+y6GS5kRwmldPDdLk93JNQzWHe0Y4MTCWsvM/e7g36THtd1y7gfdftJq/eeo1fvnq6QXN7ewfzeg2o+ejvCif9120isdeOsHg+JTj5xufmubhfcd519aVVHsKKSlw8+8u8fFE66lZn0Qy2dXWS56wpCCEVNLs93Lo5GDW969PREF8Dvgl8P8BnwZ+Afyxk0JlA43LoCbT66eGmZpWmn0Vsw/piCMwFexu66WsyE1z2GabDESEv/6dZrauLuczD76UsAKfDM5wIjCWdQ7qaG7bWc/o5DSPvdjt+Ll+3HKSwfEgt+2sn9126856JqdneHjf8aSfb3dbD03+CrwZ3uUvQpO/gsngDK9neX5Kov0gvqqqH1TV31HVr6lq7hR+mYNV3iJKC1xZXUmzpTvkWmr2e9m8soxqTyG7UuSHUFV2tfVy5caqpIcCFhe4+NpHt1PgzuOOb+8lMDb/L+qus6Oowtqq7ApxjabZ72Wbr9xxZzHA/Xs62FBTyhUbKme3XbCyjMvXreCBPZ1JbZozOD7Fy10Brs0C/0OEyI+ebC/9nUgtpkYReVhEXhGRI5FXKoTLZEQk7KjO3l8IrV0BKkry8a8oRkS4prGaZ9t7U9IR62jvCN0DY451BPNVFPPV2y+js3+UP3zoxXmLp2VriGs0IsKtO+p59dQQ+zuda335yolBXuwc4Lad9ee0Zb1tZz3H+kb5zRITH6P5zeE+pmc04/o/nI/6qhLKitxZ74dI5KfbPxOqxxQE3gZ8mwS7yInI9SLymoi0i8jdcfZ/VkReCr8OiMi0iFSKyBoR+X8ickhEDorIZxbyn0oVG2s9We2DaOkK0OTzzn7Jr26opn9kkldODjp+7oGVyxAAACAASURBVEjElJM1/Xesr+RL79/K/3uth7996rXzjo0oiGwqsxGP91+8Gk+hm/sdchYDPPB8BwXuPH7nUt85+67ftpIVJflJPf+uth5KClxcsnZF0o7pNCJCs9+b9ZFMiSiIYlX9BSCq2qGqXwLePt+kcLTTPcANwBbgFhHZEj1GVb+sqher6sXA54GnVbWfkDL6I1W9ELgC+HTs3EygsbaM04MTKXEKJpvxqWlePz00W1wM3nAWpyLcdVdbL2sqi6l32KRz+xX13LJjLV/51WF+9PKJOcd19o1S6M6jtqzQUXmcxlPo5gOXrObHLScZGE2+s3h4IsgP9nfz3uZVVJScG8xYlO/ig5f5eergac4MJadW2e62Xq7YUEWBO7uykrf5vLx6apCJYPZa5BO54uPhUt9tInKXiNwE1CYwbwfQrqpHVHUSeAi48TzjbyEUIYWqnlTV/eH3Q8Ah4NyfK2kmmyOZDp0cJDijNEU5iOvKi9hU53HcUR2cnuG5w31c3ZCajNg/f/9Wttev4LMPv8zBE/F/0XX0j7K2suQck0k2cuuOeiaDMzyyP/nO6sdfOsHI5PSbnNOx3LJjLcEZ5ft7u5Z8vuP9oxzrG80q81KEZl8FU9PKa6ey1wydiIL4Q6AE+I/AZYSK9n08gXk+IDqcoYs5HvIiUgJcDzwSZ9864BJgzxxz7xCRvSKyt6cntXH8s0X7Tmefgog4z5piqmJe3VDD88f6HW1A83LXAEMTwZS1jCxw5/HV2y9jRUkBd3x7H33DE+eM6ewbzXrzUoQtq8u5ZG0F9+/pSKqzWlW5f08Hm1eWcenauauphgo/VvHAns4lN85JhSnSKZZD6e/zKoiwmejDqjqsql2q+olwJNNzCRw73k+xue6W9wHPhs1L0ef3EFIaf6iqcQ3jqnqfqm5X1e01Namt0bKmsoQCd15W1uJv6QpQVVrA6pjOadc0VjMZnOH5o/1zzFw6u9p6EYErN6auZWRNWSFf++hl9A5P8Kn79zM1/UZ8uqrS2Z+9SXLxuG1nPUd6RnjuSPL+ji1dAQ6eGOS2nWvnXWnduqOe7oExnnl9aT/adrf1srK8aHa1nk34VxRTUZKf1X6I8yqIcDjrZbK4dXcXEF3/1w/MZQS+mbB5KUK418QjwP2q+ugizu84rjxhQ3UpbVkY69zaHcqgjv3T7txQSb5LHPVD7GrrpdnnjWvDdpJmfwV//TvN7Dnaz//48Suz23uGJxibms7aJLl4vLd5FeVFyXVW37+ng5ICFx+4ZH5r729vqaPaU7ik80/PKM8e7uXqxuqsNP2JCE0+b1aX/k7ExPQi8JiIfFRE/l3klcC8F4BGEVkvIgWElMDjsYNExAu8BXgsapsA/wQcUtX/nch/JF001HqybgUxNhl2UMdJUCspcHNZ/Qp2OeSHGByf4qXjA2mrqfOBS3x88pr1/MtvOvjuC6Hidsf7s7NI3/kIOYvX8OTBU/TGMaktlMDYFD96+SQ3XryasqL5k9UK3Hl85HI/v3z1zKKz8w90BxgYncpK81KEZr+X108PZW3P8EQURCXQRyhy6X3h13vnm6SqQeAuQqU6DgHfU9WDInKniNwZNfQm4ClVjW6qexWh5kRvjwqDfXdC/6MU01hbRtfZMcYms+cGeOVkgBk91/8Q4ZrGGg6dHKRnaOkPllieC8e0O5X/kAifu34z1zRW86c/PMC+jrNvVHHNsj4Q83HrzjVMTSfHWfzDF7sZm5rm1h1zO6djufnytSjw0AuLy6yOrGLT1cgqGTT5KgjOKIdSEDruBIlkUn8izut3Ezm4qj6hqptUdaOq/kV4272qem/UmG+p6s0x83arqqhqcyQMVlWfWOh/LhU01HpQhcNZtIqIOM2iQ1yjiUSM/Ppw8lcRu9t7KSlwcWkaY9rdrjz+7y2XsrqimDv/dR/PH+1HJGQzXk401Jaxc30lDzzfsaTkx4hzutnvpWmOeyYeaypLeMumGr77QifB6YXXJNrV1sOFq8qp9mRv6HHkemWrmSmRTOp/FpFvxr5SIVw20FiXfaGurV0BasoKqSsvirt/m8+LtzifZ153QEG09bJzfWXaY9q9JaEeEqMTQR564Tgry4soynelVSYnuO2Keo73jy2phMrejrO8fnqY23YuvBnRbTvrOT04wS9ePbOgeaOTQfZ1nM1q8xLAam8RVaUFWRvJlMi39MfAT8KvXwDlQPY8DR1mXVUprjzJqpIbrd2B8xbIc+UJVzVUsbu9J6lhkt0DYxzpHeHqDOkItqmujL/7yMVAdpfYOB/v2lpHVWnBkspwP7Cnk7JCN++7aPWC577tghpWeYsW3Mxoz9F+pqazq7xGPESEpizOqE7ExPRI1Ot+4MPANudFyw4K3Hk01Hh45UR22BhHJoK09wzPayq4prGG04MTSV0Z7Q73m8ikX4Xv3LqSf7j5Yj5zXWO6RXGEQreLD27382+HznAqsPDM5v6RSX7SepKbLvUtqt+125XHRy5fwzOv99AZ9vUkwu62XgrceexYXzn/4Ayn2eel7czQojsdppPFrPMbgdQ2vs1wmvyhULZs6C538MQgqnP7HyJEfrklM5rpmbZe6soLZxMMM4UbL/ZxZZb/Uj0ft+5Yy/SM8t1FOIsf2dfFZHCGWxdhXopw8+VrceUJDzyf+CpiV1sPO9ZVLguzX5O/ghklKx3VifgghkRkMPICfkSoR4QRptnvpXd4kpOL+IWWalq6QlU+t83Tg2FNZQnrqkqSlg8xM6P8ur2XqxqyM6Y9m6mvKuWaxmoeWqCzWFV54PlOttevYPPK8kWff6W3iOs21/L9vccTaqBzenCc108PZ0V70UTI5ozqRExMZapaHvXapKrnlMTIZZqyqEl5a3eAVd4iasviO6ijubqxmueO9CWlK9bBE4OcHZ3i2gzxP+Qat+2s52RgnF+9lnhm828O93G0d4Tbrli6weC2K+rpG5nkyYOn5h0bqQWW7f6HCHXlRdSWFWalHyKRFcRN4WS2yOcKEfmAs2JlFxeuKsedJ7R2O1eDP1m0hkt8J8LVDTWMTk7zYufSG9Hvag89mLI5pj2bue7CWmrLFpbZfP+eTipK8rlh26oln/+ahmrWVBYndP7d7b1UlRawZdXiVy2ZRrM/O3tUJ+KD+KKqzv7PVHUA+KJzImUfRfkuNtWVZfwKYnB8iiO9IwkriN/aWEWeJMcPsbutl80ry6jJ8nLa2Uq+K4+bL1/Dr17vmc0cPx9nhsZ58uApPnipPyl+gLw84ZYda3nuSP95Ax9Uld3tvVzZUE1e3vIxRW7zeTncM8zwRHY5qhNREPHGLDycYZnTnAWO6oPdISdZoslO3uJ8LlpTseQ2pGOT0+w9lv0x7dnOR3asRSAhZ/X393YRnFFuWYJzOpYPXbaGfJfw4Hmc1a+dHqJnaGLZ3SvNfi+qcDDLVhGJKIi9IvK/RWSjiGwQkb8D9jktWLaxzedlYHSKrrOLqzuTCiImsERXEBAKd23tGiAwuvimSM8f62dyeiZj8h9yFV9FMW/fXMtDLxx/UzXbWKZnlAef7+TKjVVsrElexFlNWSHv2rqSh/d1zVmbKOJ/WG4KIhIUkm0Z1YkoiD8AJoHvAt8DxoBPOylUNtKcBSn1LV0BfBXFVC2gdME1jdXM6NLKbux6vYcCVx471mV/THu2c+vOtfQOT/DzV07POeaZth66zo4tKbT1fOcPjE3xk5aTc5y7l401pazyLq+yJ7VlRazyFmW8GTqWRKKYRlT17kjPBVX9QkxhPQO4YGUZ+S7J6BugtTswb/5DLBevqcBT6F6SmWl3ey/b162guCD7Y9qznbdsqsVXcX5n8f3PdVLtKeCdW1Ym/fy/taGKDdWlcXMixqemef5oX1oLOTpJk88726grW0gkiunnIlIR9XmFiDzprFjZR6HbxeaV5RkbyRQYnaKjb3RBxdYg5Ny8YkPlotuQnhka59VTQ8v2S59tuPKEW3as4dn2UAhrLCcDY/zy1dN8ePsaR+pliQi37lzLvo6z5ySO7e84y/jUzLIJb42l2e/lSO9IVvWwT+QOqA5HLgGgqmdJrCd1ztHk99LSlZmO6ojpq9k3d6vIubi6oZrO/lE6+ha+cHw2i1tGLlc+vH0N7rz4zuKHnj+OEuor7RQfvMxPgTuPB2LqM+1q78WdJ1yRwk6DqSRSXj+bVhGJKIgZEZm9W8I9ojPvCZgBNPu8DI0HZ/sLZBIti3BQR4g4lxcT7rqrrZfKZRbTnu3Ulhfx21vq+P7e429yFgenZ3johU6ubaxxtP1qRUkB721axQ9e7GYkKuxzd1svl65dgadweQZJRr572ZQwl4iC+BNgt4h8R0S+AzwNfN5ZsbKTiPkmExNiDnQHWFtZgrdk/m5gsWysKWW1t2jBZiZVZXdbL1durFpWMe3Lgdt21nN2dIqfHXgjs/kXr57h9ODEosp6L/j8V6xleCLIj14OdSHuH5nkwInAsimvEY/K0gJ8FcUZ+XyYi0Sc1D8DtgOvEYpk+iNCkUxGDJvqyihw59HalXl+iJauwIL9DxFEhKsbq/n14V6mF9B4pu3MMGeWYUz7cuDKjVWsqyp5k5nngT2drCwv4u2bnbcgX7p2BZtXls2WAX+2vRdVlrWCgHC+1HJaQYjI7xHqA/FH4dd3gC85K1Z2ku/K48JV5RkX6to/MknX2bHz9oCYj6sbaxgcD84W+0uEZ17vmZ1rZBaRzObnj/Xz+ukhOvtGeaath49cvga3y/lmThFndWt3gJauAXa39VJW5F7SPZoNNPm9dPaPMjA6mW5REiKRO+EzwOVAh6q+DbgESKjil4hcLyKviUi7iNwdZ/9no3pOHxCRaRGpTGRuptLs83Kge3BJLR6TTURhLXYFAXBV2HG4EDPT7vZeNlSX4qtYXjHty4UPXuanwBVyFj/4QicC3LxjTcrO/4FLfBTnu7j/uc5QeY2NVSlRTukkEiRyoDs7Sn8n8tcYV9VxABEpVNVXgQvmmyQiLuAe4AZgC3CLiGyJHqOqX470nCbk13haVfsTmZupNPm9DE8EObqIiB+naE2wxPf5qPIUsnV1ecKO6ongNHuO9Jt5KYOp8hRyQ9NKHtnfxff3Hue6C+tSmqBWXpTPjRev5tEXu+geGMuJleZs5ecMDYePJREF0RXOg/gh8HMReQw4kcC8HUC7qh5R1UngIeDG84y/BXhwkXMzhtmM6gyyM7Z0BdhQXUp50cId1NFc3VjN/s6zCRUc298xwNjUdE586bOZ23bWMzQepHd4MiXO6Xjnn5oOrbavzYEfE96SfOqrSpL6fAiMTXEsTk5LMkjESX2Tqg6o6peAPwP+CUik3LcPiK4K1hXedg4iUgJcD0T6TCQ8N9NoqPFQlJ+XURnVrd2Ld1BHc21jDcEZZc+RvnnH7m7vwZUnXLHBymtkMpevW8GmOg9rKovT0qujye+l2e+lvqqE+qrSlJ8/HTT5vEl7PkzPKJ956EU+eO+v3xQynCwWFHCsqk8vYHi8uMa5DPPvA55V1f6FzhWRO4A7ANauTX8nVLcrj62rvRmTUd0zNMHJwPii8h9iuax+BYXuPHa19XLdhXXnHbu7rZdL1lRQtsRVi+EsIsI3PnY506ppC0X+6u2XzVm8bznS5PPy45aT9A1PLKguWjy+/ORr/Oq1Hv7ipm2UOpA/4qRHqAuI9nj5mds0dTNvmJcWNFdV74vUiaqpyQxzRlPYUb2QkFCniGRtJkNBFOW72LG+ct42pGdHJmnpXt4x7cuJtVUlrK9O3693X0VxUqvGZjpNSSrs+fjLJ7j36cPcunMtt+2sT4Zo5+CkgngBaBSR9SJSQEgJPB47KNyt7i3AYwudm6k0+byMTU1zpGfuxiipoqUrgAhsTVL44DWN1bSfGeZkYO5UmF8f7kPVymsYRjy2JSGj+kB3gD9++GUuX7eCL71va7JEOwfHFISqBoG7gCeBQ8D3VPWgiNwpIndGDb0JeCq6Quxcc52SNdlkUpPy1u4BNtZ4kla+4OqG+ctu7G7voazQzUX+hdd9MozlTnlRPhuqSxedUd07PMHvf2cfK0oK+MptlzlSVDGCo0HHqvqEqm5S1Y2q+hfhbfeq6r1RY76lqjcnMjdb2FDjoaTAlREJcy1dgaQmH21eWUa1p2DOfAhVZVdbL7+VAzHthrFYmvyLK/09NT3Dp+7fT+/wBPd9dLvjLXztG+wArjxh22rvgrKOneD04DhnhiaSEsEUIS9PuKqhmmfbe+MmA3b0jdJ1dszMS4ZxHpp8Xk4GxjkzNL6gef/tR6/w/NF+/tcHm5P6vZ4LUxAO0eT3cvDEIMHztHZ0moiJa6FNgubjmsYa+kYmOXTq3GzQSGMhy38wjLlpXkTp7wef7+Q7z3Xw+9du4MaLUxP1bwrCIZr9XiaCM7SdSZ+jurU7QJ7AllXJVRCRhi7xzEy723rwVRSzrsq5ctGGke1sXV2OSOJ+yr3H+vmvjx3g2k01/PH1mx2W7g1MQThEJtR+b+0aoLG2LOmtPld6i2is9ZwT7hqcnuHX7X1c01iNiJX3Noy5KC10s7HGk9Dz4WRgjDv/dT++imL+z82X4EphvoopCIdYV1VKWaE7bY5qVU1aBnU8rm6sZs/R/jclOL3cFWBoImj5D4aRAM0+Ly3d5+9AOT41ze9/Zx9jk0G+/rHti+rnshRMQThEXp6w1VeetuYgJwPj9A5PJt3/EOGaxmomgzO8cKx/dtvutl5E4KqNpiAMYz6a/F56hiY4PTgRd7+q8oVHW2npCvB3H7mYxrqyFEtoCsJRmv0VHDo5yGQw9Y7qiG0zGRnU8di5vop8l7zJD7G7vYcmn5cVpQWOnNMwlhPN82RU/9Puozz6Yjf/+bc38c6tK1Mp2iymIBykyedlMjjD66eHUn7u1u4B3HnChQ71gi4tdHPp2hWzCXPDE0Fe7ByYdWAbhnF+tqzykifE7UC5q62Hv3ziENdvXcldb2tIg3QhTEE4yHy/EJykpSvAproyivKT66CO5prGal45OUjv8ATPHe4jOKPmfzCMBCkucLGpruwcM3RH3wh3PfAijbVl/O2HL0prP3dTEA6ytrKE8iJ3yktuqCoHugOOmZciRHIdnm3vZXd7L8X5Li6rX+HoOQ1jObHNF+pRHXFUD08E+eS39yICX//YdkcqtC4EUxAOIiI0+ytSXvq76+wYZ0enHM+0bPJ58Rbns7utl2faetixvpJCt3MrFsNYbjT7vfSNTHIiMM7MjPJH33uJwz0j3HPrpazNgFwiUxAO0+T38tqpISaCqat3HzFpORXBFMGVJ1y5sYonD57iSM+IldcwjAXyRr7UAP/4yzaePHiaL7z7Qq7KEF+eKQiHafJ5mZpWXjuVOkd1S1eAfJdwwUrnw+KubqxmcDw4+94wjMS5cFU57jzh3qeP8Pf/1sbvXOrnd69al26xZjEF4TCzTcpT6Ido7R5g88rylJh7Im0qa8oKuSANcdqGkc0U5Ycc1S8dH+CiNRX8xU3bMqoKgSkIh/GvKGZFSX7KSm6oKi1dzmVQx7KmsoQtq8p519a6jLqxDSNbuGJDFXXlhXzt9sscjTpcDOl1kecAIkKTvyJlGdUdfaMMjQeT2gNiPh791JW40xiKZxjZzJ+850I++64Lkl4zLRnYCiIFNPu8vH56KCWN2SMO6m0pVBBF+S5rDmQYi8SVJxmpHMAUREpo8nuZnlFeOXlu/4Rk09odoMCdxybzBxiGsURMQaSA2YzqFPghWroGuHBVuaN9ag3DyA0cfYqIyPUi8pqItIvI3XOMeauIvCQiB0Xk6ajt/ym87YCIPCgiRU7K6iQry4uo9hQ6XnJjZkY50D2YUv+DYRjLF8cUhIi4gHuAG4AtwC0isiVmTAXwFeD9qroV+FB4uw/4j8B2Vd0GuICbnZLVaUSEJl+54yuIo30jDE8EUxbBZBjG8sbJFcQOoF1Vj6jqJPAQcGPMmFuBR1W1E0BVz0TtcwPFIuIGSoATDsrqOE3+CtrODDE6GXTsHK0O9aA2DCM3cVJB+IDjUZ+7wtui2QSsEJFficg+EfkYgKp2A38DdAIngYCqPhXvJCJyh4jsFZG9PT09Sf9PJItmn5cZhVdOOOeobukKUJSfR0ONx7FzGIaROzipIOIFxsf21nMDlwHvAd4F/JmIbBKRFYRWG+uB1UCpiNwe7ySqep+qblfV7TU1NcmTPslEzD5OZlQf6A6wZVW5hZwahpEUnHySdAFroj77OddM1AX8TFVHVLUXeAa4CHgHcFRVe1R1CngUuNJBWR2nrryIunLnHNXTM8qBEwGa/RWOHN8wjNzDSQXxAtAoIutFpICQk/nxmDGPAdeIiFtESoCdwCFCpqUrRKREQvUbrgtvz2qafBW0xOkelQyO9AwzOjnteA8IwzByB8dKbahqUETuAp4kFIX0TVU9KCJ3hvffq6qHRORnQAswA3xDVQ8AiMjDwH4gCLwI3OeUrKmi2e/lF6+eZngiiCfJjUBazEFtGEaScbQWk6o+ATwRs+3emM9fBr4cZ+4XgS86KV+qafJ5UYWD3QF2bqhK6rFbuwOUFLjYYA5qwzCShHkzU0ikPpITfoiWrgG2rfbisqJ5hmEkCVMQKaSmrJDV3qKkRzIFp2d45eSgJcgZhpFUTEGkmCa/N+kriPaeYcanZsz/YBhGUjEFkWKa/RUc7R0hMDaVtGNGViSpLPFtGMbyxxREiomEoR5M4iqitSuAp9DN+qrSpB3TMAzDFESKaXLAUf3S8QG2+crJMwe1YRhJxBREillRWsCayuKktSC9f08Hrd0B3nZBbVKOZxiGEcEURBpo8nmTUvr7+aP9fPGxg7z1ghp+75oNSZDMMAzjDUxBpIEmXwWd/aMMjE4u+hgnBsb41P37WFtZwj/cfInlPxiGkXRMQaSB2RakizQzjU1Oc8d39jIxNcN9H9uOtzg/meIZhmEApiDSwrbViy/9rarc/WgLB08M8vc3X0xDrZXWMAzDGUxBpAFvST7rqkoW5Yf4+q4jPPbSCf7LOy/gugvrHJDOMAwjhCmINNHkr1iwienp13v4q5++ynuaVvGpt250SDLDMIwQpiDSRLPPS/fAGH3DEwmNP9o7wh88sJ9NdWV8+UPNhNpkGIZhOIcpiDTRtABH9dD4FJ/89l5cecLXP7adkgJHq7QbhmEApiDSxtbV5QDz+iFmZpT/9N2XOdo7wj23XcqaypJUiGcYhmEKIl2UFeWzoaZ03ozqv/+31/m3Q6f5s/dcyJUbq1MknWEYhimItNI8T0b1T1tP8o+/bOdDl/n5+JXrUieYYRgGpiDSSpO/glOD45wZGj9n36unBvmj77/MJWsr+B83bTOntGEYKcdRBSEi14vIayLSLiJ3zzHmrSLykogcFJGno7ZXiMjDIvKqiBwSkd9yUtZ0EMmoPhBjZjo7Msknv70XT6Gbe2+/jEK3Kx3iGYaR4zimIETEBdwD3ABsAW4RkS0xYyqArwDvV9WtwIeidv8D8DNV3QxcBBxyStZ0sWVVOXny5ozq4PQMn35gP6cDE3zto5dRV16URgkNw8hlnFxB7ADaVfWIqk4CDwE3xoy5FXhUVTsBVPUMgIiUA9cC/xTePqmqAw7KmhZKC9001Hre5If4yyde5deH+/iLm7ZxydoVaZTOMIxcx0kF4QOOR33uCm+LZhOwQkR+JSL7RORj4e0bgB7gn0XkRRH5hojEbZcmIneIyF4R2dvT05Ps/4PjbPN5aekOoKo8vK+Lbz57lE9ctY4PbV+TbtEMw8hxnFQQ8byqGvPZDVwGvAd4F/BnIrIpvP1S4KuqegkwAsT1Yajqfaq6XVW319TUJE34VNHs89IzNMGTB0/xhR+0cuXGKv7k3RemWyzDMAxHFUQXEP0z2A+ciDPmZ6o6oqq9wDOE/A1dQJeq7gmPe5iQwlh2NPkrALjrgRepKy/knlsvxe2y4DLDMNKPk0+iF4BGEVkvIgXAzcDjMWMeA64REbeIlAA7gUOqego4LiIXhMddB7zioKxpY8uqclx5Qr4rj/s+up0VpQXpFskwDAMImXIcQVWDInIX8CTgAr6pqgdF5M7w/ntV9ZCI/AxoAWaAb6jqgfAh/gC4P6xcjgCfcErWdFJc4OJz11/AhavKuXBVebrFMQzDmEVUY90C2cv27dt179696RbDMAwjaxCRfaq6Pd4+M3YbhmEYcTEFYRiGYcTFFIRhGIYRF1MQhmEYRlxMQRiGYRhxMQVhGIZhxMUUhGEYhhEXUxCGYRhGXJZVopyI9AAd6ZZjDqqB3nQLcR5MvqVh8i0Nk29pLEW+elWNW+l0WSmITEZE9s6VrZgJmHxLw+RbGibf0nBKPjMxGYZhGHExBWEYhmHExRRE6rgv3QLMg8m3NEy+pWHyLQ1H5DMfhGEYhhEXW0EYhmEYcTEFYRiGYcTFFEQSEZE1IvL/ROSQiBwUkc/EGfNWEQmIyEvh139NsYzHRKQ1fO5zuitJiH8UkXYRaRGRlPUCF5ELoq7LSyIyKCJ/GDMmpddPRL4pImdE5EDUtkoR+bmItIX/XTHH3OtF5LXwtbw7hfJ9WUReDf/9fiAiFXPMPe+94KB8XxKR7qi/4bvnmJuu6/fdKNmOichLc8xNxfWL+0xJ2T2oqvZK0gtYBVwafl8GvA5siRnzVuDHaZTxGFB9nv3vBn4KCHAFsCdNcrqAU4SSeNJ2/YBrgUuBA1Hb/hdwd/j93cBfzyH/YWADUAC8HHsvOCjfOwF3+P1fx5MvkXvBQfm+BPyXBP7+abl+Mfv/Fvivabx+cZ8pqboHbQWRRFT1pKruD78fAg4BvvRKtWBuBL6tIZ4DKkRkVRrkuA44rKppzYxX1WeA/pjNNwL/En7/L8AH4kzdAbSr6hFVnQQeCs9zXD5VfUpVg+GPzwH+ZJ83Uea4fomQtjRoQAAABbNJREFUtusXQUQE+DDwYLLPmyjneaak5B40BeEQIrIOuATYE2f3b4nIyyLyUxHZmlLBQIGnRGSfiNwRZ78POB71uYv0KLmbmfuLmc7rB1Cnqich9AUGauOMyZTr+LuEVoTxmO9ecJK7wiawb85hHsmE63cNcFpV2+bYn9LrF/NMSck9aArCAUTEAzwC/KGqDsbs3k/IbHIR8H+AH6ZYvKtU9VLgBuDTInJtzH6JMyelsdAiUgC8H/h+nN3pvn6JkgnX8U+AIHD/HEPmuxec4qvARuBi4CQhM04sab9+wC2cf/WQsus3zzNlzmlxti3oGpqCSDIikk/oD3m/qj4au19VB1V1OPz+CSBfRKpTJZ+qngj/ewb4AaFlaDRdwJqoz37gRGqkm+UGYL+qno7dke7rF+Z0xOwW/vdMnDFpvY4i8nHgvcBtGjZIx5LAveAIqnpaVadVdQb4+hznTff1cwP/DvjuXGNSdf3meKak5B40BZFEwjbLfwIOqer/nmPMyvA4RGQHob9BX4rkKxWRssh7Qs7MAzHDHgc+Fo5mugIIRJayKWTOX27pvH5RPA58PPz+48Bjcca8ADSKyPrwiujm8DzHEZHrgc8B71fV0TnGJHIvOCVftE/rpjnOm7brF+YdwKuq2hVvZ6qu33meKam5B530wOfaC7ia0BKuBXgp/Ho3cCdwZ3jMXcBBQhEFzwFXplC+DeHzvhyW4U/C26PlE+AeQtEPrcD2FF/DEkIPfG/UtrRdP0KK6iQwRegX2X8AqoBfAG3hfyvDY1cDT0TNfTehqJPDkWudIvnaCdmeI/fgvbHyzXUvpEi+74TvrRZCD6xVmXT9wtu/Fbnnosam4/rN9UxJyT1opTYMwzCMuJiJyTAMw4iLKQjDMAwjLqYgDMMwjLiYgjAMwzDiYgrCMAzDiIspCMNYIOEqno4n54lIhYh8KlnjDGOhmIIwcoZw8l823fMVQCIP/kTHGcaCyKYvi2EsGBFZF66l/xVCdZzWiMhXRWRvuL7+n0eNPSYify4i+8N1/jeHt1eJyFMi8qKIfI2oGjci8p9F5ED49YdR53xVRL4R3n6/iLxDRJ4N1+8/pySDiGwVkecl1FugRUQagb8CNoa3fVlEPCLyiyj5IpU53zQufLzPisgL4WP9eez5DCMhnMj+s5e9MuUFrANmgCuitkWyTl3Ar4Dm8OdjwB+E338K+Eb4/T8S7gkAvIdQZms1cBmhjOBSwEMoo/aS8DmDQBOhH2H7gG8SUiw3Aj+MI+f/IVQ3CUK1+4vDx4nuo+AGysPvqwllTEucce8k1MRewuf/MXBtuv8W9sq+l60gjFygQ0O9LSJ8WET2Ay8CWwk1YIkQKYa2j9CDF0JNZf4VQFV/ApwNb78a+IGqjmiogOCjhEpEAxxV1VYNFaQ7CPxCVZWQQokcN5rfAF8Qkc8RqlY7FmeMAH8pIi3AvxEq3VwXZ9w7w68XCa2aNgONccYZxnlxp1sAw0gBI5E3IrIe+C/A5ap6VkS+BRRFjZ0I/zvNm78f8WrSxCunHHscCK1gJqLen/O9U9UHRGQPoRXKkyLye8CRmGG3ATXAZao6JSLHYmSPlut/qurXziOfYcyLrSCMXKOckMIIiEgdodLi8/EMoYczInIDsCJq+wdEpCRc0fMmYNdihBKRDcARVf1HQgXsmoEhQm0mI3iBM2Hl8DagPrw9dtyTwO+GewggIj4RiddQxjDOi60gjJxCVV8WkRcJmX2OAM8mMO3PgQfDZqmngc7wsfaHVyDPh8d9Q1VflFDnr4XyEeB2EZki1Iv7v6lqf9ixfYBQV7i/Bn4kInsJVfV8NSxHX/Q4Vf2siFwI/CZcGX0YuJ34PQMMY06smqthGIYRFzMxGYZhGHExBWEYhmHExRSEYRiGERdTEIZhGEZcTEEYhmEYcTEFYRiGYcTFFIRhGIYRl/8f5qHQhCWlvF8AAAAASUVORK5CYII=\n",
                        "text/plain": "<Figure size 432x288 with 1 Axes>"
                    },
                    "metadata": {
                        "needs_background": "light"
                    },
                    "output_type": "display_data"
                }
            ],
            "source": "from sklearn.neighbors import KNeighborsClassifier\nfrom sklearn import metrics\nfrom sklearn.model_selection import train_test_split\naccuracy = []\nlength = int(np.round(np.sqrt(le_y.shape[0])))+1\nfor i in range(length):\n    X_train, X_test, y_train, y_test = train_test_split(X, le_y, test_size = 0.2, random_state = i)\n    KNN = KNeighborsClassifier(n_neighbors = 16, metric = \"euclidean\").fit(X_train, y_train)\n    score = metrics.accuracy_score(y_test, KNN.predict(X_test))\n    accuracy.append(score)\nhyperparameter(length, accuracy, \"random state\")"
        },
        {
            "cell_type": "code",
            "execution_count": 78,
            "metadata": {},
            "outputs": [],
            "source": "KNN = KNeighborsClassifier(n_neighbors = 16, metric = \"euclidean\").fit(X_train, y_train)"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Decision Tree"
        },
        {
            "cell_type": "code",
            "execution_count": 29,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "0.7714285714285715"
                    },
                    "execution_count": 29,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "from sklearn.tree import DecisionTreeClassifier\n#X_train, X_test, y_train, y_test = train_test_split(X, le_y, test_size = 0.2, random_state = 4)\n\nDT = DecisionTreeClassifier(criterion = \"entropy\", max_depth = 6 ).fit(X_train, y_train)\nscore = metrics.accuracy_score(DT.predict(X_test), y_test)\nscore"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Support Vector Machine"
        },
        {
            "cell_type": "code",
            "execution_count": 67,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "Max accuracy:0.8285714285714286\nBest kernel: 0\nbest kernel: linear\n"
                },
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEGCAYAAABy53LJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3hUZfbA8e+ZFHon9A6hhNADBFFQKQJKU1fKiogKsiv2n4odRVdXd7GiUuwNUWmLSBEVGy1BSkIooYcaeickOb8/ZnBnYyATyOTOJOfzPPPM3Hvf9855uSEnt50rqooxxhjjK5fTARhjjAkuljiMMcbkiiUOY4wxuWKJwxhjTK5Y4jDGGJMroU4HkB8qVqyoderUcToMY4wJKvHx8ftVNSLr/EKROOrUqUNcXJzTYRhjTFARkW3ZzbdDVcYYY3LFEocxxphcscRhjDEmVyxxGGOMyRVLHMYYY3LFEocxxphcscRhjDEmVyxxXMDv2w8xYdEmp8MwxpiAUihuALxY03/fyUeLt3HiTDr3d2uIiDgdkjHGOM6vexwi0kNE1otIsoiMzmZ5GRH5j4isEpFEERnmmV9TRH4QkSTP/Hu9+owRkZ0istLz6uWv+J/u3ZQBMTV5/ftkXpy7DnvolTHG+HGPQ0RCgPFANyAFWC4is1R1rVezu4C1qtpbRCKA9SLyKZAOPKiqK0SkFBAvIgu8+r6iqv/yV+znhLiEF65vRlioMGHRZs6mK09e18T2PIwxhZo/D1W1A5JVdTOAiEwB+gLeiUOBUuL+TVwSOAikq+puYDeAqh4TkSSgepa++cLlEsb2jSYsxMV7v27hbEYmz/RpistlycMYUzj5M3FUB3Z4TacA7bO0eROYBewCSgEDVDXTu4GI1AFaAUu9Zo8SkVuAONx7JoeyfrmIjABGANSqVetSxoGI8NR1UYSHutx7HhmZ/KN/M0sexphCyZ/nOLL7rZr1JME1wEqgGtASeFNESv+xApGSwNfAfap61DP7baC+p/1u4N/ZfbmqTlTVGFWNiYj4U1XgXBMRRvdozN1XN2DK8h3831eryMi0cx7GmMLHn3scKUBNr+kauPcsvA0DXlT3WedkEdkCNAaWiUgY7qTxqapOO9dBVfee+ywik4DZfor/T0SEB7s3IizExbgFGziboYy7qQVhIXZVszGm8PBn4lgORIpIXWAnMBAYnKXNdqAL8LOIVAYaAZs95zzeBZJUdZx3BxGp6jkHAtAfSPDjGLJ1T5dIwkNdvPjtOtIzMnltYCvCQy15GGMKB78lDlVNF5FRwDwgBHhPVRNFZKRn+TvAWOADEVmD+9DWI6q6X0QuB4YAa0RkpWeVj6nqHOAlEWmJ+7DXVuBOf43hQkZ2rk9YiIuxs9dy9tN4xv+1NUVCQ5wIxRhj8pUUhnsTYmJi1F9PAPx48VaenJlI54YRTBjShqJhljyMMQWDiMSrakzW+XZ85RIN6VCHF69vxk8bU7n9w+WcTEt3OiRjjPErSxx5YGC7WvzrxhYs3nSAW99fzvEzljyMMQWXJY48ckObGrw6sBXx2w4x9L1lHD191umQjDHGLyxx5KE+Larx5qBWrNpxmCGTl3LkpCUPY0zBY4kjj/VsVpV3bm5D0u5jDJ68hIMn0pwOyRhj8pQlDj/oGlWZibe0YeO+4wyetIT9x884HZIxxuQZSxx+cmWjSrx/a1u2HjjBwIlL2Hf0tNMhGWNMnrDE4UcdG1Tkg2Ht2HX4FAMmLmH3kVNOh2SMMZfMEoefxdarwMe3tyP12BkGTFhCyqGTTodkjDGXxBJHPmhTuzyf3NGewyfTGDBhCdsOnHA6JGOMuWiWOPJJy5pl+Wx4LCfS0hkwYQmbU487HZIxxlwUSxz5KLp6GaaMiOVsRiYDJi5h495jTodkjDG5ZokjnzWuUpopI2IBGDhxCUm7j+bQwxhjAoslDgdEVi7FFyNiCQtxMWjSEhJ2HnE6JGOM8ZklDofUiyjJF3fGUiI8lMGTlrByx2GnQzLGGJ9Y4nBQ7Qol+OLOWMoWD+fmyUuJ23rQ6ZCMMSZHljgcVqNccb64M5ZKpYpwy3vLWLL5gNMhGWPMBVniCABVyxRjyohYqpUtxq3vL+OXjfudDskYY87LEkeAqFS6KFNGxFKnQglu+3A5P6zf53RIxhiTLUscAaRiySJ8PjyWyEolufOjeBas3et0SMYY8yd+TRwi0kNE1otIsoiMzmZ5GRH5j4isEpFEERmWU18RKS8iC0Rko+e9nD/HkN/KlQjnsztiaVKtNH/7JJ5v1+x2OiRjjPkffkscIhICjAd6AlHAIBGJytLsLmCtqrYArgT+LSLhOfQdDSxU1UhgoWe6QClTPIxPbm9Hi5plGfX578xcudPpkIwx5g/+3ONoBySr6mZVTQOmAH2ztFGglIgIUBI4CKTn0Lcv8KHn84dAPz+OwTGliobx0W3taFO7HPd/sZKv4lOcDsmYbKkq8xP3MDdhj9OhmHziz8RRHdjhNZ3imeftTaAJsAtYA9yrqpk59K2sqrsBPO+VsvtyERkhInEiEpeamnqpY3FEiSKhfDCsLR3qV+Chr1YxZdl2p0My5n/sPXqaER/HM+LjeEZ9toL1e6z+WmHgz8Qh2czTLNPXACuBakBL4E0RKe1j3wtS1YmqGqOqMREREbnpGlCKh4fy7tC2dIqMYPS0NXy8eKvTIRmDqvLF8u10HbeInzak8kC3hpQsGsoTM9aQmZmr/6omCPkzcaQANb2ma+Des/A2DJimbsnAFqBxDn33ikhVAM97gb9utWhYCBNvaUPXJpV5cmYi7/6yxemQTCG2/cBJ/jp5KY98vYYmVUsz975O3NMlkkd7Nmb51kN8tcIOqxZ0/kwcy4FIEakrIuHAQGBWljbbgS4AIlIZaARszqHvLGCo5/NQYKYfxxAwioSG8NZfW9MzugpjZ6/l7R83OR2SKWQyMpV3f9nCNa/+xOqUIzzXL5opw2OpW7EEAH9pU5OY2uV4YU4Sh06kORyt8Se/JQ5VTQdGAfOAJGCqqiaKyEgRGelpNha4TETW4L5C6hFV3X++vp4+LwLdRGQj0M0zXSiEh7p4Y1Arereoxj/nruP1hRudDskUEhv2HuOGt39j7Oy1xNYrz/z7O3FzbG1crv8eVXa5hOf6R3PsdDovfrvOwWiNv4lqwT8eGRMTo3FxcU6HkWcyMpWHvlrFtBU7GXVVAx7s3hD3hWnG5K209EzeWbSJN77fSMkioTzduyl9W1a74M/bC3OSmPDTZr4c2YG2dcrnY7Qmr4lIvKrGZJ0f6kQw5tKEuIR/3diC8BAXb/6QzNmMTEb3bGzJw+SpVTsO88jXq1m35xi9W1Tj6d5RVCxZJMd+93aNZPbq3TwxPYHZ91xOWIgVqChobIsGKZdL+Ef/ZgyJrc2Enzbz7Oy1FIa9R+N/p9Iy+MecJPq/9SuHTqYx6ZYY3hjUyqekAe4rAcf0acr6vcd4zy7kKJBsjyOIuVzCs32bEhbi4r1ft5CWnsnYvtH/c9zZmNxYvOkAj05bzdYDJxnUriaP9mpC6aJhuV5Pt6jKdG1SmVe/28i1zatSo1xxP0RrnGJ7HEFORHjyuiaM7FyfT5duZ/S01WTYdfQml46ePstj09cwaNISMhU+u6M9L1zf/KKSxjlj+rirBI2ZtTavwjQBwvY4CgAR4ZEejQgPdfH6wo2czVBevrE5oXZs2fjg+3V7eWxaAvuOneaOy+vyYPdGFAsPueT11ihXnPu6RvLCt+uYn7iH7k2r5EG0JhBY4iggRIQHujUkPET41/wNnM3I5JUBLe3EpDmvA8fP8OzstcxcuYuGlUvy9s2X0apW3habvu3yukxbsZMxsxLp2KAiJYrYr5yCwH6rFDCjrnbfwTt79W5GfbaCtPRMp0MyAUZVmbVqF91e+Yk5a3ZzX9dIZt99RZ4nDYCwEBfP9Y9m15HTdt9RAWKJowC6s3N9nrouinmJe/nbJ/GcPpvhdEgmQOw+corhH8Vxz+e/U7N8cWbffQX3dW1IeKj/fhW0rVOeATE1efeXLazbc9Rv32PyjyWOAuq2y+vyXL9oFq7bx/CP4ix5FHKZmcpnS7fTfdxP/JK8nyeubcK0v11Goyql8uX7R/dsTKmioTwxPcGKIBYAljgKsJtja/PSDc35JXk/t32wnJNp6U6HZBywdf8JBk9ewmPT1xBdvQzz7uvEHVfUIyQfL9suVyKcR3s1IW7bIb6M35FzBxPQLHEUcDe1rcm4m1qwZPMBbn1vOcfPWPIoLDIylUk/babHaz+RuPMoL1zfjM+Gt6d2hRKOxHNj6xq0q1OeF75dx0ErghjULHEUAv1b1eC1ga2I336IW95dytHTZ50OyfjZ+j3HuP6tX3l+ThKXN6jIggc6M6hdLUfL0pwrgnj8dDovzElyLA5z6SxxFBK9W1Rj/OBWrNl5hJsnL+XwSfuLryBKS8/klQUbuO6Nn9lx6BSvD2rFpFtiqFKmqNOhAdCwcinuuKIeX8ansGzLQafDMRfJEkch0iO6Ku/c3IZ1u48xeNJSO1xQwKzccZjr3viZ1xZu5NpmVfnugc70aXHhSrZOuKdLA6qXLcYTM9bY5eJByhJHIdOlSWUmDY1hU+pxBk1cQuqxM06HZC7RybR0npu9luvf+pVjp9N579YYXh3YivIlwp0OLVvFw0N5pk9TNuw9bk+zDFKWOAqhzg0jeP/Wtmw/eJKBExez9+hpp0MyF+m35P30ePVnJv+yhUHtajH//k5c3biy02HlqGtUZbpHVea1hRvYcfCk0+GYXLLEUUhd1qAiHwxry54jpxkwYTG7Dp9yOiSTC0dOnWX016sZPHkpLoEpI2J5vn8zSl1CUcL89nSfprhEGDMr0R4JEGQscRRi7etV4KPb23PgeBoDJi62v/yCxIK1e+n+yiKmxu3gzk71+PbeTsTWq+B0WLlWvWwx7usaycJ1+5i/dq/T4ZhcsMRRyLWpXY5P7mjPkZNnGTBhMdsOnHA6JHMe+4+fYdRnKxj+URzliocz466OPNqrSZ5UsnXKsI51aVylFGNmJXLC7jEKGpY4DC1qluWz4bGcOpvBTRMWsyn1uNMhGS+qyozfd9Jt3CLmJ+7lwW4NmTXqcprXKOt0aJcsLMTF8/2j2X3kNK9ZEcSg4dfEISI9RGS9iCSLyOhslj8kIis9rwQRyRCR8iLSyGv+ShE5KiL3efqMEZGdXst6+XMMhUV09TJ8PiKWjExlwIQlbNh7zOmQDLDr8Clu+2A5932xkjoVS/DNPZdzd5dIvxYlzG9tapdnYFt3EcSk3VYEMRiIv05KiUgIsAHoBqQAy4FBqprt48BEpDdwv6penc16dgLtVXWbiIwBjqvqv3yNJSYmRuPi4i5uIIVM8j73PR7pmcont7cnqlppp0MqlDIzlU+Xbeef364jI1N56JpGDL2sTr7Wl8pPh06k0WXcIupUKM5XIy+zxx8HCBGJV9WYrPP9+WdLOyBZVTerahowBeh7gfaDgM+zmd8F2KSq2/wQo8miQaVSfHFnB4qEuhg0aQlrUo44HVKhs2X/CQZOWsKTMxJoUdNdlPC2y+sW2KQB7iKIj/Vqworth5kaZ0UQA50/E0d1wPsnIMUz709EpDjQA/g6m8UD+XNCGSUiq0XkPRHJ9ukzIjJCROJEJC41NTX30RdidSuWYOqdHShZJJTBk5fw+/ZDTodUKKRnZPLOok30ePUnknYf5aUbmvPJ7e2pVaG406HlixtaV6ddXXcRxAPH7cbUQOZT4hCR2iLS1fO5mIj4UsQ/uz+PzndcrDfwq6r+T/EaEQkH+gBfes1+G6gPtAR2A//OboWqOlFVY1Q1JiIiwodwjbea5YvzxZ2xlCsezpB3l7F8q9UV8qe1u47S/63fePHbdXRuGMF3D3TmprY1A65ciD+JCM/3i+bEmXT+MWed0+GYC8gxcYjIcOArYIJnVg1ghg/rTgFqek3XAHadp212exUAPYEVqvrHRd6quldVM1Q1E5iE+5CY8YMa5Yoz9c4OVCpVhKHvLWPxpgNOh1TgnEnP4N/z19PnzV/YfeQU4we3ZsKQNlQuHRhFCfNbZOVSjOhUj69XpLBks/28BSpf9jjuAjoCRwFUdSNQyYd+y4FIEanr2XMYCMzK2khEygCdgZnZrONP5z1EpKrXZH8gwYdYzEWqUqYoU+6MpXrZYgz7YBk/b7TDfnklftshrn39F974Ppk+Laux4P7OXNu8aqHay8jO3VdHUqNcMZ6YkWBFEAOUL4njjOfkNgAiEsr5Dzn9QVXTgVHAPCAJmKqqiSIyUkRGejXtD8xX1f+588xz3qMbMC3Lql8SkTUishq4CrjfhzGYS1CpVFGmjIilToUS3P5hHD+s2+d0SEHtxJl0nvlPIje+8xsnz6Tz/rC2jLupJeUCtChhfisWHsKzfZuSvO84k37e7HQ4Jhs5Xo4rIi8Bh4FbgLuBvwNrVfVx/4eXN+xy3Lxx6EQaQ95byvo9xxg/uDXdm1ZxOqSg8/PGVB6dtoaUQ6e4pUNtHu7RmJJFQp0OKyCN/DieHzfsY8H9nalZvnBcIBBoLuVy3EeAVGANcCcwB3gib8MzwaBciXA+vSOWqGpl+PunK/hm9W6nQwoaR06e5eGvVjHk3WWEhbiYemcHnu0bbUnjAp7qHYVLhKdmJlgRxABzwZ9aEXEBq1U1GveJaFPIlSkWxie3t2PY+8u5+/MVpGe2pG/LbK+yNh5zE/bw5MwEDp5I429X1ufeLpEUDQve+lL5pVrZYjzQrSHPfZPEvMQ99IiumnMnky8uuMfhuXJplYjUyqd4TBAoVTSMD29rR7u65bnvi5V8aTdsZSv12Bnu+nQFIz+JJ6JkEWbe1ZFHejS2pJELt15WhyZVS/PMf9Zy3IogBgxfDlVVBRJFZKGIzDr38ndgJrCVKBLK+7e2o2P9ijz01Wo+W7rd6ZAChqrydXwKXcctYsHavTx0TSNmjupIdPUyTocWdEJDXDzXz10E8dUFG5wOx3j4coD1Gb9HYYJSsfAQJg+NYeQn8Tw2fQ3pmZnc0qGO02E5aufhUzw2bQ2LNqTSpnY5/nlDcxpUKul0WEGtTe1yDGpXi/d/28r1rWtY/bQAkOMeh6ouAtYBpTyvJM88YygaFsKEIW3oFlWZp2YmMrmQXj6Zmal8tHgr3cctYvnWg4zpHcWXd3awpJFHHunRiLLFwnh8xhoyM+1EudN8uXP8JmAZ8BfgJmCpiNzo78BM8CgSGsJbf21Nr2ZVeO6bJMb/kOx0SPlqU+pxBkxczFMzE2lduxzz7uvErR3rWoXXPFS2uLsI4u/bDzNluZ1Tc5ovh6oeB9qq6j4AEYkAvsNdhsQYwP1AntcHtiIsZBUvz1vP2YxM7u0SWaDvgj6bkcmknzfz6ncbKRrq4uUbm3NjmxoFesxOur51dabG7eDFb5Po3rQyFUsWcTqkQsuXk+Ouc0nD44CP/UwhExriYtxNLbmhdQ1e/W4j/5q/vsBef5+w8wj9xv/KS3PX06VxJb57sDN/iSlcRQnzm4jwfP9oTp3N4B9zkpwOp1DzZY9jrojM4781owYA3/ovJBPMQlzCyzc2JzxUGP/DJtLSM3msV5MC8wv19NkM3vh+I+8s2ky54uG8/dfW9Gxm9xfklwaV3EUQx/+wiRvb1OCy+hWdDqlQyjFxqOpDInI9cDnuUukTVXW63yMzQcvlEp7v14ywEBeTft7C2Qzl6d5RQZ884rYe5OGvV7M59QQ3tqnBE9c2oWxxqy+V30ZdFcmsVbt4ckYC397bqUA9RjdY5Jg4RKQuMEdVp3mmi4lIHVXd6u/gTPByuYRn+jQlPMTF5F+2cCY9k+f7RQflCeMTZ9J5ed56Ply8lWplivHRbe3o1NCe8eKUYuEhPNsnmmEfLGfSz5u566oGTodU6PhyqOpL4DKv6QzPvLZ+icgUGCLC49c2ISzUxds/buJsRib/vKF5UD0CddGGVB6btoZdR04xtEMdHrqmESWsvpTjrmpciZ7RVXh94UZ6N69WaJ6SGCh82ccL9S6r7vls++fGJyLCw9c04t4ukXwVn8KDU1eSnhH4z1g4fDKNB6euYuh7yygS5uLLOzswpk9TSxoB5KneUYS6hKdmWRHE/ObL/4JUEemjqrMARKQvsN+/YZmCRES4v1tDwkNdnkt1lVcHtiQsJDCPTX+7ZjdPzkzk0Mk0Rl3VgFFXN7D6UgGoapliPNC9EWNnr2Vuwh67SCEf+ZI4RgKfisibuE+O78D9bA5jcuWuqxoQHuLi+TlJnM3I5I3BrSgSGji/kPcdPc1TMxOZm7iHptVK8+FtbWlazepLBbKhHWrzdXwKY/6TyBUNI6xMfT7xpeTIJlWNBaKAKFW9TFUL163BJs8M71SPMb2jmL92L3/7ZAWnz2Y4HRKqypdxO+g6bhHfr9/HIz0aM/OujpY0gkBoiIvn+0ez79gZxs23Ioj5xZeSI/eKSGngBPCKiKwQke7+D80UVLd2rMvz/aP5ft0+hn8Ux6k055LHjoMnueW9ZTz01WoaVSnFt/dewd+urE9ogB5GM3/WqlY5BrerxQe/bSFh5xGnwykUfPnfcZuqHgW6A5WAYcCLfo3KFHh/bV+bl25szi/J+7ntg+WcTMvfZy1kZCof/LqFa179iRXbDjG2b1O+GNGB+hFWlDAYPXxNY8qXCOfxGQlkWBFEv/MlcZy7drIX8L6qrvKaZ8xFuymmJuNuasHSLQcY+t4yjp0+my/fm7zvGDdNWMyY/6ylbZ3yzLu/E0M61AnKe0yMW5niYTx+bRNW7TjM58vs2TD+5kviiBeR+bgTxzwRKQX4dD2liPQQkfUikiwio7NZ/pCIrPS8EkQkQ0TKe5ZtFZE1nmVxXn3Ki8gCEdnoeS/n21BNIOrfqgavD2rFiu2HGfLuMo6c8l/yOJuRyZvfb6TXa7+wKfU4425qwQfD2lKjnN0DUBD0a1mdDvUq8NLcdaQeO+N0OAWaL4njdmA07gq5J3HfwzHs3EIRaZpdJxEJAcYDPXGfWB8kIlHebVT1ZVVtqaotgUeBRap60KvJVZ7lMV7zRgMLVTUSWOiZNkHsuubVGD+4NYm7jnDz5KUcPpmWc6dcSth5hD5v/sq/5m+gW9PKLLi/M9e3tkq2BYmIMLafFUHMD75cVZWpqitU9bBn+oCqrvZq8vF5urYDklV1s+emwSlA3wt81SD+W0jxQvoCH3o+fwj086GPCXA9oqvwzs1tWL/nGIMmLeXA8bz5i/H02Qxe/HYdfcf/yv7jZ5gwpA3jB7cmopSV5C6IGlQqycjO9Zn++05+S7bbzfwlLy4dOd+fbNVx3/NxTopn3p9XIFIc6AF87TVbgfkiEi8iI7zmV1bV3QCe90rnWecIEYkTkbjU1FTfRmIc1aVJZSYPjWFz6nEGTVrCvmOnL2l9y7YcpNdrP/POok3c2LoG393fmWuaVsmjaE2guuuqBtQqX5wnZiZwJt35y70LorxIHOe7hCG7hHK+tr2BX7Mcpuqoqq1xH+q6S0Q65Soo1YmqGqOqMRERVpAuWHRqGMH7t7Zlx8FTDJy4hD1Hcp88jp9J58kZCdw0YTFpGZl8cnt7/nljc8oUD/NDxCbQFA0L4dm+TdmceoKJiwrno4z9zZ8Xq6cANb2mawC7ztN2IFkOU6nqLs/7PmA67kNfAHtFpCqA5937IVOmALisQUU+vK0de4+cZsDExew8fMrnvj+s30f3cYv4ZOk2butYl/n3d+LySHtmQ2FzZaNKXNusKm/8kMy2AyecDqfAyYvEcb4zmcuBSBGpKyLhuJPDrKyNRKQM0BmY6TWvhOfqLUSkBO57SBI8i2cBQz2fh3r3MwVHu7rl+ej29hw8nsaACYvZcfDkBdsfOpHGA1+sZNj7yylRJJSvRl7GU72jKB5uJSgKqyeviyI8xMVTMxOtCGIe8+XO8a9F5FoRybatpxxJdvPTgVHAPCAJmKqqiSIyUkRGejXtD8xXVe8/CyoDv4jIKmAZ8I2qzvUsexHoJiIbgW7YzYgFVpva5fh0eHuOnU5nwITFbN3/578cVZXZq3fRddwiZq3axT1XN2D2PZfTprZdpV3YVSlTlAe6NWTRhlTmrNnjdDgFiuSUiUWkK+7Lb2NxP4fjA1Vdlw+x5ZmYmBiNi4vLuaEJSIm7jjDk3WWEuoTPhsfSoJL77u69R0/z5IwE5q/dS7PqZXjpxuY0qVra4WhNIEnPyKTv+F9JPXaGhQ92plRRO8+VGyISn+V2CMC3y3G/U9W/Aq2BrcACEflNRIaJiG0F43dNq5Xh8+GxZCoMnLiYdXuO8sXy7XQdt4hFG1J5tGdjpv/9Mksa5k/cRRCbkXr8DOMWWBHEvOLTOQ4RqQDcCtwB/A68hjuRLPBbZMZ4aVSlFFNGxOIS4brXf+GRr9fQpGpp5t7XiTs7W1FCc34ta5blr+1r8eFvW60IYh7x5RzHNOBnoDjQW1X7qOoXqno3YBXhTL5pUKkkU+/sQGy9CjzXL5opw2OpW7GE02GZIPDQuSKI09dYEcQ84MufaW+qapSqvnDuxrtzsjv2ZYw/1alYgk/uaM/NsbWtKKHxWZliYTx5XRSrUo7wmRVBvGS+JI4mIlL23ISIlBORv/sxJmOMyXN9WlSjYwN3EcRLrUpQ2PmSOIafq1MFoKqHgOH+C8kYY/KeiDC2bzRnzmby/DdWBPFS+JI4XOJVQtRT9TbcfyEZY4x/1Isoycgr6zNz5S5+tSKIF82XxDEPmCoiXUTkatylQebm0McYYwLS36+sT+0KxXliRkJAPPM+GPmSOB4Bvgf+BtyF+xkYD/szKGOM8ZeiYSGM7RvNlv0nmGBFEC9KjoV8VDUTeNvzMsaYoNepYQTXNa/K+B+T6duyGnXssu5c8eU+jkgR+UpE1orI5nOv/AjOGGP85cnroigS4uLJmQlWBDGXfDlU9T7uvY104CrgI87/1D9jjAkKlUsX5cHuDfl5435mr96dcwfzB18SRzFVXYi7IOI2VR0DXO3fsIwxxv+GdKhDs+pleHb2Wo6ePut0OEHDl8Rx2lNSfaOIjBKR/pznccM2vfEAABRwSURBVK3GGBNMQlzC8/2j2X/8DOPmWxFEX/mSOO7DXafqHqANcDP/fZCSMcYEteY1yjIktjYfLd7K6pTDObY3OSQOz81+N6nqcVVNUdVhqnqDqi7Jp/iMMcbv/u+aRlQoWYTHpydYEUQfXDBxqGoG0Mb7znFjjCloShd1F0Fcs/MIny7d5nQ4Ac+XQ1W/AzNFZIiIXH/u5e/AjDEmP/VuXpXLG1Tk5bnr2XfUiiBeiC+JozxwAPeVVL09r+v8GZQxxuQ3EWFsv2jOZGQy1oogXpAvd44Py49AjDHGaXUrluDvV9bn1e82clNMDa6IjHA6pIDky53j74vIe1lfvqxcRHqIyHoRSRaR0dksf0hEVnpeCSKSISLlRaSmiPwgIkkikigi93r1GSMiO7369crdkI0x5vxGdq5PnQrFedKKIJ6XL4eqZgPfeF4LgdLA8Zw6ea7IGg/0BKKAQSIS5d1GVV9W1Zaq2hJ4FFikqgdx36X+oKo2AWKBu7L0feVcP1Wd48MYjDHGJ0XDQhjbL5qtB07y9o+bnA4nIPlyqOpr72kR+Rz4zod1twOSVXWzp98UoC+w9jztB+Eu2Y7nEbW7PZ+PiUgSUP0CfY0xJs9cERlBnxbVePvHTfRrVd2ebZ+FL3scWUUCtXxoVx3Y4TWd4pn3JyJSHOgBfJ3NsjpAK2Cp1+xRIrLac9is3HnWOUJE4kQkLjU11YdwjTHmv564rglFQl08OcOKIGblyzmOYyJy9NwL+A/uZ3Tk2DWbeef71+8N/Oo5TOX93SVxJ5P7VPWoZ/bbQH2gJe69kn9nt0JVnaiqMaoaExFhJ7iMMblTqVRRHurRiF+S9zNr1S6nwwkoOSYOVS2lqqW9Xg2zHr46jxSgptd0DeB8//oD8RymOkdEwnAnjU9VdZpXPHtVNcPznJBJuA+JGWNMnvtr+9o0r1GG575J4sgpK4J4ji97HP1FpIzXdFkR6efDupcDkSJSV0TCcSeHWdmsvwzQGZjpNU+Ad4EkVR2XpX1Vr8n+QIIPsRhjTK6FuITn+zXjwPEz/Hv+eqfDCRi+nON4WlWPnJtQ1cPA0zl1UtV0YBTuZ5YnAVNVNVFERorISK+m/YH5qnrCa15HYAhwdTaX3b4kImtEZDXu54Pc78MYjDHmojSrUYZbOtTh4yXbWLXDiiCC+xkbF24gslpVm2eZt0ZVm/k1sjwUExOjcXFxTodhjAlSR0+fpeu/F1GpdBFm3nU5Ia7CUb5PROJVNSbrfF/2OOJEZJyI1BeReiLyChCf9yEaY0xgOlcEMWHnUT5evNXpcBznS+K4G0gDvgCmAqeAu/wZlDHGBJrrmlflisiK/Gv+BvYW8iKIvlxVdUJVR5+7tFVVH8tyPsIYYwo8EWFs32jSMjIZO7tw34vsy1VVC0SkrNd0ORGZ59+wjDEm8NSpWIK7rmzA7NW7WbSh8N5Y7MuhqoqeK6kAUNVD2DPHjTGF1Mgr61GvYgmemll4iyD6kjgyReSPEiOeEiB2/70xplAqEuougrjtwEneKqRFEH1JHI8Dv4jIxyLyMbAIdyVbY4wplDo2qEjfltV458dNbErNsVh4gePLyfG5QAywHveVVQ/ivrLKGGMKrcevbUKRsMJZBNGXk+N34H4Ox4Oe18fAGP+GZYwxga1SqaI83KMxv206UOiKIPpyqOpeoC2wTVWvwl3ivPBeTmCMMR6D29WiRY0yjJ29liMnC08RRF8Sx2lVPQ0gIkVUdR3QyL9hGWNM4AtxCc/3b8bBE2m8PH+d0+HkG18SR4rnPo4ZwAIRmcn5y6MbY0yhEl29DEMvq8OnS7ezspAUQfTl5Hh/VT2sqmOAJ3GXO/elrLoxxhQKD3RrSKVSRXh8+hrSMzKdDsfvcvXoWFVdpKqzVDXNXwEZY0ywKVU0jKeua0rirqN8tHib0+H43cU8c9wYY0wWvZpVoXPDCMYt2MCeIwW7CKIlDmOMyQMiwrN9m3K2EBRBtMRhjDF5pHaFEoy6qgHfrNnNj+v3OR2O31jiMMaYPDSicz3qRZTgqZmJBbYIoiUOY4zJQ0VCQ3iubzTbD55k/A/JTofjF35NHCLSQ0TWi0iyiIzOZvlDIrLS80oQkQwRKX+hviJS3vOMkI2e93L+HIMxxuTWZQ0q0r9Vdd5ZtInkfQWvCKLfEoeIhADjgZ5AFDBIRKK826jqy6raUlVb4q64u0hVD+bQdzSwUFUjcdfQ+lNCMsYYpz3WqwnFwkIKZBFEf+5xtAOSVXWz576PKUDfC7QfBHzuQ9++wIeezx9iNyMaYwJQRKkiPNyjMYs3H2DGyp1Oh5On/Jk4qgM7vKZTPPP+RESKAz2Ar33oW1lVdwN43rN9GqGIjBCROBGJS021mozGmPw3uF0tWtYsy3OzkwpUEUR/Jg7JZt759td6A7+q6sGL6JstVZ2oqjGqGhMREZGbrsYYkydcLuH5/tEcOpnGS/MKThFEfyaOFKCm13QNzl8ccSD/PUyVU9+9IlIVwPNecC+WNsYEvabVynDrZXX5bNl2Vmw/5HQ4ecKfiWM5ECkidUUkHHdymJW1kYiUAToDM33sOwsY6vk8NEs/Y4wJOA90b0jlUkV5fHpCgSiC6LfEoarpwChgHpAETFXVRBEZKSIjvZr2B+ar6omc+noWvwh0E5GNQDfPtDHGBKySRUJ5uncUSbuP8mEBKIIoBe0ysezExMRoXFyc02EYYwoxVWXYB8tZvuUg3z3YmaplijkdUo5EJF5VY7LOtzvHjTEmH4gIz/aJJj1TefY/wV0E0RKHMcbkk1oVinNPl0i+TdjDD+uC97oeSxzGGJOPhl9Rj/oRJXhqVgKn0oKzCKIlDmOMyUfhoS6e69eMHQdP8eYPG50O56JY4jDGmHzWoX4Frm9dnYk/bSZ53zGnw8k1SxzGGOOAx3o1oXh4KI9PD74iiJY4jDHGARVLFuGRHo1ZuuUg01YEVxFESxzGGOOQgW1r0qpWWf4xJ4nDJ9OcDsdnljiMMcYhLpfwfL9mHD51ln/OXe90OD6zxGGMMQ6KqlaaYZfV4fNl24nfFhxFEC1xGGOMw+7r1pCqZYry+PQ1QVEE0RKHMcY47FwRxHV7jvHBb1udDidHljiMMSYAXNO0Clc3rsS4BRvYdfiU0+FckCUOY4wJACLCM32akqmBXwTREocxxgSImuWLc/fVkcxN3MPCpL1Oh3NeljiMMSaADL+iHg0qleTpWYkBWwTREocxxgSQ8FAXz/eLJuXQKd74PjCLIFriMMaYANO+XgVuaF2DiT9tZsPewCuCaInDGGMC0GO9GlOiSChPzAi8IoiWOIwxJgBVKFmER3s2ZtmWg3wdYEUQ/Zo4RKSHiKwXkWQRGX2eNleKyEoRSRSRRZ55jTzzzr2Oish9nmVjRGSn17Je/hyDMcY45aaYmrT2FEE8dCJwiiD6LXGISAgwHugJRAGDRCQqS5uywFtAH1VtCvwFQFXXq2pLVW0JtAFOAtO9ur5ybrmqzvHXGIwxxkkul/B8/2YcOXWWf85d53Q4f/DnHkc7IFlVN6tqGjAF6JulzWBgmqpuB1DV7J7e3gXYpKrb/BirMcYEpCZVS3P75XWZsnwH8dsOOh0O4N/EUR3Y4TWd4pnnrSFQTkR+FJF4Ebklm/UMBD7PMm+UiKwWkfdEpFx2Xy4iI0QkTkTiUlNTL3YMxhjjuHu7RFKtTFEen57A2QAogujPxCHZzMt6aUAo7kNR1wLXAE+KSMM/ViASDvQBvvTq8zZQH2gJ7Ab+nd2Xq+pEVY1R1ZiIiIiLHoQxxjitRJFQnu7TlHV7jvH+r1ucDseviSMFqOk1XQPYlU2buap6QlX3Az8BLbyW9wRWqOof996r6l5VzVDVTGAS7kNixhhToHWPqkzXJpV49buN7HS4CKI/E8dyIFJE6nr2HAYCs7K0mQlcISKhIlIcaA8keS0fRJbDVCJS1WuyP5CQ55EbY0yAERGe7u0ugvjMrERHY/Fb4lDVdGAUMA93MpiqqokiMlJERnraJAFzgdXAMmCyqiYAeBJJN2BallW/JCJrRGQ1cBVwv7/GYIwxgaRm+eLc26Uh89fuZcFa54ogSqDdkegPMTExGhcX53QYxhhzyc5mZHLt6z9z4kwGCx7oRPHwUL99l4jEq2pM1vl257gxxgSRsBAXz/Vrxs7Dp3h9YbIjMVjiMMaYINOubnn+0qYGk3/ezPo9+V8E0RKHMcYEoUd7NaFk0VCemLGGzMz8PeVgicMYY4JQ+RLhPNqzMcu3HuKrFSn5+t2WOIwxJkj9pU1NYmqX44U5SRzMxyKIljiMMSZIuVzCc/2jOXY6nRe/Tcq5Q159b759kzHGmDzXuIq7COLUuBSWb82fIoiWOIwxJsjd2zWS6mWL8fj0NflSBNEShzHGBLni4aGM6dOUDXuP8+4v/i+CaInDGGMKgG5RlenapDKvfbeRlEMn/fpdljiMMaaAGNPH/ZDVMbPW+vV7LHEYY0wBUaNcce7rGsl3SXuZn7jHb99jicMYYwqQ2y6vS6PKpRgzK5ETZ9L98h2WOIwxpgAJC3HxXP9odh05zesLN/rlOyxxGGNMAdO2TnkGxNRk8i9bWLfnaJ6v33+F3I0xxjhmdM/G7DpyCn88cskShzHGFEDlSoTz8e3t/bJuO1RljDEmVyxxGGOMyRVLHMYYY3LFr4lDRHqIyHoRSRaR0edpc6WIrBSRRBFZ5DV/q4is8SyL85pfXkQWiMhGz3s5f47BGGPM//Jb4hCREGA80BOIAgaJSFSWNmWBt4A+qtoU+EuW1Vylqi1VNcZr3mhgoapGAgs908YYY/KJP/c42gHJqrpZVdOAKUDfLG0GA9NUdTuAqu7zYb19gQ89nz8E+uVRvMYYY3zgz8RRHdjhNZ3imeetIVBORH4UkXgRucVrmQLzPfNHeM2vrKq7ATzvlbL7chEZISJxIhKXmpp6yYMxxhjj5s/7OCSbeVlvRQkF2gBdgGLAYhFZoqobgI6quktEKgELRGSdqv7k65er6kRgIkBMTIwfboExxpjCyZ+JIwWo6TVdA9iVTZv9qnoCOCEiPwEtgA2qugvch69EZDruQ18/AXtFpKqq7haRqkCOh7fi4+P3i8i2ixxHRWD/RfYNNDaWwFNQxgE2lkB1KWOpnd1MfyaO5UCkiNQFdgIDcZ/T8DYTeFNEQoFwoD3wioiUAFyqeszzuTvwrKfPLGAo8KLnfWZOgahqxMUOQkTispycD1o2lsBTUMYBNpZA5Y+x+C1xqGq6iIwC5gEhwHuqmigiIz3L31HVJBGZC6wGMoHJqpogIvWA6SJyLsbPVHWuZ9UvAlNF5HZgO3++EssYY4wf+bVWlarOAeZkmfdOlumXgZezzNuM+5BVdus8gPuciDHGGAfYneM5m+h0AHnIxhJ4Cso4wMYSqPJ8LKL+qLlrjDGmwLI9DmOMMbliicMYY0yuWOIAROQ9EdknIgnnWS4i8rqnWONqEWmd3zH6yoexXCkiRzzFI1eKyFP5HaMvRKSmiPwgIkmeApj3ZtMmKLaLj2MJlu1SVESWicgqz1ieyaZNsGwXX8YSFNsF3PUBReR3EZmdzbK83SaqWuhfQCegNZBwnuW9gG9x3w0fCyx1OuZLGMuVwGyn4/RhHFWB1p7PpYANQFQwbhcfxxIs20WAkp7PYcBSIDZIt4svYwmK7eKJ9QHgs+zizettYnscgLpLmRy8QJO+wEfqtgQo67lrPeD4MJagoKq7VXWF5/MxIIk/1zoLiu3i41iCguff+rhnMszzynqFTbBsF1/GEhREpAZwLTD5PE3ydJtY4vCNLwUbg0kHz+75tyLS1OlgciIidYBWuP8i9BZ02+UCY4Eg2S6eQyIrcZf7WaCqQbtdfBgLBMd2eRV4GPeN1NnJ021iicM3vhRsDBYrgNqq2gJ4A5jhcDwXJCIlga+B+1T1aNbF2XQJ2O2Sw1iCZruoaoaqtsRdf66diERnaRI028WHsQT8dhGR64B9qhp/oWbZzLvobWKJwze+FGwMCqp69Nzuubrv7A8TkYoOh5UtEQnD/Yv2U1Wdlk2ToNkuOY0lmLbLOap6GPgR6JFlUdBsl3PON5Yg2S4dgT4ishX3c4+uFpFPsrTJ021iicM3s4BbPFcmxAJH1PNMkGAjIlXEUwRMRNrh/hk44GxUf+aJ8V0gSVXHnadZUGwXX8YSRNslQtxP7kREigFdgXVZmgXLdslxLMGwXVT1UVWtoap1cBeT/V5Vb87SLE+3iV9rVQULEfkc99UTFUUkBXga94ky1F1baw7uqxKSgZPAMGcizZkPY7kR+JuIpAOngIHquewiwHQEhgBrPMegAR4DakHQbRdfxhIs26Uq8KG4Hw3tAqaq6mzxKl5K8GwXX8YSLNvlT/y5TazkiDHGmFyxQ1XGGGNyxRKHMcaYXLHEYYwxJlcscRhjjMkVSxzGGGNyxRKHMXlAROrIeSoS++n7jufcyhj/sMRhjINExO6lMkHHfmiNyWMiUg93eZGRwDNABO6broar6joR+QB3BeNWwAoRqQAcBWKAKsDDqvqVZ10PATcBRYDpqvp0Pg/HmD+xPQ5j8pCINMKdNIYB/wDuVtU2wP8Bb3k1bQh0VdUHPdNVgcuB64AXPevqDkQC7YCWQBsR6ZQf4zDmQmyPw5i8EwHMBG4AtgGXAV96Sh2Be6/hnC9VNcNreoaqZgJrRaSyZ153z+t3z3RJ3InkJ/+Eb4xvLHEYk3eO4H7mQUfP+2FPye7snMgyfcbrs3i9v6CqE/I0SmMukR2qMibvpAH9gFtwH3LaIiJ/gT+e+dwil+ubB9zmeY4HIlJdRCrlZcDGXAzb4zAmD6nqCc+DdRYAnwC3i8gTuCsUTwFW5WJd80WkCbDYc7jrOHAz7qfVGeMYq45rjDEmV+xQlTHGmFyxxGGMMSZXLHEYY4zJFUscxhhjcsUShzHGmFyxxGGMMSZXLHEYY4zJlf8H2KZ7oiKJ+mAAAAAASUVORK5CYII=\n",
                        "text/plain": "<Figure size 432x288 with 1 Axes>"
                    },
                    "metadata": {
                        "needs_background": "light"
                    },
                    "output_type": "display_data"
                }
            ],
            "source": "from sklearn import svm\nkernel = ['linear', 'poly', 'rbf', 'sigmoid']\nSVM_acc = []\nfor c, i in enumerate(kernel):\n    X_train, X_test, y_train, y_test = train_test_split(X, le_y, test_size = 0.2, random_state = c)\n    SVM = svm.SVC(kernel= i).fit(X_train, y_train)\n    score = metrics.accuracy_score(SVM.predict(X_test), y_test)\n    SVM_acc.append(score)\nindex, _ = hyperparameter(len(kernel), SVM_acc, \"kernel\")\nprint(f\"best kernel: {kernel[index]}\")"
        },
        {
            "cell_type": "code",
            "execution_count": 68,
            "metadata": {},
            "outputs": [],
            "source": "SVM = svm.SVC(kernel= \"linear\").fit(X_train, y_train)"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Logistic Regression"
        },
        {
            "cell_type": "code",
            "execution_count": 162,
            "metadata": {},
            "outputs": [],
            "source": "from sklearn.linear_model import LogisticRegression\nLog_acc = []\nsolver = ['liblinear', 'newton-cg', 'lbfgs', 'sag', 'saga']\n\ncross_validation = [1,2,3,4,5]\nfor i, c in zip(solver, cross_validation):\n    X_train, X_test, y_train, y_test = train_test_split(X, le_y, test_size = 0.2, random_state = 4)\n    LR = LogisticRegression(C = 0.01, solver = i).fit(X_train, y_train)\n    score = metrics.accuracy_score(LR.predict(X_test), y_test)\n    Log_acc.append(score)"
        },
        {
            "cell_type": "code",
            "execution_count": 163,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "Max accuracy:0.7857142857142857\nBest Solver: 1\nnewton-cg\n"
                },
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAe00lEQVR4nO3deZhddZ3n8fcnK9lICKlIqAqETTBsISkQxREBlygiRCGBUdu2p0Uc6daeHlrRfsa255+eYR67e2xHmnHBnlZMhbCJAYItYIuKdRMCZAEJEbyVBFLZN7JU1Xf+uKfgpnKr6lRSp07VvZ/X89TDPVvdbx2q7jfnd875HEUEZmZmXQ3LuwAzMxuc3CDMzKwiNwgzM6vIDcLMzCpygzAzs4pG5F1Af5oyZUrMmDEj7zLMzIaMZcuWbY6IukrLqqpBzJgxg0KhkHcZZmZDhqRXulvmISYzM6vIDcLMzCpygzAzs4rcIMzMrCI3CDMzq8gNwszMKnKDMDOziqrqPggbGI+ufo3nWrbnXYaZJcaOHsFNl57W79/XDcL6ZMfrB7n5R8vZ39aBlHc1ZgYwZfxoNwjL30+e2cD+tg4euPkSzmuYlHc5ZpYhn4OwPmkqFDnrhAmcWz8x71LMLGNuEJbamo07ebZlB/MbpyOPL5lVPTcIS62pUGTU8GHMu6A+71LMbAC4QVgq+9vaue/p9bxv5ls4btyovMsxswHgBmGp/Gz1JrbtPcj8C6fnXYqZDRA3CEtlYaHIiROP4V2nT8m7FDMbIJk2CElzJb0gaa2kL1dYfoukFcnXSkntkiYny/5C0qpk/l2SjsmyVuvehu2v8+8vtnLtnAaGD/PJabNakVmDkDQc+BbwQWAmcIOkmeXrRMRtETErImYBtwJPRMRWSfXAnwONEXEOMBy4PqtarWd3L2shAq5r9PCSWS3J8gjiImBtRKyLiAPAj4Gre1j/BuCusukRwBhJI4CxwIbMKrVudXQEi5YVeedpxzN98ti8yzGzAZRlg6gHimXTLcm8w0gaC8wFFgNExHrgfwF/ADYCOyJiaYa1Wjd+s24Lxa2vs8Anp81qTpYNotJgdXSz7lXAkxGxFUDScZSONk4BTgTGSfpExTeRbpRUkFRobW3th7KtXFOhyIRjRvCBs0/IuxQzG2BZNogWoPyfnQ10P0x0PYcOL70X+H1EtEbEQeAe4J2VNoyIOyKiMSIa6+rq+qFs67Tj9YM8tPJVrplVzzEjh+ddjpkNsCwbRDNwhqRTJI2i1AQe6LqSpInApcD9ZbP/AFwsaaxKmQ5XAGsyrNUqeCAJ5pvvk9NmNSmzNNeIaJN0M/AIpauQvhcRqyTdlCy/PVl1HrA0IvaUbfuUpLuB5UAb8DRwR1a1WmVNzUXeNu1Yzqk/Nu9SzCwHmcZ9R8QSYEmXebd3mb4TuLPCtl8DvpZhedaD1Rt28tz6HXztqpkO5jOrUb6T2irqDOa7ZpaD+cxqlRuEHWZ/Wzv3rVjP+852MJ9ZLXODsMM8uvo1tu89yAKfnDaraW4QdpiFzUXqJ43hEgfzmdU0Nwg7xPrtr/PLtZv5mIP5zGqeG4Qd4u5CEsw3pyHvUswsZ24Q9obOYL5LTncwn5m5QViZX6/bQsu2133ntJkBbhBWpqlQ5FgH85lZwg3CANixNwnmu8DBfGZW4gZhADzwzHoOOJjPzMq4QRgACwtFZk47lnPqJ+ZdipkNEm4QxqoNO1i5fifzG31pq5m9yQ3CWFRoYdSIYVxzgYP5zOxNbhA1bt/Bdu59ej0fOPsEJo11MJ+ZvckNosY9uvo1drx+0MNLZnYYN4ga11RIgvlOczCfmR3KDaKGtWzbyy/XbubaOQ0MczCfmXXhBlHD7l7WAsB1Hl4yswrcIGpUR0ewqNDCJadNoeE4B/OZ2eHcIGrUr17awvrtrzP/Qt85bWaVuUHUqKZCkYljRvL+mW/JuxQzG6TcIGrQjr0HeXjVq1wz60QH85lZt9wgatD9STDfdQ7mM7MeuEHUoIXNRc4+0cF8ZtYzN4gas3L9DlZt2MkCn5w2s164QdSYRYUio0YM4+rzHcxnZj1zg6gh+w62c9+KDcw9+wQmjh2ZdzlmNsi5QdSQpW8E83l4ycx65wZRQ5qaS8F87zzt+LxLMbMhwA2iRhS37uXJlzZzXaOD+cwsnUwbhKS5kl6QtFbSlyssv0XSiuRrpaR2SZOTZZMk3S3peUlrJL0jy1qrXWcw37VzHMxnZulk1iAkDQe+BXwQmAncIGlm+ToRcVtEzIqIWcCtwBMRsTVZ/I/AwxFxFnA+sCarWqtdR0dw97IW3nW6g/nMLL0sjyAuAtZGxLqIOAD8GLi6h/VvAO4CkHQs8G7guwARcSAitmdYa1V78qXNpWA+n5w2sz7IskHUA8Wy6ZZk3mEkjQXmAouTWacCrcD3JT0t6TuSxnWz7Y2SCpIKra2t/Vd9FWkqtDBxzEje52A+M+uDLBtEpTOh0c26VwFPlg0vjQBmA9+OiAuAPcBh5zAAIuKOiGiMiMa6urqjrbnqbN97gEdWvcq8C+odzGdmfZJlg2gBysc0GoAN3ax7PcnwUtm2LRHxVDJ9N6WGYX10/4oNSTCfT06bWd9k2SCagTMknSJpFKUm8EDXlSRNBC4F7u+cFxGvAkVJZyazrgBWZ1hr1VrYXOSc+mM5+0QH85lZ34zI6htHRJukm4FHgOHA9yJilaSbkuW3J6vOA5ZGxJ4u3+LPgB8mzWUd8Omsaq1WK9fvYPXGnfz3q8/OuxQzG4IyaxAAEbEEWNJl3u1dpu8E7qyw7QqgMcPyql5TEsz3EQfzmdkR8J3UVWrfwXbue3o9HzzHwXxmdmTcIKrUI6teZee+Nt/7YGZHzA2iSjUVijQcN4Z3nOpgPjM7Mm4QVai4dS9Prt3CdXOmO5jPzI6YG0QVWrSsBQmu9b0PZnYU3CCqTHtHcHehyLtOn0L9pDF5l2NmQ1iqBiHpZEnvTV6PkTQh27LsSD25djMbduxjwYU+OW1mR6fXBiHpM5SiLv45mdUA3JdlUXbkmgpFJo11MJ+ZHb00RxCfBy4BdgJExIvA1CyLsiOzbc8Blq56jWtm1TN6hIP5zOzopGkQ+5PnOQAgaQTdp7Jaju5fsZ4D7R2+98HM+kWaBvGEpK8AYyS9D1gE/CTbsqyvIoKFhRbOrZ/IzBOPzbscM6sCaRrElyg9vOc54LOUspX+OsuirO9WbdjJmo07me+T02bWT3oM65M0DHg2Is4B/u/AlGRHYmFzkdEjhvGR80/MuxQzqxI9HkFERAfwjKSTBqgeOwL7DrZz34okmG+Mg/nMrH+kifueBqyS9FtKj/4EICI+kllV1iePrHqVXQ7mM7N+lqZBfD3zKuyoLGwuMn3yGC52MJ+Z9aNeT1JHxBPA88CE5GtNMs8GgeLWvfzqJQfzmVn/S3Mn9Xzgt8B1wHzgKUnXZl2YpbOoUCwF881xMJ+Z9a80Q0xfBS6MiE0AkuqAn1GK37ActXcEdy9r4T+cUceJDuYzs36W5j6IYZ3NIbEl5XaWsV92BvP55LSZZSDNEcTDkh4B7kqmFwAPZVeSpdVUKHLc2JG8d6ajscys//XaICLiFkkfBd4FCLgjIu7NvDLr0bY9B3h01Wt8/OKTHMxnZpnotUFIOgVYEhH3JNNjJM2IiJezLs66d18SzOfnPphZVtKcS1gEdJRNtyfzLCcRwcLmIuc1TOSsExzMZ2bZSNMgRpTHfSevR2VXkvXmufU7eP7VXb5z2swylaZBtEp6I1ZD0tXA5uxKst40FUrBfFc5mM/MMpTmKqabgB9K+idKJ6mLwB9lWpV1a9/Bdu5fsYEPnTvNwXxmlqk0VzG9BFwsaTygiNiVfVnWnYdXloL5rmv0ndNmlq00URtfkHQspSTXv5e0XNL7sy/NKlnYXOSkyWO5+BQH85lZttKcg/iTiNgJvB+YCnwa+LtMq7KK/rBlL79et4Xr5jQ4mM/MMpemQXR+En0I+H5EPFM2r+cNpbmSXpC0VtKXKyy/RdKK5GulpHZJk8uWD5f0tKQH07xftVu0LAnm8/CSmQ2ANA1imaSllBrEI5ImcOh9ERVJGg58C/ggMBO4QdLM8nUi4raImBURs4BbgSciYmvZKl8A1qT7UapbZzDfu8+oY9pEB/OZWfbSNIj/BHyZUqLrXkr3QHy6c6Gks7vZ7iJgbUSsS+6d+DFwdQ/vcwNv5j0hqQG4EvhOihqr3r+/2MrGHft857SZDZg0DwzqiIjlEbE9md4SEc+WrfL/utm0ntIlsZ1aknmHkTQWmAssLpv9D8BfkeJopRYsKrQwedwo3vu2t+RdipnViP6I7e7ufESl+dHNulcBT3YOL0n6MLApIpb1+ubSjZIKkgqtra2pCh5qtu45wNLVr3LNrHpGjXDSupkNjP74tOnuQ78FKB8PaQA2dLPu9ZQNLwGXAB+R9DKloanLJf1rxTePuCMiGiOisa6urk+FDxX3Pb2eg+3h4SUzG1BZ/nO0GThD0imSRlFqAg90XUnSROBS4P7OeRFxa0Q0RMSMZLufR8QnMqx10IoImgpFzm+YyJknTMi7HDOrIf3RIA5UmhkRbcDNwCOUrkRqiohVkm6SdFPZqvOApRGxpx9qqTrPtiTBfD56MLMBluZ5EIuB7wEPRcRhJ4wj4uLuto2IJcCSLvNu7zJ9J3BnD9/jceDx3uqsVk2FIseMdDCfmQ28NEcQ3wb+I/CipL+TdFbGNVni9QPtPLBiAx86ZxrHHuNgPjMbWGkuc/1ZRHwcmA28DDwq6VeSPi3Jn1oZenjVRnbtb+M6P/fBzHKQ6hyEpOOBPwb+FHga+EdKDePRzCozFjYXOfn4sVx86uTeVzYz62dpzkHcA5xF6Ya4qyJiY7JooaRClsXVsle27OE367byX9//ViQH85nZwEvzwKB/ioifV1oQEY39XI8lFhVaGCa4do6Hl8wsH2mGmN4maVLnhKTjJP3nDGuqeZ3BfJe+tY4TJh6TdzlmVqPSNIjPdOYwAUTENuAz2ZVkv3ixlVd37mO+T06bWY7SNIhhKhsET2K8R2VXki0qFJk8bhRXOJjPzHKUpkE8AjRJukLS5ZQykx7OtqzatWX3fh5d/RrzLnAwn5nlK81J6i8BnwU+RymhdSl+RkNm7k2C+Ty8ZGZ567VBJPEa306+LENvBPNNn+RgPjPLXa9jGJLOkHS3pNWS1nV+DURxteaZlh387rXdLPDRg5kNAmkGub9P6eihDbgM+Be6f4qcHYXOYL4Pnz8t71LMzFI1iDER8W+AIuKViPgb4PJsy6o9rx9o5ycrNvChcx3MZ2aDQ5qT1PskDaOU5nozsB6Ymm1ZteehlaVgPp+cNrPBIs0RxBeBscCfA3OATwCfyrKoWrSwuciM48fy9lMczGdmg0OPRxDJTXHzI+IWYDfw6QGpqsa8vHkPT/1+K7d84EwH85nZoNHjEUREtANz5E+tTC1aVmSY4GOzG/IuxczsDWnOQTwN3C9pEfDGc6Mj4p7MqqohncF87zlzqoP5zGxQSdMgJgNbOPTKpQDcIPrBL37Xyms79/P1j/jowcwGlzR3Uvu8Q4aaCkWOHzeKy89yMJ+ZDS5pnij3fUpHDIeIiD/JpKIasmX3fn625jU+9Y4ZDuYzs0EnzRDTg2WvjwHmARuyKae2vBHMd6HvfTCzwSfNENPi8mlJdwE/y6yiGhERLGwuMmv6JN76FgfzmdngcyTjGmcAJ/V3IbVmRXE7L27azQIfPZjZIJXmHMQuDj0H8SqlZ0TYUWgqtDBm5HA+fJ6D+cxscEozxOTxj36290AbP3mmFMw3wcF8ZjZIpXkexDxJE8umJ0m6JtuyqttDz73K7v1tzG/0vQ9mNnilOQfxtYjY0TkREduBr2VXUvVbWChyypRxXORgPjMbxNI0iErrpLk81ir4/eY9/Pb3W7muscHBfGY2qKVpEAVJ35B0mqRTJf09sCzrwqrVooKD+cxsaEjTIP4MOAAsBJqA14HPp/nmkuZKekHSWklfrrD8Fkkrkq+VktolTZY0XdJjktZIWiXpC335oQartvYOFi9v4bIzp/KWYx3MZ2aDW5qrmPYAh3249yZ5lsS3gPcBLUCzpAciYnXZ974NuC1Z/yrgLyJiq6TRwF9GxHJJE4Blkh4t33Yo+sWLncF8vvfBzAa/NFcxPSppUtn0cZIeSfG9LwLWRsS6iDgA/Bi4uof1bwDuAoiIjRGxPHm9C1gD1Kd4z0GtqbmFKeNHccXb/MRWMxv80gwxTUmuXAIgIraR7pnU9UCxbLqFbj7kJY0F5gKLKyybAVwAPNXNtjdKKkgqtLa2pigrH5uTYL55F9QzcriD+cxs8EvzSdUh6Y1ojeQD+7B01woqXaLT3XZXAU9GxNZDvoE0nlLT+GJE7Ky0YUTcERGNEdFYV1eXoqx83Lt8PW0dwfxGDy+Z2dCQ5nLVrwK/lPREMv1u4MYU27UA5Z+GDXSfAns9yfBSJ0kjKTWHHw71p9dFBE2FIhecNIkzHMxnZkNEr0cQEfEw0Ai8QOlKpr+kdCVTb5qBMySdImkUpSbwQNeVkru0LwXuL5sn4LvAmoj4Ror3GtSe7gzm89GDmQ0hacL6/hT4AqUjgBXAxcCvOfQRpIeJiDZJNwOPAMOB70XEKkk3JctvT1adByxNrpbqdAnwSeA5SSuSeV+JiCWpf7JBZFGhyJiRw7nSwXxmNoSkGWL6AnAh8JuIuEzSWcDX03zz5AN9SZd5t3eZvhO4s8u8X1L5HMaQUwrm28iV5zmYz8yGljQnqfdFxD4ASaMj4nngzGzLqh5L3gjm8/CSmQ0taY4gWpL7IO4DHpW0DT9yNLWm5iKnThnHhTOOy7sUM7M+SXMn9bzk5d9IegyYCDycaVVVYl3rbn778la+NPcsB/OZ2ZDTp1TWiHii97Ws06JlLQwfJj42e8jfBG5mNci39Gakrb2DxctauOzMOqY6mM/MhiA3iIw88btWNu3az3U+OW1mQ5QbREYWNheZMn4Ul5/lYD4zG5rcIDLQums/P39+Ex+d3eBgPjMbsvzplYF7n25Jgvn81DgzG7rcIPpZKZivhdknTeL0qQ7mM7Ohyw2iny3/w3bWbtrNggt9ctrMhjY3iH62qFBk7KjhXHneiXmXYmZ2VNwg+tGe/W385JkNXHnuNMaP7tM9iGZmg44bRD9a8txG9hxo9/CSmVUFN4h+1FQocmrdOOac7GA+Mxv63CD6ybrW3TS/vI35jdMdzGdmVcENop80FUrBfB91MJ+ZVQk3iH7Q1t7B4uUtXHbmVKZOcDCfmVUHN4h+8PgLrbTu2u87p82sqrhB9IOFhSJTxo/mMgfzmVkVcYM4Spt27ePnz2/iY7PrHcxnZlXFn2hH6d7l62nvCD/3wcyqjhvEUSgF8xWZc/JxnD51fN7lmJn1KzeIo7D8D9t4qXUPC3z0YGZVyA3iKDQ1tyTBfNPyLsXMrN+5QRyhPfvbePDZDXz4vGmMczCfmVUhN4gj9FMH85lZlXODOEJNzaVgvtknOZjPzKqTG8QReKl1N4VXtrHAwXxmVsXcII5AU6HI8GFinoP5zKyKuUH00cH2DhYvW8/lZzmYz8yqW6YNQtJcSS9IWivpyxWW3yJpRfK1UlK7pMlpts3L4y+0snn3fub73gczq3KZNQhJw4FvAR8EZgI3SJpZvk5E3BYRsyJiFnAr8EREbE2zbV4WNhepmzCay86sy7sUM7NMZXkEcRGwNiLWRcQB4MfA1T2sfwNw1xFuOyA27drHYy9s4qOz6xnhYD4zq3JZfsrVA8Wy6ZZk3mEkjQXmAouPYNsbJRUkFVpbW4+66J7ckwTzeXjJzGpBlg2i0vWf0c26VwFPRsTWvm4bEXdERGNENNbVZTfs0xnM13jycZxW52A+M6t+WTaIFqD8n9oNwIZu1r2eN4eX+rrtgFj2yjbWte5hvu+cNrMakWWDaAbOkHSKpFGUmsADXVeSNBG4FLi/r9sOpKZCkXGjhnPluQ7mM7PakFnKXES0SboZeAQYDnwvIlZJuilZfnuy6jxgaUTs6W3brGrtze79bTz47EauOu9EB/OZWc3I9NMuIpYAS7rMu73L9J3AnWm2zcuSZzey90C7h5fMrKb4Ws0UFhaKnFY3jtknTcq7FDOzAeMG0Yu1m3az7JVtLLjQwXxmVlvcIHqxqFBkxDAx74KGvEsxMxtQbhA9ONjeweLlLVx+1lTqJozOuxwzswHlBtGDx57fxObdB3zntJnVJDeIHjQVSsF873Ewn5nVIDeIbmzauY/HXmjlY7MbHMxnZjXJn3zdWPxGMJ9PTptZbXKDqCAiWFQoctGMyZzqYD4zq1FuEBUUXtnGus17uM5HD2ZWw9wgKmhqToL5znMwn5nVLjeILnbvb+Onz23kqvNPZOwoB/OZWe1yg+jip89ucDCfmRluEIdZ2Fzk9KnjuWC6g/nMrLa5QZRZu2kXy/+wnQWNDuYzM3ODKNNUaCkF882uz7sUM7PcuUEkDrZ3cM/yFq5421SmjHcwn5mZG0Ti5w7mMzM7hBtEoqm5yNQJo7n0rQ7mMzMDNwgAXtu5j8de2MTH5jiYz8yskz8NgcXLW+gIPLxkZlam5htEKZivhYtOmcwpU8blXY6Z2aBR81kSew+0c9GMybzrjCl5l2JmNqjUfIMYN3oE/+Pa8/Iuw8xs0Kn5ISYzM6vMDcLMzCpygzAzs4rcIMzMrCI3CDMzq8gNwszMKnKDMDOzitwgzMysIkVE3jX0G0mtwCtHuPkUYHM/ltNfXFffuK6+cV19U411nRwRFWOsq6pBHA1JhYhozLuOrlxX37iuvnFdfVNrdXmIyczMKnKDMDOzitwg3nRH3gV0w3X1jevqG9fVNzVVl89BmJlZRT6CMDOzitwgzMysoppqEJK+J2mTpJXdLJek/y1praRnJc0eJHW9R9IOSSuSr/82QHVNl/SYpDWSVkn6QoV1BnyfpaxrwPeZpGMk/VbSM0ldX6+wTh77K01dufyOJe89XNLTkh6ssCyXv8kUdeX1N/mypOeS9yxUWN6/+ysiauYLeDcwG1jZzfIPAQ8BAi4Gnhokdb0HeDCH/TUNmJ28ngD8DpiZ9z5LWdeA77NkH4xPXo8EngIuHgT7K01dufyOJe/9X4AfVXr/vP4mU9SV19/ky8CUHpb36/6qqSOIiPgFsLWHVa4G/iVKfgNMkjRtENSVi4jYGBHLk9e7gDVAfZfVBnyfpaxrwCX7YHcyOTL56noVSB77K01duZDUAFwJfKebVXL5m0xR12DVr/urphpECvVAsWy6hUHwwZN4RzJE8JCkswf6zSXNAC6g9K/Pcrnusx7qghz2WTIssQLYBDwaEYNif6WoC/L5HfsH4K+Ajm6W5/X71VtdkM/+CmCppGWSbqywvF/3lxvEoVRh3mD4l9ZySnkp5wPfBO4byDeXNB5YDHwxInZ2XVxhkwHZZ73Ulcs+i4j2iJgFNAAXSTqnyyq57K8UdQ34/pL0YWBTRCzrabUK8zLdXynryutv8pKImA18EPi8pHd3Wd6v+8sN4lAtwPSy6QZgQ061vCEidnYOEUTEEmCkpCkD8d6SRlL6EP5hRNxTYZVc9llvdeW5z5L33A48DsztsijX37Hu6sppf10CfETSy8CPgcsl/WuXdfLYX73WldfvV0RsSP67CbgXuKjLKv26v9wgDvUA8EfJlQAXAzsiYmPeRUk6QZKS1xdR+v+2ZQDeV8B3gTUR8Y1uVhvwfZamrjz2maQ6SZOS12OA9wLPd1ktj/3Va1157K+IuDUiGiJiBnA98POI+ESX1QZ8f6WpK6ffr3GSJnS+Bt4PdL3ysV/314gjrnYIknQXpasPpkhqAb5G6YQdEXE7sITSVQBrgb3ApwdJXdcCn5PUBrwOXB/JJQsZuwT4JPBcMn4N8BXgpLLa8thnaerKY59NA34gaTilD4ymiHhQ0k1ldeWxv9LUldfv2GEGwf5KU1ce++stwL1JXxoB/CgiHs5yfzlqw8zMKvIQk5mZVeQGYWZmFblBmJlZRW4QZmZWkRuEmZlV5AZh1geSvqpSIuqzKiVqvr2HdR+XNOgecG+WVk3dB2F2NCS9A/gwpSTZ/cmds6MG4H2HR0R71u9j1pWPIMzSmwZsjoj9ABGxOSI2SLpCpecGPKfSsz1Gl28k6XOS/mfZ9B9L+mby+hMqPathhaR/Tm5mQ9JuSX8r6SngHQP3I5q9yQ3CLL2lwHRJv5P0fyRdKukY4E5gQUScS+mo/HNdtrsb+GjZ9AJgoaS3Ja8vSYL02oGPJ+uMo/R8kLdHxC+z+5HMuucGYZZSEs42B7gRaAUWAp8Ffh8Rv0tW+wGlB0CVb9cKrJN0saTjgTOBJ4Erku/XnESGXAGcmmzWTimM0Cw3Pgdh1gfJuYDHgcclPQd8KuWmC4H5lELy7o2ISMLefhARt1ZYf5/PO1jefARhlpKkMyWdUTZrFvAaMEPS6cm8TwJPVNj8HuAa4AZKzQLg34BrJU1Nvv9kSSdnUrzZEfARhFl644FvJtHZbZQSM28E7gIWSRoBNAO3d90wIrZJWk3p2dm/TeatlvTXlJ4QNgw4CHweeGVAfhqzXjjN1czMKvIQk5mZVeQGYWZmFblBmJlZRW4QZmZWkRuEmZlV5AZhZmYVuUGYmVlF/x8dl8egVl125QAAAABJRU5ErkJggg==\n",
                        "text/plain": "<Figure size 432x288 with 1 Axes>"
                    },
                    "metadata": {
                        "needs_background": "light"
                    },
                    "output_type": "display_data"
                }
            ],
            "source": "ind, _ = hyperparameter(len(solver), Log_acc, \"Solver\" )\nprint(solver[ind])"
        },
        {
            "cell_type": "code",
            "execution_count": 173,
            "metadata": {},
            "outputs": [],
            "source": "Log_acc = []\nc_value = [0.001, 0.01, 0.1, 1, 10, 100, 1000]\n\ncross_validation = [1,2,3,4,5,6,7]\nfor i, c in zip(c_value, cross_validation):\n    X_train, X_test, y_train, y_test = train_test_split(X, le_y, test_size = 0.2, random_state = 4)\n    LR = LogisticRegression(C = i, solver = \"lbfgs\").fit(X_train, y_train)\n    score = metrics.accuracy_score(LR.predict(X_test), y_test)\n    Log_acc.append(score)"
        },
        {
            "cell_type": "code",
            "execution_count": 175,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "Max accuracy:0.7857142857142857\nBest C_value: 0\nbest C_value: 0.001\n"
                },
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEHCAYAAAC0pdErAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZhU9Zn28e/dDc0mO8gOjcaoqAjaAjYmTtwGI0hIjAHFgBk1JKLZxoxmMq9J3lmS14yZuIdoAI0BF1SMGpcYNYKINIoRRBQQpUUFRURBZHveP6pwWlINRXdVn67q+3NdddFnq7rrSuynz+8853cUEZiZme2uJOkAZmbWOLlAmJlZRi4QZmaWkQuEmZll5AJhZmYZNUs6QC516dIlysvLk45hZlYwFi5c+E5EdM20ragKRHl5OVVVVUnHMDMrGJJeq22bh5jMzCwjFwgzM8vIBcLMzDJygTAzs4xcIMzMLCMXCDMzy8gFwszMMiqq+yDq6g/zX2fHzp1Jx8iJls1LGTmwJ63KSpOOYmYFzgUC+L/3vchH23YkHSNnbprzKr8552j6dW6TdBQzK2AqpgcGVVRURF3upH7nw4/zkCYZi17fwA/ueJ6dEfzP1wZx4qHdko5kZo2YpIURUZFxmwtE8Vm9fjPfunUhi9/YyEUnfIbvnvRZSkuUdCwza4T2VCB8kboI9enUmjsnVfK1ij5c/ZflTJz6DOs3bU06lpkVGBeIItWyeSm/OGMgP//yEcx/dT2jrp7D36o3JB3LzAqIC0SRGzukL3dOOhaAM66fx4xnXqeYhhXNLH9cIJqAgb07cN9FxzHswM5cdtcL/Musv7GliLq2zCw/XCCaiI5typg68RguPuEz3F5VzVeuf4rV6zcnHcvMGjEXiCaktER8/5SDuWlCBavXb2bk1XN4bNnapGOZWSPlAtEEnXhoN/540XH07NCKb0xbwK8eeZmdO31dwsw+zQWiierXuQ13fauSLw/uza8ffYVvTF/Ahs1uhTWz/+UC0YS1Kivll18dyH+MOZy5y99h5NVzWPzG+0nHMrNGwgWiiZPE2UP7cfs3j2XHzuDL1z/F7VWrk45lZo1AXguEpBGSlklaLunSDNsvkbQo/VosaYekTult35O0JL1+hqSW+cza1A3u25H7LjqOY8o78sM7/8Zld7kV1qypy1uBkFQKXAucCgwAxkkaUHOfiLgiIgZFxCDgMuCJiFgvqRdwMVAREYcDpcDYfGW1lM77teDmbwzl2/9wIDOeWc2Zv5lH9XtuhTVrqvJ5BjEEWB4RKyNiKzATGL2H/ccBM2osNwNaSWoGtAbW5C2pfaK0RPxwxCFMOedoXl23iZFXz+GvL69LOpaZJSCfBaIXUHMwuzq97u9Iag2MAGYBRMQbwC+B14E3gfcj4uFajr1AUpWkqnXr/IssV045rDv3XnQc3du1ZMLUZ7j60VfcCmvWxOSzQGSaX7q23zCjgLkRsR5AUkdSZxv9gZ5AG0njMx0YEVMioiIiKrp27ZqD2LZL/y5tuOvblYw+sif//cjLnH9zFe9/tC3pWGbWQPJZIKqBPjWWe1P7MNFYPj28dBLwakSsi4htwF1AZV5S2h61LmvGr742iJ+NPoy/vrKOUVfP4cU1G5OOZWYNIJ8FYgFwkKT+kspIFYF7d99JUnvgeGB2jdWvA8MktZYk4ERgaR6z2h5I4uvHljPzgmP5ePsOxlw3l1kLq5OOZWZ5lrcCERHbgcnAQ6R+ud8eEUskTZI0qcauY4CHI2JTjWPnA3cCzwIvpHNOyVdWy87R/Tpy30WfY3DfDvzgjuf58T0v8PF2t8KaFSs/ctT22fYdO7ni4WX85omVHNmnA9effRQ9O7RKOpaZ1YEfOWo51ay0hMtOPZQbxh/FirUfMvLqOcxd/k7Sscwsx1wgrM5GHN6D2ZOH07lNGefcNJ/rHl/up9WZFREXCKuXA7vuxz0XDue0gT35fw8u44JbFrJxi1thzYqBC4TVW5sWzbhq7CAuHzWAx15ay+lXz+Glt9wKa1boXCAsJyRx7vD+zLhgGJu27mDMtU9xz3NvJB3LzOrBBcJy6pjyTtx/0XEc0as9371tEZfPXszW7TuTjmVmdeACYTm3f7uW3Hr+UM47rj/T573G2CnzeOv9LUnHMrN95AJhedG8tIQfjxzANWcN5qW3PmDk1U8yb8W7Sccys33gAmF5NXJgT+6dPJz2rZoz/qb5TPnrCrfCmhUIFwjLu8/s35bZk4/jHw/rxn8+8BLfvvVZPnArrFmj5wJhDWK/Fs249qyj+PFph/Lwi28z+tq5vPL2B0nHMrM9cIGwBiOJ8z53ALeeN5SNH21n9LVz+ePzflCgWWPlAmENbtgBnbn/4uM4tEc7LprxHD/744ts2+FWWLPGxgXCEtGtXUtmnD+MiZXl/G7uq5z126dZu9GtsGaNiQuEJaasWQk/Of0wfj12EIvf2MhpV8/hmVfXJx3LzNJcICxxowf14p4Lh9O2RTPG/fZpbnxypVthzRoBFwhrFA7u3pbZk4dz0qH78+/3L2XyjOfY9PH2pGOZNWkuENZotG3ZnBvGH82lpx7Cn154kzNumOeL12YJcoGwRkUSk44/kCvPHMTSNzfy8JK3k45k1mTltUBIGiFpmaTlki7NsP0SSYvSr8WSdkjqJOngGusXSdoo6bv5zGqNy6gje9KnUyumP7Uq6ShmTVbeCoSkUuBa4FRgADBO0oCa+0TEFRExKCIGAZcBT0TE+ohYVmP90cBm4O58ZbXGp7REfH1YOc+sWs+SNe8nHcesScrnGcQQYHlErIyIrcBMYPQe9h8HzMiw/kRgRUS8loeM1oidWdGHVs1LfRZhlpB8FohewOoay9XpdX9HUmtgBDArw+axZC4cu469QFKVpKp169bVI641Nu1bN2fMUb24Z9Ea1m/amnQcsyYnnwVCGdbV1tw+CpgbEZ+6S0pSGXA6cEdtHxIRUyKiIiIqunbtWuew1jhNrCxn6/adzFzwetJRzJqcfBaIaqBPjeXeQG0zs9V2lnAq8GxEuJWlifpst7ZUHtiZ3897je1ueTVrUPksEAuAgyT1T58JjAXu3X0nSe2B44HZGd6jtusS1oRMrCxnzftbeORF/51g1pDyViAiYjswGXgIWArcHhFLJE2SNKnGrmOAhyNiU83j09clTgbuyldGKwwnHtqN3h1bMdUXq80aVLN8vnlEPAA8sNu6G3ZbngZMy3DsZqBzHuNZgSgtEV8/th//+cBLvLhmIwN6tks6klmT4DuprSCcWdGHls1L3PJq1oBcIKwgdGhdxpjBvbln0Ru855ZXswbhAmEFY0JlPz7evpOZC1bvfWczqzcXCCsYh3Rvx7EHdOb3T7vl1awhuEBYQZlQWc4bGz7iz0vd8mqWby4QVlBOOnR/enVoxTRfrDbLOxcIKyjNSks459h+PL1yPUvf3Jh0HLOi5gJhBWfsMamW15vnrUo6illRc4GwgtOhdRlfGtSLu597gw2b3fJqli8uEFaQJlSWs2XbTm5zy6tZ3rhAWEE6tEc7hvbvxM3zXmPHztpmkTez+nCBsIJ17nC3vJrlkwuEFayTDu1Gz/YtPT+TWZ5kVSAk9ZN0UvrnVpLa5jeW2d6lWl7LeWrFuyx764Ok45gVnb0WCEnnA3cCv0mv6g3ck89QZtkae0wfWjQrYfq8VUlHMSs62ZxBXAgMBzYCRMQrwP75DGWWrY5t0i2vz77B+5u3JR3HrKhkUyA+johPms0lNQPcNmKNxoTKcj7atoPbq9zyapZL2RSIJyT9CGgl6WTgDuCP+Y1llr0BPdsxpH8nps9b5ZZXsxzKpkD8C7AOeAH4JqlHiP44mzeXNELSMknLJV2aYfslkhalX4sl7ZDUKb2tg6Q7Jb0kaamkY7P/WtbUTKwsp/q9j/jLS2uTjmJWNPb4TGpJJcDfIuJw4Lf78saSSoFrgZOBamCBpHsj4sVd+0TEFcAV6f1HAd+LiPXpzb8GHoyIMySVAa335fOtaTllQDd6tG/JtKde5eQB3ZKOY1YU9ngGERE7gecl9a3Dew8BlkfEyvQ1jJnA6D3sPw6YASCpHfB54KZ0jq0RsaEOGayJaFZawvhh/Zi7/F1eedstr2a5kM0QUw9giaRHJd2765XFcb2AmlcNq9Pr/o6k1sAIYFZ61QGkhrWmSnpO0o2S2tRy7AWSqiRVrVu3LotYVqzGDelLWbMSPyvCLEeyKRA/BUYCPwP+u8Zrb5RhXW1XEEcBc2sMLzUDjgKuj4jBwCbg765hAETElIioiIiKrl27ZhHLilWnNmWMPrIndz37Bu9/5JZXs/raa4GIiCeAl4C26dfS9Lq9qQb61FjuDaypZd+xpIeXahxbHRHz08t3kioYZnu0q+X1Dre8mtVbNndSnwk8A3wVOBOYL+mMLN57AXCQpP7pi8xjgb8bmpLUHjgemL1rXUS8BayWdHB61YnAi7sfa7a7w3u155jyjp7l1SwHshli+lfgmIiYEBFfJ3Xx+d/2dlBEbAcmAw8BS4HbI2KJpEmSJtXYdQzwcERs2u0tLgJulfQ3YBDwn1lkNWNiZX9eX7+Zx9zyalYve2xzTSuJiJr/pb1LlpP8RcQDpO6bqLnuht2WpwHTMhy7CKjI5nPMajrlsG50b9eS6fNWcZJbXs3qLJtf9A9KekjSREkTgfuBP+U3llndNS8t4Zxj+/HkK++wfK1bXs3qKpuL1JeQmsl1IHAkMCUifpjvYGb1MfaYPpQ1K2H6U68lHcWsYGVzkbo/8EBEfD8ivkfqjKI838HM6qPzfi04/ciezHq2mo1b3PJqVhfZDDHdAeyssbwjvc6sUZtYWc7mrTu4o6o66ShmBSmbAtGs5nTf6Z/L8hfJLDcO79Wein4duXneKna65dVsn2VTINZJOn3XgqTRwDv5i2SWOxMqy3nt3c08/rJbXs32VTYFYhLwI0mvS1pNavrvb+Y3lllujDi8O93atWDq3FVJRzErONl0Ma2IiGHAAGBARFRGxPL8RzOrv+alJYwfuqvl9cOk45gVlGy6mL6Tnn57E/ArSc9KOiX/0cxyY9zQvpSVlnDzvFVJRzErKNkMMX0jIjYCpwD7A+cCP89rKrMc6rJfC0Ye2YNZC93yarYvsikQu6bt/iIwNSKeJ/NU3maN1rmV/dm0dQd3uuXVLGvZFIiFkh4mVSAektSWT98XYdboHdG7PUf17eCWV7N9kE2B+CdSD+s5JiI2k7oH4txdGyUdlqdsZjk1cXh/Vr27mSde9pMHzbKRTRfTzoh4dtczoSPi3Yj4W41dbslbOrMcOvXw7uzftoUfSWqWpaym7d4LX4+wgtC8tITxw/rxxMvrWLHOLa9me5OLAuEBXSsY44akWl5vmedZXs32JhcFwqxgdG3bgpEDe3BH1Wo+cMur2R7lokBs3fsuZo3HhMpyNm3dwayFbnk125Ns7qSeJek0SRn3TU/DYVYwjuzTgcF9OzB93mtueTXbg2zOIK4HzgJekfRzSYdk++aSRkhaJmm5pEszbL9E0qL0a7GkHZI6pbetkvRCeltV1t/ILAsTK8t59Z1N/PUVt7ya1SabNtc/R8TZwFHAKuARSU9JOldS89qOk1QKXAucSmqiv3GSBuz23ldExKCIGARcBjwREetr7PKF9PaKff5mZntw6uE96OqWV7M9yuoahKTOwETgPOA54NekCsYjezhsCLA8IlamHzI0Exi9h/3HATOyyWNWX2XNSjh7aF8eX7aOV9/ZlHQcs0Ypm2sQdwFPAq2BURFxekTcFhEXAfvt4dBewOoay9XpdZk+ozUwAphVY3UAD0taKOmCPeS7QFKVpKp16zxcYNk7a2hfmpeK6T6LMMsomzOIayJiQET8V0S8WXPDXoZ+Mt1AV9sVwVHA3N2Gl4ZHxFGkhqgulPT5TAdGxJSIqIiIiq5du+4hjtmn7d+2Jacd0YM7F1bz4cfbk45j1uhkUyAOldRh14KkjpK+ncVx1UCfGsu9gTW17DuW3YaXImJN+t+1wN2khqzMcmri8P58+PF2t7yaZZBNgTh/1zxMABHxHnB+FsctAA6S1F9SGakicO/uO0lqDxwPzK6xrk161lgktSH1LIrFWXym2T4Z1KcDR/bpwPR5nuXVbHfZFIgSSZ8MF6W7k8r2dlBEbAcmAw8BS4HbI2KJpEmSJtXYdQzwcETUvFLYDZgj6XngGeD+iHgwi6xm++zcynJWrtvEk8vfSTqKWaOiiD3/1STpCqAcuIHUNYRJwOqI+EHe0+2jioqKqKryLRO2b7Zu30nlz//CwN7t+d3EY5KOY9agJC2s7XpyNmcQ/wL8BfgWcCHwKPDD3MUzS9aultfHlq1llVtezT6R7fMgro+IMyLiKxHxm4jY0RDhzBrK2UP7Uipxs2d5NftENvdBHCTpTkkvSlq569UQ4cwayv7tWnJaepbXTW55NQOyG2KaSmo+pu3AF4Cb8VPkrAhNqCzng4+3c9ezbnk1g+wKRKuIeJTUBe3XIuInwAn5jWXW8Ab36cDA3u2Z9tQq9ta8YdYUZFMgtqSn+n5F0mRJY4D985zLrMFJYmJlOSvWbWKOW17NsioQ3yU1D9PFwNHAeGBCPkOZJeW0gT3osl+Z52cyYy8FIn1T3JkR8WFEVEfEuelOpqcbKJ9Zg2rRrJSzhvTl0ZfW8vq7m5OOY5aoPRaIdDvr0TXvpDYrdmcP65dueV2VdBSzRGUzxPQcMFvSOZK+vOuV72BmSenWriWnHtGD29zyak1cNgWiE/Auqc6lUenXyHyGMkvaxMp+fLBlO3c/90bSUcwS02xvO0TEuQ0RxKwxOapvR47o1Z7pT63i7KF98SirNUV7LRCSppLhQT8R8Y28JDJrBCQxobKcf77jeZ5a8S7DP9Ml6UhmDS6bIab7gPvTr0eBdsCH+Qxl1hiMHNiDzm3KmDp3VdJRzBKRzRBTzedEI2kG8Oe8JTJrJFo2L2XckL5c+/hyVq/fTJ9OrZOOZNagsjmD2N1BQN9cBzFrjMYP60eJW16ticpmNtcPJG3c9QL+SOoZEWZFr3v7low4vDu3LVjN5q1uebWmJZvnQbSNiHY1Xp/dfdjJrJidW1nORre8WhOUzRnEGEntayx3kPSl/MYyazyO7teRw3q2Y7pnebUmJptrEJdHxPu7FiJiA3B5Nm8uaYSkZZKWS7o0w/ZLJC1KvxZL2iGpU43tpZKek3RfNp9nlg+7Znl9+e0Pmbfi3aTjmDWYbApEpn2yuX+iFLgWOBUYAIyTNKDmPhFxRUQMiohBwGXAExGxvsYu3wGWZpHRLK9GHdmTTm3KmOZZXq0JyaZAVEm6UtKBkg6Q9CtgYRbHDQGWR8TKiNgKzARG72H/ccCMXQuSegOnATdm8VlmeZVqee3Dn5e+zer1nuXVmoZsCsRFwFbgNuB24CPgwiyO6wWsrrFcnV73dyS1BkYANS9+/w/wQ2Dnnj5E0gWSqiRVrVu3LotYZnUzflg/JPH7p19LOopZg8imi2lTRFwaERXp148iYlMW751p8prarvCNAubuGl6SNBJYGxF7PVOJiCm7snXt2jWLWGZ106N9K0Yc1p2ZC1bz0dYdSccxy7tsupgekdShxnJHSQ9l8d7VQJ8ay72BNbXsO5Yaw0vAcOB0SatIDU2dIOn3WXymWV5NqCzn/Y+2cc8it7xa8ctmiKlLunMJgIh4j+yeSb0AOEhSf0llpIrAvbvvlG6hPR6YXeMzLouI3hFRnj7uLxExPovPNMurY8o7MqBHO6bNdcurFb9sCsROSZ9MrSGpnNqHij4REduBycBDpDqRbo+IJZImSZpUY9cxwMNZDluZJWpXy+uytz/g6ZXr936AWQHT3v4KkjQCmAI8kV71eeCCiMhmmKlBVVRURFVVVdIxrMht2baDY//rUYb078RvzqlIOo5ZvUhaGBEZ/4+czUXqB4EKYBmpTqYfkOpkMmuSWjYvZeyQvjzy4ttUv+eWVyte2VykPo/UcyB+kH7dAvwkv7HMGrddLa+3uOXVilg21yC+AxwDvBYRXwAGA77hwJq0Xh1accqAbtzmllcrYtkUiC0RsQVAUouIeAk4OL+xzBq/iZXlbNi8jdluebUilU2BqE7fB3EP8Iik2dR+P4NZkzGkfycO6d6WaZ7l1YpUNhepx0TEhoj4CfBvwE2Ap/u2Jk8S5w4v56W3PmD+q255teKzT48cjYgnIuLe9OR7Zk3e6EG96NC6OdM9y6sVobo8k9rM0lo2L2XsMX15aMlbvLHB3d9WXFwgzOpp/LDURAOe5dWKjQuEWT317tiaUwZ0Z8Yzr7Nlm1terXi4QJjlwIR0y+u9i9zgZ8XDBcIsB4YdkGp5neqWVysiLhBmOSCJCZXlLH1zIwtWvZd0HLOccIEwy5EvDepF+1bNmfbUq0lHMcsJFwizHGlVVsrYY/rw0JK3WeOWVysCLhBmOTR+WD8iwi2vVhRcIMxyqE+n1px0aDe3vFpRcIEwy7GJw8t5b/M27n3eLa9W2PJaICSNkLRM0nJJl2bYfomkRenXYkk7JHWS1FLSM5Kel7RE0k/zmdMsl449oDMHd2vLdLe8WoHLW4GQVApcC5wKDADGSRpQc5+IuCIiBkXEIOAy4ImIWA98DJwQEUcCg4ARkoblK6tZLu1qeV2yZiNVr7nl1QpXPs8ghgDLI2JlevbXmcDoPew/DpgBECkfptc3T7/8p5gVjC8N7km7ls2Y5llerYDls0D0AlbXWK5Or/s7kloDI4BZNdaVSloErAUeiYj5ecxqllOty5oxdkhfHlz8Fm++75ZXK0z5LBDKsK62s4BRwNz08FJqx4gd6aGn3sAQSYdn/BDpAklVkqrWrfOjsq3xOGdYP3ZGcOvTrycdxaxO8lkgqoE+NZZ7U/ujSseSHl7aXURsAB4ndYaRafuUiKiIiIquXbvWPa1Zjrnl1Qpdszy+9wLgIEn9gTdIFYGzdt9JUnvgeGB8jXVdgW0RsUFSK+Ak4Bd5zGqWFxMry3nkxbe57vEVVPTrmHSceistEUf360jL5qVJR7EGkLcCERHbJU0GHgJKgd9FxBJJk9Lbb0jvOgZ4OCI21Ti8BzA93QlVAtweEfflK6tZvlQe2JlDurflqkdfSTpKzhzSvS03jD+a8i5tko5ieaZi6tOuqKiIqqqqpGOYfcr6TVt59Z0P975jAah+7yMuv3cJO3YGV545iJMHdEs6ktWTpIURUZFpWz6HmMwM6NSmjE5tOiUdIyeO7gdH9e3It25dyPk3VzH5C5/heyd/ltKSTD0pVug81YaZ7ZM+nVpz56RKvlbRh2seW87Eqc+wftPWpGNZHrhAmNk+a9m8lF+cMZCff/kI5r+6npFXPcmi1RuSjmU55gJhZnU2dkhfZk2qRBJn3jCPW+e/5vmniogLhJnVyxG923PfRccx7MDO/Ovdi7nkzr/5vo8i4QJhZvXWsU0ZUycew8UnHsSdC6v58nVP8fq7m5OOZfXkAmFmOVFaIr5/8mf53cQKqt/bzMirn+Sxl9YmHcvqwQXCzHLqhEO6cd9Fn6NXx9acO20BVz7yMjt2+rpEIXKBMLOc69u5NXd/u5KvHNWbqx59hW9MW8B7boUtOC4QZpYXLZuX8suvDuQ/xhzOvBXvMvLqObxQ/X7SsWwfuECYWd5I4uyh/bh90rFEBF+54SluW+DpzwuFC4SZ5d2gPh247+LPMaS8E/8y6wUuneVW2ELgAmFmDaJTmzKmf2MIF37hQGYuWM1Xb5jH6vVuhW3MXCDMrMGUlohL/vEQfvv1Cla9s4lR18zhiZf9JMjGygXCzBrcyQO68ceLjqN7u5ZMnPoMVz36CjvdCtvouECYWSLKu7Th7m8P50uDenHlIy9z3s1VvL95W9KxrAYXCDNLTKuyUq4880h+NvownnxlHaOumcOSNW6FbSxcIMwsUZL4+rHl3PbNY9m6fSdfvu4p7lxYnXQswwXCzBqJo/p25L6Lj+Oovh355zue51/vfoGPt7sVNkl5LRCSRkhaJmm5pEszbL9E0qL0a7GkHZI6Seoj6TFJSyUtkfSdfOY0s8ahy34tuOWfhjDp+AO5df7rnPmbp3ljw0dJx2qylK+He0gqBV4GTgaqgQXAuIh4sZb9RwHfi4gTJPUAekTEs5LaAguBL9V27C4VFRVRVVWV0+9hZsl4cPFb/PMdz1PWrISrxg7muIO6JB2pKElaGBEVmbbl8wxiCLA8IlZGxFZgJjB6D/uPA2YARMSbEfFs+ucPgKVArzxmNbNGZsTh3bl38nC67FfG1383n2sfW+5W2AaWzwLRC1hdY7maWn7JS2oNjABmZdhWDgwG5tdy7AWSqiRVrVvnG27MiskBXffj7m8PZ+TAnlzx0DIuuGUh73/kVtiGks8CoQzraiv/o4C5EbH+U28g7UeqaHw3IjZmOjAipkRERURUdO3atV6BzazxadOiGb8eO4jLRw3g8WVrGX3NHJa+mfHXgeVYPgtENdCnxnJvYE0t+44lPby0i6TmpIrDrRFxV14SmllBkMS5w/sz84JhbN66gzHXzeXu59wKm2/5LBALgIMk9ZdURqoI3Lv7TpLaA8cDs2usE3ATsDQirsxjRjMrIBXlnbjv4uMY2LsD37vtef7P7MVs3b4z6VhFK28FIiK2A5OBh0hdZL49IpZImiRpUo1dxwAPR8SmGuuGA+cAJ9Rog/1ivrKaWeHYv21Lbj1vKOd/rj83z3uNr02Zx5vvuxU2H/LW5poEt7maNS0PvPAml9zxPK3KSrlq3GAqD3Qr7L5Kqs3VzCyvvnhED2ZPHk6H1mWMv3E+NzyxgmL6ozdpLhBmVtA+s39b7rlwOKce3oOf/+klvvX7Z/lgi1thc8EFwswK3n4tmnHNWYP58WmH8sjStxl9zVxefvuDpGMVPBcIMysKkjjvcwfwh/OGsnHLdkZfM5d7n6+ts96y4QJhZkVl6AGduf/i4zisZzsunvEcP/3jErbtcCtsXbhAmFnR6dauJTMuGMa5w8uZOncV46Y8zdqNW5KOVXBcIMysKDUvLeHyUYdx1bjBLFmzkS9eNYf5K99NOlZBcYEws4r/eE8AAAXqSURBVKJ2+pE9mT15OO1aNuOsG+dz45Mr3QqbJRcIMyt6n+3WltmTh3PSofvz7/cvZfIfnuPDj7cnHavRa5Z0ADOzhtC2ZXNuGH80U/66kl88+BKLVm+gX+fWScfKifatmnP9+KNz/r4uEGbWZEjim8cfyBG923PdYyuK5pnX23bkZ8jMBcLMmpzKA7t43qYs+BqEmZll5AJhZmYZuUCYmVlGLhBmZpaRC4SZmWXkAmFmZhm5QJiZWUYuEGZmlpGKadIqSeuA1+p4eBfgnRzGSVKxfJdi+R7g79IYFcv3gPp9l34R0TXThqIqEPUhqSoiKpLOkQvF8l2K5XuAv0tjVCzfA/L3XTzEZGZmGblAmJlZRi4Q/2tK0gFyqFi+S7F8D/B3aYyK5XtAnr6Lr0GYmVlGPoMwM7OMXCDMzCyjJl8gJP1O0lpJi5POUh+S+kh6TNJSSUskfSfpTHUlqaWkZyQ9n/4uP006U31IKpX0nKT7ks5SH5JWSXpB0iJJVUnnqQ9JHSTdKeml9H8zxyadqS4kHZz+32PXa6Ok7+bs/Zv6NQhJnwc+BG6OiMOTzlNXknoAPSLiWUltgYXAlyLixYSj7TNJAtpExIeSmgNzgO9ExNMJR6sTSd8HKoB2ETEy6Tx1JWkVUBERBX9zmaTpwJMRcaOkMqB1RGxIOld9SCoF3gCGRkRdbxj+lCZ/BhERfwXWJ52jviLizYh4Nv3zB8BSoFeyqeomUj5MLzZPvwryLxlJvYHTgBuTzmIpktoBnwduAoiIrYVeHNJOBFbkqjiAC0RRklQODAbmJ5uk7tLDMouAtcAjEVGo3+V/gB8CO5MOkgMBPCxpoaQLkg5TDwcA64Cp6aG/GyW1STpUDowFZuTyDV0gioyk/YBZwHcjYmPSeeoqInZExCCgNzBEUsEN/0kaCayNiIVJZ8mR4RFxFHAqcGF6eLYQNQOOAq6PiMHAJuDSZCPVT3qY7HTgjly+rwtEEUmP188Cbo2Iu5LOkwvpU//HgREJR6mL4cDp6bH7mcAJkn6fbKS6i4g16X/XAncDQ5JNVGfVQHWNs9I7SRWMQnYq8GxEvJ3LN3WBKBLpC7s3AUsj4sqk89SHpK6SOqR/bgWcBLyUbKp9FxGXRUTviCgndfr/l4gYn3CsOpHUJt38QHo45hSgIDv/IuItYLWkg9OrTgQKrpljN+PI8fASpE61mjRJM4B/ALpIqgYuj4ibkk1VJ8OBc4AX0mP3AD+KiAcSzFRXPYDp6a6MEuD2iCjoFtEi0A24O/V3CM2AP0TEg8lGqpeLgFvTQzMrgXMTzlNnkloDJwPfzPl7N/U2VzMzy8xDTGZmlpELhJmZZeQCYWZmGblAmJlZRi4QZmaWkQuEmZll5AJhtgeSukuaKWmFpBclPSDpszl43/JCn2Leip8LhFkt0nen3w08HhEHRsQA4EekbhozK3ouEGa1+wKwLSJu2LUiIhZFxJO77yjpNklfrLE8TdJX0mcKT0p6Nv2qzHDsREnX1Fi+T9I/pH8+RdK89LF3pCdjNGsQLhBmtTuc1IOXsjET+Bp8MrPmicADpKYrPzk9C+rXgKuy/XBJXYAfAyelj68Cvp91erN6avJzMZnlyJ+AqyS1IDXz7F8j4iNJ7YFrJA0CdgD7cv1iGDAAmJueA6kMmJfb2Ga1c4Ewq90S4IxsdoyILZIeB/6R1JnCrpk1vwe8DRxJ6ox9S4bDt/Pps/mW6X9F6mFJ4/Y5uVkOeIjJrHZ/AVpIOn/XCknHSDq+lv1nkpoV9HPAQ+l17YE3I2Inqdl2SzMctwoYJKlEUh/+9zkLTwPDJX0m/dmtc9FBZZYtFwizWkRqquMxwMnpNtclwE+ANbUc8jCpZx3/OSK2ptddB0yQ9DSp4aVNGY6bC7wKvAD8Etj1bPF1wERghqS/kSoYh9T/m5llx9N9m5lZRj6DMDOzjHyR2mwfSDoCuGW31R9HxNAk8pjlk4eYzMwsIw8xmZlZRi4QZmaWkQuEmZll5AJhZmYZ/X9fDGsopMRPxgAAAABJRU5ErkJggg==\n",
                        "text/plain": "<Figure size 432x288 with 1 Axes>"
                    },
                    "metadata": {
                        "needs_background": "light"
                    },
                    "output_type": "display_data"
                }
            ],
            "source": "ind, _ = hyperparameter(len(c_value), Log_acc, \"C_value\" )\nprint(\"best C_value:\", c_value[ind])"
        },
        {
            "cell_type": "code",
            "execution_count": 206,
            "metadata": {},
            "outputs": [],
            "source": "LR = LogisticRegression(C = 0.001, solver = \"liblinear\").fit(X_train, y_train)"
        },
        {
            "cell_type": "code",
            "execution_count": 237,
            "metadata": {},
            "outputs": [
                {
                    "name": "stderr",
                    "output_type": "stream",
                    "text": "/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/model_selection/_validation.py:552: FitFailedWarning: Estimator fit failed. The score on this train-test partition for these parameters will be set to nan. Details: \nTraceback (most recent call last):\n  File \"/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/model_selection/_validation.py\", line 531, in _fit_and_score\n    estimator.fit(X_train, y_train, **fit_params)\n  File \"/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py\", line 1304, in fit\n    solver = _check_solver(self.solver, self.penalty, self.dual)\n  File \"/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py\", line 443, in _check_solver\n    \"got %s penalty.\" % (solver, penalty))\nValueError: Solver lbfgs supports only 'l2' or 'none' penalties, got l1 penalty.\n\n  FitFailedWarning)\n/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/model_selection/_validation.py:552: FitFailedWarning: Estimator fit failed. The score on this train-test partition for these parameters will be set to nan. Details: \nTraceback (most recent call last):\n  File \"/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/model_selection/_validation.py\", line 531, in _fit_and_score\n    estimator.fit(X_train, y_train, **fit_params)\n  File \"/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py\", line 1304, in fit\n    solver = _check_solver(self.solver, self.penalty, self.dual)\n  File \"/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py\", line 443, in _check_solver\n    \"got %s penalty.\" % (solver, penalty))\nValueError: Solver lbfgs supports only 'l2' or 'none' penalties, got l1 penalty.\n\n  FitFailedWarning)\n/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/model_selection/_validation.py:552: FitFailedWarning: Estimator fit failed. The score on this train-test partition for these parameters will be set to nan. Details: \nTraceback (most recent call last):\n  File \"/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/model_selection/_validation.py\", line 531, in _fit_and_score\n    estimator.fit(X_train, y_train, **fit_params)\n  File \"/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py\", line 1304, in fit\n    solver = _check_solver(self.solver, self.penalty, self.dual)\n  File \"/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py\", line 443, in _check_solver\n    \"got %s penalty.\" % (solver, penalty))\nValueError: Solver lbfgs supports only 'l2' or 'none' penalties, got l1 penalty.\n\n  FitFailedWarning)\n/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/model_selection/_validation.py:552: FitFailedWarning: Estimator fit failed. The score on this train-test partition for these parameters will be set to nan. Details: \nTraceback (most recent call last):\n  File \"/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/model_selection/_validation.py\", line 531, in _fit_and_score\n    estimator.fit(X_train, y_train, **fit_params)\n  File \"/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py\", line 1304, in fit\n    solver = _check_solver(self.solver, self.penalty, self.dual)\n  File \"/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py\", line 443, in _check_solver\n    \"got %s penalty.\" % (solver, penalty))\nValueError: Solver lbfgs supports only 'l2' or 'none' penalties, got l1 penalty.\n\n  FitFailedWarning)\n/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/model_selection/_validation.py:552: FitFailedWarning: Estimator fit failed. The score on this train-test partition for these parameters will be set to nan. Details: \nTraceback (most recent call last):\n  File \"/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/model_selection/_validation.py\", line 531, in _fit_and_score\n    estimator.fit(X_train, y_train, **fit_params)\n  File \"/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py\", line 1304, in fit\n    solver = _check_solver(self.solver, self.penalty, self.dual)\n  File \"/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py\", line 443, in _check_solver\n    \"got %s penalty.\" % (solver, penalty))\nValueError: Solver lbfgs supports only 'l2' or 'none' penalties, got l1 penalty.\n\n  FitFailedWarning)\n/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/model_selection/_validation.py:552: FitFailedWarning: Estimator fit failed. The score on this train-test partition for these parameters will be set to nan. Details: \nTraceback (most recent call last):\n  File \"/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/model_selection/_validation.py\", line 531, in _fit_and_score\n    estimator.fit(X_train, y_train, **fit_params)\n  File \"/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py\", line 1304, in fit\n    solver = _check_solver(self.solver, self.penalty, self.dual)\n  File \"/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py\", line 443, in _check_solver\n    \"got %s penalty.\" % (solver, penalty))\nValueError: Solver lbfgs supports only 'l2' or 'none' penalties, got l1 penalty.\n\n  FitFailedWarning)\n/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/model_selection/_validation.py:552: FitFailedWarning: Estimator fit failed. The score on this train-test partition for these parameters will be set to nan. Details: \nTraceback (most recent call last):\n  File \"/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/model_selection/_validation.py\", line 531, in _fit_and_score\n    estimator.fit(X_train, y_train, **fit_params)\n  File \"/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py\", line 1304, in fit\n    solver = _check_solver(self.solver, self.penalty, self.dual)\n  File \"/opt/conda/envs/Python-3.7-main/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py\", line 443, in _check_solver\n    \"got %s penalty.\" % (solver, penalty))\nValueError: Solver lbfgs supports only 'l2' or 'none' penalties, got l1 penalty.\n\n  FitFailedWarning)\n"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "tuned hpyerparameters :(best parameters)  {'C': 0.001, 'penalty': 'l2'}\naccuracy : 0.7718253968253967\n"
                }
            ],
            "source": "from sklearn.model_selection import GridSearchCV\nX_train, X_test, y_train, y_test = train_test_split(X, le_y, test_size = 0.2, random_state = c)\ngrid={\"C\":np.logspace(-3,3,7), \"penalty\":[\"l1\",\"l2\"]}# l1 lasso l2 ridge\nlogreg=LogisticRegression()\nlogreg_cv=GridSearchCV(logreg,grid,cv=10)\nlogreg_cv.fit(X_train,y_train)\n\nprint(\"tuned hpyerparameters :(best parameters) \",logreg_cv.best_params_)\nprint(\"accuracy :\",logreg_cv.best_score_)"
        },
        {
            "cell_type": "code",
            "execution_count": 238,
            "metadata": {},
            "outputs": [],
            "source": "LR = LogisticRegression(C = 0.001, penalty = \"l2\").fit(X_train, y_train)"
        },
        {
            "cell_type": "code",
            "execution_count": 239,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "Log loss: 8.95470488690319\n0.7407407407407407\n"
                }
            ],
            "source": "print(\"Log loss:\", log_loss(test_le_y, LR.predict(testX)))\nprint(metrics.accuracy_score(test_le_y, LR.predict(testX)))"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Model Evaluation using Test set"
        },
        {
            "cell_type": "code",
            "execution_count": 35,
            "metadata": {},
            "outputs": [],
            "source": "from sklearn.metrics import jaccard_score\nfrom sklearn.metrics import f1_score\nfrom sklearn.metrics import log_loss"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "First, download and load the test set:"
        },
        {
            "cell_type": "code",
            "execution_count": 36,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "--2020-11-08 12:29:04--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv\nResolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196\nConnecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.\nHTTP request sent, awaiting response... 200 OK\nLength: 3642 (3.6K) [text/csv]\nSaving to: \u2018loan_test.csv\u2019\n\nloan_test.csv       100%[===================>]   3.56K  --.-KB/s    in 0s      \n\n2020-11-08 12:29:05 (78.8 MB/s) - \u2018loan_test.csv\u2019 saved [3642/3642]\n\n"
                }
            ],
            "source": "!wget -O loan_test.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### Load Test set for evaluation "
        },
        {
            "cell_type": "code",
            "execution_count": 89,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n      <th>dayofweek</th>\n      <th>weekend</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>1</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>50</td>\n      <td>Bechalor</td>\n      <td>1</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>5</td>\n      <td>5</td>\n      <td>PAIDOFF</td>\n      <td>300</td>\n      <td>7</td>\n      <td>2016-09-09</td>\n      <td>2016-09-15</td>\n      <td>35</td>\n      <td>Master or Above</td>\n      <td>0</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>21</td>\n      <td>21</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-10</td>\n      <td>2016-10-09</td>\n      <td>43</td>\n      <td>High School or Below</td>\n      <td>1</td>\n      <td>5</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>24</td>\n      <td>24</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-10</td>\n      <td>2016-10-09</td>\n      <td>26</td>\n      <td>college</td>\n      <td>0</td>\n      <td>5</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>35</td>\n      <td>35</td>\n      <td>PAIDOFF</td>\n      <td>800</td>\n      <td>15</td>\n      <td>2016-09-11</td>\n      <td>2016-09-25</td>\n      <td>29</td>\n      <td>Bechalor</td>\n      <td>0</td>\n      <td>6</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           1             1     PAIDOFF       1000     30     2016-09-08   \n1           5             5     PAIDOFF        300      7     2016-09-09   \n2          21            21     PAIDOFF       1000     30     2016-09-10   \n3          24            24     PAIDOFF       1000     30     2016-09-10   \n4          35            35     PAIDOFF        800     15     2016-09-11   \n\n    due_date  age             education  Gender  dayofweek  weekend  \n0 2016-10-07   50              Bechalor       1          3        0  \n1 2016-09-15   35       Master or Above       0          4        1  \n2 2016-10-09   43  High School or Below       1          5        1  \n3 2016-10-09   26               college       0          5        1  \n4 2016-09-25   29              Bechalor       0          6        1  "
                    },
                    "execution_count": 89,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "testdf."
        },
        {
            "cell_type": "code",
            "execution_count": 124,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "(54, 12)"
                    },
                    "execution_count": 124,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "testdf = pd.read_csv('loan_test.csv')\n# testdf.dropna(axis=0, inplace = True)\n\ntestdf['due_date'] = pd.to_datetime(testdf['due_date'])\ntestdf['effective_date'] = pd.to_datetime(testdf['effective_date'])\ntestdf['dayofweek'] = testdf['effective_date'].dt.dayofweek\ntestdf['weekend'] = testdf['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\ntestdf['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\ntestFeature = testdf[['Principal','terms','age','Gender','weekend']]\ntestFeature = pd.concat([testFeature, pd.get_dummies(testdf['education'])], axis=1)\ntestFeature.drop(['Master or Above'], axis = 1,inplace=True)\ntestFeature = pd.concat([testFeature, testdf[\"loan_status\"]], axis=1)\ntestFeature.dropna(axis=0, inplace = True)\ntestFeature\ntestFeature.shape\ntestdf.shape"
        },
        {
            "cell_type": "code",
            "execution_count": 140,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>weekend</th>\n      <th>Bechalor</th>\n      <th>High School or Below</th>\n      <th>college</th>\n      <th>loan_status</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>50</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>300</td>\n      <td>7</td>\n      <td>35</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>43</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>26</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>PAIDOFF</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>800</td>\n      <td>15</td>\n      <td>29</td>\n      <td>0</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n0       1000     30   50       1        0         1                     0   \n1        300      7   35       0        1         0                     0   \n2       1000     30   43       1        1         0                     1   \n3       1000     30   26       0        1         0                     0   \n4        800     15   29       0        1         1                     0   \n\n   college loan_status  \n0        0     PAIDOFF  \n1        0     PAIDOFF  \n2        0     PAIDOFF  \n3        1     PAIDOFF  \n4        0     PAIDOFF  "
                    },
                    "execution_count": 140,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "testFeature.head()"
        },
        {
            "cell_type": "code",
            "execution_count": 142,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "(54, 8)"
                    },
                    "execution_count": 142,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "testX = testFeature[['Principal','terms','age','Gender','weekend',\"Bechalor\", \"High School or Below\", \"college\"]]\ntestX = preprocessing.StandardScaler().fit(testX).transform(testX)\ntestX.shape"
        },
        {
            "cell_type": "code",
            "execution_count": 143,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "array([[ 0.49362588,  0.92844966,  3.05981865,  1.97714211, -1.30384048,\n         2.39791576, -0.79772404, -0.86135677],\n       [-3.56269116, -1.70427745,  0.53336288, -0.50578054,  0.76696499,\n        -0.41702883, -0.79772404, -0.86135677],\n       [ 0.49362588,  0.92844966,  1.88080596,  1.97714211,  0.76696499,\n        -0.41702883,  1.25356634, -0.86135677],\n       [ 0.49362588,  0.92844966, -0.98251057, -0.50578054,  0.76696499,\n        -0.41702883, -0.79772404,  1.16095912],\n       [-0.66532184, -0.78854628, -0.47721942, -0.50578054,  0.76696499,\n         2.39791576, -0.79772404, -0.86135677],\n       [-1.24479571, -0.78854628,  0.19650211, -0.50578054,  0.76696499,\n        -0.41702883,  1.25356634, -0.86135677],\n       [ 0.49362588, -0.78854628, -1.31937134, -0.50578054,  0.76696499,\n        -0.41702883, -0.79772404,  1.16095912],\n       [ 0.49362588,  0.92844966,  0.02807173, -0.50578054,  0.76696499,\n         2.39791576, -0.79772404, -0.86135677],\n       [-0.66532184, -0.78854628, -0.81408019,  1.97714211,  0.76696499,\n        -0.41702883, -0.79772404,  1.16095912],\n       [ 0.49362588, -0.78854628,  0.87022365, -0.50578054,  0.76696499,\n        -0.41702883, -0.79772404,  1.16095912],\n       [-0.66532184, -0.78854628, -1.31937134, -0.50578054,  0.76696499,\n        -0.41702883,  1.25356634, -0.86135677],\n       [-3.56269116, -1.70427745,  0.53336288, -0.50578054,  0.76696499,\n        -0.41702883, -0.79772404,  1.16095912],\n       [ 0.49362588,  0.92844966, -0.14035865, -0.50578054,  0.76696499,\n         2.39791576, -0.79772404, -0.86135677],\n       [ 0.49362588,  0.92844966,  0.87022365,  1.97714211,  0.76696499,\n        -0.41702883, -0.79772404,  1.16095912],\n       [ 0.49362588,  0.92844966,  0.87022365,  1.97714211,  0.76696499,\n        -0.41702883,  1.25356634, -0.86135677],\n       [ 0.49362588,  0.92844966,  0.19650211, -0.50578054,  0.76696499,\n        -0.41702883, -0.79772404,  1.16095912],\n       [-0.66532184, -0.78854628,  1.88080596, -0.50578054,  0.76696499,\n         2.39791576, -0.79772404, -0.86135677],\n       [ 0.49362588, -1.70427745,  0.02807173,  1.97714211,  0.76696499,\n         2.39791576, -0.79772404, -0.86135677],\n       [ 0.49362588, -0.78854628, -0.98251057, -0.50578054,  0.76696499,\n        -0.41702883,  1.25356634, -0.86135677],\n       [ 0.49362588, -1.70427745, -0.47721942, -0.50578054,  0.76696499,\n        -0.41702883,  1.25356634, -0.86135677],\n       [ 0.49362588,  0.92844966, -0.30878904, -0.50578054,  0.76696499,\n        -0.41702883, -0.79772404,  1.16095912],\n       [ 0.49362588, -1.70427745, -0.81408019, -0.50578054,  0.76696499,\n        -0.41702883,  1.25356634, -0.86135677],\n       [-3.56269116, -1.70427745,  0.87022365, -0.50578054, -1.30384048,\n        -0.41702883, -0.79772404, -0.86135677],\n       [ 0.49362588, -0.78854628, -0.47721942, -0.50578054, -1.30384048,\n        -0.41702883, -0.79772404,  1.16095912],\n       [ 0.49362588, -0.78854628, -0.98251057, -0.50578054, -1.30384048,\n         2.39791576, -0.79772404, -0.86135677],\n       [-0.66532184,  0.92844966, -0.64564981, -0.50578054, -1.30384048,\n        -0.41702883, -0.79772404,  1.16095912],\n       [ 0.49362588,  0.92844966,  1.03865404, -0.50578054, -1.30384048,\n        -0.41702883, -0.79772404,  1.16095912],\n       [ 0.49362588,  0.92844966,  2.38609711, -0.50578054, -1.30384048,\n        -0.41702883, -0.79772404,  1.16095912],\n       [ 0.49362588,  0.92844966,  0.19650211, -0.50578054, -1.30384048,\n         2.39791576, -0.79772404, -0.86135677],\n       [ 0.49362588,  0.92844966, -0.47721942, -0.50578054, -1.30384048,\n        -0.41702883, -0.79772404,  1.16095912],\n       [ 0.49362588,  0.92844966, -0.47721942, -0.50578054, -1.30384048,\n        -0.41702883, -0.79772404,  1.16095912],\n       [ 0.49362588, -0.78854628,  0.70179327, -0.50578054, -1.30384048,\n        -0.41702883,  1.25356634, -0.86135677],\n       [ 0.49362588,  0.92844966, -0.47721942, -0.50578054, -1.30384048,\n        -0.41702883, -0.79772404,  1.16095912],\n       [ 0.49362588,  0.92844966, -0.30878904, -0.50578054, -1.30384048,\n        -0.41702883, -0.79772404,  1.16095912],\n       [ 0.49362588, -0.78854628,  0.70179327, -0.50578054, -1.30384048,\n        -0.41702883,  1.25356634, -0.86135677],\n       [ 0.49362588,  0.92844966, -0.47721942, -0.50578054, -1.30384048,\n        -0.41702883, -0.79772404,  1.16095912],\n       [ 0.49362588,  0.92844966, -0.64564981, -0.50578054, -1.30384048,\n        -0.41702883,  1.25356634, -0.86135677],\n       [-0.66532184, -0.78854628, -1.48780173, -0.50578054, -1.30384048,\n        -0.41702883, -0.79772404,  1.16095912],\n       [ 0.49362588,  0.92844966,  1.03865404,  1.97714211, -1.30384048,\n        -0.41702883,  1.25356634, -0.86135677],\n       [ 0.49362588,  0.92844966, -0.30878904,  1.97714211, -1.30384048,\n        -0.41702883, -0.79772404,  1.16095912],\n       [ 0.49362588,  0.92844966,  0.19650211, -0.50578054,  0.76696499,\n        -0.41702883,  1.25356634, -0.86135677],\n       [ 0.49362588, -0.78854628, -0.14035865,  1.97714211,  0.76696499,\n        -0.41702883,  1.25356634, -0.86135677],\n       [-0.66532184, -0.78854628,  1.54394519, -0.50578054,  0.76696499,\n        -0.41702883, -0.79772404,  1.16095912],\n       [ 0.49362588,  0.92844966, -0.30878904, -0.50578054,  0.76696499,\n        -0.41702883, -0.79772404,  1.16095912],\n       [-0.66532184, -0.78854628, -0.98251057,  1.97714211,  0.76696499,\n        -0.41702883,  1.25356634, -0.86135677],\n       [ 0.49362588,  0.92844966, -1.99309288, -0.50578054,  0.76696499,\n        -0.41702883,  1.25356634, -0.86135677],\n       [ 0.49362588, -0.78854628, -0.98251057, -0.50578054,  0.76696499,\n        -0.41702883,  1.25356634, -0.86135677],\n       [ 0.49362588,  0.92844966, -1.31937134,  1.97714211,  0.76696499,\n        -0.41702883,  1.25356634, -0.86135677],\n       [-0.66532184, -0.78854628, -0.81408019, -0.50578054,  0.76696499,\n        -0.41702883, -0.79772404,  1.16095912],\n       [ 0.49362588,  0.92844966,  0.02807173, -0.50578054,  0.76696499,\n        -0.41702883,  1.25356634, -0.86135677],\n       [-0.66532184, -0.78854628, -0.47721942, -0.50578054,  0.76696499,\n        -0.41702883, -0.79772404,  1.16095912],\n       [ 0.49362588,  0.92844966,  0.87022365, -0.50578054,  0.76696499,\n        -0.41702883,  1.25356634, -0.86135677],\n       [-0.66532184, -0.78854628,  0.70179327, -0.50578054,  0.76696499,\n        -0.41702883,  1.25356634, -0.86135677],\n       [ 0.49362588,  0.92844966,  0.19650211, -0.50578054, -1.30384048,\n        -0.41702883,  1.25356634, -0.86135677]])"
                    },
                    "execution_count": 143,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "testX"
        },
        {
            "cell_type": "code",
            "execution_count": 145,
            "metadata": {},
            "outputs": [],
            "source": "testy = testFeature['loan_status'].values\ntest_le_y = preprocessing.LabelEncoder()\ntest_le_y.fit(testy)\ntest_le_y = test_le_y.transform(testy)"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Jaccard"
        },
        {
            "cell_type": "code",
            "execution_count": 158,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "KNN: 0.7307692307692307\nDecision Tree: 0.7115384615384616\nSVM: 0.7037037037037037\nLog regression: 0.7407407407407407\n"
                }
            ],
            "source": "print(\"KNN:\", jaccard_score(KNN.predict(testX), test_le_y))\nprint(\"Decision Tree:\", jaccard_score(DT.predict(testX), test_le_y))\nprint(\"SVM:\", jaccard_score(SVM.predict(testX), test_le_y))\nprint(\"Log regression:\", jaccard_score(LR.predict(testX), test_le_y))"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# F1"
        },
        {
            "cell_type": "code",
            "execution_count": 159,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "KNN: 0.7983539094650205\nDecision Tree: 0.773966752086208\nSVM: 0.7954911433172304\nLog regression: 0.851063829787234\n"
                }
            ],
            "source": "print(\"KNN:\", f1_score(KNN.predict(testX), test_le_y, average = \"weighted\"))\nprint(\"Decision Tree:\", f1_score(DT.predict(testX), test_le_y, average = \"weighted\"))\nprint(\"SVM:\", f1_score(SVM.predict(testX), test_le_y, average = \"weighted\"))\nprint(\"Log regression:\", f1_score(LR.predict(testX), test_le_y, average = \"weighted\"))"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Log loss"
        },
        {
            "cell_type": "code",
            "execution_count": 240,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "Log loss: 8.95470488690319\n"
                }
            ],
            "source": "print(\"Log loss:\", log_loss(test_le_y, LR.predict(testX)))"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Report\nYou should be able to report the accuracy of the built model using different evaluation metrics:"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "| Algorithm          | Jaccard | F1-score | LogLoss |\n|--------------------|---------|----------|---------|\n| KNN                | 0.73    | 0.79     | NA      |\n| Decision Tree      | 0.71    | 0.77     | NA      |\n| SVM                | 0.70    | 0.79     | NA      |\n| LogisticRegression | 0.74    | 0.85     | 8.95       |"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "<h2>Want to learn more?</h2>\n\nIBM SPSS Modeler is a comprehensive analytics platform that has many machine learning algorithms. It has been designed to bring predictive intelligence to decisions made by individuals, by groups, by systems \u2013 by your enterprise as a whole. A free trial is available through this course, available here: <a href=\"http://cocl.us/ML0101EN-SPSSModeler\">SPSS Modeler</a>\n\nAlso, you can use Watson Studio to run these notebooks faster with bigger datasets. Watson Studio is IBM's leading cloud solution for data scientists, built by data scientists. With Jupyter notebooks, RStudio, Apache Spark and popular libraries pre-packaged in the cloud, Watson Studio enables data scientists to collaborate on their projects without having to install anything. Join the fast-growing community of Watson Studio users today with a free account at <a href=\"https://cocl.us/ML0101EN_DSX\">Watson Studio</a>\n\n<h3>Thanks for completing this lesson!</h3>\n\n<h4>Author:  <a href=\"https://ca.linkedin.com/in/saeedaghabozorgi\">Saeed Aghabozorgi</a></h4>\n<p><a href=\"https://ca.linkedin.com/in/saeedaghabozorgi\">Saeed Aghabozorgi</a>, PhD is a Data Scientist in IBM with a track record of developing enterprise level applications that substantially increases clients\u2019 ability to turn data into actionable knowledge. He is a researcher in data mining field and expert in developing advanced analytic methods like machine learning and statistical modelling on large datasets.</p>\n\n<hr>\n\n<p>Copyright &copy; 2018 <a href=\"https://cocl.us/DX0108EN_CC\">Cognitive Class</a>. This notebook and its source code are released under the terms of the <a href=\"https://bigdatauniversity.com/mit-license/\">MIT License</a>.</p>"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": ""
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": ""
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": ""
        }
    ],
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3.7",
            "language": "python",
            "name": "python3"
        },
        "language_info": {
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "file_extension": ".py",
            "mimetype": "text/x-python",
            "name": "python",
            "nbconvert_exporter": "python",
            "pygments_lexer": "ipython3",
            "version": "3.7.9"
        }
    },
    "nbformat": 4,
    "nbformat_minor": 2
}