{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"http://hilpisch.com/tpq_logo.png\" width=\"350px\" align=\"right\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# EPAT Session 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Executive Program in Algorithmic Trading**\n",
    "\n",
    "**_Event-based Backtesting_**\n",
    "\n",
    "Dr. Yves J. Hilpisch | The Python Quants GmbH | http://tpq.io\n",
    "\n",
    "<img src=\"http://hilpisch.com/images/tpq_bootcamp.png\" width=\"350px\" align=\"left\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from pylab import plt\n",
    "plt.style.use('ggplot')\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Financial Data Class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class FinancialData(object):\n",
    "    def __init__(self, symbol):\n",
    "        self.symbol = symbol\n",
    "        self.prepare_data()\n",
    "    \n",
    "    def prepare_data(self):\n",
    "        self.raw = pd.read_csv('http://hilpisch.com/tr_eikon_eod_data.csv',\n",
    "                               index_col=0, parse_dates=True)\n",
    "        self.data = pd.DataFrame(self.raw[self.symbol])\n",
    "        self.data['Returns'] = np.log(self.data / self.data.shift(1))\n",
    "        \n",
    "    def plot_data(self, cols=None):\n",
    "        if cols is None:\n",
    "            cols = [self.symbol]\n",
    "        self.data[cols].plot(figsize=(10, 6), title=self.symbol)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "fd = FinancialData('AAPL.O')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fd.data.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fd.data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fd.plot_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Event-based View on Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# vectorized data handling = complete data set in a single step\n",
    "# fd.data['AAPL.O'].plot(figsize=(10, 6));\n",
    "fd.plot_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for bar in range(10):\n",
    "    print(bar)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# event-based view on data = going bar by bar \"through time\"\n",
    "for bar in range(10):\n",
    "    print(bar, fd.data['AAPL.O'].iloc[bar])\n",
    "    time.sleep(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# event-based view on data = going bar by bar \"through time\"\n",
    "for bar in range(10):\n",
    "    print(bar, str(fd.data['AAPL.O'].index[bar])[:10], fd.data['AAPL.O'].iloc[bar])\n",
    "    time.sleep(.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Backtesting Base Class"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are going to implement a **base class** for event-based backtesting with:\n",
    "\n",
    "* `__init__`\n",
    "* `prepare_data` (`FinancialBase`)\n",
    "* `plot_data` (`FinancialBase`)\n",
    "* `get_date_price`\n",
    "* `print_balance`\n",
    "* `place_buy_order`\n",
    "* `place_sell_order`\n",
    "* `close_out`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import math"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "amount = 5000\n",
    "price = 27.85"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "amount / price  # --> vectorized backtesting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "units = math.floor(amount / price)  # --> event-based backtesting\n",
    "units"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cost = units * price\n",
    "cost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "amount - cost  # cash left"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class BacktestingBase(FinancialData):\n",
    "    def __init__(self, symbol, amount, verbose=True):\n",
    "        super(BacktestingBase, self).__init__(symbol)\n",
    "        self.amount = amount  # current cash balance\n",
    "        self.initial_amount = amount  # initial invest/cash\n",
    "        self.verbose = verbose\n",
    "        self.units = 0\n",
    "        self.trades = 0\n",
    "        \n",
    "    def get_date_price(self, bar):\n",
    "        date = str(self.data[self.symbol].index[bar])[:10]\n",
    "        price = self.data[self.symbol].iloc[bar]\n",
    "        return date, price\n",
    "    \n",
    "    def print_balance(self, bar):\n",
    "        date, price = self.get_date_price(bar)\n",
    "        print('%s | current cash balance is %8.2f' % (date, self.amount))\n",
    "                 \n",
    "    def place_buy_order(self, bar, units=None, amount=None):\n",
    "        date, price = self.get_date_price(bar)\n",
    "        if amount is not None:\n",
    "            units = math.floor(amount / price)\n",
    "        self.amount -= units * price  # here tc can be included\n",
    "        self.units += units\n",
    "        self.trades += 1\n",
    "        if self.verbose is True:\n",
    "            print('%s | buying %3d units for    %8.2f' % (date, units, price))\n",
    "            self.print_balance(bar)\n",
    "        \n",
    "    def place_sell_order(self, bar, units=None, amount=None):\n",
    "        date, price = self.get_date_price(bar)\n",
    "        if amount is not None:\n",
    "            units = math.floor(amount / price)\n",
    "        self.amount += units * price\n",
    "        self.units -= units\n",
    "        self.trades += 1\n",
    "        if self.verbose is True:\n",
    "            print('%s | selling %3d units for   %8.2f' % (date, units, price))\n",
    "            self.print_balance(bar)\n",
    "    \n",
    "    def close_out(self, bar):\n",
    "        date, price = self.get_date_price(bar)\n",
    "        self.amount += self.units * price\n",
    "        print(50 * '=')\n",
    "        print('Closing out the position.')\n",
    "        print(50 * '=')\n",
    "        if self.units != 0:\n",
    "            self.trades += 1\n",
    "            print('%s | selling %3d units for   %8.2f' % (date, self.units, price))\n",
    "        self.units -= self.units\n",
    "        self.print_balance(bar)\n",
    "        perf = ((self.amount - self.initial_amount) / self.initial_amount) * 100\n",
    "        print('%s | net performance         %8.2f' % (date, perf))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bb = BacktestingBase('AAPL.O', 10000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bb.data.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bb.get_date_price(177)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bb.print_balance(210)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bb.place_buy_order(209, units=15)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(bb.units, bb.trades)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bb.place_buy_order(260, amount=2000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(bb.units, bb.trades)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bb.place_sell_order(300, units=40)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bb.place_sell_order(350, amount=500)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(bb.units, bb.trades)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bb.close_out(400)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Long Only Backtesting Class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class LongOnlyBacktest(BacktestingBase):\n",
    "    # def __init__(self, *args):\n",
    "    #    super(LongOnlyBacktest, self).__init__(*args)\n",
    "    \n",
    "    def run_strategy(self, SMA1, SMA2):\n",
    "        print('\\n\\nRunning strategy for %s | SMA1=%d | SMA2=%d' % (self.symbol, SMA1, SMA2))\n",
    "        print(50 * '=')\n",
    "        self.units = 0\n",
    "        self.trades = 0\n",
    "        self.position = 0\n",
    "        self.amount = self.initial_amount\n",
    "        self.results = self.data.copy()\n",
    "        self.results['SMA1'] = self.results[self.symbol].rolling(SMA1).mean()\n",
    "        self.results['SMA2'] = self.results[self.symbol].rolling(SMA2).mean()\n",
    "        \n",
    "        for bar in range(SMA2 - 1, len(self.results)):\n",
    "            \n",
    "            if self.position == 0:\n",
    "                if self.results['SMA1'].iloc[bar] > self.results['SMA2'].iloc[bar]:\n",
    "                    # self.place_buy_order(bar, units=100)\n",
    "                    self.place_buy_order(bar, amount=self.amount * 0.8)\n",
    "                    # self.place_buy_order(bar, amount=5000)\n",
    "                    date, price = self.get_date_price(bar)\n",
    "                    self.entry_cost = self.units * price\n",
    "                    # place whatever logic reflects your strategy\n",
    "                    self.position = 1\n",
    "            \n",
    "            elif self.position == 1:\n",
    "                if self.results['SMA1'].iloc[bar] < self.results['SMA2'].iloc[bar]:\n",
    "                    # self.place_sell_order(bar, units=100)\n",
    "                    self.place_sell_order(bar, units=self.units)\n",
    "                    self.position = 0\n",
    "                # stop loss logic\n",
    "                else:\n",
    "                    date, price = self.get_date_price(bar)\n",
    "                    current_position_value = self.units * price\n",
    "                    if (current_position_value - self.entry_cost) / self.entry_cost <= -0.05:\n",
    "                        self.place_sell_order(bar, units=self.units)\n",
    "                        self.position = -2  # position indicating a previous stop\n",
    "                        self.entry_cost = 0\n",
    "                        self.trades += 1\n",
    "                        self.wait_days = 10\n",
    "                        if self.verbose:\n",
    "                            print('Closing out due to stop loss.')\n",
    "            \n",
    "            elif self.position == -2 and self.wait_days > 0:\n",
    "                self.wait_days -= 1\n",
    "                if self.wait_days == 0:\n",
    "                    self.position = 0\n",
    "                    \n",
    "        self.close_out(bar)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sma = LongOnlyBacktest('AAPL.O', 10000, True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sma.run_strategy(42, 252)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sma = LongOnlyBacktest('AAPL.O', 10000, True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sma.run_strategy(42, 252)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sma.run_strategy(30, 180)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from itertools import product"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for sym in sma.raw.columns.values:\n",
    "    print(sym)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for sym in ['AAPL.O', 'MSFT.O']:\n",
    "    sma = LongOnlyBacktest(sym, 10000, False)\n",
    "    for SMA1, SMA2 in product([30, 42], [180, 252]):\n",
    "        sma.run_strategy(SMA1, SMA2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Long-Short Strategies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class LongShortBacktest(BacktestingBase):\n",
    "    \n",
    "    def run_strategy(self, SMA1, SMA2):\n",
    "        print('\\n\\nRunning strategy for %s | SMA1=%d | SMA2=%d' % (self.symbol, SMA1, SMA2))\n",
    "        print(50 * '=')\n",
    "        self.units = 0\n",
    "        self.trades = 0\n",
    "        self.position = 0\n",
    "        self.entry_value = 0\n",
    "        self.amount = self.initial_amount\n",
    "        self.results = self.data.copy()\n",
    "        self.results['SMA1'] = self.results[self.symbol].rolling(SMA1).mean()\n",
    "        self.results['SMA2'] = self.results[self.symbol].rolling(SMA2).mean()\n",
    "        \n",
    "        for bar in range(SMA2 - 1, len(self.results)):\n",
    "            date, price = self.get_date_price(bar)\n",
    "            current_position_value = self.units * price\n",
    "            diff = current_position_value - self.entry_value\n",
    "            rdiff = diff / self.entry_value\n",
    "            rdiff = rdiff if self.position >= 0 else -rdiff\n",
    "            if self.verbose:\n",
    "                print('%s | %8.2f | %8.2f | %8.3f | %7.3f' %\n",
    "                  (date, self.entry_value, current_position_value, diff, rdiff))\n",
    "            \n",
    "            if self.position in [0, -1, -2]:\n",
    "                if self.results['SMA1'].iloc[bar] > self.results['SMA2'].iloc[bar]:\n",
    "                    if self.position == -1:\n",
    "                        self.place_buy_order(bar, amount=-self.units)\n",
    "                    # self.place_buy_order(bar, amount=5000)\n",
    "                    self.place_buy_order(bar, amount=self.amount * 0.8)\n",
    "                    date, price = self.get_date_price(bar)\n",
    "                    self.entry_value = self.units * price\n",
    "                    self.position = 1\n",
    "                elif self.entry_value != 0:\n",
    "                    if (current_position_value - self.entry_value) / -self.entry_value <= -0.075:\n",
    "                        self.place_buy_order(bar, units=-self.units)\n",
    "                        self.position = -2\n",
    "                        self.entry_value = 0\n",
    "                        if self.verbose:\n",
    "                            print('Closing out short position due to stop loss.')\n",
    "            \n",
    "            elif self.position in [0, 1, 2]:\n",
    "                if self.results['SMA1'].iloc[bar] < self.results['SMA2'].iloc[bar]:\n",
    "                    if self.position == 1:\n",
    "                        self.place_sell_order(bar, amount=self.units)\n",
    "                    # self.place_sell_order(bar, amount=5000)\n",
    "                    self.place_sell_order(bar, amount=self.amount * 0.8)\n",
    "                    self.entry_value = self.units * price\n",
    "                    self.position = -1\n",
    "                elif self.entry_value != 0:\n",
    "                    if (current_position_value - self.entry_value) / self.entry_value <= -0.075:\n",
    "                        self.place_sell_order(bar, units=self.units)\n",
    "                        self.position = 2\n",
    "                        self.entry_value = 0\n",
    "                        if self.verbose:\n",
    "                            print('Closing out long position due to stop loss.')\n",
    "                    \n",
    "        self.close_out(bar)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sma = LongShortBacktest('AAPL.O', 10000, False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sma.run_strategy(42, 252)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for sym in ['AAPL.O', 'MSFT.O']:\n",
    "    sma = LongShortBacktest(sym, 10000, False)\n",
    "    for SMA1, SMA2 in product([30, 42], [180, 252]):\n",
    "        sma.run_strategy(SMA1, SMA2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some improvements (as an exercise):\n",
    "\n",
    "* include different signals (momentum)\n",
    "* include proportional and fixed transaction costs\n",
    "* allow for different time periods for the backtest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"http://hilpisch.com/tpq_logo.png\" width=\"350px\" align=\"right\">"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}