Last active
September 14, 2021 05:36
-
-
Save pbugnion/a4a24d6f54fe9f6239aae90569197e9e to your computer and use it in GitHub Desktop.
Revisions
-
pbugnion revised this gist
Jun 5, 2018 . 1 changed file with 4 additions and 2 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -28,7 +28,7 @@ pip install apache-airflow mkdir -p $AIRFLOW_HOME wget --output-document $AIRFLOW_HOME/airflow.cfg $SOURCE/airflow.cfg echo "export AIRFLOW_HOME=$AIRFLOW_HOME" | sudo tee /etc/profile.d/airflow.sh @@ -46,6 +46,8 @@ script: AIRFLOW_HOME=/project/airflow SOURCE=https://gist.github.com/pbugnion/a4a24d6f54fe9f6239aae90569197e9e/raw echo "export AIRFLOW_HOME=$AIRFLOW_HOME" | sudo tee /etc/profile.d/airflow.sh sudo sv stop jupyter wget --output-document /tmp/airflow-run $SOURCE/run @@ -84,7 +86,7 @@ wget --output-document /project/stock-price/fetch_stock.py $SOURCE/fetch_stock.p wget --output-document /project/stock-price/publish_report.py $SOURCE/publish_report.py wget --output-document /project/stock-price/plot-stock-price.ipynb $SOURCE/plot-stock-price.ipynb wget --output-document /project/airflow/dags/stock_price.py $SOURCE/stock_price_dag.py ``` Then create a report by publishing the `plot-stock-price` notebook in -
pbugnion revised this gist
Jun 5, 2018 . 1 changed file with 1 addition and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -82,6 +82,7 @@ mkdir -p /project/stock-price /project/stock-price/data wget --output-document /project/stock-price/fetch_stock.py $SOURCE/fetch_stock.py wget --output-document /project/stock-price/publish_report.py $SOURCE/publish_report.py wget --output-document /project/stock-price/plot-stock-price.ipynb $SOURCE/plot-stock-price.ipynb wget --output-document /project/airflow/stock_price.py $SOURCE/stock_price_dag.py ``` -
pbugnion revised this gist
Jun 5, 2018 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -30,7 +30,7 @@ mkdir -p $AIRFLOW_HOME wget --output-document /project/airflow/airflow.cfg $SOURCE/a0ec0a53459bbfd52a61c4eb766cac22c79d5f3e/airflow.cfg echo "export AIRFLOW_HOME=$AIRFLOW_HOME" | sudo tee /etc/profile.d/airflow.sh airflow initdb ``` -
pbugnion revised this gist
Jun 5, 2018 . 1 changed file with 4 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -91,6 +91,10 @@ Then create a report by publishing the `plot-stock-price` notebook in UUID in the URL when you are viewing the report). Edit the DAG definition with the correct UUID. The file `/project/airflow/dags/stock_price.py` is the airflow DAG definition: it defines what tasks are available in the DAG, and how they depend on each other. This pipeline is scheduled to run every day at midnight. You can also run the pipeline for a particular day or set of days by running the following command in a *new* terminal on the instance running the -
pbugnion revised this gist
Jun 5, 2018 . 3 changed files with 22 additions and 4 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -86,6 +86,11 @@ wget --output-document /project/stock-price/publish_report.py $SOURCE/publish_re wget --output-document /project/airflow/stock_price.py $SOURCE/stock_price_dag.py ``` Then create a report by publishing the `plot-stock-price` notebook in `/project/stock-price`. Get the report ID for that report (the last UUID in the URL when you are viewing the report). Edit the DAG definition with the correct UUID. This pipeline is scheduled to run every day at midnight. You can also run the pipeline for a particular day or set of days by running the following command in a *new* terminal on the instance running the @@ -97,3 +102,5 @@ airflow backfill stock_price -s 2018-03-01 -e 2018-03-06 This will fetch the stock price for dates between the 1st and the 6th of March 2018.  LoadingSorry, something went wrong. Reload?Sorry, we cannot display this file.Sorry, this file is invalid so it cannot be displayed.This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -2,6 +2,8 @@ from airflow.operators.bash_operator import BashOperator from datetime import datetime, timedelta REPORT_ID = 'b0b5e1c5-d7c9-4ae5-8b02-ce0f4a7ddaee' default_args = { 'owner': 'airflow', 'depends_on_past': False, @@ -19,18 +21,27 @@ ) fetch_command = ( '/opt/anaconda/envs/Python3/bin/python ' '/project/stock-price/fetch_stock.py GOOG {{ ds }} {{ ds }}' ) fetch_operator = BashOperator( task_id='fetch_price', bash_command=fetch_command, dag=dag ) publish_operator = BashOperator( task_id='publish', bash_command=( '/opt/anaconda/envs/Python3/bin/python ' '/project/stock-price/publish_report.py ' f'{REPORT_ID} ' '/project/stock-price/plot-stock-price.ipynb' ), dag=dag ) publish_operator.set_upstream(fetch_operator) -
pbugnion revised this gist
Jun 5, 2018 . 1 changed file with 3 additions and 2 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -7,10 +7,11 @@ from nbconvert.preprocessors import ExecutePreprocessor import sml.auth import sml.config HUDSON_URL = sml.config.url_for_service('hudson') TAVERN_URL = sml.config.url_for_service('tavern') class InputError(Exception): -
pbugnion revised this gist
Jun 5, 2018 . 1 changed file with 20 additions and 7 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,18 +1,17 @@ import os import argparse import requests import nbformat from nbconvert.preprocessors import ExecutePreprocessor import sml.auth HUDSON_URL = 'https://hudson.platform.asidata.science' TAVERN_URL = 'https://tavern.platform.asidata.science' class InputError(Exception): pass @@ -58,7 +57,7 @@ def run(report_id, notebook, run_notebook=True): reconstructed_notebook_path = '/'.join(reconstructed_notebook_path) # Authenticate and publish auth_headers = sml.auth.auth_headers() user_id_response = requests.get( HUDSON_URL + '/authenticate', headers=auth_headers @@ -75,6 +74,20 @@ def run(report_id, notebook, run_notebook=True): json=body, headers=auth_headers) create_version_response.raise_for_status() def parse_command_line(): parser = argparse.ArgumentParser( description='Run and publish a notebook') parser.add_argument('report_id', help='ID of the report to update') parser.add_argument( 'notebook_path', help='Path of the notebook to publish' ) args = parser.parse_args() return (args.report_id, args.notebook_path) if __name__ == '__main__': report_id, notebook_path = parse_command_line() run(report_id, notebook_path) -
pbugnion revised this gist
Jun 5, 2018 . 1 changed file with 4 additions and 4 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -86,14 +86,14 @@ wget --output-document /project/stock-price/publish_report.py $SOURCE/publish_re wget --output-document /project/airflow/stock_price.py $SOURCE/stock_price_dag.py ``` This pipeline is scheduled to run every day at midnight. You can also run the pipeline for a particular day or set of days by running the following command in a *new* terminal on the instance running the *airflow* webserver: ``` airflow backfill stock_price -s 2018-03-01 -e 2018-03-06 ``` This will fetch the stock price for dates between the 1st and the 6th of March 2018. -
pbugnion revised this gist
Jun 5, 2018 . 1 changed file with 25 additions and 12 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,16 +1,23 @@ # Airflow on SherlockML [Apache Airflow](https://airflow.apache.org/) is an open source tool for creating task pipelines. It lets you define sets of tasks and dependencies between those tasks, and then takes care of the execution.  Airflow can be a useful add-on to SherlockML: you can schedule periodic jobs that automatically fetch data or retrain a model or publish a report. By defining dependencies between those, you can have a resilient, automated data processing pipeline. ## Installing Airflow in SherlockML ### Setting up a project To set up a project, create an environment called `airflow-setup-project` with the following entry in the scripts tab: ```bash AIRFLOW_HOME=/project/airflow @@ -32,7 +39,8 @@ Then apply this environment to any Jupyter server within the project. ### Running the Airflow webserver Create an environment called `airflow-webserver` with a single shell script: ``` AIRFLOW_HOME=/project/airflow @@ -53,13 +61,17 @@ sudo sv start airflow sleep 20 # Wait for airflow to start ``` When this environment is applied to a server in the workspace, it will stop the Jupyter server and replace it with Airflow. You can then click on the server name to open the Airflow dashboard. ### Install examples This gist contains an example with an airflow pipeline that fetches the Google stock price for a given date, saves it in a CSV in the SherlockML workspace (we could also save it in, say, InfluxDB) and publishes a SherlockML report with a graph of the stock price over time, including that date. To install the demo, run the following commands in a terminal: @@ -74,13 +86,14 @@ wget --output-document /project/stock-price/publish_report.py $SOURCE/publish_re wget --output-document /project/airflow/stock_price.py $SOURCE/stock_price_dag.py ``` You can then run the pipeline by running the following command in a *new* terminal on the instance running the *airflow* webserver: ``` airflow backfill stock_price -s 2018-03-01 -e 2018-03-06 ``` This will fetch the stock price for dates between the 1st and the 6th of March 2018. This pipeline is scheduled to run every day at midnight. -
pbugnion revised this gist
Jun 5, 2018 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -4,7 +4,7 @@ task pipelines. It lets you define sets of tasks and dependencies between those tasks, and then takes care of the execution.  ## Installing Airflow in SherlockML -
pbugnion revised this gist
Jun 5, 2018 . 2 changed files with 3 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -4,6 +4,8 @@ task pipelines. It lets you define sets of tasks and dependencies between those tasks, and then takes care of the execution.  ## Installing Airflow in SherlockML ### Setting up a project @@ -81,4 +83,4 @@ airflow backfill stock_price -s 2018-03-01 -e 2018-03-06 This will fetch the stock price for dates between the 1st and the 6th of March 2018. This pipeline is scheduled to run every day at midnight. LoadingSorry, something went wrong. Reload?Sorry, we cannot display this file.Sorry, this file is invalid so it cannot be displayed. -
pbugnion revised this gist
Jun 5, 2018 . No changes.There are no files selected for viewing
-
pbugnion revised this gist
Jun 5, 2018 . 1 changed file with 3 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -79,4 +79,6 @@ the *airflow* webserver: airflow backfill stock_price -s 2018-03-01 -e 2018-03-06 ``` This will fetch the stock price for dates between the 1st and the 6th of March 2018. This pipeline is scheduled to run every day at midnight. -
pbugnion revised this gist
Jun 5, 2018 . 1 changed file with 5 additions and 5 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -19,7 +19,7 @@ "outputs": [], "source": [ "stock_price = 'GOOG'\n", "datastore = Path('/project/stock-price/data')" ] }, { @@ -31,8 +31,8 @@ "name": "stdout", "output_type": "stream", "text": [ "start date: 2018-01-02 00:00:00\n", "end date: 2018-01-10 00:00:00\n" ] } ], @@ -53,9 +53,9 @@ "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgMAAAF3CAYAAAA4gEgdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xd8VOed7/HPTx1QB4miQjFgwMZUY9wdd3uTuDsm7ont7G76vpJd7+bu3Xu3pd211ymbLNiOvYbYwY4TO4lLsJMYXEQHA6YLVGgSSIAKqvPcP+ZAhCzEABqdOTPf9+ull2bOnHP4Pcxo5jvPOc9zzDmHiIiIJK4kvwsQERERfykMiIiIJDiFARERkQSnMCAiIpLgFAZEREQSnMKAiIhIglMYEBERSXAKAyIiIglOYUBERCTBKQyIiIgkuBS/C4iGIUOGuFGjRvldhoiISL9ZuXLlfudcwelsG5dhYNSoUaxYscLvMkRERPqNmVWc7rY6TCAiIpLgFAZEREQSnMKAiIhIglMYEBERSXAKAyIiIglOYUBERCTBKQyIiIgkOIUBERGRBKcwICIikuAUBkRERBKcwoCIiEiCi8trE4iICHSGHNtqGqlraqMgK52CrHSyM1IwM79LkxijMCAiEicONLaypuogqysPsrqqnrVVh2hs7ThunbSUJAoy0ynMTqcgM/1YSCjMyjh2uyAr/FhaijqPE4XCgIhIALV3hti0p4FVlfWsrqxnddVBKg40A5CcZEwcnsUt04qYVprL0OwM9je2UtvQ5aexlYoDzayoqKeuqa3HfyN3YOrJg0NmOrkDU9XbEHAKAyIiAbD3UMuxD/3VlfV8WH2I1o4QAIVZ6UwvzeOzs0qZVprH5KIcBqQlR7zv9s7Qx8JCzXG3W1hZWU/N4dZj/2ZXqcnWJSwc38NQ2CU0FGSlk5EaeV3SfxQGRERiTEt7J+t3HTrW3b+68iB7DrUA4W7+yUU53Dt7JNNK85hWmsvwnIwz+maempzE8JwBDM8Z0Ot6zjkaWzuOCwrHBYfGVqrrm1lTVc+Bpjac+/g+sjNSuoSFjOPCQmH2n2/nDUwjKUm9Df1FYUBExEfOOSrrmsMf/N43/492H6YjFP4kLc0fyKzR+UwryWVaaR4Th2f7dizfzMjKSCUrI5WzCjJ7XbejM0RdU9vxwaGxlZrDLdR6vRAfVh+k5nArR9o7P7Z9SpIxJPP4cxgKs9O54uwCZozMj1YTE5a5nqJbwM2cOdOtWLHC7zJERD6moaWdD6sPhT/4Kw+yuurgsWP2g9KSmVKSy7TSXKaV5DG1NJchmek+Vxx9Ta0d3Q5NtHjBofVYcKhpaOVAYyshB5eNL+DrV49jWmme36XHFDNb6ZybeTrbqmdARCRKQiHHttrGP3/wVx5kS03Dse7zcYWZXD2x8Fh3/7jCLJITsGt8UHoKg9JTGDVkUK/rNbd18NwHFfz0ne3c8l/vc+WEQr5+9XgmF+f0U6XxSz0DIiJ9pK6pjTVVf/7gX1t1kAZvaF/OgFSmleYy3fvgP684l5wBqT5XHEyNrR08+/5O5i4u59CRdq6ZNJSvXz2eSSOy/S7NV2fSM6AwICJyGo4O7Vt97MO/np1dhvZNGJZ1rLt/Wmkuo4cM0vC7Pna4pZ2fvbuTJ98tp6GlgxsnD+NrV49n/NAsv0vzhcJANwoDItLXehvaV5CVzvTS8Al+00pymVycw8A0HYXtL4ea23nq3XKefm8nTW0dfPK8EXz1qnGMLez9JMd4ozDQjcKAiJyJXof2JSdxblH2seP800rzGHGGQ/ukb9Q3tTF3STnPvr+TlvZObp5axFeuGnfScxHihcJANwoDIhKpkw3tK8kfcKyrPzy0L4v0FE2cE8v2N7Yyd3E5//PBTto7HbdNL+LLV46jJH+g36VFlcJANwoDInIivQ3tG5iWzJTi3GMf/FNLcinIiv+hffGqpqGFn/6pnPlLKwiFHHfMLOFLV46lKLf3yZWCSmGgG4UBEYGTD+0bW5h5bDKfaaW5jB+amEP74t3eQy3815+28cKyKhyOu84v5YufGMuwnAy/S+tTCgPdKAyIJKZIhvYd7fKfUqKhfYlm18Ej/PiP21i4vIqkJOPuC0r5qyvOojArPkKBwkA3CgMi8U9D++R0VdU186M/bOOlVdWkJhv3zh7JFy4/K/CzPSoMdKMwIBJ/NLRP+trO/U384A9b+fXqXaSnJHP/RaP4wmVjyBuU5ndpp0VhoBuFAZFg09A+6U/baxt54q2t/ObD3QxMTeZzl4zmoUvGkDMwWIeRFAa6URgQCQ4N7ZNYsWVfA0+8tZXfrdtDVkYKD10yhgcvGUV2RjBCgcJANwoDIrFLQ/sk1m3cc5j/fGsLb27YR86AVB65bAz3XzSKzPTYPvSkMNCNwoBIbDjZ0L6zCgYxrTTv2MV7xhVmkpKc5G/RIp71uw7x+KItvL2phryBqXzh8rO478KRMXs+isJANwoDIv6ob2pjTdWfu/vXVB4/tG9qSZdv/cW5gTsmK4lpTdVBHl+0hXe21DIkM42/vPws7pk9kozU2DpcpTDQjcKASPS1d4bYvLfhuO7+HfubAEgymDAs+9gH/7TSXEYPHkSSJvSRAFtZUcfji7by7rb9FGal89dXnMVds0pjJhQoDHSjMCDS9/Ydbjmuu//DXQdpaQ8P7RuS2WVoX2kuk4tyGBTjx1dFTldZ+QEeW7SFZTvqGJ6TwRc/MZY7Z5aQluLvIS6FgW4UBkTOTEt7Jxt2Hzr2wb+6sp7dXYb2nVOU3eUM/1yKcgdoaJ8kFOccH2w/wH8s2sLKinqKcgfw5SvHctuMYlJ9Ou9FYaAbhQGRyDnnqKo7ctxMfh/tOUx7Z/i9oThvwLHJfKaV5jJpRLaG9ol4nHMs3rqfxxZtYW3VQUrzB/KVq8Zx89QR/X4yrMJANwoDIifW2NrBh1UHj83kt7ryIAe6DO07rzjn2If/1NLcuJm3XSSanHP8cXMNjy3awvpdhxk9ZBBfvWocn5oyot8ufqUw0I3CgEhYKOTYXtt43Ex+m/d9fGjf0Tn8xw/V0D6RM+Gc4/cf7ePxRVvYtLeBsYWZfO3qcdx47vCon0CrMNCNwoAkqt6G9mVnpBw3ha+G9olETyjkeGPDXh5ftIWtNY2cPTSLr18zjmsnDYtaKFAY6EZhQBKBhvaJxL7OkON36/bwn29toby2iUnDs/mba8Zz1cTCPj/pVmGgG4UBiUca2icSXJ0hxytrdvHE21upONDMecU5/PNN5zK1JLfP/o0zCQN6txCJQZEM7fvsrJEa2icSEMlJxq3Ti/n0lBG8vHoXP/zDVlJiqKdOYUDEZ5EM7ZsxKp+HNLRPJPBSkpO4c2YJt00v7rdRBpFQGBDxkXOOzz+7gj9sqgFgQGoyU0pyeOjSMRraJxLHYikIgMKAiK+W76znD5tqeOCiUdwxs5izh2ZpaJ+I9DuFAREfzV1cTt7AVP7u+gkMSFPXv4j4Q19BRHyyvbaRtzbu494LRykIiIivFAZEfPLkkh2kpSRx34Uj/S5FRBKcwoCID/Y3tvLLVdXcNr2YIZnpfpcjIglOYUDEB//zQQVtHSEeunS036WIiCgMiPS3I22dPPfBTq6eOJSzCjL9LkdERGFApL+9tKqa+uZ2HrlsjN+liIgACgMi/aoz5HhqSTlTSnI5f1Se3+WIiAAKAyL9atFH+9h5oJlHLh2jawmISMxQGBDpR/OWlFOSP4DrzhnqdykiIscoDIj0k5UVdaysqOfzF4/WlMMiElOi9o5kZk+bWY2Zre+y7A4z22BmITOb2W39vzezbWa22cyu67L8em/ZNjN7NFr1ikTbvMU7yBmQyh0zS/wuRUTkONH8evIMcH23ZeuBW4HFXRea2STgLuAcb5v/MrNkM0sGfgzcAEwC5njrigTKzv1NvPnRXu6ZXcqgdF0SRERiS9TelZxzi81sVLdlG4GeTpy6CXjBOdcK7DCzbcAs77Ftzrlyb7sXvHU/ilbdItHw1Ls7SE1K4v4LR/ldiojIx8TKgcsioKrL/Wpv2YmWiwRGXVMbL66s4pZpRRRmZ/hdjojIx8RKGOhpjJXrZfnHd2D2iJmtMLMVtbW1fVqcyJmYX1ZBS7umHhaR2BUrYaAa6HpWVTGwu5flH+Ocm+ucm+mcm1lQUBC1QkVORUt7J8++v5MrJxQybmiW3+WIiPQoVsLAq8BdZpZuZqOBccAyYDkwzsxGm1ka4ZMMX/WxTpFT8vKqXRxoauPhSzX1sIjErqidQGhmzwNXAEPMrBr4J6AO+CFQAPzOzNY4565zzm0ws4WETwzsAL7onOv09vMl4E0gGXjaObchWjWL9KVQyPHkknImF+Uwe0y+3+WIiJxQNEcTzDnBQ786wfr/BvxbD8tfA17rw9JE+sXbm2oo39/ED+ZM09TDIhLTYuUwgUjcmbe4nKLcAdx47jC/SxER6ZXCgEgUrK6sZ9nOOj53iaYeFpHYp3cpkSh4cskOsjJS+Mz5mnpYRGKfwoBIH6s80Mzr6/dw9wUjydTUwyISAAoDIn3s6fd2kJxkPHDRKL9LERGJiMKASB862NzGL5ZX8ekpRQzL0dTDIhIMCgMifWjB0kqOtHfy8GWaelhEgkNhQKSPtHZ08rP3dnLZ+AImDMv2uxwRkYgpDIj0kVdW72Z/YyuPaOphEQkYhQGRPhAKOeYuKWfi8GwuHjvY73JERE6JwoBIH3hnSy3bahp55LLRmnpYRAJHYUCkD8xdXM6w7Aw+ed4Iv0sRETllCgMiZ2hd9SE+KD/A5y4ZRaqmHhaRANI7l8gZmreknMz0FO6aVep3KSIip0VhQOQMVNc387t1e5gzq4TsjFS/yxEROS0KAyJn4Gfv7cSABy/WJEMiElwKAyKn6dCRdl5YVsknzxvOiNwBfpcjInLaFAZETtPzyyppauvkIU0yJCIBpzAgchraOkL87L0dXDx2MOcW5fhdjojIGVEYEDkNv1m7m32HW3lYvQIiEgcUBkROkXOOeUvKOXtoFpePL/C7HBGRM6YwIHKKlmzdz6a9DTx0qaYeFpH4oDAgcormLSmnMCudT0/V1MMiEh8UBkROwUe7D7Nk634evHg06SnJfpcjItInFAZETsGTS8oZlJbMZy/Q1MMiEj8UBkQitOfQEV5du5vPnF9KzgBNPSwi8UNhQCRCz7y3Ewc8ePEov0sREelTCgMiEWhoaefnSyu5cfJwSvIH+l2OiEifUhgQicAvllfR0NrBw5fqgkQiEn8UBkROor0zxNPv7mD2mHzOK871uxwRkT6nMCByEq+t28PuQy08cpmmHhaR+KQwINIL5xxzF5cztjCTK8YX+l2OiEhUKAyI9OL97QfYsPswD186mqQkTT0sIvFJYUCkF3MXlzMkM52bphb5XYqISNQoDIicwOa9DbyzpZYHLhpJRqqmHhaR+KUwIHIC85aUMyA1mbsvGOl3KSIiUaUwINKDfYdbeGXNLu6cWUzeoDS/yxERiSqFAZEePPP+TjpDjs9dokmGRCT+KQyIdNPY2sGCsgquP3cYIwcP8rscEZGoUxgQ6Wbh8ioOt3Tw8KWaZEhEEoPCgEgXHZ0hnnp3B+ePymNaaZ7f5YiI9AuFAZEuXl+/l10Hj6hXQEQSisKAiOfo1MOjhwzi6olD/S5HRKTfKAyIeJbuqGPdrkM8pKmHRSTBKAyIeOYtLid/UBq3TS/2uxQRkX6lMCACbKtp4O1NNdx3oaYeFpHEozAgAjy5ZAfpKUncO1tTD4tI4lEYkIRX09DCy6t2cfuMYgZnpvtdjohIv1MYkIT33AcVtIdCfF5TD4tIglIYkITW3NbBc2UVXDNxKGMKMv0uR0TEFwoDktBeWlnNweZ2HrlMkwyJSOJSGJCE1RlyPLlkB9NKc5kxUlMPi0jiUhiQhPX7DXuprGvmC5eNwUyTDIlI4lIYkITknOO/F5czcvBArpk0zO9yRER8pTAgCWllRT1rqg7y0CWjSdbUwyKS4BQGJCHNXVxO3sBUbp9R4ncpIiK+UxiQhFNe28iijfu4d/ZIBqRp6mEREYUBSThPvbuD1OQk7r1wlN+liIjEBIUBSSgHGlt5aWU1t00voiBLUw+LiIDCgCSY58oqaO0I8flLNMmQiMhRCgOSMFraO/mfDyq4emIhYws19bCIyFFRCwNm9rSZ1ZjZ+i7L8s1skZlt9X7necvzzOxXZvahmS0zs3O7bHO9mW02s21m9mi06pX498tV1dQ1tfHwpeoVEBHpKpo9A88A13db9ijwtnNuHPC2dx/gH4A1zrnzgPuAJwDMLBn4MXADMAmYY2aTolizxKmQN/XwlOIcZo3O97scEZGYErUw4JxbDNR1W3wT8Kx3+1ngZu/2JMLhAOfcJmCUmQ0FZgHbnHPlzrk24AVvHyKn5K2N+9ixv4mHNfWwiMjH9Pc5A0Odc3sAvN+F3vK1wK0AZjYLGAkUA0VAVZftq71lIqdk3pJyivMGcP05mnpYRKS7WDmB8DtAnpmtAb4MrAY6gJ6+wrmedmBmj5jZCjNbUVtbG71KJXBWVdazfGc9n79kNCnJsfKSFxGJHSn9/O/tM7Phzrk9ZjYcqAFwzh0GHgSwcB/uDu9nINB1vthiYHdPO3bOzQXmAsycObPHwCCJad7icrIzUrhzpqYeFhHpSX9/TXoVuN+7fT/wCoCZ5ZpZmrf8IWCxFxCWA+PMbLT3+F3ePkQiUnGgiTc27OWe2SMZlN7f2VdEJBhOGgbMbKiZPWVmr3v3J5nZ5yPY7nngA+BsM6v2tvkOcI2ZbQWu8e4DTAQ2mNkmwiMHvgrgnOsAvgS8CWwEFjrnNpxqIyVxPfXuDlKSjAcuGuV3KSIiMSuSr0rPAD8DvuXd3wL8Aniqt42cc3NO8NBVPaz7ATDuBPt5DXgtgjpFjlPf1MbCFVXcPLWIwuwMv8sREYlZkRwmGOKcWwiE4Ni39c6oViXSB+aXVdDSHuLhyzTJkIhIbyIJA01mNhjvLH4zmw0cimpVImeopb2TZz/YyRVnFzB+aJbf5YiIxLRIDhP8DeGT9s4ys/eAAuD2qFYlcoZ+vXoX+xvbeERTD4uInNRJw4BzbpWZXQ6cTXjc/2bnXHvUKxM5TaGQY96Scs4Zkc2FZw32uxwRkZgXyWiCLwKZzrkNzrn1QKaZ/XX0SxM5PX/cXMP22iYe0dTDIiIRieScgYedcweP3nHO1QMPR68kkTMzd3E5I3IyuHHycL9LEREJhEjCQJJ1+XrlXUkwrZf1RXyztuogS3fU8blLRpOqqYdFRCISyQmEbwILzeynhEcU/CXwRlSrEjlN85aUk5WewmfO19TDIiKRiiQM/B3wBeCvCJ9A+HvgyWgWJXI6quqaeW3dHh6+dAxZGal+lyMiEhiRjCYIAT/xfkRi1tPv7SDJjAcuHuV3KSIigXLCMGBmC51zd5rZOnq4bLBz7ryoViZyCg41t/OL5VV8esoIhucM8LscEZFA6a1n4Kve70/2RyEiZ2LBsgqa2zo19bCIyGk4YRhwzu3xRg485Zy7uh9rEjklrR2dPPPeTi4dN4SJw7P9LkdEJHB6HXvlnOsEms0sp5/qETllr67ZTU1DK4+oV0BE5LREMpqgBVhnZouApqMLnXNfiVpVIhFyLjz18IRhWVwydojf5YiIBFIkYeB33o9IzHlnSy1b9jXy2J1TNPWwiMhpimRo4bNmlgZMIDyqYLNzri3qlYlEYN6ScoZlZ/DJ80b4XYqISGBFcqGiG4HtwA+AHwHbzOyGaBcmcjLrdx3ivW0HePDiUaSlaOphEZHTFclhgseATzjntgGY2VmEDxu8Hs3CRE7mySXlZKanMOeCUr9LEREJtEi+TtUcDQKecqAmSvWIRGT3wSP85sM93HV+CdmaelhE5IxE0jOwwcxeAxYSPmfgDmC5md0K4Jx7OYr1iXzM+9v386+/3QjAg5eM9rkaEZHgiyQMZAD7gMu9+7VAPvApwuFAYUD6xea9DXzn9Y38cXMtI3Iy+OGcaRTlauphEZEzFcloggf7oxCRE9l7qIXHFm3mpZXVDEpP4dEbJvDARaPISE32uzQRkbgQSc+AiC8Ot7Tz3+9s56l3dxAKwYMXj+ZLnxhL3qA0v0sTEYkrCgMSc9o6QixYWsEP3t5KfXM7N00dwTeuPZuS/IF+lyYiEpdOGgbMLN0519ptWb5zri56ZUkics7xu3V7+N4bm6msa+bCMYP5hxsnMrlYl8YQEYmmSHoGXjazm51z7QBmNhz4LTAjqpVJQikrP8C3X9/E2qqDnD00i589eD5XjC/QFMMiIv0gkjDwa+BFM7sNKAFeBb4R1aokYWzd18B339jEWxtrGJadwfdvP49bpxeTnKQQICLSXyIZTTDPuzbBr4FRwBecc+9HuzCJb/sOt/D4oi0sXFHFoLQU/vb6s3nwotEMSNMIARGR/nbCMGBmf9P1LuFegTXAbDOb7Zx7LNrFSfxpaGln7uJy5i0ppzPkuP+iUXz5ynHka4SAiIhveusZyOp2/1cnWC5yUu2dIZ5fVskTb23lQFMbnzxvON+87mxGDh7kd2kiIgnvhGHAOfd/+7MQiU/OOV5fv5fvv7mZHfubuGB0Pk/fOJEpJbl+lyYiIp5IhhYuAu5wzh307ucBLzjnrot2cRJsy3fW8e+vbWR15UHGFWby9AMz+cTZhRohICISYyIZTVBwNAgAOOfqzawwijVJwG2raeS7b2xi0Uf7GJqdzndvm8xt04tJSY7kIpkiItLfIgkDnWZW6pyrBDCzkYQvUCRynJqGFv7zra38YnkVA1KT+ca14/ncJaMZmKaJLkVEYlkk79LfAt41s3e8+5cBj0SvJAmaxtYO5nkjBNo6Qtw7eyRfvnIsgzPT/S5NREQiEMk8A2+Y2XRgtrfo6865/dEtS4KgvTPEC8ureOKtLexvbOMvJodHCIwaohECIiJBEmn/7UWEewSO+m0UapGAcM7x5oZ9fO+NTZTvb2LWqHzm3jeB6aV5fpcmIiKnIZLRBN8BzgcWeIu+amYXO+f+PqqVSUxaWVHHv7+2iZUV9YwtzGTefTO5eqJGCIiIBFkkPQM3AlOdcyEAM3sWWA0oDCSQ8tpGvvfGZt7YsJeCrHS+fetk7pihEQIiIvEg0sMEucDRSxbrerIJpLahlSfe3sLzy6rISEnib64Zz0OXaoSAiEg8ieQd/dvAajP7I+FrFFwG/ENUqxLfNbV28OSSHcxdvJ2WjhCfnVXKV64aR0GWRgiIiMSbSEYTPG9mfyJ83oABf+ec2xvtwsQfHZ0hFq6o5vG3tlDb0Mr15wzjm9efzVkFmX6XJiIiURLJCYRvO+euAl7tYZnECecciz7ax3ff2MT22iZmjMzjp/dMZ8bIfL9LExGRKOvtEsYZwEBgiHc9gqOni2cDI/qhNuknqyrr+fZrG1m+s54xQwbx3/fO4NpJQzVCQEQkQfTWM/AF4GuEP/hX8ucwcBj4cZTrkn6wY38T339zE6+t28uQzHT+9eZz+cz5JaRqhICISELp7RLGTwBPmNmXnXM/7MeaJMr2N7byw7e3smBpJWkpSXz1qnE8ctkYBqVrhICISCKK5N1/r5llOecazOx/AdOBf3XOrYpybdLHOkOOn76znZ/8aTtH2ju56/wSvnr1OAqzMvwuTUREfBRJGPhH59yLZnYJcB3w/4CfABdEtTLpcy+trOL7b27m6olDefSGCYwt1AgBERGBSA4Od3q//wL4iXPuFSAteiVJNDjneK6sgvFDM5l33wwFAREROSaSMLDLzP4buBN4zczSI9xOYsja6kOs33WYe2aP1CgBERE5TiQf6ncCbwLXO+cOAvnAN6NalfS5+WUVDExL5pZpRX6XIiIiMSaSGQibgZe73N8D7IlmUdK3Dja38Zu1u7ltRjFZGal+lyMiIjFG3f0J4KWV1bR2hLjngpF+lyIiIjFIYSDOhUKOBUsrmV6ay6QR2X6XIyIiMUhhIM69v/0AO/Y3cc9s9QqIiEjPFAbi3PyyCvIGpnLj5OF+lyIiIjFKYSCO7T3UwqKN+7hjZgkZqcl+lyMiIjFKYSCOvbC8ks6Q47OzSv0uRUREYpjCQJzq6AzxwrIqLh03hFFDBvldjoiIxDCFgTj11sYa9h5u0YmDIiJyUgoDcWrB0gqG52Rw1YRCv0sREZEYF7UwYGZPm1mNma3vsizfzBaZ2Vbvd563PMfMfmNma81sg5k92GWb+731t5rZ/dGqN57s2N/Ekq37mTOrlJRk5T0REeldND8pngGu77bsUeBt59w44G3vPsAXgY+cc1OAK4D/MLM0M8sH/onw5ZJnAf90NEDIif18aQUpScZd55f4XYqIiARA1MKAc24xUNdt8U3As97tZ4Gbj64OZFn4cnqZ3nYdwHXAIudcnXOuHljExwOGdNHS3smLK6u59pyhFGZn+F2OiIgEwEkvVNTHhnoXOsI5t8fMjh7Q/hHwKrAbyAI+45wLmVkRUNVl+2pAl93rxe8+3MPB5nZdh0BERCIWKweUrwPWACOAqcCPzCwbsB7WdT3twMweMbMVZraitrY2epXGuPlLKxhTMIgLzxrsdykiIhIQ/R0G9pnZcADvd423/EHgZRe2DdgBTCDcE9D1wHcx4d6Dj3HOzXXOzXTOzSwoKIhaA2LZ+l2HWF15kLsvGEn4iIuIiMjJ9XcYeBU4OiLgfuAV73YlcBWAmQ0FzgbKgTeBa80szztx8FpvmfRgwdIKMlKTuH16sd+liIhIgETtnAEze57wyIAhZlZNeFTAd4CFZvZ5wgHgDm/1fwGeMbN1hA8N/J1zbr+3n38Blnvr/bNzrvtJiQIcbmnn16t386nzRpAzMNXvckREJECiFgacc3NO8NBVPay7m/C3/p728zTwdB+WFpd+tWoXR9o7NeOgiIicslg5gVDOgHOO+WUVTC7KYUpJrt/liIhIwCgMxIFlO+rYWtPIPbN1dUIRETl1CgNxYP7SSrIyUvj0FE3BICIip05hIOBqG1p5Y/0ebp9RzIC0ZL/LERGRAFIYCLiFK6po73TcrRkHRUTkNCkMBFhnyPHzpZVcOGYwYwsz/S4iKFGcAAARwklEQVRHREQCSmEgwN7ZUsOug0c0nFBERM6IwkCAzS+rpCArnWvPGep3KSIiEmAKAwFVVdfMHzfXcNf5JaQm62kUEZHTp0+RgPr5skoMmDNLcwuIiMiZURgIoNaOThYur+LKCUMZkTvA73JERCTgFAYC6I31eznQ1KYZB0VEpE8oDATQgrJKSvMHctm4Ar9LERGROKAwEDCb9zawbGcdd19QSlKS+V2OiIjEAYWBgFmwtIK0lCTumFnidykiIhInFAYCpKm1g5dX7eIvJg8nf1Ca3+WIiEicUBgIkFfW7KaxtUMnDoqISJ9SGAgI5xzzyyqYMCyL6aV5fpcjIiJxRGEgIFZXHeSjPYe5Z/ZIzHTioIiI9B2FgYCYX1bBoLRkbp5W5HcpIiISZxQGAqC+qY3ffriHW6YXkZme4nc5IiISZxQGAuDFlVW0dYR0qWIREYkKhYEYFwo5FiytZObIPCYMy/a7HBERiUMKAzHu3W37qTjQrF4BERGJGoWBGDe/rIL8QWncMHmY36WIiEicUhiIYXsOHeGtjfu4c2YJ6SnJfpcjIiJxSmEghj2/rAoH3H2BZhwUEZHoURiIUe2dIV5YVsnl4wsoyR/odzkiIhLHFAZi1Fsf7aOmoZV7LtCJgyIiEl0KAzFq/tIKinIH8IkJhX6XIiIicU5hIAaV1zby3rYDzJlVQnKSrkMgIiLRpTAQgxYsrSQlybjz/BK/SxERkQSgMBBjjrR18uKKKq47dxiFWRl+lyMiIglAYSDG/ObD3Rxu6dCJgyIi0m8UBmLMgrIKxhZmMntMvt+liIhIglAYiCHrqg+xtvoQ91xQiplOHBQRkf6hMBBD5pdVMCA1mVtnFPtdioiIJBCFgRhx6Eg7r6zdxU1TR5Cdkep3OSIikkAUBmLEy6uqaWkP6VLFIiLS7xQGYoBzjgVLK5lSksu5RTl+lyMiIglGYSAGlJXXsa2mkXt0dUIREfGBwkAMmL+0gpwBqXxqygi/SxERkQSkMOCzmoYW3ly/l9tnFJORmux3OSIikoAUBny2cHkVHSHH3TpEICIiPlEY8FFnyPHzpZVcPHYwYwoy/S5HREQSlMKAj/6wqYbdh1p0HQIREfGVwoCP5pdVUJiVztWThvpdioiIJDCFAZ9UHmhm8dZa5swqJTVZT4OIiPhHn0I+WbCsgiQz5szSiYMiIuIvhQEftHZ08uKKaq6eWMiwnAy/yxERkQSnMOCD19ftpa6pTdchEBGRmKAw4IP5ZRWMGjyQi88a4ncpIiIiCgP9bdPew6yoqOfuC0aSlGR+lyMiIqIw0N/ml1WQlpLE7TOK/S5FREQEUBjoV42tHfxq1S4+ed5w8gal+V2OiIgIoDDQr369ehdNbZ06cVBERGKKwkA/cc4xv6yCScOzmVaS63c5IiIixygM9JOVFfVs2tvAPbNHYqYTB0VEJHYoDPST+WUVZKWncNPUEX6XIiIichyFgX5woLGV19bt5dbpRQxKT/G7HBERkeMoDPSDF1dW09YZ4m6dOCgiIjFIYSDKQiHHz5dWMmt0PuOHZvldjoiIyMdELQyY2dNmVmNm67ssyzezRWa21fud5y3/ppmt8X7Wm1mnmeV7j11vZpvNbJuZPRqteqNl8dZaKuuaNZxQRERiVjR7Bp4Bru+27FHgbefcOOBt7z7Oue8756Y656YCfw+845yrM7Nk4MfADcAkYI6ZTYpizX1uflklQzLTuP6cYX6XIiIi0qOohQHn3GKgrtvim4BnvdvPAjf3sOkc4Hnv9ixgm3Ou3DnXBrzg7SMQdh08wh827ePOmSWkpeiIjIiIxKb+/oQa6pzbA+D9Luz6oJkNJNyb8EtvURFQ1WWVam9ZILywrBIHzJlV6ncpIiIiJxRrX1c/BbznnDvao9DT7Dyupw3N7BEzW2FmK2pra6NWYKTaO0O8sLyKT5xdSEn+QL/LEREROaH+DgP7zGw4gPe7ptvjd/HnQwQQ7gko6XK/GNjd046dc3OdczOdczMLCgr6sOTT8/sN+6htaOWe2eoVEBGR2NbfYeBV4H7v9v3AK0cfMLMc4PKuy4DlwDgzG21maYTDwqv9VOsZea5sJ0W5A7h8fOHJVxYREfFRNIcWPg98AJxtZtVm9nngO8A1ZrYVuMa7f9QtwO+dc01HFzjnOoAvAW8CG4GFzrkN0aq5r2yraaCsvI67Z5eSnKTrEIiISGyL2ty4zrk5J3joqhOs/wzh4Yjdl78GvNZnhfWD+WWVpCYbd84sOfnKIiIiPou1EwgDr7mtg1+uquaGc4czJDPd73JEREROSmGgj/1m7W4aWjo046CIiASGwkAfm19WyfihmZw/Ks/vUkRERCKiMNCH1lYdZN2uQ9wzeyRmOnFQRESCQWGgD80vq2BgWjK3TAvMJIkiIiIKA33lUHM7v/lwNzdNLSIrI9XvckRERCKmMNBHXlpVTUt7SDMOiohI4CgM9AHnHAuWVjCtNJdzRuT4XY6IiMgpURjoAx9sP0B5bRP3XKDhhCIiEjwKA33gubIKcgem8hfnDfe7FBERkVOmMHCG9h1u4fcf7ePOmSVkpCb7XY6IiMgpUxg4Qy8sq6Iz5PjsLJ04KCIiwaQwcAY6OkM8v6ySS8cNYdSQQX6XIyIicloUBs7A25tq2Hu4RdchEBGRQFMYOAPzyyoYnpPBVRMK/S5FRETktCkMnKad+5tYsnU/d51fSkqy/htFRCS49Cl2mn6+rJLkJOOuWSV+lyIiInJGFAZOQ0t7Jy+uqOLaSUMZmp3hdzkiIiJnRGHgNLy2bg/1ze06cVBEROKCwsBpmF9WwZghg7jorMF+lyIiInLGFAZO0Ybdh1hVeZC7Z4/EzPwuR0RE5IwpDJyi+WWVZKQmcfv0Yr9LERER6RMKA6egoaWdV9bs4lPnjSBnYKrf5YiIiPQJhYFT8KvVu2hu69SJgyIiElcUBiLknGN+WQWTi3KYUpLrdzkiIiJ9RmEgQst31rNlXyP3zNbVCUVEJL4oDERoflkFWRkpfGrKCL9LERER6VMKAxHY39jK6+v3cNv0YgampfhdjoiISJ9SGIjAwhVVtHc6HSIQEZG4pDBwEp0hx8+XVjJ7TD5jC7P8LkdERKTPqc/7JA4faWfi8GxunVbkdykiIiJRoTBwEnmD0ph330y/yxAREYkaHSYQERFJcAoDIiIiCU5hQEREJMEpDIiIiCQ4hQEREZEEpzAgIiKS4BQGREREEpzCgIiISIJTGBAREUlwCgMiIiIJTmFAREQkwSkMiIiIJDiFARERkQRnzjm/a+hzZlYLVPTxbocA+/t4n36Il3aA2hKr4qUt8dIOUFtiVV+3ZaRzruB0NozLMBANZrbCORf4axnHSztAbYlV8dKWeGkHqC2xKpbaosMEIiIiCU5hQEREJMEpDERurt8F9JF4aQeoLbEqXtoSL+0AtSVWxUxbdM6AiIhIglPPgIiISKJzzgXyBygB/ghsBDYAX/WW5wOLgK3e7zxv+QTgA6AV+Ea3fX3d28d64Hkg4wT/5v3efrcC93dZ/m9AFdB4kppnAOuAbcAPAPPaUenV1QJ8COQGrR3e8ieARq8djcA/BvE56fb62gc44B8C3JbHury+jgA/DWJbujwne73a9gDfC1o7vOWvAg3ec9IGVAfxOfGWXwcc6vL6+o8gtsV7fS0Dmr22rAeyY7wdPa4HpAO/8Nq3FBjV236cc4EOA8OB6d7tLGALMInwm8Oj3vJHge96twuB873/vG902U8RsAMY4N1fCDzQw7+XD5R7v/O820dfFLO9ek72xC0DLvReeK8DN3jb/TWQ4rWjDngyaO3wlo/r8px8EzgYxOeky+vrBuBNwmFtW4Db8v+Ax+Pkb+UR4C1gsNeOS4LWjh7ev35AeKx54J4Tb/mfgC95t28j/GEauLZ4220ALif8d7IX+EmMt6PH9Qh/phwN/XcBv+htP8654B4mcM7tcc6t8m43EO4hKAJuAp71VnsWuNlbp8Y5txxo72F3KcAAM0sBBgK7e1jnOmCRc67OOVdPOCFe7+27zDm3p7d6zWw4kO2c+8CFn6H/AW722vFfzrkOrx1bgdFBa4e3/dajz4lXSz0BfE687fcAnwf+FggRfl4C2RbCvTS7vH0F+m8FuBr4jnPugNeOAUFrh7f9HufcKjMz4BZgDQF8TryHW4Fa73Ya4S80gWuLt10JsNj7O1kL3Bir7TjJel1rfgm4ynutnVBgw0BXZjYKmEa4O2To0f8c73dhb9s653YR/uZUSbjb8ZBz7vc9rFpEuDvmqGpvWaSKvG1OuL3XjsmEu5UC2Q4z+zczqwIeAFIJ6HNiZp8Gdjnn1hL+w54c1LZ4vmRmH5rZQmB6gNsyHrjUzFYTfjNtD2g7jrqUcGgeT3Cfk68B3/f+7h8n/G07qG1ZD3zaey+eRfhbe6y2ozfH9u2c6yB8GGdwbxsEPgyYWSbwS+BrzrnDp7F9HuEUNRoYAQwys3t6WrWHZe5U/qnetvfa8R7hcwaeOoX9Ht0+JtrhnPsWMBHIBN4L4nNiZgOBbwH/23teCoBvBbEt3u+fAGcBFxP+8NkS4LakEH4zDgH/G/jZKew3vPPYaMdR9xLuIg7y+9dfET5GPpHwIYL9AW7L54CvED5c8Cbh8zlOST+2o9cyTnXfgQ4DZpZKOAgscM697C3e53UDHe0OqjnJbq4Gdjjnap1z7cDLwEVmdoGZrfF+Pk04tZV02a6Ynrt+jtaW3GX7f/a2L+5pe68dHxD+lnOl13UVuHZ0acsvgaeBc7zFQWvLWYT/kNcS7v5MAf7FzIYFsC045/YR/lt/ifD5KEO9dQLXFsKHO2YQ/pv/HuFQsD+A7cDMMoD7gKeD/P5F+CS4Vwn/3f+I8N9OUNuyHegA/pFw2Nwew+3ozbF9W/hQRQ7hwzcnlHKSHcYsMzPC36A3Ouce6/LQq4RfnN/xfr9ykl1VArO9b4NHgKuAFc65pcDULv9ePvDvXuoDuBb4+xPt1DnX2XV7bx8NZjabcBfafcAPvXa8TjhBTnDONQexHd7ycYT/iDYS/oPZFMS2OOfWmdlQwsfc6ggfI5zpnNtvZoFqi7d8OPBdws9LHeGuUAjY8+L9reQAIefcY2Y2nvAx6heC1A5vuQG/BWqdc//UZZNAPSfeQ7u9tmwkHKC3BrEt3nOywGvHfwLPAD8l3OMRk+3oxdH/+w+A24E/eF8yT8yd5AzDWP0hfBaxI9ytvsb7uZHwcZG3Cb8g3wbyvfWHEU5Lhwmf5V5N+CQSgP9L+INrPfAckH6Cf/NzhM8q3wY82GX597z9hbzf/+cE28/0/o3thBO0dWlHG+EXzhHgtaC1w1v+J68tRwgfo9oQxOekh9dXG+FhSEFtyxvdnpf1QWxLl+ek3mtLM+GzuwPVjm6vr2oC/P7lLf+rLq+vJmBzENvS5Tk5Ogx3XwDa0eN6QAbworffZcCYnrbv+qMZCEVERBJcoM8ZEBERkTOnMCAiIpLgFAZEREQSnMKAiIhIglMYEBERSXAKAyJyyszs/5jZN3p5/GYzm9SfNYnI6VMYEJFouJnwletEJAA0z4CIRMTMvkV4trYqwtM0ryQ8idEjhGcC3EZ4rv2phGekO+T93Obt4seEr/PQDDzsnNuEiMQEhQEROSkzm0F4etYLCE9jvorwVK0/c+HLCWNm/wrsc8790MyeAX7rnHvJe+xt4C+dc1vN7ALg2865K/u/JSLSk8Bem0BE+tWlwK+cd+0M7xoNAOd6ISCX8JUq3+y+oYWv/HgR8KL9+ZLq6VGvWEQipjAgIpHqqRvxGeBm59xaM3sAuKKHdZKAg865SC+yIiL9TCcQikgkFgO3mNkAM8sCPuUtzwL2WPjS1Xd3Wb/BewwXvrb9DjO7A8JX7DOzKf1XuoicjM4ZEJGIdDmBsILwFdI+InyVur/1lq0DspxzD5jZxcA8wleAu53wVdV+AgwHUoEXnHMnuya7iPQThQEREZEEp8MEIiIiCU5hQEREJMEpDIiIiCQ4hQEREZEEpzAgIiKS4BQGREREEpzCgIiISIJTGBAREUlw/x+PRNyK0Gy07wAAAABJRU5ErkJggg==\n", "text/plain": [ "<matplotlib.figure.Figure at 0x7f37eb86b358>" ] }, "metadata": {}, -
pbugnion revised this gist
Jun 5, 2018 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -29,7 +29,7 @@ publish_operator = BashOperator( task_id='publish', bash_command='/opt/anaconda/envs/Python3/bin/python /project/stock-price/publish_report.py', dag=dag ) -
pbugnion revised this gist
Jun 5, 2018 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -10,7 +10,7 @@ 'email_on_retry': False, 'retries': 1, 'retry_delay': timedelta(minutes=5), 'schedule_interval': '@daily' } dag = DAG( -
pbugnion revised this gist
Jun 5, 2018 . 2 changed files with 6 additions and 7 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -36,21 +36,19 @@ Create an environment called `airflow-webserver` with a single shell script: AIRFLOW_HOME=/project/airflow SOURCE=https://gist.github.com/pbugnion/a4a24d6f54fe9f6239aae90569197e9e/raw sudo sv stop jupyter wget --output-document /tmp/airflow-run $SOURCE/run sudo mkdir -p /etc/service/airflow sudo mv /tmp/airflow-run /etc/service/airflow/run sudo chown root:root /etc/service/airflow/run sudo chmod a+x /etc/service/airflow/run sleep 7 # Wait for runit to pick up new service sudo sv start airflow sleep 20 # Wait for airflow to start ``` When this environment is applied to a server in the workspace, it will stop the Jupyter server and replace it with Airflow. You can then click on the server name to open the Airflow dashboard. This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -9,7 +9,8 @@ 'email_on_failure': False, 'email_on_retry': False, 'retries': 1, 'retry_delay': timedelta(minutes=5), 'schedule': '@daily' } dag = DAG( -
pbugnion revised this gist
Jun 5, 2018 . No changes.There are no files selected for viewing
-
pbugnion revised this gist
Jun 5, 2018 . 2 changed files with 85 additions and 2 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1 +1,84 @@ # Airflow on SherlockML [Apache Airflow](https://airflow.apache.org/) is an open source tool for creating task pipelines. It lets you define sets of tasks and dependencies between those tasks, and then takes care of the execution. ## Installing Airflow in SherlockML ### Setting up a project To set up a project, create an environment called `airflow-setup-project` with the following entry in the scripts tab: ```bash AIRFLOW_HOME=/project/airflow SOURCE=https://gist.github.com/pbugnion/a4a24d6f54fe9f6239aae90569197e9e/raw/ source activate Python3 pip install apache-airflow mkdir -p $AIRFLOW_HOME wget --output-document /project/airflow/airflow.cfg $SOURCE/a0ec0a53459bbfd52a61c4eb766cac22c79d5f3e/airflow.cfg echo "export AIRFLOW_HOME=$AIRFLOW_HOME" > /etc/profile.d/airflow.sh airflow initdb ``` Then apply this environment to any Jupyter server within the project. ### Running the Airflow webserver Create an environment called `airflow-webserver` with a single shell script: ``` AIRFLOW_HOME=/project/airflow SOURCE=https://gist.github.com/pbugnion/a4a24d6f54fe9f6239aae90569197e9e/raw mkdir -p $AIRFLOW_HOME wget --output-document /project/airflow/airflow.cfg $SOURCE/airflow.cfg echo "export AIRFLOW_HOME=$AIRFLOW_HOME" | sudo tee /etc/profile.d/airflow.sh sudo sv stop jupyter wget --output-document /tmp/airflow-run $SOURCE/run sudo mkdir -p /etc/service/airflow sudo mv /tmp/airflow-run /etc/service/airflow/run sleep 7 # Wait for runit to pick up new service sudo sv start airflow ``` When this environment is applied to a server in the workspace, it will stop the Jupyter server and replace it with Airflow. You can then click on the server name to open the Airflow dashboard. ### Install examples This gist contains an example with an airflow pipeline that fetches the Google stock price for a given date, saves it in a CSV in the SherlockML workspace (we could also save it in, say, InfluxDB) and publishes a SherlockML report with a graph of the stock price over time, including that date. To install the demo, run the following commands in a terminal: ```bash SOURCE=https://gist.github.com/pbugnion/a4a24d6f54fe9f6239aae90569197e9e/raw mkdir -p /project/stock-price /project/stock-price/data wget --output-document /project/stock-price/fetch_stock.py $SOURCE/fetch_stock.py wget --output-document /project/stock-price/publish_report.py $SOURCE/publish_report.py wget --output-document /project/airflow/stock_price.py $SOURCE/stock_price_dag.py ``` You can then run the pipeline by running the following command in a *new* terminal on the instance running the *airflow* webserver: ``` airflow backfill stock_price -s 2018-03-01 -e 2018-03-06 ``` This will fetch the stock price for dates between the 1st and the 6th of March 2018. This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -10,7 +10,7 @@ logging.basicConfig(level=logging.INFO) QUANDL_URL = 'https://www.quandl.com/api/v3/datasets/WIKI/{}.json' DATASTORE = Path('/project/stock-price/data') Configuration = namedtuple( 'Configuration', -
pbugnion revised this gist
Jun 5, 2018 . 1 changed file with 101 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,101 @@ { "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "from pathlib import Path\n", "\n", "import matplotlib.pyplot as plt\n", "import pandas as pd" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "stock_price = 'GOOG'\n", "datastore = Path('/project/data')" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "start date: 2018-01-04 00:00:00\n", "end date: 2018-02-23 00:00:00\n" ] } ], "source": [ "with open(datastore / f'{stock_price}.csv') as f:\n", " df = pd.read_csv(\n", " f, parse_dates=['date'], infer_datetime_format=True\n", " )\n", "\n", "print(f'start date: {min(df.date)}')\n", "print(f'end date: {max(df.date)}')" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAf4AAAF3CAYAAABE0Ck1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3Xd4XNW59/3vrW4126qustywDTY2trDp2NiUEBLSIBhCOZCQAklO8pD25iR5Us5zOKknJCSEEAfIARMSakILNsV0d3DvtlxkdduyZNVZ7x+zZYSsMpI12jOj3+e65pqZNXv23NuSdc9ae617m3MOERERGRji/A5ARERE+o8Sv4iIyACixC8iIjKAKPGLiIgMIEr8IiIiA4gSv4iIyACixC8iIjKAKPGLiIgMIEr8IiIiA4gSv4iIyACS4HcA4ZKTk+MKCwv9DkNERKRfrFq1qsI5l9vddjGb+AsLC1m5cqXfYYiIiPQLM9sTynYa6hcRERlAlPhFREQGECV+ERGRAUSJX0REZABR4hcRERlAlPhFREQGECV+ERGRAUSJX0REZABR4hcRERlAlPhFREQGkLAlfjNbZGZlZra+TdtVZrbBzAJmVtSm/TozW9vmFjCzGd5rr5jZljav5YUrZhERkVgXzlr99wO/BR5s07Ye+ATwh7YbOuceAh4CMLNpwFPOubVtNrnOOafC+yInwTnH2r2HONrQTHyckRgfF7yPiyMh3kiIMxLi47x7IyEujsR4+8C2CXGGmfl9KCJyEsKW+J1zy8yssF3bJqC7PxwLgcXhiktkoPrdKzv42QtbTno/CXHvfxk4/oXB+/KQGB/HjWeP4aZzx/ZBxCISDpF4db5PA1e2a/uzmbUAjwE/cc65/g9LJHo9sWYfP3thCx+dPoLrzx5Dc4ujORDw7h3NLYHgfSBAU4ujxWtrfdzU0bat+2hta3G8u+8Qv166jWtmF5CSGO/3YYtIByIq8ZvZHKDOObe+TfN1zrn9ZpZBMPFfzwdPH7R9/63ArQAFBQXhDlckKry5o4Jv/v09zh6Xzc+vmk5SQvjm9L62rZzr/7ScFzYc5MoZI8P2OSLSe5E2q/8a2g3zO+f2e/c1wMPA7M7e7Jy71zlX5Jwrys3NDWugItFga2kNn//LKgqz07jn+llhTfoA547PoSArlYfeKQ7r54hI70VM4jezOOAq4JE2bQlmluM9TgSuIDhBUES6UXqknpsWLWdQYjz33zybwYMSw/6ZcXHGwtkFLN9VxfaymrB/noj0XDiX8y0G3gImmdk+M7vFzD5uZvuAs4FnzOyFNm+5ANjnnNvZpi0ZeMHM3gPWAvuBP4YrZpFYcbShmZvvX8GhY00suulMRg4Z1G+ffVXRKBLjjYff2dtvnykioQvnrP6Fnbz0RCfbvwKc1a6tFpjVt5GJxLbmlgC3PbSazQdruO/GIqaOHNyvn5+Tnswlpw3jsdX7+OZlkzTJTyTCRMxQv4icPOcc//Hkel7dWs5PPjaVeZP8qXd13ewCDh9r4tl1Jb58voh0TolfJIb87pUdPLJiL7fPm8DC2f6tbDl7fDZjc9J4WJP8RCKOEr9IhKhrbGZ72dFev/+Z90r42Qtb+PgZI/k/l5zSh5H1nJmxcPZoVu6pZmtp6JP8Dh9rYkd57/8NRKR7SvwiEeJH/9jIpf+zjNe3VfT4vdvLavjG399l1pih3PnJaRFRVvdTs0aTFB8Xcq+/vqmFa//4NvN/8So3LFrOWzsqUa0ukb6nxC8SARqbAzy7roSWgONLD61iZw96vTX1Tdz6l1WkJsVz97UzSU6IjMl0WWlJXDY1OMnvWGNLt9v/8B8b2XDgCJ85q4CNBw6z8I9v8/HfvckLGw4SCOgLgEhfUeIXiQCvbSvnSH0zP77yNBLi4/jsAys5XNfU7fucc3zz7++xp7KO3yycybDBKf0QbeiunVNATX0z/3jvQJfbPbZqH4uXF/OlueP5ycem8fq3LuInH5tKVW0jn//LKi7+1as8unIvjc2BfopcJHYp8YtEgGfeK2HwoEQ+fWYBf7h+Fnur6/jSw6toauk60f3xtZ08t/4g375sMmePz+6naEM3Z2wW43O7nuS3+eARvvvkOs4al8XXLw7OTUhJjOczZ43hpf9zIXctPIPkhHi++ff3uOCnL/PYqn39Fb5ITFLiF/FZfVML/9pYyqWn5ZOUEMeZhVn8v49P443tlfzoHxs7fd+bOyq487nNXD5tGJ89PzKvhhec5FfA2r2H2HjgyAmv19Q38cX/XU1GSiJ3LTyDhPgP/klKiI/jo9NH8MxXzuOBm2eTnZ7ENx97Tz1/kZOgxC/is2Vbyzna0MyHTx9xvO2qotF8/sJx/OXtPTz41u4T3lNy+BhffngNY3PS+OmnpkfEZL7OfGrWKJIS4nh4+Z4PtDvn+PZj6yiuquO3C88gL6Pz0xRmxoWn5HLdnDG0BBxVtY3hDlskZinxi/jsn++VMDQ1kXPaDdV/89LJLJiSzw//sZFlW8uPtzc2B/jSQ6upb2rhD9fPIj05oi6yeYIhqUl8eNpwnlxzgNqG5uPt97+5m2fWlfCNSycxZ1xopymy05MAqDjaEJZYRQYCJX4RH9U3tbBkUymXTR1GYrth7vg443+umcHEvHRue3j18TX+P3lmI2uKD/Gzq6YzIS/Dj7B77No5BRxtaOYf7wYn+a3aU81/PrOJBVPyufX8cSHvJ8dL/JXq8Yv0mhK/iI9e3lxGXWMLV7QZ5m8rPTmB+24sIjkhjs8+sII/v7GLB9/aw+fOH8vl04b3c7S9VzRmKBPz0nl4eTFVtY3c/vBqhg9J4RdXTScuLvTTFFlpyQBUqscv0mtK/CI++ue6ErLTkpgzNqvTbUYNTeUP1xdx4FA9P/zHRuaMzeJbl03uxyhPnplx7ZwC3tt3mOvue4fK2kZ+f90sBqf27FLBrUP9lUfV4xfpLSV+EZ/UNTbz0qYyPjRt2Amz2dubNWYov/z0dOaMzeI31544+z0afOKMUSQnxLGp5Aj/9yOn9eqqgRnJCSTFx1FRqx6/SG9F9qwgkRj20uYyjjW18OFpHQ/zt3fF6SM6PSUQDQanJvKNSydx5FgTC2eP7tU+zIzs9CT1+EVOghK/iE/++W4JuRnJzO5imD/WfLYHE/k6E0z86vGL9Fb0jReKxICjDc28vKWMy6cOI74Hk9sEstOSNatf5CQo8Yv4YOmmUhqaA1wxPXqH7v2ioX6Rk6PEL+KDf75XwrDMFGYVDPU7lKiTk55MZW2DLtkr0ktK/CL97Eh9E69uKefyacN7tIZdgrLTkqhvClAXwqV+ReRESvwi/WzJxlIaWwJ8+PToKcATSbLTW4v4aLhfpDeU+EX62T/fK2HkkEHMLBjidyhR6Xi9fq3lF+kVJX6RfnS4ronXtpVz+bRhEX1FvUiWk6Yev8jJUOIX6UcvbDxIU4uL6kI8fnu/bK96/CK9ocQv0o+eea+E0VmDOH1Uz8vVSlBWmq7QJ3IylPhF+kl1bSNvbK/gw9NGaJj/JKQkxpORnECFevwivaLEL9JPXthwkOaA4wrN5j9pKuIj0ntK/CL94GhDM39YtpNxuWmcNiLT73CiXrZXxEdEek6JX6QffP/J9eyprOX/fXyahvn7QHaaevwivaXELxJmj63ax+Nr9vOV+RM5a1y23+HEhOz0ZCqU+EV6RYlfJIx2lB/le0+tZ87YLL580US/w4kZOelJVNU2EAioXr9ITynxi4RJfVMLtz+8huSEOH59zRm6/G4fyk5LIuDg0LEmv0MRiTpK/CJh8l/PbmJTyRF+cfV0hg1O8TucmJJ1vF6/JviJ9FTYEr+ZLTKzMjNb36btKjPbYGYBMytq015oZsfMbK13u6fNa7PMbJ2ZbTezu0wzoyQKvLDhIA+8tYdbzhvLRZPz/Q4n5uR4RXx0nl+k58LZ478fuKxd23rgE8CyDrbf4Zyb4d2+0Kb998CtwETv1n6fIhFl/6FjfPPv7zFt5GC+ddlkv8OJScev0KclfSI9FrbE75xbBlS1a9vknNsS6j7MbDiQ6Zx7yznngAeBj/VtpCJ9p7klwFcWr6El4PjNwjNIStDZtHB4v16/evwiPRVJf5XGmtkaM3vVzM732kYC+9pss89rE4lI/7NkG6v2VPOfH59KYU6a3+HErKGpSZjpHL9IbyT4HYCnBChwzlWa2SzgSTM7DejofH6n63fM7FaCpwUoKCgIS6AinXl9WwV3v7KdTxeN5soZ+n4aTvFxRlZqEhW6UI9Ij0VEj9851+Ccq/QerwJ2AKcQ7OGParPpKOBAF/u51zlX5Jwrys3NDWfIIh9QXtPA1x5dy/jcdH7w0VP9DmdACNbrV49fpKciIvGbWa6ZxXuPxxGcxLfTOVcC1JjZWd5s/huAp3wMVeQEgYDj64+u5cixJn577RmkJkXKQFpsy05L1jl+kV4I53K+xcBbwCQz22dmt5jZx81sH3A28IyZveBtfgHwnpm9C/wd+IJzrnVi4BeB+4DtBEcCngtXzCK9ce9rO3ltWwXf/8ipTB6mC/D0l+z0JCo11C/SY2HrmjjnFnby0hMdbPsY8Fgn+1kJTO3D0ET6zOrian7+whYunzaMa2drXkl/yklP1lC/SC9ExFC/SDQ6fKyJryxew7DBKfzXJ07XVff6WXZaEkfqm2lsDvgdikhUUeIX6QXnHN9+7D0OHq7nroVnMHhQot8hDTitRXyqNNwv0iNK/CK98PDyYp5bf5A7Lp3EzIKhfoczILUW8anQcL9Ijyjxi/TQ5oNH+NE/NnLBKbncev44v8MZsHJaq/epxy/SI0r8Ij1Q19jM7Q+vIXNQIr+8ejpxutSub7LTdIU+kd7QgmORHvjh0xvZUX6U/71lDjneOWbxh+r1i/SOevwiIXpq7X7+unIvt82dwLkTcvwOZ8BLT04gKSGOCl2hT6RHlPhFQrC7opbvPrGeojFD+fcFE/0ORwAzIyctST1+kR5S4hfpRmNzgC8vXkN8nPHrhWeQEK//NpEiW0V8RHpM5/hFuvHfz29m3f7D/OH6WYwcMsjvcKQNle0V6Tl1XUS6sHRTKX96fRc3nj2GS08b5nc40o4u1CPSc0r8Ip04eLieO/72LqcOz+Q7l0/xOxzpQE56EhVHG3DO+R2KSNRQ4hfpQEvA8dVH1tDQHOA3155BSmK83yFJB7LSkmhoDlDb2OJ3KCJRQ4lfpAO/eWkb7+yq4sdXTmV8brrf4UgnWuv1a4KfSOiU+EXaeXtnJXct3cYnZo7kk7NG+R2OdOH9ev06zy8SKiV+kTaqahv56iNrKMxO48dXTvU7HOlGjsr2ivSYlvOJeJxz3PG3d6mubWLRTWeSlqz/HpEuWxfqEekx9fhFPIve2M1Lm8v47oencNqIwX6HIyHISmut168ev0iolPhFgPf2HeLO5zZxyan53HD2GL/DkRClJMaTkZygc/wiPaDELwNeTX0TX168htz0ZH76qdMx06V2o4mq94n0jE5iyoDmnOO7T6xnX/Ux/nrrWQxJTfI7JOkh1esX6Rn1+GVA+9vKfTz97gG+tmAiRYVZfocjvZCtK/SJ9IgSvwxY28tq+P7T6zl3QjZfnDvB73Ckl7LTkzXUL9IDSvwyINU3tXD7w2tIS0rgV1fPID5O5/WjVU56ElW1DQQCqtcvEgolfhmQfvzPjWw+WMMvPz2DvMwUv8ORk5CdlkTAwaFjTX6HIhIVlPglJuwoP8qHfv0a//38ZsqO1He57bPrSnjonWI+f+E4Ljwlt58ilHBRvX6RntGsfokJd7+0ne1lNWw+eIQ/vbaLj58xks9dMI4JeR+8wM7eqjq+9dh7zBg9hDsumeRTtNKX2tbrn5jvczAiUUCJX6Levuo6nnr3ADedU8j1Z43hvtd38reV+/jryr0smJLHrReM58zCoTQHHF95ZA0Av1l4BonxGvCKBTmtPf5a9fhFQqHEL1Hvvtd2YcAt541lxJBB/ORj0/jaglN48K09PPjWbq7+w1vMGD2EUUMHsab4EHdfO5PRWal+hy19JPt42V7N7BcJhRK/RLXKow08sqKYj50xkhFDBh1vz05P5msXn8IXLhzP31bt5b7XdrF27yGunVPAh08f7mPE0teGpCYRZzrHLxIqJX6Jag+8tYf6pgBfuHBch68PSornhrMLuW7OGNbureb0UUP6OUIJt/g4IystiQqt5RcJiRK/RK3ahmYeeHM3l5yaz4S8jC63jY8zZo1RZb5YlZ2msr0iodLsJolai5cXc/hYE1+YO97vUMRn2ekq2ysSKiV+iUqNzQHue20XZ43LYmbBUL/DEZ9lpekKfRJdiivraG4J+PLZYUv8ZrbIzMrMbH2btqvMbIOZBcysqE37xWa2yszWefcXtXntFTPbYmZrvVteuGKW6PHk2v0cPFKvGvsCBJf0VWioX6JEIOD46N2v8/2nN/jy+eHs8d8PXNaubT3wCWBZu/YK4CPOuWnAjcBf2r1+nXNuhncrC0ewEj0CAcc9r+7g1OGZXDAxx+9wJAJkpyVRU99MQ3OL36GIdGt7+VEO1TVxxmh/JhuHLfE755YBVe3aNjnntnSw7Rrn3AHv6QYgxcySwxWbRLcXN5Wys7yWL84dj5kuriPvl+2t0nC/RIEVu4Op8UyfLgUeief4Pwmscc61Hbf7szfM/z3TX/oBzTnH717ZwZjsVD40dZjf4UiEaC3bqwl+Eg1W7KoiNyOZMdn+FBKLqMRvZqcB/w18vk3zdd4pgPO92/VdvP9WM1tpZivLy8vDG6z44u2dVby79xCfO38cCSq5K56c4/X6dZ5fIt+K3dXMLszybcQyYv5ymtko4AngBufcjtZ259x+774GeBiY3dk+nHP3OueKnHNFubm66los+v2rO8hJT+ZTs0b5HYpEkOy01iv0qccvkW3/oWPsP3SMMwv9W40UEYnfzIYAzwDfcc690aY9wcxyvMeJwBUEJwjKALR+/2GWbS3n5vMKSUmM9zsciSDHh/p1oR6JcCu98/tFPp3fhzBW7jOzxcBcIMfM9gE/IDjZ7zdALvCMma11zl0K3A5MAL5nZt/zdnEJUAu84CX9eGAJ8MdwxSydW1NczYYDR0hKiCMpPu74fWK750kJ798S443k+Pjjj092aP6eV3eQkZzAZ84a00dHJbEiPTmBpIQ49fgl4i3fVUVGcgJThmf6FkPYEr9zbmEnLz3RwbY/AX7Syfaz+iwo6ZVAwHHrX1ZRXnNyvak4w/sSEEdyR18c2r/Wpj0hznh2XQm3XjCezJTEPjoyiRVmRk5aEhVK/BLhVuyuYuaYocTH+TdPXbX6pVsbDhyhvKaBH370NC6anEdTS4DGlgCNzQGaWgI0NLc+djQ2B2hsaaGp2dHQZptGb5vW9zV28drRhuYPtDe1OBqaAxRkpXLzeYV+/3NIhMpOT9ZQv0S06tpGtpYe5coZI32NQ4lfuvXKlmDNpMunDSc3Q+UVJDKpXr9EupV7qgH/1u+3iojJfRLZXtlazrSRg5X0JaJlpyWrgI9EtJW7q0iKj+P0UYN9jUOJX7p0qK6RNcXVzJ2k5ZES2XLSk6g42oBzzu9QRDq0fHcVp48a7PuqJCV+6dJr2yoIOJT4JeJlpyfR0BygtlH1+iXyHGtsYd2+w5w51t9hflDil268sqWcIamJzBitS99KZHu/iI8m+EnkWbO3muaAY7bP5/dBiV+6EAg4Xt1axvkTc31deiISiuzjZXt1nl8iz4pd1ZjBzDH+d6KU+KVTGw4coeJoI3NP0TC/RL6cdPX4JXKt3FPFpPwMBg/yvw6JEr90qnUZ3wVK/BIF3i/bqx6/RJbmlgCr91QzOwLO74MSv3RBy/gkmmSltV6aVz1+iSwbS45Q29ji+/r9Vkr80qHWZXzzNJtfokRyQjwZKQk6xy8RZ/mu4IV5lPgloi3zlvFdOCnP71BEQpaTnqyhfok4K3dXMzprEMMGp/gdCqDEL514ZUuZt4xviN+hiIQsKy1JQ/0SUZxzrNhdFTG9fVDilw4EAo5lW8u1jE+iTnaa6vVLZNlZUUtlbWNErN9vpcQvJ9AyPolWukKfRJoVref3I2RGPyjxSwdal/FdqIl9EmVy0pOoqm2kJaB6/RIZlu+uIjstiXE5aX6HcpwSv5zg5S1lnD5q8PGCKCLRIjstiYBDV+mTiLFydzVFhUMxi5zTpkr88gGH6hpZu/eQhvklKp06Ini50xW7q3yORGLZgUPH+OTv3+RfGw52uV3pkXqKq+oiamIfKPFLO1rGJ9FsZsEQhqQmsmRjqd+hSAz7xb+2smpPNV96aDXPrSvpdLvW9fuRUrGvlRK/fICW8Uk0S4iPY96kPF7eUqbz/BIWmw8e4fE1+/jMWQVMHz2E2xev4R/vHuhw2xW7q0hNiufU4Zn9HGXXlPjluNZlfBdoGZ9EsQVT8qmua2J1cbXfoUgM+unzW8hITuAbl0zmgZtnM6tgKF99ZA1Prd1/wrbLd1Uxs2AoCfGRlWojKxrx1foDh4PL+DSbX6LYBafkkBhvLNmk4X7pW2/vrOSlzWV8ad4EBqcmkp6cwP03n8mcsdl87a9reWzVvuPbHj7WxJbSmog7vw9K/NLGK1vKAV2NT6JbRkoic8Zm6zy/9CnnHHc+t5lhmSncdE7h8fbUpAQW3XQm54zP4Y6/v8ujK/cCsHpPNc7BmWOH+hRx55T45bhXtIxPYsT8KXnsKK9ld0Wt36FIjHhhw0HW7j3E1y8+hZTE+A+8NigpnvtuLOK8CTl88+/vsXh5Mct3V5EQZ5wxWolfesg5x9bSGh5+p5glG0vZXnaUhuaWPv8cLeOTWLJgSj6AhvulTzS3BPjp81uYmJfOJ2aO7HCblMR4/nhDEfMm5fKdx9fxyPJipo4czKCk+A6391OC3wHIiZxzbCqp4bn1JTy7roQd5R/stcQZjBgyiMLsNMZkp1KYncb4vDQumJjb60kkrcv45k7WMj6JfqOzUpmUn8GSTaV89vxxfocjUe7RlfvYWVHLH28o6vJvbEpiPPdcP4vbHlrNkk1lXFUUeef3QYk/YjjnWL//CM+uL+G5dSXsrqwjzuCscdncdO5Yzp+QQ3VdI7sra9ldUceeylp2VdbxzLoSDtU1ATC7MIvfXHsG+Zk9v/TjK1vKGJqayPRRWsYnsWH+lDz+sGwnh+uaGJya6Hc4EqXqGpv5nyVbKRozlAVTuu8YJSfE87vrZvHgW7v56PQR4Q+wF5T4feScY+3eQzy3/iDPrithX/Ux4uOMc8Zn8/kLx3PJqflktznfXkgaZxSceL7oUF0j/9pYyg+e2sCH73qNX19zBudOyAk5hsdX7+eF9QeZPyVfy/gkZiw4NZ/fvbKDV7aWceWMjodnRbrz5zd2U1bTwO+umxly2d2khLiIHmlS4u9ngYBjdXE1z647yPPrSzhwuJ7EeOO8CTl8Zf5ELp6Sz9C0pB7tc0hqElcXjeaM0UP44kOruf5P7/C1Badw27wJxHWRyLccrOF7T65n+e4qzigYwh2XTDrZwxOJGDNGDSEnPYklm5T4pXeqahu555UdXHxqPkURuCyvt5T4+0FLwLFidxXPrSvhufUHKatpICkhjgsm5nLHpZOYPyWfwYNOfihyYn4GT912Lt99Yh2/eHErK/dU86tPzyCr3ReJ2oZmfr10G396fRcZKQn89yencdWs0V1+SRCJNnFxxrxJeTy/4SBNLQESI6yIikS+u1/eTm1jM9+8NLY6RUr8YdLcEuCdXVU8u66EFzYcpOJoI8kJwXKiH5o2jIsm55GR0vfnHdOSE/jVp2dQVJjFj/6xkSvueo3fXjeTmQVDcc7x/PqD/OifGyk5XM81Z47mm5dNPuGLgUisWHBqPn9btY8Vu6s4Z3xop79EAPZW1fGXt/Zw1azRTMzP8DucPtVt4jezfOD/ASOccx8ys1OBs51zfwp7dFGmqSXAmzsqec5L9tV1TQxKjOeiKXlcPnU4cyflkpYc/u9aZsZnzhrD9FFD+NLDq7j6nrf49wUTWbG7mle3lnPq8Ex+e+1MZo2JvPWlIn3p/Ik5JCXEsWRjmRK/9MivXtyKGfz7xRP9DqXPhZKF7gf+DHzXe74V+CugxO/ZXVHLb1/ezosbSzl8rIn05ATmT8njQ1OHc+Epub6t45w2ajD/vP187vj7u/z8X1vJSE7gBx85levPGhNxtaNFwiE1KYFzxmezdHMp37tiSkRdE10i096qOh5ZUcwTa/fz+QvGM3zwIL9D6nOhJP4c59yjZvYdAOdcs5mFVEHGzBYBVwBlzrmpXttVwP8FpgCznXMr22z/HeAWoAX4inPuBa/9MuDXQDxwn3PuzhCPr1/8f0+sY+3eQ1w2dRiXTx3OeRNzTqjs5JfBqYnce/0slm4KVuXL68VSP5FoNn9KPt97cj07yo8yIS+2hmylbzS1BFi6qYzFy4tZtq0cAy6eks+X5o33O7SwCCXx15pZNuAAzOws4HCI+78f+C3wYJu29cAngD+03dA7hXANcBowAlhiZqd4L98NXAzsA1aY2dPOuY0hxhBWh481sXxXFZ+7YBzfumyy3+F0yMxYcGq+32GI+GLBlDy+9yQs2VSmxC8fsLeqjr+u2MujK/dSVtPAsMwUvnLRRD595mhGDIm9nn6rUBL/14GngfFm9gaQC3wqlJ0755aZWWG7tk1AR0NuVwKPOOcagF1mth2Y7b223Tm303vfI962EZH4X91aTnPAHS8RKiKRZfjgQZw2IpMlG0v5woWx2YOT0HXUu583KY+FswuYO6n31U+jSbeJ3zm32swuBCYBBmxxzjWFIZaRwNttnu/z2gD2tmufE4bP75Wlm0rJTktixmhVvBOJVPOn5PPbl7ZRVduoVSwD1EDt3XcklFn9twEPOec2eM+HmtlC59zv+jiWjmbdODq+kJDrcAdmtwK3AhQUFPRdZJ1oagnw8uYyLjltmCreiUSwi6fkc9fSbby8uYxPzhrldzjST9S771goQ/2fc87d3frEOVdtZp8D+jrx7wNGt3k+CjjgPe6s/QOcc/cC9wIUFRV1+OWgL63cXc2R+mYN84tEuKkjM8lKNQBmAAAgAElEQVTPTGbJplIl/gGgs9791WeOZuQA6913JJTEH2dm5pxrndwXD4RjrOxp4GEz+yXByX0TgeUERwImmtlYYD/BCYDXhuHze2zJplKS4uM4f6LWB4tEMjPjosn5PL12Pw3NLSQnRMaqG+k76t2HLpTE/wLwqJndQ3CI/QvA86Hs3MwWA3OBHDPbB/wAqAJ+Q3CS4DNmttY5d6lzboOZPUpw0l4zcJtzrsXbz+1eHPHAotbTDn5yzrF0UynnTMjul6I8InJyFkzJY/HyYt7ZWcUFp+T6HY70EfXuey6UjPUt4PPAFwn2vv8F3BfKzp1zCzt56YlOtv9P4D87aH8WeDaUz+wvO8pr2V1Zxy0RfAUmEXnfuRNySEmMY8mmUiX+KNdR737upDyuVe8+JKHM6g8Av/du4lmyqRSA+ZO7vz6ziPgvJTGe8ybksnRTGT/8qFMVvyjUUe/+y97MfPXuQ9dp4jezR51zV5vZOjqYRe+cOz2skUW4pZtKOW1E5oBbBiISzRZMyWPJplI2H6xhyvBMv8OREKh33/e66vF/1bu/oj8CiSZVtY2s2lPN7RfF3sUbRGLZRVOCI3RLNpYq8Uc49e7Dp9PE75wr8Wbw/8k5t6AfY4p4L28uI+CCvQcRiR55GSlMHz2EJZvL+PJ8fXGPRIGA47tPrueRFcXq3YdJl+f4nXMtZlZnZoOdc6HW5495SzaVkp+ZzNQRg/0ORUR6aMHkPH7x4lbKaurJy9BFqyKJc44f/XMji5cXc9M5hXzugnHq3YdBKF+f6oF1ZvYnM7ur9RbuwCJVQ3MLy7aWc9HkfOJUrU8k6rResOrlzWU+RyLt/f7VHdz/5m5uOW8sP/jIqUr6YRLKcr5nvJsA7+ysoraxhYtP1TC/SDSaPCyDkUMG8eLGMj59ZvhLe0toHl25l58+v4UrZ4zgu5dP0aqLMAplOd8DZpYETCY4u3+Lc64x7JFFqCWbSklJjOOc8arWJxKNzIz5U/J4dOVe6ptaSElUFT+/LdlYynceX8f5E3P42aemazQ1zLod6jezy4EdwF3Ab4HtZvahcAcWiYLV+so4b0Ku/liIRLH5U/Kpbwrw5o4Kv0MZ8FbtqeK2h1czdUQm93xmFkkJmsAXbqH8C/8SmOecm+ucuxCYB/wqvGFFpk0lNew/dEzD/CJR7qxxWaQlxfPiRp3n99PW0hpuvn8lI4cMYtFNZ6r8eT8JJfGXOee2t3m+ExiQ/1uWetX65qlan0hUS06I54JTcnlpcyne9ceknx04dIwbFy0nOSGOB26eTXZ6st8hDRihJP4NZvasmd1kZjcC/wBWmNknzOwTYY4voizZXMaM0UO0BEgkBsyfkk/pkQbW7z/idygDTnVtIzcsWs7RhmYeuHk2o7NS/Q5pQAkl8acApcCFBK+0Vw5kAR9hAFX1KztSz7t7D6loj0iMmDcplziDF72RPOk/X/3rWoqr6rjvhiJVUPRBKLP6/60/Aol0L3lrfudPyfc5EhHpC9npycwsGMrSTaV8/eJT/A5nwDh8rInXtpVz+7wJzBmX7Xc4A5KmT4ZoyaZSRg4ZxORhGX6HIiJ9ZP6UfDYcOELJ4WN+hzJgrNpThXNoSbSPlPhDUN/UwuvbK1gwJU9FJURiSOupu6WbBuR8ZV+8s6uKxHjjjIIhfocyYIWyjv+EqZZmlhWecCLTG9srqG8KaJhfJMZMyEtnTHYqS3Sev98s31XF6aOGqBaKj0Lp8T9uZomtT8xsOPBi+EKKPEs2lZKenMCccQPq+45IzDMz5k/O580dldQ1NvsdTsyra2xm3b7DzB6rv6V+CiXxPwn8zczizawQeAH4TjiDijQVRxu5cFIuyQn6hioSaxZMyaOxOcBr21TFL9zWFh+iOeCU+H0Wyqz+P3q1+p8ECoHPO+feDHdgkeSPNxTR3BLwOwwRCYMzx2aRkZLAko2lXHraML/DiWnv7KoizmDWmKF+hzKgdZr4zezrbZ8Co4G1wFlmdpZz7pfhDi6SJMRrHqRILEqMj2PupDxe3lJGIOB0gZgwWr6rilNHZJKZktj9xhI2XWWzjDa3dOAJYHubNhGRmLBgSh4VRxtZu++Q36HErMbmAKuLq5ldqLX7fuu0x++c+2F/BiIi4pe5p+QRH2cs2VjKzAINQ4fDuv2HaGgOMHus/n39FspyvhfNbEib50PN7IXwhiUi0n8GpyZyZuFQrecPo+W7qgE4s1AT+/wWyonrXOfc8fEv51w1oIL1IhJTFkzJZ0tpDXur6vwOJSYt31XJhLx0XYUvAoSS+FvMrKD1iZmNAXQdSxGJKa0FulTMp++1BBwrd1drGV+ECCXxfxd43cz+YmZ/AZYxwNbxi0jsG5uTxvjcNA33h8GmkiPUNDQzR4k/InSb+J1zzwMzgb96t1nOOZ3jF5GYs2BKPu/sqqSmvqnX+7jzuc28vFlfHtpavqsK0Pn9SBHq4vRzgLne7axwBSMi4qf5U/JpanEs29q7Kn4lh49xz6s7+NuqvX0cWXRbsbuKUUMHMWLIIL9DEUKb1X8n8FVgo3f7qpn9V7gDExHpbzMLhjA0NbHX5/lf3lwOwM7y2r4MK6o551i+q0rn9yNItyV7gcuBGc65AICZPQCsQef5RSTGJMTHMW9SHi9tKaO5JdDjip0vbQ5+YdhdWasqgJ4d5bVU1jbq/H4ECfW3uu2FkweHIxARkUgwf0o+h+qaWF3csyp+9U0tvLG9ksyUBOqbAhw8Uh+mCKNL6/n92WNVsS9ShJL4/wtYY2b3e739VV6biEjMueCUHBLjrcfD/W/vrORYUwsL5wRXP2u4P2j5rkpyM5IpzE71OxTxhDKrfzHBCX2Pe7ezvTYRkZiTkZLIWeOye5z4X9pcxqDEeK6dHUz8uyqOhiO8qLN8VxWzC7Mw02mPSBHK5L6lzrkS59zTzrmnnHMHzWxpCO9bZGZlZra+TVuWVwJ4m3c/1Gv/hpmt9W7rzazFzLK813ab2TrvtZUnc7AiIqGYPzmPneW17CwPLXk753hpcxnnTsihICuVQYnx7KxQj39fdR0HDtdrYl+E6TTxm1mKl3xzvPr8Wd6tEBgRwr7vBy5r1/ZtYKlzbiKw1HuOc+5nzrkZzrkZBCcNvuqcq2rzvnne60WhHpiISG+1VvELtZjPtrKj7Ks+xvwpeZgZY3PS2KXE3+b8vhJ/JOmqx/95gufzJ3v3rbengLu727FzbhlQ1a75SuAB7/EDwMc6eOtCQKcSRMQ3o7NSmTwsI+Th/pe8gj3zJgUvYzIuN03n+Akm/syUBCbl60rukaTTxO+c+7Vzbixwh3NunHNurHeb7pz7bS8/L985V+Ltv4R2F/sxs1SCowSPtQ0F+JeZrTKzW7vauZndamYrzWxleXl5L0MUEYH5U/JYuaeaQ3WN3W770qYyTh2eybDBKQCMy0ljX3UdDc0t4Q4zorWu39eyxsgSyqz+g2aWAWBm/2Fmj5vZzDDF8xHgjXbD/Oc652YCHwJuM7MLOnuzc+5e51yRc64oNzc3TCGKyEAwf0o+LQHHK1u67kQcrmtiVXE186e8348Zm5tGwDGgr/RXVlPPzopalemNQKEk/u8552rM7DzgUoJD9L/v5eeVmtlwAO++/Qm0a2g3zO+cO+DdlwFPALN7+dkiIiGbMWoIOelJ3Q73v7qtnJaAY97kNok/Jx0Y2Ev6Vu6uBnR+PxKFdFle7/7DwO+dc08BSb38vKeBG73HNxKcLwCAmQ0GLmzXltZmtCENuAQ4vkpARCRc4uKMiybn8erWcppaAp1u99KmUrLTkpg+6v06Z2Nz0gAG9Mz+5buqGJQYz9SRqvkWaUJJ/PvN7A/A1cCzZpYcyvvMbDHwFjDJzPaZ2S3AncDFZrYNuNh73urjwL+cc23/p+QTvCTwu8By4BnvaoEiImE3f0o+NfXNrNjVfp5yUEvA8crWci6clEt8m/PYgwclkpOexK4B3ON/Z1cVs8YMJbGHZY8l/EKp1X81wQl3P3fOHfKG6L/R3Zuccws7eWl+J9vfT3AJYNu2ncD0EGIUEelz50/MISkhjhc3lXLOhJwTXl9TXM2huiYumpx3wmsDeUnf4bomNh88wtcWnOJ3KNKBUCr31TnnHnfObfOelzjn/hX+0ERE/JWalMC547NZuqkM59wJr7+0uYyEOOP8iSdOJh6Xk87OAVq9b+WeKpzT+f1IpTEYEZEuzJ+ST3FVHdvLTkziL20uo6hwKIMHJZ7w2tjcNCqONnL4WFN/hBlRlu+uIjHemDF6SPcbS79T4hcR6ULrMr0X283u33/oGJsP1nQ4zA/vT/DbPQCH+5fvqmL6qCGkJMb7HYp0QIlfRKQLwwcPYurIzBPK977sVeu7aHJ+h+8b5yX+gXae/2hDM+v2HdYwfwRT4hcR6cb8yfmsLq6m8mjD8baXNpdRkJXK+Ny0Dt9TkJ1KnBHyhX5ixds7KmkOOM7rYDKkRAYlfhGRbiyYko9z79fkr29q4c0dFVw0Oa/Ty80mJ8QzamjqgFvL//r2ClIS45hVONTvUKQTSvwiIt2YOjKT/Mzk48P9b+2opL4p0On5/VYDcUnfsm3lzBmbTXKCzu9HKiV+EZFumBnzp+SzbFs59U0tLN1cSmpSPHPGdX0euzXxd7QUMBbtP3SMneW1nD9Rw/yRTIlfRCQEC6bkUdfYwts7K3l5cznnTcjptlc7PjeNusYWSo80dLldrHh9W/CCRhecooukRTIlfhGREJwzPoeUxDh+98oO9h861u0wP7S5WM8AKeSzbFsF+ZnJTMxL9zsU6YISv4hICFIS4zl/Yi7Lvbr980JJ/LkDZ0lfS8DxxvYKzpuQ2+mER4kMSvwiIiFa4BXzCU72S+l2++GZKaQkxg2Ii/VsOHCYQ3VNOr8fBZT4RURCNG9yHonxxiWnDgtp+7g4ozA7bUAs6XttWwUA52r9fsQL5ep8IiIC5GWk8NxXL2B01qCQ3zMuN41NJTVhjCoyvLatnFOHZ5Kbkex3KNIN9fhFRHpgQl56j9aoj81Jo7iqjqaWQBij8ldtQzOr9lRrmD9KKPGLiITRuJx0WgKOvVV1focSNst3VdHU4jq8PLFEHiV+EZEwap3ZvzOGJ/gt21ZOckIcRSrTGxWU+EVEwmggXKXvtW0VzB6bpcvwRgklfhGRMBqSmsTQ1MSYndlfcvgY28uOcoGG+aOGEr+ISJiNy01nV4xW72tdxneeJvZFDSV+EZEwG5uTFrPn+F/bVkFOejKTh2X4HYqESIlfRCTMxuakUVbTwNGGZr9D6VMBr0zv+RNzVKY3iijxi4iEWesEv90xdp5/Y8kRqmobtX4/yijxi4iE2bjc1qv0xVbiP35+X2V6o4oSv4hImI3JTsUMdpbH1gS/17aVM3lYBnkhXLBIIocSv4hImKUkxjNi8KCYWst/rLGFlbtVpjcaKfGLiPSDcblpMZX439lVSWNLgPO0fj/qKPGLiPSDcTlp7CqvxTnndyh94rVtFSQlxDG7MMvvUKSHlPhFRPrB2Jw0ahqaKT/a4HcofeL1bRXMLsxiUJLK9EYbJX4RkX4w1pvZvysGCvmUHqlnS2mNqvVFKSV+EZF+EEsX62ldxqeJfdFJiV9EpB+MGDKIpIS4mEj8r28rJzstiSnDMv0ORXohwe8AREQGgvg4ozA7lR1RMNS//9Axqo42Ut/cQn1TCw1NAeqb379fti1YpjcuTmV6o1FYE7+ZLQKuAMqcc1O9tizgr0AhsBu42jlXbWZzgaeAXd7bH3fO/ch7z2XAr4F44D7n3J3hjFtEJBzG5qSxvSyyi/j879t7+I8n13e73WWnDeuHaCQcwt3jvx/4LfBgm7ZvA0udc3ea2be959/yXnvNOXdF2x2YWTxwN3AxsA9YYWZPO+c2hjl2EZE+NS43nZc2l9HcEiAhPvLOtL6w4SDff2o9cyfl8pk5Y0hOjCMlMZ7khOB9SkI8yYlxpCbFk5GS6He40kthTfzOuWVmVtiu+Upgrvf4AeAV3k/8HZkNbHfO7QQws0e8fSjxi0hUGZuTRlOLY/+hY4zJTvM7nA9YubuKryxew7RRQ/jddTNJTdKZ4Fjlx1fOfOdcCYB3n9fmtbPN7F0ze87MTvPaRgJ722yzz2sTEYkqrTP7d0bYef7tZTXc8sBKRgwZxKIbi5T0Y1wkjTWtBsY456YDvwGe9No7mj3SYekrM7vVzFaa2cry8vIwhSki0jtjWxN/BM3sLz1Sz42LVpAYH8eDN88mOz3Z75AkzPxI/KVmNhzAuy8DcM4dcc4d9R4/CySaWQ7BHv7oNu8fBRzoaMfOuXudc0XOuaLcXNWPFpHIkpWWxOBBieyqiIwJfkfqm7hx0XIO1TVy/7+dyeisVL9Dkn7gR+J/GrjRe3wjwZn8mNkwMzPv8WwvtkpgBTDRzMaaWRJwjbcPEZGoYmaMyU5lT2Wd36HQ0NzCrQ+uZHvZUe65fhZTRw72OyTpJ+FezreY4ES+HDPbB/wAuBN41MxuAYqBq7zNPwV80cyagWPANS54NYtmM7sdeIHgcr5FzrkN4YxbRCRc8jNT2Fvlb+IPBBxff/Rd3t5Zxa8+PZ3zdYW9ASXcs/oXdvLS/A62/S3BpX8d7edZ4Nk+DE1ExBf5mcms3F3l2+c75/jJM5t45r0Svv2hyXz8jFG+xSL+iKTJfSIiMS8/I4XquiYamlt8+fz7XtvFojd2cdM5hXz+gnG+xCD+UuIXEelHeZnBWfNlR/r/8rxPrd3Pfz67iQ9PG873rzgVb1qVDDBK/CIi/SgvMwWAspr+TfxvbK/gjr+9y5yxWfzi6umqsz+AKfGLiPSj/Awv8R+p77fP3HDgMJ//yyrG56Zz7w1FpCTG99tnS+RR4hcR6Uf53lB/aT8l/r1Vddz05xVkpiRw/7/NZvAg1dgf6JT4RUT60dDUJBLijNJ+GOqvqm3kxkXLaWhq4YGbZzNscErYP1Minwoyi4j0o7g4Iy8jOeyT+441tnDLAyvYd+gYD312DhPzM8L6eRI91OMXEelneZkplNWEb6i/uSXA7Q+vZu3eQ9x1zQzOLMwK22dJ9FHiFxHpZ/mZyWE7x++c43tPrWfp5jJ+9NHTuGzq8LB8jkQvJX4RkX6Wl5FCaZiG+n+9dBuLl+/ltnnjuf7swrB8hkQ3JX4RkX6Wn5nM4WNN1Df1bfW+xcuL+Z8l2/jUrFHcccmkPt23xA4lfhGRftZaxKe8D2f2L9lYynefWMfcSbn81yemqSqfdEqJX0Skn+V7ib+vzvOvLq7m9sWrmTpyMHdfO5PEeP1pl87pt0NEpJ/lZbQW8Tn5Hv+O8qPccv8K8jNTWHTTmaQla5W2dE2JX0Skn+Ufr9d/cj3+siP13PCn5cSZ8eDNs8lJT+6L8CTG6auhiEg/G5qaSGK8nVSPv6a+iZv+vILqukYeufUsxmSn9WGEEsvU4xcR6WdmRl5GSq8v1NPYHOAL/7uKraU1/O66mZw+akgfRyixTD1+EREf5GUmU9qLof5AwPGNv7/LG9sr+flV05k7KS8M0UksU49fRMQH+RkpvarX/8Bbu3lq7QG+cekkPjVrVN8HJjFPiV9ExAe9Ldu7fFcVY3PS+NLc8WGISgYCJX4RER/kZaZwpL6ZY409q963p7KOwuxUFeiRXlPiFxHxQeta/p4s6XPOsaeyVjP45aQo8YuI+OD9tfyhn+evrG2ktrGFMdmp4QpLBgAlfhERH/SmbO+eyjoAJX45KUr8IiI+yM/sedne4qpaAAqyNNQvvafELyLig8GDEklKiOtREZ89lXWYweisQWGMTGKdEr+IiA+C1fuSe3SOv7iyjuGZKSQnxIcxMol1SvwiIj7Jz0zp2Tn+qjoKdH5fTpISv4iIT3paxGdPZR1jdH5fTpISv4iIT/J6ULb3aEMzFUcb1OOXk6bELyLik7zMZGoamqlrbO5222It5ZM+osQvIuKT/AyviE8Ivf7WpXyFqtonJ0mJX0TEJz0p4tNavEdD/XKywpb4zWyRmZWZ2fo2bVlm9qKZbfPuh3rt15nZe97tTTOb3uY9u81snZmtNbOV4YpXRKS/5bUW8QlhSd+eqjqGpiaSmZIY7rAkxoWzx38/cFm7tm8DS51zE4Gl3nOAXcCFzrnTgR8D97Z73zzn3AznXFEY4xUR6VfvD/V33+MvrqyjQMP80gfClvidc8uAqnbNVwIPeI8fAD7mbfumc67aa38bGBWuuEREIkXmoASSE+JCKuKzp6qWMVka5peT19/n+POdcyUA3n1eB9vcAjzX5rkD/mVmq8zs1n6IUUSkX5hZSEV8GpsD7K8+phn90icS/A6gLTObRzDxn9em+Vzn3AEzywNeNLPN3mhCR++/FbgVoKCgIOzxioicrLyM7ov47D90jICDAvX4pQ/0d4+/1MyGA3j3Za0vmNnpwH3Alc65ytZ259wB774MeAKY3dnOnXP3OueKnHNFubm5YToEEZG+k5/ZfRGfPZXBpXxjdI5f+kB/J/6ngRu9xzcCTwGYWQHwOHC9c25r68ZmlmZmGa2PgUuA9YiIxIi8zO4v1FNcFVzKV6ihfukDYRvqN7PFwFwgx8z2AT8A7gQeNbNbgGLgKm/z7wPZwO/MDKDZm8GfDzzhtSUADzvnng9XzCIi/S0/M4WjDc0cbWgmPbnjP8l7KusYlBhPbkZyP0cnsShsid85t7CTl+Z3sO1ngc920L4TmN6+XUQkVuR5ybzsSD3puekdbrOnso6CrFS8TpDISVHlPhERH71fva/z4f7iqlpV7JM+o8QvIuKjfK96X1lNxzP7nXMUV9VpDb/0GSV+EREf5WV2faGespoG6psCWsMvfUaJX0TERxnJCaQkxnW6ln93RXApn8r1Sl9R4hcR8dHx6n2dLOnb4y3l01C/9BUlfhERn+VnpHR6oZ7iyjri44yRQwf1c1QSq5T4RUR81lURnz1VdYwcMojEeP25lr6h3yQREZ/lZQQv1OOcO+G14spaTeyTPqXELyLis/zMZOoaWzja0HzCa3uq6nRxHulTSvwiIj5rLeLTfrj/8LEmDtU1qccvfUqJX0TEZ3leEZ/2S/qKK4Mz+guytJRP+o4Sv4iIz/IyOi7is/v45XjV45e+o8QvIuKz/M56/FWtPX4lfuk7SvwiIj5LT04gNSn+hHP8eypryUlPJq2Ty/WK9IYSv4iIz45X72vX499TWadhfulzSvwiIhEgNyP5hHP8xVVK/NL3lPhFRCJAsF7/+z3++qYWDh6pZ4xm9EsfU+IXEYkA+V6Pv7V6377qOpzTjH7pe0r8IiIRID8zhWNNLdR41fv2tK7hV+KXPqbELyISAVqL+LRepa818etyvNLXlPhFRCJAaxGfUm+C357KWtKTE8hKS/IzLIlBSvwiIhGgtYhPmTfBr/XiPGbmZ1gSg5T4RUQiQF7mB3v8xVrDL2GixC8iEgHSkxNIS4qn9Eg9LQHH3uo6TeyTsFDiFxGJEPmZKZQdaaDk8DGaWhyF2VrDL31PiV9EJELkZSZTVlN//HK8mtEv4aDELyISIfIyUig90sCeKq3hl/BR4hcRiRD5mcmUHqlnd2UtifHG8MGD/A5JYpASv4hIhMjPTKGhOcC6fYcZPTSV+Dgt5ZO+p8QvIhIhWpf0rSk+pGF+CRslfhGRCJGXESzic6ypRRP7JGyU+EVEIkS+1+MHKNBSPgkTJX4RkQjR2uMHLeWT8FHiFxGJEGnJCWQkJwBQmKPEL+ER1sRvZovMrMzM1rdpyzKzF81sm3c/1Gs3M7vLzLab2XtmNrPNe270tt9mZjeGM2YRET/lZiZjBqOGKvFLeIS7x38/cFm7tm8DS51zE4Gl3nOADwETvdutwO8h+EUB+AEwB5gN/KD1y4KISKzJz0hhWGYKKYnxfociMSohnDt3zi0zs8J2zVcCc73HDwCvAN/y2h90zjngbTMbYmbDvW1fdM5VAZjZiwS/TCwOZ+wiIn644ewxVNY2+h2GxLCwJv5O5DvnSgCccyVmlue1jwT2ttlun9fWWbuISMz50LThfocgMS6SJvd1VKLKddF+4g7MbjWzlWa2sry8vE+DExERiQV+JP5Sbwgf777Ma98HjG6z3SjgQBftJ3DO3eucK3LOFeXm5vZ54CIiItHOj8T/NNA6M/9G4Kk27Td4s/vPAg57pwReAC4xs6HepL5LvDYRERHpobCe4zezxQQn5+WY2T6Cs/PvBB41s1uAYuAqb/NngcuB7UAd8G8AzrkqM/sxsMLb7ketE/1ERESkZyw4iT72FBUVuZUrV/odhoiISL8ws1XOuaLutoukyX0iIiISZkr8IiIiA4gSv4iIyACixC8iIjKAKPGLiIgMIEr8IiIiA4gSv4iIyACixC8iIjKAxGwBHzMrB/a0acoBKnwKp7/E+jHG+vGBjjFW6BijXzQe3xjnXLcXqonZxN+ema0MpaJRNIv1Y4z14wMdY6zQMUa/WD4+DfWLiIgMIEr8IiIiA8hASvz3+h1AP4j1Y4z14wMdY6zQMUa/mD2+AXOOX0RERAZWj19EREScc77cgNHAy8AmYAPwVa89C3gR2ObdD/XaJwNvAQ3AHe329TVvH+uBxUBKJ595o7ffbcCNbdr/E9gLHO0m5lnAOmA7cBfvj5j8DNgMvAc8AQyJseOb7sW2DvgHkBnFP8NOtwOuBjZ6cTwcjccIpALPeL+PG4A727z2Be9nuBZ4HTg1Go/R2+554F3vs+4B4ruJOSKOsaufT6j/H73Xvgxs8fbx0yg+xg5/3nTwuxptx9fVdkAy8FfvZ/sOUNjV73tf3/rtgzr4RxkOzPQeZwBbvR/uT4Fve+3fBv7be5wHnOn9otzRZj8jgV3AIO/5o8BNHXxeFrDTux/qPW79BTnLi6e7PzbLgbMBA54DPuS1XwIkeI//27vF0vGtAC70HmqKUMEAAAbASURBVN8M/DiKf4YdbgdMBNa02V9eNB4jwT8287zHScBrbX6OmW22+yjwfDQeY9tjIfi7+hhwjfe8s5gj4hi7+vn04P/jPGAJkByJv6s9PMbO/j+e8LsabcfX1XbAl4B7vMfXAH/t6ve9r2++DfU750qcc6u9xzUEv8WNBK4EHvA2ewD4mLdNmXNuBdDUwe4SgEFmlkDwH/tAB9tcCrzonKtyzlUT/GZ4mbfvt51zJV3Fa2bDCf4yvuWCP60H28T2L+dcs7fp28CoWDo+YBKwzHv8IvBJb79RdYzdbPc54G5vvzjnyqLxGJ1zdc65l73HjcBqYJT3/EibTdMAF43H2O5YEgj+UXXe885ijohj7Orn01Y3/x+/SLD32NAaazQeo/d6hz/vjn5Xo+34utmubcx/B+abmXUQQ1hExDl+MysEziA45JHf+ovg3ed19V7n3H7g50AxUAIcds79q4NNRxIcUmq1z2sL1UjvPd29/2aC386Pi4HjW0/wWzfAVQSH3D4gSo6xK6cAp5jZG2b2tpld1n6DaDtGMxsCfARY2qbtNjPbQbCX9JUO3lNIlByjmb0AlAE1BP94EkrMkXKMHf182r2/s/+PpwDnm9k7ZvaqmZ0ZpcfYpa5+V6Pt+DrY7vi+vU7jYSC7q330Jd8Tv5mlExyq+/d23/JCff9Qgt+exgIjgDQz+0xHm3bQ5jpo6/Sjunu/mX0XaAYeatMWC8d3M3Cbma0iOMTW2C7GaDnGriQQHO6fCywE7vP+s7bGGFXH6PVkFgN3Oed2Ht+Rc3c758YD3wL+o917ouoYnXOXEhz+TQYuCjHGiDjGzn4+Ib4/geCQ81nAN4BH2/YWo+gYu9TZ72q0HV8n24Xzb1m3fE38ZpZI8Af4kHPuca+51Bvmah3uKutmNwuAXc65cudcE/A4cI6ZzTGztd7towS/rbXtqY6i4+Gd1tji27z/R9772w7nfOD9ZnYjcAVwnTc0FzPH55zb7Jy7xDk3i+Av8I42+4mmY+zKPuAp51yTc24XwYlTE6P4GO/l/2/vbkLkJsMAjv/fZf3ERa0W7VXQgwoWFApWL56kIKygJ1FqUSmoeBHPaykoFUqligUPbtVDsaJQevGwF4/1g9qKiF0pgiAiSlUseunj4Xm3OzudmWZhdrpp/j8IDUkmeZ/JJE82fZIXTkXEviEfPcTyreO2xkhE/AscIU/kI9u8zmJcsX9Web75Gfgk0jHgHPle+bbF2NT532pL4xt0LJ5fd70wuB74o9G3MQ4xwYKC3oG84nkf2Nc3/Q1WFmrs6Zs/x8pCjS1kxeS1dZ0HgRcHbG8DWdBxYx1OAxv6lrlYQdEX5FX2UrHNtjr9YbIafONlGt9S8dBUjWlHW2Mctlzdhwfr+M3kbbib2hgjsJs8OU71Tb+9Z/wR4Ms27kfgOmBTHZ8mq6NfGNXm9RTjsP2ziuNxJ7Crjt9Rf6uljTGOOB4v+K22Mb5hywHPs7K476Mm39O4holtaMAX8gB5a+ME+cjGcWAbebJdIB+fWOj5om8lr5L+As7U8aXK3lfJRya+BT6gVrsO2OYO8vGJReDpnul76vrO1X/nhnz+vrqNH4G3WH7cbbEefEtxHLjM4nuJrKD9AXi9Z3obYxy4HHkC2EtewJ1kuUq8VTGSf5UEWfi01N5n6rw3yRPecfKxqLtaGuMtZFI8UePZz/JTNcPavC5iHLV/VnE8Xgl8WOd9DTzU4hiHHY8X/FbbFt+o5YCrgcN1vceA25rkzXENvrlPkqQOueTFfZIkaXJM/JIkdYiJX5KkDjHxS5LUISZ+SZI6xMQvadVKKXOllJdHzJ8tpdw5yTZJasbEL2ktzJI9p0laZ3yOX1IjtS+Kp8iXVf0GfEV2LvIc+VKZReBJYDNwtM77k9qbI/A2sBE4CzwbEd9Psv2Skolf0kWVUu4F5snXnU6Tb4w7ALwXEb/XZXYDv0bE/lLKPHA0Ij6u8xaAnRFxqpSyBXgtIhp1rCNpvKYvdQMktcKDwKcRcRaglHKkTr+7JvwbyHfof9b/wdqb2v3A4Z5O5K5a8xZLGsjEL6mpQbcH54HZiPimlLKd7Na43xRwJiI2r13TJDVlcZ+kJj4HHi2lXFNKmSF7TAOYAX6p3aU+0bP833UekX2mny6lPA5Q0j2Ta7qkXv4fv6RGeor7fiJ7OvsO+Ad4pU47CcxExPZSylbgXeA/4DGy97V3gE3AFcChiFhNn+ySxsTEL0lSh3irX5KkDjHxS5LUISZ+SZI6xMQvSVKHmPglSeoQE78kSR1i4pckqUNM/JIkdcj/7CZz8TL112YAAAAASUVORK5CYII=\n", "text/plain": [ "<matplotlib.figure.Figure at 0x7f47f6df7128>" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.figure(figsize=(8, 6))\n", "plt.xlabel('date')\n", "plt.ylabel('stock price')\n", "plt.plot(df.date, df.price);" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python [conda env:Python3]", "language": "python", "name": "conda-env-Python3-py" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.4" } }, "nbformat": 4, "nbformat_minor": 2 } -
pbugnion revised this gist
Jun 5, 2018 . 1 changed file with 12 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,12 @@ #!/bin/sh # Runit unit file for Airflow webserver set -e cd "$PROJECT_HOME" exec /sbin/setuser sherlock env \ AIRFLOW_HOME=/project/airflow \ PATH=/opt/anaconda/envs/Python3/bin:opt/anaconda/bin:"$PATH" \ airflow webserver -
pbugnion revised this gist
Jun 5, 2018 . 4 changed files with 572 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,353 @@ # Airflow configuration file # /project/airflow/airflow.cfg [core] # The home folder for airflow, default is ~/airflow airflow_home = /project/airflow # The folder where your airflow pipelines live, most likely a # subfolder in a code repository # This path must be absolute dags_folder = /project/airflow/dags # The folder where airflow should store its log files # This path must be absolute base_log_folder = /project/airflow/logs # Airflow can store logs remotely in AWS S3 or Google Cloud Storage. Users # must supply a remote location URL (starting with either 's3://...' or # 'gs://...') and an Airflow connection id that provides access to the storage # location. remote_base_log_folder = remote_log_conn_id = # Use server-side encryption for logs stored in S3 encrypt_s3_logs = False # DEPRECATED option for remote log storage, use remote_base_log_folder instead! s3_log_folder = # The executor class that airflow should use. Choices include # SequentialExecutor, LocalExecutor, CeleryExecutor executor = SequentialExecutor # The SqlAlchemy connection string to the metadata database. # SqlAlchemy supports many different database engine, more information # their website sql_alchemy_conn = sqlite:////project/airflow/airflow.db # The SqlAlchemy pool size is the maximum number of database connections # in the pool. sql_alchemy_pool_size = 5 # The SqlAlchemy pool recycle is the number of seconds a connection # can be idle in the pool before it is invalidated. This config does # not apply to sqlite. sql_alchemy_pool_recycle = 3600 # The amount of parallelism as a setting to the executor. This defines # the max number of task instances that should run simultaneously # on this airflow installation parallelism = 8 # The number of task instances allowed to run concurrently by the scheduler dag_concurrency = 16 # Are DAGs paused by default at creation dags_are_paused_at_creation = True # When not using pools, tasks are run in the "default pool", # whose size is guided by this config element non_pooled_task_slot_count = 128 # The maximum number of active DAG runs per DAG max_active_runs_per_dag = 16 # Whether to load the examples that ship with Airflow. It's good to # get started, but you probably want to set this to False in a production # environment load_examples = False # Where your Airflow plugins are stored plugins_folder = /home/sherlock/workspace/airflow/plugins # Secret key to save connection passwords in the db fernet_key = IrY20uqM6R-OGwrRaWFh4rgdh5hD0n2p7a5jgd-nhxw= # Whether to disable pickling dags donot_pickle = False # How long before timing out a python file import while filling the DagBag dagbag_import_timeout = 30 # The class to use for running task instances in a subprocess task_runner = BashTaskRunner # If set, tasks without a `run_as_user` argument will be run with this user # Can be used to de-elevate a sudo user running Airflow when executing tasks default_impersonation = # What security module to use (for example kerberos): security = # Turn unit test mode on (overwrites many configuration options with test # values at runtime) unit_test_mode = False [cli] # In what way should the cli access the API. The LocalClient will use the # database directly, while the json_client will use the api running on the # webserver api_client = airflow.api.client.local_client endpoint_url = http://localhost:8888 [api] # How to authenticate users of the API auth_backend = airflow.api.auth.backend.default [operators] # The default owner assigned to each new operator, unless # provided explicitly or passed via `default_args` default_owner = Airflow default_cpus = 1 default_ram = 512 default_disk = 512 default_gpus = 0 [webserver] # The base url of your website as airflow cannot guess what domain or # cname you are using. This is used in automated emails that # airflow sends to point links to the right web server base_url = http://localhost:8888 # The ip specified when starting the web server web_server_host = 127.0.0.1 # The port on which to run the web server web_server_port = 8888 # Paths to the SSL certificate and key for the web server. When both are # provided SSL will be enabled. This does not change the web server port. web_server_ssl_cert = web_server_ssl_key = # Number of seconds the gunicorn webserver waits before timing out on a worker web_server_worker_timeout = 120 # Number of workers to refresh at a time. When set to 0, worker refresh is # disabled. When nonzero, airflow periodically refreshes webserver workers by # bringing up new ones and killing old ones. worker_refresh_batch_size = 1 # Number of seconds to wait before refreshing a batch of workers. worker_refresh_interval = 30 # Secret key used to run your flask app secret_key = temporary_key # Number of workers to run the Gunicorn web server workers = 4 # The worker class gunicorn should use. Choices include # sync (default), eventlet, gevent worker_class = sync # Log files for the gunicorn webserver. '-' means log to stderr. access_logfile = - error_logfile = - # Expose the configuration file in the web server expose_config = False # Set to true to turn on authentication: # http://pythonhosted.org/airflow/security.html#web-authentication authenticate = False # Filter the list of dags by owner name (requires authentication to be enabled) filter_by_owner = False # Filtering mode. Choices include user (default) and ldapgroup. # Ldap group filtering requires using the ldap backend # # Note that the ldap server needs the "memberOf" overlay to be set up # in order to user the ldapgroup mode. owner_mode = user # Default DAG orientation. Valid values are: # LR (Left->Right), TB (Top->Bottom), RL (Right->Left), BT (Bottom->Top) dag_orientation = LR # Puts the webserver in demonstration mode; blurs the names of Operators for # privacy. demo_mode = False # The amount of time (in secs) webserver will wait for initial handshake # while fetching logs from other worker machine log_fetch_timeout_sec = 5 # By default, the webserver shows paused DAGs. Flip this to hide paused # DAGs by default hide_paused_dags_by_default = False [email] email_backend = airflow.utils.email.send_email_smtp [smtp] # If you want airflow to send emails on retries, failure, and you want to use # the airflow.utils.email.send_email_smtp function, you have to configure an # smtp server here smtp_host = localhost smtp_starttls = True smtp_ssl = False # Uncomment and set the user/pass settings if you want to use SMTP AUTH # smtp_user = airflow # smtp_password = airflow smtp_port = 25 smtp_mail_from = [email protected] [celery] # This section only applies if you are using the CeleryExecutor in # [core] section above # The app name that will be used by celery celery_app_name = airflow.executors.celery_executor # The concurrency that will be used when starting workers with the # "airflow worker" command. This defines the number of task instances that # a worker will take, so size up your workers based on the resources on # your worker box and the nature of your tasks celeryd_concurrency = 16 # When you start an airflow worker, airflow starts a tiny web server # subprocess to serve the workers local log files to the airflow main # web server, who then builds pages and sends them to users. This defines # the port on which the logs are served. It needs to be unused, and open # visible from the main web server to connect into the workers. worker_log_server_port = 8793 # The Celery broker URL. Celery supports RabbitMQ, Redis and experimentally # a sqlalchemy database. Refer to the Celery documentation for more # information. broker_url = sqla+mysql://airflow:airflow@localhost:3306/airflow # Another key Celery setting celery_result_backend = db+mysql://airflow:airflow@localhost:3306/airflow # Celery Flower is a sweet UI for Celery. Airflow has a shortcut to start # it `airflow flower`. This defines the IP that Celery Flower runs on flower_host = 0.0.0.0 # This defines the port that Celery Flower runs on flower_port = 5555 # Default queue that tasks get assigned to and that worker listen on. default_queue = default [scheduler] # Task instances listen for external kill signal (when you clear tasks # from the CLI or the UI), this defines the frequency at which they should # listen (in seconds). job_heartbeat_sec = 5 # The scheduler constantly tries to trigger new tasks (look at the # scheduler section in the docs for more information). This defines # how often the scheduler should run (in seconds). scheduler_heartbeat_sec = 5 # after how much time should the scheduler terminate in seconds # -1 indicates to run continuously (see also num_runs) run_duration = -1 # after how much time a new DAGs should be picked up from the filesystem min_file_process_interval = 0 dag_dir_list_interval = 300 # How often should stats be printed to the logs print_stats_interval = 30 child_process_log_directory = /home/sherlock/workspace/airflow/logs/scheduler # Local task jobs periodically heartbeat to the DB. If the job has # not heartbeat in this many seconds, the scheduler will mark the # associated task instance as failed and will re-schedule the task. scheduler_zombie_task_threshold = 300 # Turn off scheduler catchup by setting this to False. # Default behavior is unchanged and # Command Line Backfills still work, but the scheduler # will not do scheduler catchup if this is False, # however it can be set on a per DAG basis in the # DAG definition (catchup) catchup_by_default = True # Statsd (https://github.com/etsy/statsd) integration settings statsd_on = False statsd_host = localhost statsd_port = 8125 statsd_prefix = airflow # The scheduler can run multiple threads in parallel to schedule dags. # This defines how many threads will run. However airflow will never # use more threads than the amount of cpu cores available. max_threads = 2 authenticate = False [mesos] # Mesos master address which MesosExecutor will connect to. master = localhost:5050 # The framework name which Airflow scheduler will register itself as on mesos framework_name = Airflow # Number of cpu cores required for running one task instance using # 'airflow run <dag_id> <task_id> <execution_date> --local -p <pickle_id>' # command on a mesos slave task_cpu = 1 # Memory in MB required for running one task instance using # 'airflow run <dag_id> <task_id> <execution_date> --local -p <pickle_id>' # command on a mesos slave task_memory = 256 # Enable framework checkpointing for mesos # See http://mesos.apache.org/documentation/latest/slave-recovery/ checkpoint = False # Failover timeout in milliseconds. # When checkpointing is enabled and this option is set, Mesos waits # until the configured timeout for # the MesosExecutor framework to re-register after a failover. Mesos # shuts down running tasks if the # MesosExecutor framework fails to re-register within this timeframe. # failover_timeout = 604800 # Enable framework authentication for mesos # See http://mesos.apache.org/documentation/latest/configuration/ authenticate = False # Mesos credentials, if authentication is enabled # default_principal = admin # default_secret = admin [kerberos] ccache = /tmp/airflow_krb5_ccache # gets augmented with fqdn principal = airflow reinit_frequency = 3600 kinit_path = kinit keytab = airflow.keytab [github_enterprise] api_rev = v3 [admin] # UI to hide sensitive variable fields when set to True hide_sensitive_variable_fields = True This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,104 @@ import json import csv import logging from pathlib import Path import argparse from collections import namedtuple import requests logging.basicConfig(level=logging.INFO) QUANDL_URL = 'https://www.quandl.com/api/v3/datasets/WIKI/{}.json' DATASTORE = Path('/project/data') Configuration = namedtuple( 'Configuration', ['stock', 'start_date', 'end_date'] ) def fetch_stock_info(stock, start_date, end_date): response = requests.get( QUANDL_URL.format(stock), params={'start_date': start_date, 'end_date': end_date} ) response.raise_for_status() json = response.json() data = json['dataset']['data'] stock_price = {} for entry in data: date = entry[0] closing = entry[4] stock_price[date] = float(closing) return stock_price def read_current_data(store): stock_prices = {} try: with open(store) as f: reader = csv.reader(f) next(reader) # skip header for row in reader: date, price = row stock_prices[date] = price except FileNotFoundError: return {} return stock_prices def write_data(store, stock_prices): sorted_data = sorted(stock_prices.items(), key=lambda it: it[0]) with open(store, 'w') as f: writer = csv.writer(f) writer.writerow(['date', 'price']) writer.writerows(sorted_data) def store_update(store, new_prices): stock_prices = read_current_data(store) stock_prices.update(new_prices) write_data(store, stock_prices) def fetch_and_update_stocks(stock, start_date, end_date): logging.info( f'Fetching stocks for symbol {stock} ' f'between {start_date} and {end_date}' ) prices = fetch_stock_info(stock, start_date, end_date) logging.info(f'Fetched {len(prices)} stock prices.') store = DATASTORE / f'{stock}.csv' store_update(store, prices) logging.info(f'Written information back to data store {store}.') def parse_command_line(): parser = argparse.ArgumentParser( description='Fetch and store stock information') parser.add_argument('symbol') parser.add_argument( 'start_date', help='start date, in format 2018-01-22' ) parser.add_argument( 'end_date', help='end date, in format 2018-01-29' ) args = parser.parse_args() return Configuration( args.symbol.upper(), args.start_date, args.end_date ) if __name__ == '__main__': configuration = parse_command_line() logging.info(f'Using configuration {configuration}.') fetch_and_update_stocks( configuration.stock, configuration.start_date, configuration.end_date )
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,80 @@ import os import requests import sherlockml.auth import nbformat from nbconvert.preprocessors import ExecutePreprocessor HUDSON_URL = 'https://hudson.platform.asidata.science' TAVERN_URL = 'https://tavern.platform.asidata.science' REPORT_ID = 'b0b5e1c5-d7c9-4ae5-8b02-ce0f4a7ddaee' NOTEBOOK_PATH = '/project/stock-price/plot-stock-price.ipynb' class InputError(Exception): pass def run(report_id, notebook, run_notebook=True): """Publish a report in the current project. Parameters ---------- report_id: str SherlockML report ID. notebook: str Full path to notebook on SherlockML. run_notebook: bool, optional Run notebook before publishing (default: True). """ # Run the notebook if run_notebook: with open(notebook) as f: nb = nbformat.read(f, as_version=4) ep = ExecutePreprocessor(kernel_name='Python3') ep.preprocess(nb, {'metadata': {'path': os.path.dirname(notebook)}}) with open(notebook, 'wt') as f: nbformat.write(nb, f) # Make sure notebook path starts with /project split_notebook_path = notebook.split('/') if split_notebook_path[0] == '' and split_notebook_path[1] == 'project': pass else: raise InputError('notebook path must start with /project') # Get rid of the project bit reconstructed_notebook_path = [] for i, item in enumerate(split_notebook_path): if i < 2: continue reconstructed_notebook_path.append(item) reconstructed_notebook_path = '/'.join(reconstructed_notebook_path) # Authenticate and publish auth_headers = sherlockml.auth.auth_headers() user_id_response = requests.get( HUDSON_URL + '/authenticate', headers=auth_headers ) user_id_response.raise_for_status() user_id = user_id_response.json()['account']['userId'] body = { 'notebook_path': reconstructed_notebook_path, 'author_id': user_id, 'draft': False } create_version_response = requests.post( TAVERN_URL + '/report/{}/version'.format(report_id), json=body, headers=auth_headers) create_version_response.raise_for_status() run(REPORT_ID, NOTEBOOK_PATH) This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,35 @@ from airflow import DAG from airflow.operators.bash_operator import BashOperator from datetime import datetime, timedelta default_args = { 'owner': 'airflow', 'depends_on_past': False, 'start_date': datetime(2017, 1, 1), 'email_on_failure': False, 'email_on_retry': False, 'retries': 1, 'retry_delay': timedelta(minutes=5) } dag = DAG( 'stock_price', default_args=default_args ) fetch_command = '/opt/anaconda/envs/Python3/bin/python /project/stock-price/fetch_stock.py GOOG {{ ds }} {{ ds }}' fetch_operator = BashOperator( task_id='fetch_price', bash_command=fetch_command, dag = dag ) publish_operator = BashOperator( task_id='publish', bash_command='/opt/anaconda/envs/Python3/bin/python /project/stock-price/publish.py', dag=dag ) publish_operator.set_upstream(fetch_operator) -
pbugnion created this gist
Jun 5, 2018 .There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1 @@ # Airflow on SherlockML