Skip to content

Instantly share code, notes, and snippets.

@kernullist
Forked from lukassup/zipapp.md
Created October 22, 2019 05:18
Show Gist options
  • Save kernullist/ab0f44667f0b69b41b6c9a5aa3c68adc to your computer and use it in GitHub Desktop.
Save kernullist/ab0f44667f0b69b41b6c9a5aa3c68adc to your computer and use it in GitHub Desktop.

Revisions

  1. @lukassup lukassup revised this gist Apr 2, 2018. 1 changed file with 2 additions and 0 deletions.
    2 changes: 2 additions & 0 deletions zipapp.md
    Original file line number Diff line number Diff line change
    @@ -2,6 +2,8 @@

    [What's a `zipapp`?](https://docs.python.org/3/library/zipapp.html)

    This concept is very much like `.jar` or `.war` archives in Java.

    > NOTE: The built `.pyz` _zipapp_ can run on both Python 2 & 3 but you can only build `.pyz` _zipapps_ with Python 3.5 or later.
    ## Initial setup
  2. @lukassup lukassup revised this gist Apr 2, 2018. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion zipapp.md
    Original file line number Diff line number Diff line change
    @@ -127,7 +127,7 @@ gunicorn.app.wsgiapp:run
    ```
    So the command line flag for my _zipapp_ entrypoint will be `-m 'gunicorn.app.wsgiapp:run'`.

    I want to use Python 3 so I also set the interpreter with `-p /usr/bin/env python3`.
    I want to use Python 3 so I also set the interpreter with `-p '/usr/bin/env python3'`.

    Let's run our _zipapp_ command
    ```
  3. @lukassup lukassup revised this gist Apr 2, 2018. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion zipapp.md
    Original file line number Diff line number Diff line change
    @@ -121,7 +121,7 @@ $ python3 -m zipapp APP_DIR -m ENTRYPOINT_MODULE:ENTRYPOINT_FUNCTION -p PYTHON_I

    I'm going to use [`gunicorn`](gunicorn.org/) to serve my Flask app because it's pure Python and will work accross any supported platform without worrying about binary compilation (e.g. `uwsgi`).

    When you run `gunicorn` command on the command line [this is the entrypoint](benoitc/gunicorn@4371ff2ed4babed2ffcc8076926c612732fe01d0/setup.py#L107) that gets called:
    When you run `gunicorn` command on the command line [this is the entrypoint](https://github.com/benoitc/gunicorn/blob/4371ff2ed4babed2ffcc8076926c612732fe01d0/setup.py#L107) that gets called:
    ```
    gunicorn.app.wsgiapp:run
    ```
  4. @lukassup lukassup revised this gist Apr 2, 2018. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion zipapp.md
    Original file line number Diff line number Diff line change
    @@ -49,7 +49,7 @@ $ tree -L 1 --dirsfirst -F .

    Let's create a new directory called `app` which will be packaged into a _zipapp_ as a whole.

    In order to create a _zipapp_ you must have all dependencies installed in the same directory (e.g our `app` directory), not in a virtualenv. I'm going to create a [`requirements.txt`](https://pip.pypa.io/en/stable/user_guide/#requirements-files) file to pin the versions of each installed dependent Python package that I've just installed in my virtualenv and then install all of them into the new `app` directory.
    In order to create a _zipapp_ you must have all dependencies installed in the same directory as your application source code (e.g our `app` directory), not in a virtualenv. I'm going to create a [`requirements.txt`](https://pip.pypa.io/en/stable/user_guide/#requirements-files) file to pin the versions of each installed dependent Python package that I've just installed in my virtualenv and then install all of them into the new `app` directory.

    Alternatively, you could skip the virtualenv step and run `pip install -t ./app Flask gunicorn` if you don't want to pin versions.

  5. @lukassup lukassup revised this gist Apr 2, 2018. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion zipapp.md
    Original file line number Diff line number Diff line change
    @@ -47,7 +47,7 @@ $ tree -L 1 --dirsfirst -F .

    ## Install all dependencies to project directory

    Let's create a new directory called `app` which will be packaged as a _zipapp_ as a whole.
    Let's create a new directory called `app` which will be packaged into a _zipapp_ as a whole.

    In order to create a _zipapp_ you must have all dependencies installed in the same directory (e.g our `app` directory), not in a virtualenv. I'm going to create a [`requirements.txt`](https://pip.pypa.io/en/stable/user_guide/#requirements-files) file to pin the versions of each installed dependent Python package that I've just installed in my virtualenv and then install all of them into the new `app` directory.

  6. @lukassup lukassup revised this gist Apr 2, 2018. 1 changed file with 63 additions and 54 deletions.
    117 changes: 63 additions & 54 deletions zipapp.md
    Original file line number Diff line number Diff line change
    @@ -2,14 +2,14 @@

    [What's a `zipapp`?](https://docs.python.org/3/library/zipapp.html)

    > NOTE: the built `.pyz` archive can run on Python 2 & 3 but you can only build such archives with Python 3.5+.
    > NOTE: The built `.pyz` _zipapp_ can run on both Python 2 & 3 but you can only build `.pyz` _zipapps_ with Python 3.5 or later.
    ## Initial setup

    You should have a Flask application in `flaskr`
    There is a single subdirectory called `flaskr` in the beginning. It's an example [Flask](http://flask.pocoo.org/) app that I'm going to package into a Python _zipapp_.

    ```
    tree -L 2 --dirsfirst -F .
    $ tree -L 2 --dirsfirst -F .
    .
    └── flaskr/
    ├── flaskr/
    @@ -26,18 +26,18 @@ tree -L 2 --dirsfirst -F .

    ## Install to virtualenv

    ```bash
    virtualenv -p python3 venv
    source ./venv/bin/activate
    pip install Flask gunicorn
    Now let's create a new [Python virtual environment](https://virtualenv.pypa.io/en/stable/) next to `flaskr` subdirectory and install `Flask` and `gunicorn` into the new virtual environment:

    ```
    $ virtualenv -p python3 venv
    $ source ./venv/bin/activate
    $ pip install Flask gunicorn
    ```

    Now your project should look like this
    Now there should be two subdirectories: `flaskr` and `venv`:

    ```bash
    tree -L 1 --dirsfirst -F .
    ```
    ```
    $ tree -L 1 --dirsfirst -F .
    .
    ├── flaskr/
    └── venv/
    @@ -47,38 +47,43 @@ tree -L 1 --dirsfirst -F .

    ## Install all dependencies to project directory

    ```bash
    mkdir app
    cd app
    pip freeze > requirements.txt
    pip install -t . -r requirements.txt
    Let's create a new directory called `app` which will be packaged as a _zipapp_ as a whole.

    In order to create a _zipapp_ you must have all dependencies installed in the same directory (e.g our `app` directory), not in a virtualenv. I'm going to create a [`requirements.txt`](https://pip.pypa.io/en/stable/user_guide/#requirements-files) file to pin the versions of each installed dependent Python package that I've just installed in my virtualenv and then install all of them into the new `app` directory.

    Alternatively, you could skip the virtualenv step and run `pip install -t ./app Flask gunicorn` if you don't want to pin versions.

    ```
    $ mkdir app
    $ cd app
    $ pip freeze > requirements.txt
    $ pip install -t . -r requirements.txt
    ```

    ## Add application code

    ```bash
    cp -r ../flaskr .
    Let's copy our application code into the new `app` directory:
    ```
    $ cp -r ../flaskr .
    ```

    or install it this way if you have a nice distributable Python package that has a `setup.py`

    ```bash
    pip install -t . ../flaskr
    Or install it if you have a nice distributable Python package that has a `setup.py`:
    ```
    $ pip install -t . ../flaskr
    ```

    ## Cleanup

    ```bash
    rm -rf ./__pycache__ ./*.dist-info
    cd ..
    To save disk space you can remove extra `pip` files and cache before creating a _zipapp_:
    ```
    $ rm -rf ./__pycache__ ./*.dist-info
    $ cd ..
    ```

    Now your project should look like this
    Now your project should look like this.

    ```bash
    tree -L 2 --dirsfirst -F .
    ```
    ```
    $ tree -L 2 --dirsfirst -F .
    .
    ├── app/
    │ ├── click/
    @@ -109,44 +114,53 @@ tree -L 2 --dirsfirst -F .

    ## Create a `.pyz` app archive

    Set the entrypoint of the archive to `gunicorn.app.wsgiapp:run` with the `-m` flag which is the same exact function when you run the `gunicorn` command. Also set the interpreter to `/usr/bin/env python3` with the `-p` flag.

    ```bash
    python3 -m zipapp app -m 'gunicorn.app.wsgiapp:run' -p '/usr/bin/env python3'
    _Zipapp_ module command has this syntax:
    ```
    $ python3 -m zipapp APP_DIR -m ENTRYPOINT_MODULE:ENTRYPOINT_FUNCTION -p PYTHON_INTERPRETER
    ```

    You get an executable `.pyz` archive with all dependencies bundled inside.
    I'm going to use [`gunicorn`](gunicorn.org/) to serve my Flask app because it's pure Python and will work accross any supported platform without worrying about binary compilation (e.g. `uwsgi`).

    ```bash
    ls -lh app.pyz
    When you run `gunicorn` command on the command line [this is the entrypoint](benoitc/gunicorn@4371ff2ed4babed2ffcc8076926c612732fe01d0/setup.py#L107) that gets called:
    ```
    gunicorn.app.wsgiapp:run
    ```
    -rwxr--r-- 1 user user 4.0M Dec 18 09:38 app.pyz
    So the command line flag for my _zipapp_ entrypoint will be `-m 'gunicorn.app.wsgiapp:run'`.

    I want to use Python 3 so I also set the interpreter with `-p /usr/bin/env python3`.

    Let's run our _zipapp_ command
    ```
    ```bash
    file app.pyz
    $ python3 -m zipapp app -m 'gunicorn.app.wsgiapp:run' -p '/usr/bin/env python3'
    ```

    You should get an executable `.pyz` archive with all dependencies bundled inside:
    ```
    $ ls -lh app.pyz
    -rwxr--r-- 1 user user 4.0M Dec 18 09:38 app.pyz
    $ file app.pyz
    app.pyz: a /usr/bin/env python3 script executable (binary data)
    ```

    ## Run the app from archive

    ```bash
    ./app.pyz flaskr:app
    ```
    In order to run my app I have to provide the entrypoint for `gunicorn` like I would run any WSGI Python app. In my case it's `flaskr:app`:
    ```
    $ ./app.pyz flaskr:app
    [2017-12-18 09:38:35 +0200] [39081] [INFO] Starting gunicorn 19.7.1
    [2017-12-18 09:38:35 +0200] [39081] [INFO] Listening at: http://127.0.0.1:8000 (39081)
    [2017-12-18 09:38:35 +0200] [39081] [INFO] Using worker: sync
    [2017-12-18 09:38:35 +0200] [39084] [INFO] Booting worker with pid: 39084
    ```

    Hooray, it's running! 🎉

    ## BONUS: Default entrypoint

    Let's say I don't want to add `flaskr:app` each time I want to run this archive.
    You can do it by creating an entrypoint `app/__main__.py` with this content:
    Let's say I don't want to set my `flaskr:app` entrypoint each time I run the `.pyz` archive.

    You can do that by creating a custom `app` entrypoint `app/__main__.py` with this content:
    ```python
    # -*- coding: utf-8 -*-
    import sys
    @@ -155,21 +169,16 @@ sys.argv.append('flaskr:app')
    sys.exit(run())
    ```

    Repackage your app

    ```bash
    python3 -m zipapp app -p '/usr/bin/env python3'
    The repackage your app
    ```

    Run it

    ```bash
    ./app.pyz
    $ python3 -m zipapp app -p '/usr/bin/env python3'
    ```

    And run it
    ```
    $ ./app.pyz
    [2017-12-18 13:35:30 +0200] [44258] [INFO] Starting gunicorn 19.7.1
    [2017-12-18 13:35:30 +0200] [44258] [INFO] Listening at: http://127.0.0.1:8000 (44258)
    [2017-12-18 13:35:30 +0200] [44258] [INFO] Using worker: sync
    [2017-12-18 13:35:30 +0200] [44261] [INFO] Booting worker with pid: 44261
    ```

  7. @lukassup lukassup revised this gist Dec 18, 2017. 1 changed file with 67 additions and 0 deletions.
    67 changes: 67 additions & 0 deletions zipapp.md
    Original file line number Diff line number Diff line change
    @@ -4,6 +4,26 @@

    > NOTE: the built `.pyz` archive can run on Python 2 & 3 but you can only build such archives with Python 3.5+.
    ## Initial setup

    You should have a Flask application in `flaskr`

    ```
    tree -L 2 --dirsfirst -F .
    .
    └── flaskr/
    ├── flaskr/
    ├── flaskr.egg-info/
    ├── venv/
    ├── LICENSE
    ├── MANIFEST.in
    ├── README.rst
    ├── requirements.txt
    └── setup.py
    4 directories, 5 files
    ```

    ## Install to virtualenv

    ```bash
    @@ -12,6 +32,19 @@ source ./venv/bin/activate
    pip install Flask gunicorn
    ```

    Now your project should look like this

    ```bash
    tree -L 1 --dirsfirst -F .
    ```
    ```
    .
    ├── flaskr/
    └── venv/
    2 directories, 0 files
    ```

    ## Install all dependencies to project directory

    ```bash
    @@ -40,6 +73,40 @@ rm -rf ./__pycache__ ./*.dist-info
    cd ..
    ```

    Now your project should look like this

    ```bash
    tree -L 2 --dirsfirst -F .
    ```
    ```
    .
    ├── app/
    │ ├── click/
    │ ├── flask/
    │ ├── gunicorn/
    │ ├── jinja2/
    │ ├── markupsafe/
    │ ├── werkzeug/
    │ ├── itsdangerous.py
    │ └── requirements.txt
    ├── flaskr/
    │ ├── flaskr/
    │ ├── flaskr.egg-info/
    │ ├── venv/
    │ ├── LICENSE
    │ ├── MANIFEST.in
    │ ├── README.rst
    │ ├── requirements.txt
    │ └── setup.py
    └── venv/
    ├── bin/
    ├── include/
    ├── lib/
    └── pip-selfcheck.json
    15 directories, 8 files
    ```

    ## Create a `.pyz` app archive

    Set the entrypoint of the archive to `gunicorn.app.wsgiapp:run` with the `-m` flag which is the same exact function when you run the `gunicorn` command. Also set the interpreter to `/usr/bin/env python3` with the `-p` flag.
  8. @lukassup lukassup revised this gist Dec 18, 2017. 1 changed file with 2 additions and 0 deletions.
    2 changes: 2 additions & 0 deletions zipapp.md
    Original file line number Diff line number Diff line change
    @@ -2,6 +2,8 @@

    [What's a `zipapp`?](https://docs.python.org/3/library/zipapp.html)

    > NOTE: the built `.pyz` archive can run on Python 2 & 3 but you can only build such archives with Python 3.5+.
    ## Install to virtualenv

    ```bash
  9. @lukassup lukassup revised this gist Dec 18, 2017. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion zipapp.md
    Original file line number Diff line number Diff line change
    @@ -23,7 +23,6 @@ pip install -t . -r requirements.txt

    ```bash
    cp -r ../flaskr .
    cd ..
    ```

    or install it this way if you have a nice distributable Python package that has a `setup.py`
    @@ -36,6 +35,7 @@ pip install -t . ../flaskr

    ```bash
    rm -rf ./__pycache__ ./*.dist-info
    cd ..
    ```

    ## Create a `.pyz` app archive
  10. @lukassup lukassup revised this gist Dec 18, 2017. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion zipapp.md
    Original file line number Diff line number Diff line change
    @@ -29,7 +29,7 @@ cd ..
    or install it this way if you have a nice distributable Python package that has a `setup.py`

    ```bash
    pip install -t . ../flask
    pip install -t . ../flaskr
    ```

    ## Cleanup
  11. @lukassup lukassup revised this gist Dec 18, 2017. 1 changed file with 33 additions and 1 deletion.
    34 changes: 33 additions & 1 deletion zipapp.md
    Original file line number Diff line number Diff line change
    @@ -71,4 +71,36 @@ app.pyz: a /usr/bin/env python3 script executable (binary data)
    [2017-12-18 09:38:35 +0200] [39081] [INFO] Listening at: http://127.0.0.1:8000 (39081)
    [2017-12-18 09:38:35 +0200] [39081] [INFO] Using worker: sync
    [2017-12-18 09:38:35 +0200] [39084] [INFO] Booting worker with pid: 39084
    ```
    ```

    ## BONUS: Default entrypoint

    Let's say I don't want to add `flaskr:app` each time I want to run this archive.
    You can do it by creating an entrypoint `app/__main__.py` with this content:

    ```python
    # -*- coding: utf-8 -*-
    import sys
    from gunicorn.app.wsgiapp import run
    sys.argv.append('flaskr:app')
    sys.exit(run())
    ```

    Repackage your app

    ```bash
    python3 -m zipapp app -p '/usr/bin/env python3'
    ```

    Run it

    ```bash
    ./app.pyz
    ```
    ```
    [2017-12-18 13:35:30 +0200] [44258] [INFO] Starting gunicorn 19.7.1
    [2017-12-18 13:35:30 +0200] [44258] [INFO] Listening at: http://127.0.0.1:8000 (44258)
    [2017-12-18 13:35:30 +0200] [44258] [INFO] Using worker: sync
    [2017-12-18 13:35:30 +0200] [44261] [INFO] Booting worker with pid: 44261
    ```

  12. @lukassup lukassup revised this gist Dec 18, 2017. 1 changed file with 12 additions and 1 deletion.
    13 changes: 12 additions & 1 deletion zipapp.md
    Original file line number Diff line number Diff line change
    @@ -17,7 +17,6 @@ mkdir app
    cd app
    pip freeze > requirements.txt
    pip install -t . -r requirements.txt
    rm -rf ./__pycache__ ./*.dist-info
    ```

    ## Add application code
    @@ -27,6 +26,18 @@ cp -r ../flaskr .
    cd ..
    ```

    or install it this way if you have a nice distributable Python package that has a `setup.py`

    ```bash
    pip install -t . ../flask
    ```

    ## Cleanup

    ```bash
    rm -rf ./__pycache__ ./*.dist-info
    ```

    ## Create a `.pyz` app archive

    Set the entrypoint of the archive to `gunicorn.app.wsgiapp:run` with the `-m` flag which is the same exact function when you run the `gunicorn` command. Also set the interpreter to `/usr/bin/env python3` with the `-p` flag.
  13. @lukassup lukassup revised this gist Dec 18, 2017. 1 changed file with 2 additions and 0 deletions.
    2 changes: 2 additions & 0 deletions zipapp.md
    Original file line number Diff line number Diff line change
    @@ -29,6 +29,8 @@ cd ..

    ## Create a `.pyz` app archive

    Set the entrypoint of the archive to `gunicorn.app.wsgiapp:run` with the `-m` flag which is the same exact function when you run the `gunicorn` command. Also set the interpreter to `/usr/bin/env python3` with the `-p` flag.

    ```bash
    python3 -m zipapp app -m 'gunicorn.app.wsgiapp:run' -p '/usr/bin/env python3'
    ```
  14. @lukassup lukassup revised this gist Dec 18, 2017. 1 changed file with 2 additions and 0 deletions.
    2 changes: 2 additions & 0 deletions zipapp.md
    Original file line number Diff line number Diff line change
    @@ -1,5 +1,7 @@
    # Python zipapp web apps

    [What's a `zipapp`?](https://docs.python.org/3/library/zipapp.html)

    ## Install to virtualenv

    ```bash
  15. @lukassup lukassup created this gist Dec 18, 2017.
    59 changes: 59 additions & 0 deletions zipapp.md
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,59 @@
    # Python zipapp web apps

    ## Install to virtualenv

    ```bash
    virtualenv -p python3 venv
    source ./venv/bin/activate
    pip install Flask gunicorn
    ```

    ## Install all dependencies to project directory

    ```bash
    mkdir app
    cd app
    pip freeze > requirements.txt
    pip install -t . -r requirements.txt
    rm -rf ./__pycache__ ./*.dist-info
    ```

    ## Add application code

    ```bash
    cp -r ../flaskr .
    cd ..
    ```

    ## Create a `.pyz` app archive

    ```bash
    python3 -m zipapp app -m 'gunicorn.app.wsgiapp:run' -p '/usr/bin/env python3'
    ```

    You get an executable `.pyz` archive with all dependencies bundled inside.

    ```bash
    ls -lh app.pyz
    ```
    ```
    -rwxr--r-- 1 user user 4.0M Dec 18 09:38 app.pyz
    ```
    ```bash
    file app.pyz
    ```
    ```
    app.pyz: a /usr/bin/env python3 script executable (binary data)
    ```

    ## Run the app from archive

    ```bash
    ./app.pyz flaskr:app
    ```
    ```
    [2017-12-18 09:38:35 +0200] [39081] [INFO] Starting gunicorn 19.7.1
    [2017-12-18 09:38:35 +0200] [39081] [INFO] Listening at: http://127.0.0.1:8000 (39081)
    [2017-12-18 09:38:35 +0200] [39081] [INFO] Using worker: sync
    [2017-12-18 09:38:35 +0200] [39084] [INFO] Booting worker with pid: 39084
    ```