-
-
Save zennro/9485830 to your computer and use it in GitHub Desktop.
Revisions
-
datagrok revised this gist
Apr 8, 2012 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -192,7 +192,7 @@ This lets the user control their PS1 and it works everywhere, no matter how many While using `activate` is intended only a convenience and is [not necessary to work within a virtual environment][1], most of programmers I know treat it as a black box and never do without it. I suspect that, in part, the complexity of the script is what prevents more programmers from avoiding it. Perhaps the worst part about a popular, useful tool like virtualenv using this antipattern is that many other programmers are adopting it as normative and using it for their own work. [virtualenvwrapper][] and [dustinlacewell/capn][capn] are two examples. Stop doing this, everyone! [1]: https://github.com/pypa/virtualenv/issues/247#issuecomment-4853286 -
datagrok revised this gist
Mar 31, 2012 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -19,7 +19,7 @@ The approach of modifying the user's current environment suffers from a number o Virtualenv's `activate` suffers from a number of other warts as well: - You can't simply run the script; you have to learn and employ your shell's "source this script" builtin. Many non-experts frequently stumble over this distinction. Doing away with the recommendation to source a shell script should make virtualenv easier to use. # This file must be used with "source bin/activate" *from bash* # you cannot run it directly -
datagrok revised this gist
Mar 31, 2012 . 1 changed file with 2 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -28,6 +28,8 @@ Virtualenv's `activate` suffers from a number of other warts as well: - Its default means to display whether or not a virtual environment is currently active (modifying the user's PS1 variable) is fragile. On Debian and Ubuntu boxes it becomes confusing if one enters a subshell, or uses a tool like `screen` or `tmux`. - It is not executable, and not meant to be used as an executable, yet it lives in a a directory named `bin`. ## Doing It Right Entering and exiting a virtual environment should be like using `ssh` to connect to another machine. When you're done, a simple `exit` should restore you to your original, unmodified environment. -
datagrok revised this gist
Mar 31, 2012 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -38,7 +38,7 @@ Another advantage to the subshell approach is that it is far simpler than the ho Finally, employing a prompt context variable instead of messing with PS1 would allow the user to define how that information is presented. ## A better `activate`: "`inve`" To differentiate, I'm calling this approach "inve" as in "inside this virtual environment, ..." I'll happily take name suggestions. -
datagrok revised this gist
Mar 31, 2012 . 1 changed file with 3 additions and 5 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -126,9 +126,7 @@ Unfortunately, I don't know if this "eval the output of a command" technique wor ### A system-level `inve` I find it convenient to employ a "system-level" `inve` script that lives in my system `$PATH`, that I can run from anywhere within any virtual environment, and without specifying the full path to 'ENV/bin/inve'. This goes against the intention that "virtualenvs are self-sufficient once created" so I'm not advocating this technique be used _instead_ of `ENV/bin/inve`. #!/bin/sh @@ -164,7 +162,7 @@ Also, until an `inve`-like script gets created in virtualenv `bin/` directories, unset PYTHON_HOME exec "${@:-$SHELL}" Until an `inve`-like script gets created in virtualenv `bin/` directories, this system-level script will allow you to immediately use the subshell technique with all existing virtualenvs. If ever the `inve` script does land in virtualenv's `bin/`, this system level script could be simply a helper that searches for and invokes `ENV/bin/inve`: # Locate the root of the current virtualenv ... (same as above) ... @@ -192,7 +190,7 @@ This lets the user control their PS1 and it works everywhere, no matter how many While using `activate` is intended only a convenience and is [not necessary to work within a virtual environment][1], most of programmers I know treat it as a black box and never do without it. I suspect that, in part, the complexity of the script is what prevents more programmers from avoiding it. Perhaps the worst part about a popular, useful tool like virtualenv using this antipattern is that many other programmers are adopting it as normative and using it for their own work. [virtualenvwrapper][] and [dustinlacewell/capn][capn] are two examples who have had the their brains eaten by the "source bin/activate" worms. Stop doing this, everyone! [1]: https://github.com/pypa/virtualenv/issues/247#issuecomment-4853286 -
datagrok revised this gist
Mar 31, 2012 . 1 changed file with 86 additions and 16 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -2,9 +2,9 @@ I'm a Python programmer and frequently work with the excellent [virtualenv][] tool by Ian Bicking. Virtualenv is a great tool on the whole but there is one glaring problem: the `activate` script that virtualenv provides as a convenience to enable its functionality requires you to _source_ it with your shell to invoke it. The `activate` script sets some environment variables in your _current_ environment and defines for you a `deactivate` _shell function_ which will (attempt to) help you to undo those changes later. This pattern is abhorrently wrong and un-unix-y. `activate` should instead do what `ssh-agent` does, and launch a sub-shell or sub-command with a modified environment. ## Problems @@ -24,24 +24,27 @@ Virtualenv's `activate` suffers from a number of other warts as well: # This file must be used with "source bin/activate" *from bash* # you cannot run it directly - In an attempt to preserve the user's old environment, it declares \_OLD_VIRTUAL_PATH, \_OLD_VIRTUAL_PYTHONHOME, and \_OLD_VIRTUAL_PS1, and must define how to restore them upon deactivation. If you happen to want to modify `activate` to override more variables specific to your environment, you have to do the same. - Its default means to display whether or not a virtual environment is currently active (modifying the user's PS1 variable) is fragile. On Debian and Ubuntu boxes it becomes confusing if one enters a subshell, or uses a tool like `screen` or `tmux`. ## Doing It Right Entering and exiting a virtual environment should be like using `ssh` to connect to another machine. When you're done, a simple `exit` should restore you to your original, unmodified environment. An example of a program that does this the Right Way is `ssh-agent`. In order to communicate the port that it uses to other programs, it must set some variables into the environment. It provides an option to do what virtualenv does, but the better way is to simply ask `ssh-agent` to launch your command for you, with a modified environment. `ssh-agent $SHELL` will launch a sub-shell for you with its environment already modified appropriately for `ssh-agent`. Most Debian and Ubuntu machines even launch X11 this way; see `/etc/X11/Xsession.d/90x11-common_ssh-agent`. Another advantage to the subshell approach is that it is far simpler than the hoops virtualenv jumps through to activate and deactivate an environment. There's no need to set \_OLD\_ variables since the former environment is restored automatically. There's no need for a `deactivate` function. Finally, employing a prompt context variable instead of messing with PS1 would allow the user to define how that information is presented. ## `inve`: a better `activate` To differentiate, I'm calling this approach "inve" as in "inside this virtual environment, ..." I'll happily take name suggestions. ### Launching a subcommand with a modified environment How do we make an executable like `ssh-agent` that launches a subcommand with a modified environment? Easy. Call this `my_launcher`: #!/bin/sh export MY_VAR=xyz @@ -59,31 +62,87 @@ Let's now examine `bin/activate` to see what we can throw away if we assume that That's it. Three lines, down from 76. Down from 187 if you count all variants for other shells. Wrap this with the launcher technique above, call it `inve`, and `./bin/inve $SHELL` spawns a new subshell in the active virtualenv. What if you want a no-argument invocation to default to spawning an activated shell? This is the entire script: #!/bin/sh export VIRTUAL_ENV="/home/mike/var/virtualenvs/myvirtualenv" export PATH="$VIRTUAL_ENV/bin:$PATH" unset PYTHON_HOME exec "${@:-$SHELL}" Now `bin/inve` does what `bin/activate` _should_. By the way: this works for _all shells_. bash, zsh, csh, fish, ksh, and anything else, with one script. ## More hacks ### Re-enabling current environment modification Some users source `bin/activate` from within their own shell scripts, which I don't find quite as offensive. `ssh-agent` also supports this style of use. It too has to deal with the syntax differences between shells to do so. It's not hard to enable this; here's one proposal. #!/bin/sh # As above, do what's needed to activate export VIRTUAL_ENV="/home/mike/var/virtualenvs/myvirtualenv" export PATH="$VIRTUAL_ENV/bin:$PATH" unset PYTHON_HOME # If the first argument is -s or -c, do what ssh-agent does if [ "$1" = "-s" ]; then cat <<- DONE export VIRTUAL_ENV="$VIRTUAL_ENV"; export PATH="$PATH"; unset PYTHON_HOME; DONE elif [ "$1" = "-c" ]; then cat <<- DONE setenv VIRTUAL_ENV "$VIRTUAL_ENV"; setenv PATH "$PATH"; unset PYTHON_HOME; DONE # Otherwise, launch a shell or subcommand else exec "${@:-$SHELL}" fi Now `inve` supports the same -s and -c options that `ssh-agent` does. Where one might previously have written a script like this: #!/bin/sh source ./activate ... (commands) ... One would now write instead: #!/bin/sh eval `./inve -s` ... (commands) ... Or, for csh: #!/bin/csh eval `./inve -c` ... (commands) ... Unfortunately, I don't know if this "eval the output of a command" technique works for all possible shells. ### A system-level `inve` I find it convenient to employ a "system-level" `inve` script that lives in my system `$PATH`, that I can run from anywhere within any virtual environment, and without specifying the full path to 'ENV/bin/inve'. It is a useful feature for many that virtualenvs are "self-sufficient once created" so I'm not advocating this technique be used instead of the above. Also, until an `inve`-like script gets created in virtualenv `bin/` directories, this one script will allow you to use the `inve` technique with all existing virtualenvs. #!/bin/sh # inve # # usage: inve [COMMAND [ARGS]] # # For use with Ian Bicking's virtualenv tool. Attempts to find the root of # a virtual environment. Then, executes COMMAND with ARGS in the context of # the activated environment. If no COMMAND is given, activate defaults to a # subshell. # First, locate the root of the current virtualenv while [ "$PWD" != "/" ]; do # Stop here if this the root of a virtualenv if [ \ -x bin/python \ @@ -105,9 +164,16 @@ With some cleverness, this `activate` could be made into a system-level command unset PYTHON_HOME exec "${@:-$SHELL}" If ever the `inve` script does land in virtualenv's `bin/`, this system level script could be simply a helper that searches for and invokes it like this: # Locate the root of the current virtualenv ... (same as above) ... # Activate exec bin/inve "$@" ### Don't mess with my prompt But what about the prompt? Build a PS1 that does the right thing everywhere without needing to be modified to suit a particular purpose. I tend to have a function that collects all the context info this way, in my .bashrc: function ps1_context { # For any of these bits of context that exist, display them and append @@ -124,8 +190,12 @@ This lets the user control their PS1 and it works everywhere, no matter how many ## Conclusion While using `activate` is intended only a convenience and is [not necessary to work within a virtual environment][1], most of programmers I know treat it as a black box and never do without it. I suspect that, in part, the complexity of the script is what prevents more programmers from avoiding it. Perhaps the worst part about a popular, useful tool like virtualenv using this antipattern is that many other programmers are adopting it as normative and using it for their own work. [virtualenvwrapper][] and [dustinlacewell/capn][capn] are two examples who have had the their brains eaten by the virtualenv bin/activate worms. Stop doing this, everyone! [1]: https://github.com/pypa/virtualenv/issues/247#issuecomment-4853286 [virtualenv]: http://www.virtualenv.org [subcommander]: https://github.com/datagrok/subcommander [capn]: https://github.com/dustinlacewell/capn -
datagrok revised this gist
Mar 28, 2012 . 1 changed file with 13 additions and 3 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -73,6 +73,15 @@ With some cleverness, this `activate` could be made into a system-level command #!/bin/sh # activate # # usage: activate [COMMAND [ARGS]] # # For use with Ian Bicking's virtualenv tool. Attempts to find the root of # a virtual environment. Then, executes COMMAND with ARGS in the context of # the activated environment. If no COMMAND is given, activate defaults to a # subshell. # First, locate the virtualenv while [ `pwd` != '/' ]; do # Stop here if this the root of a virtualenv @@ -85,12 +94,13 @@ With some cleverness, this `activate` could be made into a system-level command fi cd .. done if [ "$PWD" = "/" ]; then echo "Could not activate: no virtual environment found." >&2 exit 1 fi # Activate export VIRTUAL_ENV="$PWD" export PATH="$VIRTUAL_ENV/bin:$PATH" unset PYTHON_HOME exec "${@:-$SHELL}" @@ -108,7 +118,7 @@ But what about the prompt? Build a PS1 that does the right thing everywhere with done } export PS1="$(ps1_context)"'\u@\h:\w\$ ' This lets the user control their PS1 and it works everywhere, no matter how many subshells or screen sessions you're nested into. This is the only piece that has to be customized per-shell. @@ -119,4 +129,4 @@ Perhaps the worst part about a popular, useful tool like virtualenv using this a [virtualenv]: http://www.virtualenv.org [subcommander]: https://github.com/datagrok/subcommander [capn]: https://github.com/dustinlacewell/capn [virtualenvwrapper]: http://www.doughellmann.com/projects/virtualenvwrapper/ -
datagrok created this gist
Mar 25, 2012 .There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,122 @@ # Virtualenv's `bin/activate` is Doing It Wrong I'm a Python programmer and frequently work with the excellent [virtualenv][] tool by Ian Bicking. Virtualenv is great on the whole but there is one glaring problem: it requires you to _source_ with your shell an `activate` script before you can work within a virtualenv environment. The `activate` script sets some environment variables in your _current_ environment and defines for you a _shell function_ which will attempt to undo those changes. This is abhorrently wrong and un-unix-y. It should instead do what `ssh-agent` does, and launch a sub-shell or sub-command with a modified environment. ## Problems The approach of modifying the user's current environment suffers from a number of problems: - It breaks if you don't use a supported shell. - A separate activate script must be maintained for each supported shell syntax. - What do you do if you use no shell at all? (I.E. run programs in a virtualenv from a GUI.) - If the `deactivate` script fails to un-set an environment variable, it may contaminate other environments. - If you want to edit `deactivate` or any other function sourced into your environment, you have to kill your shell and re-source the script to see the changes take effect. - If you change the current directory from one to another virtual environment and forget to carefully `deactivate` and `activate` as you do so, you may end up using libraries from or making changes in the wrong one! Virtualenv's `activate` suffers from a number of other warts as well: - You can't simply run the script; you have to learn and employ your shell's "source this script" builtin: # This file must be used with "source bin/activate" *from bash* # you cannot run it directly - In an attempt to preserve the user's old environment, it declares \_OLD_VIRTUAL_PATH, \_OLD_VIRTUAL_PYTHONHOME, and \_OLD_VIRTUAL_PS1, and must define how to restore them upon deactivation. If you want to modify `activate` to override others, you have to do the same. - It messes with your prompt. For those of us with a PS1 setting in our system-level shell rc files, this gets clobbered if we start a subshell, or attempt to use `tmux` or `gnu screen` in an active virtualenv. Then it appears, incorrectly, that the virtualenv is deactivated. ## Doing It Right Entering and exiting a virtual environment should be like using `ssh` to connect to another machine. When you're done, a simple `exit` should restore you to your original, unmodified environment. An example of a program that does this the Right Way is `ssh-agent`. In order to communicate the port that it uses to other programs, it must set some variables into the environment. It provides an option to do what virtualenv does, but the better way is to simply ask `ssh-agent` to launch your command for you, with a modified environment. `ssh-agent $SHELL` will launch a sub-shell for you with its environment already modified appropriately for `ssh-agent`. Most Debian and Ubuntu machines even launch X11 this way; see `/etc/X11/Xsession.d/90x11-common_ssh-agent`. Another advantage to the subshell approach is that it is far simpler than the hoops virtualenv jumps through to activate and deactivate an environment. There's no need to set \_OLD_ variables since the former environment is restored automatically. There's no need for a `deactivate` function. Messing with the prompt is also bad news. A prompt should simply detect the state of its environment and present that to the user. What if a user has a fancy, customized prompt? What if a user wants to change how the prompt reports the virtualenv status? ## A better `activate` ### Launching a subcommand with a modified environment How do we make a command that launches a subcommand with a modified environment? Easy. Call this `my_launcher`: #!/bin/sh export MY_VAR=xyz exec "$@" Calling "my_launcher firefox" will launch firefox with MY_VAR set to 'xyz' in its environment. The environment where "my_launcher" is called from will not be disturbed. ### Simplifying `activate` Let's now examine `bin/activate` to see what we can throw away if we assume that the system takes care of restoring the environment for us when we `exit`. We don't need the `deactivate` shell function at all. We don't need any \_OLD_ variables. We don't mess with the prompt. What's left? export VIRTUAL_ENV="/home/mike/var/virtualenvs/myvirtualenv" export PATH="$VIRTUAL_ENV/bin:$PATH" unset PYTHON_HOME That's it. Three lines, down from 76. Down from 187 if you count all variants for other shells. Wrap this with the launcher technique above, call it `activate`, and `./bin/activate $SHELL` spawns a new subshell in the active virtualenv. What if you want a no-argument invocation to default to spawning an activated shell? This is the entire script: #!/bin/sh export VIRTUAL_ENV="/home/mike/var/virtualenvs/myvirtualenv" export PATH="$VIRTUAL_ENV/bin:$PATH" unset PYTHON_HOME exec "${@:-$SHELL}" Now 'bin/activate' does what it _should_. By the way: this works for _all shells_. bash, zsh, csh, fish, ksh, and anything else, with one script. With some cleverness, this `activate` could be made into a system-level command so you need not keep a copy in every virtualenv, and when calling it you could do so from within any directory in your virtualenv, and omit the "bin/". Something like this: #!/bin/sh # First, locate the virtualenv while [ `pwd` != '/' ]; do # Stop here if this the root of a virtualenv if [ \ -x bin/python \ -a -e lib/python*/site.py \ -a -e include/python*/Python.h ] then break fi cd .. done if [ "`pwd`" = "/" ]; then echo "Could not activate: no virtual environment found." >&2 exit 1 fi # Activate export PATH="$VIRTUAL_ENV/bin:$PATH" unset PYTHON_HOME exec "${@:-$SHELL}" ### Don't mess with my prompt But what about the prompt? Build a PS1 that does the right thing everywhere without needing to be hacked to a particular purpose. I tend to have a function that collects all the context info this way, in my .bashrc: function ps1_context { # For any of these bits of context that exist, display them and append # a space. virtualenv=`basename "$VIRTUAL_ENV"` for v in "$debian_chroot" "$virtualenv" "$PS1_CONTEXT"; do echo -n "${v:+$v }" done } PS1="$(ps1_context)"'\u@\h:\w\$ ' This lets the user control their PS1 and it works everywhere, no matter how many subshells or screen sessions you're nested into. This is the only piece that has to be customized per-shell. ## Conclusion Perhaps the worst part about a popular, useful tool like virtualenv using this antipattern is that many other programmers are adopting it as normative and using it for their own work. [virtualenvwrapper][] and [dustinlacewell/capn][capn] are two examples who have had the their brains eaten by the virtualenv bin/activate worms. Stop doing this, everyone! [virtualenv]: http://www.virtualenv.org [subcommander]: https://github.com/datagrok/subcommander [capn]: https://github.com/dustinlacewell/capn [virtualenvwrapper]: http://www.doughellmann.com/projects/virtualenvwrapper/