# Virtualenv's `bin/activate` is Doing It Wrong

I'm a Python programmer and frequently work with the excellent [virtualenv][] tool by Ian Bicking.

Virtualenv is a great tool on the whole but there is one glaring problem: the `activate` script that virtualenv provides as a convenience to enable its functionality requires you to _source_ it with your shell to invoke it. The `activate` script sets some environment variables in your _current_ environment and defines for you a `deactivate` _shell function_ which will (attempt to) help you to undo those changes later.

This pattern is abhorrently wrong and un-unix-y. `activate` should instead do what `ssh-agent` does, and launch a sub-shell or sub-command with a modified environment.

## Problems

The approach of modifying the user's current environment suffers from a number of problems:

- It breaks if you don't use a supported shell.
- A separate activate script must be maintained for each supported shell syntax.
- What do you do if you use no shell at all? (I.E. run programs in a virtualenv from a GUI.)
- If the `deactivate` script fails to un-set an environment variable, it may contaminate other environments.
- If you want to edit `deactivate` or any other function sourced into your environment, you have to kill your shell and re-source the script to see the changes take effect.
- If you change the current directory from one to another virtual environment and forget to carefully `deactivate` and `activate` as you do so, you may end up using libraries from or making changes in the wrong one!

Virtualenv's `activate` suffers from a number of other warts as well:

- You can't simply run the script; you have to learn and employ your shell's "source this script" builtin. Many non-experts frequently stumble over this distinction. Doing away with the recommendation to source a shell script should make virtualenv easier to use.

		# This file must be used with "source bin/activate" *from bash*
		# you cannot run it directly

- In an attempt to preserve the user's old environment, it declares \_OLD_VIRTUAL_PATH, \_OLD_VIRTUAL_PYTHONHOME, and \_OLD_VIRTUAL_PS1, and must define how to restore them upon deactivation. If you happen to want to modify `activate` to override more variables specific to your environment, you have to do the same.

- Its default means to display whether or not a virtual environment is currently active (modifying the user's PS1 variable) is fragile. On Debian and Ubuntu boxes it becomes confusing if one enters a subshell, or uses a tool like `screen` or `tmux`.

- It is not executable, and not meant to be used as an executable, yet it lives in a a directory named `bin`.

## Doing It Right

Entering and exiting a virtual environment should be like using `ssh` to connect to another machine. When you're done, a simple `exit` should restore you to your original, unmodified environment.

An example of a program that does this the Right Way is `ssh-agent`. In order to communicate the port that it uses to other programs, it must set some variables into the environment. It provides an option to do what virtualenv does, but the better way is to simply ask `ssh-agent` to launch your command for you, with a modified environment. `ssh-agent $SHELL` will launch a sub-shell for you with its environment already modified appropriately for `ssh-agent`. Most Debian and Ubuntu machines even launch X11 this way; see `/etc/X11/Xsession.d/90x11-common_ssh-agent`.

Another advantage to the subshell approach is that it is far simpler than the hoops virtualenv jumps through to activate and deactivate an environment. There's no need to set \_OLD\_ variables since the former environment is restored automatically. There's no need for a `deactivate` function.

Finally, employing a prompt context variable instead of messing with PS1 would allow the user to define how that information is presented.

## A better `activate`: "`inve`"

To differentiate, I'm calling this approach "inve" as in "inside this virtual environment, ..." I'll happily take name suggestions.

### Launching a subcommand with a modified environment

How do we make an executable like `ssh-agent` that launches a subcommand with a modified environment? Easy. Call this `my_launcher`:

	#!/bin/sh
	export MY_VAR=xyz
	exec "$@"

Calling "my_launcher firefox" will launch firefox with MY_VAR set to 'xyz' in its environment. The environment where "my_launcher" is called from will not be disturbed.

### Simplifying `activate`

Let's now examine `bin/activate` to see what we can throw away if we assume that the system takes care of restoring the environment for us when we `exit`. We don't need the `deactivate` shell function at all. We don't need any \_OLD_ variables. We don't mess with the prompt. What's left?

	export VIRTUAL_ENV="/home/mike/var/virtualenvs/myvirtualenv"
	export PATH="$VIRTUAL_ENV/bin:$PATH"
	unset PYTHON_HOME

That's it. Three lines, down from 76. Down from 187 if you count all variants for other shells.

Wrap this with the launcher technique above, call it `inve`, and `./bin/inve $SHELL` spawns a new subshell in the active virtualenv. What if you want a no-argument invocation to default to spawning an activated shell? This is the entire script:

	#!/bin/sh
	export VIRTUAL_ENV="/home/mike/var/virtualenvs/myvirtualenv"
	export PATH="$VIRTUAL_ENV/bin:$PATH"
	unset PYTHON_HOME
	exec "${@:-$SHELL}"

Now `bin/inve` does what `bin/activate` _should_. By the way: this works for _all shells_. bash, zsh, csh, fish, ksh, and anything else, with one script.

## More hacks

### Re-enabling current environment modification

Some users source `bin/activate` from within their own shell scripts, which I don't find quite as offensive.

`ssh-agent` also supports this style of use. It too has to deal with the syntax differences between shells to do so. It's not hard to enable this; here's one proposal.

	#!/bin/sh
	
	# As above, do what's needed to activate
	export VIRTUAL_ENV="/home/mike/var/virtualenvs/myvirtualenv"
	export PATH="$VIRTUAL_ENV/bin:$PATH"
	unset PYTHON_HOME

	# If the first argument is -s or -c, do what ssh-agent does
	if [ "$1" = "-s" ]; then cat <<- DONE
		export VIRTUAL_ENV="$VIRTUAL_ENV";
		export PATH="$PATH";
		unset PYTHON_HOME;
	DONE
	elif [ "$1" = "-c" ]; then cat <<- DONE
		setenv VIRTUAL_ENV "$VIRTUAL_ENV";
		setenv PATH "$PATH";
		unset PYTHON_HOME;
	DONE

	# Otherwise, launch a shell or subcommand
	else
		exec "${@:-$SHELL}"
	fi

Now `inve` supports the same -s and -c options that `ssh-agent` does. Where one might previously have written a script like this:

	#!/bin/sh
	source ./activate
	... (commands) ...

One would now write instead:

	#!/bin/sh
	eval `./inve -s`
	... (commands) ...

Or, for csh:

	#!/bin/csh
	eval `./inve -c`
	... (commands) ...

Unfortunately, I don't know if this "eval the output of a command" technique works for all possible shells.

### A system-level `inve`

I find it convenient to employ a "system-level" `inve` script that lives in my system `$PATH`, that I can run from anywhere within any virtual environment, and without specifying the full path to 'ENV/bin/inve'. This goes against the intention that "virtualenvs are self-sufficient once created" so I'm not advocating this technique be used _instead_ of `ENV/bin/inve`.

	#!/bin/sh

	# inve
	#
	# usage: inve [COMMAND [ARGS]]
	#
	# For use with Ian Bicking's virtualenv tool. Attempts to find the root of
	# a virtual environment. Then, executes COMMAND with ARGS in the context of
	# the activated environment. If no COMMAND is given, activate defaults to a
	# subshell.

	# First, locate the root of the current virtualenv
	while [ "$PWD" != "/" ]; do
		# Stop here if this the root of a virtualenv
		if [ \
			-x bin/python \
			-a -e lib/python*/site.py \
			-a -e include/python*/Python.h ]
		then
			break
		fi
		cd ..
	done
	if [ "$PWD" = "/" ]; then
		echo "Could not activate: no virtual environment found." >&2
		exit 1
	fi

	# Activate
	export VIRTUAL_ENV="$PWD"
	export PATH="$VIRTUAL_ENV/bin:$PATH"
	unset PYTHON_HOME
	exec "${@:-$SHELL}"

Until an `inve`-like script gets created in virtualenv `bin/` directories, this system-level script will allow you to immediately use the subshell technique with all existing virtualenvs. If ever the `inve` script does land in virtualenv's `bin/`, this system level script could be simply a helper that searches for and invokes `ENV/bin/inve`:

	# Locate the root of the current virtualenv
	... (same as above) ...
	# Activate
	exec bin/inve "$@"

### Don't mess with my prompt

But what about the prompt? Build a PS1 that does the right thing everywhere without needing to be modified to suit a particular purpose. I tend to have a function that collects all the context info this way, in my .bashrc:

	function ps1_context {
		# For any of these bits of context that exist, display them and append
		# a space.
		virtualenv=`basename "$VIRTUAL_ENV"`
		for v in "$debian_chroot" "$virtualenv" "$PS1_CONTEXT"; do
			echo -n "${v:+$v }"
		done
	}

	export PS1="$(ps1_context)"'\u@\h:\w\$ '

This lets the user control their PS1 and it works everywhere, no matter how many subshells or screen sessions you're nested into. This is the only piece that has to be customized per-shell.

## Conclusion

While using `activate` is intended only a convenience and is [not necessary to work within a virtual environment][1], most of programmers I know treat it as a black box and never do without it. I suspect that, in part, the complexity of the script is what prevents more programmers from avoiding it.

Perhaps the worst part about a popular, useful tool like virtualenv using this antipattern is that many other programmers are adopting it as normative and using it for their own work. [virtualenvwrapper][] and [dustinlacewell/capn][capn] are two examples. Stop doing this, everyone!

# Post-Script Update: Other Projects

I wrote this back in March 2012, and cobbled together at that time some scripts that worked well-enough for me to get away from virtualenv's bad behavior. After some encouragement and feedback from the community I started a patch to the virtualenv project that would make it use this method, and added some adapter code that would avoid breaking the workflow for those used to the current-shell-modification behavior, but I still haven't finished it. Meanwhile, some great projects have sprung up and properly implemented this idea:

- berdario's [invewrapper][]
- sashahart's [vex][]

I'm super happy about this. I don't mind parallel implementations; it's all open source and we can build upon one another's good ideas.

[1]: https://github.com/pypa/virtualenv/issues/247#issuecomment-4853286

[virtualenv]: http://www.virtualenv.org
[subcommander]: https://github.com/datagrok/subcommander
[capn]: https://github.com/dustinlacewell/capn
[virtualenvwrapper]: http://www.doughellmann.com/projects/virtualenvwrapper/
[invewrapper]: https://github.com/berdario/invewrapper
[vex]: https://github.com/sashahart/vex