Using Virtual Environments
To get started, if you’re not using Python 3, you’ll want to install the virtualenv
tool with pip
:
If you are using Python 3, then you should already have the venv module from the standard library installed.
Note: From here on out, we’ll assume you’re using the newer venv
tool, since there are few differences between it and virtualenv
with regard to the actual commands. In reality, though, they are very different tools.
Start by making a new directory to work with:
$ mkdir python-virtual-environments && cd python-virtual-environments
Create a new virtual environment inside the directory:
# Python 2:
$ virtualenv env
# Python 3
$ python3 -m venv env
Note: By default, this will not include any of your existing site packages.
The Python 3 venv
approach has the benefit of forcing you to choose a specific version of the Python 3 interpreter that should be used to create the virtual environment. This avoids any confusion as to which Python installation the new environment is based on.
From Python 3.3 to 3.4, the recommended way to create a virtual environment was to use the pyvenv
command-line tool that also comes included with your Python 3 installation by default. But on 3.6 and above, python3 -m venv
is the way to go.
In the above example, this command creates a directory called env
, which contains a directory structure similar to this:
├── bin
│ ├── activate
│ ├── activate.csh
│ ├── activate.fish
│ ├── easy_install
│ ├── easy_install-3.5
│ ├── pip
│ ├── pip3
│ ├── pip3.5
│ ├── python -> python3.5
│ ├── python3 -> python3.5
│ └── python3.5 -> /Library/Frameworks/Python.framework/Versions/3.5/bin/python3.5
├── include
├── lib
│ └── python3.5
│ └── site-packages
└── pyvenv.cfg
Here’s what each folder contains:
bin
: files that interact with the virtual environment
include
: C headers that compile the Python packages
lib
: a copy of the Python version along with a site-packages
folder where each dependency is installed
Further, there are copies of, or symlinks to, a few different Python tools as well as to the Python executables themselves. These files are used to ensure that all Python code and commands are executed within the context of the current environment, which is how the isolation from the global environment is achieved. We’ll explain this in more detail in the next section.
More interesting are the activate scripts in the bin
directory. These scripts are used to set up your shell to use the environment’s Python executable and its site-packages by default.
In order to use this environment’s packages/resources in isolation, you need to “activate” it. To do this, just run the following:
$ source env/bin/activate
(env) $
Notice how your prompt is now prefixed with the name of your environment (env
, in our case). This is the indicator that env
is currently active, which means the python
executable will only use this environment’s packages and settings.
To show the package isolation in action, we can use the bcrypt module as an example. Let’s say we have bcrypt
installed system-wide but not in our virtual environment.
Before we test this, we need to go back to the “system” context by executing deactivate
:
Now your shell session is back to normal, and the python
command refers to the global Python install. Remember to do this whenever you’re done using a specific virtual environment.
Now, install bcrypt
and use it to hash a password:
$ pip -q install bcrypt
$ python -c "import bcrypt; print(bcrypt.hashpw('password'.encode('utf-8'), bcrypt.gensalt()))"
$2b$12$vWa/VSvxxyQ9d.WGgVTdrell515Ctux36LCga8nM5QTW0.4w8TXXi
Here’s what happens if we try the same command when the virtual environment is activated:
$ source env/bin/activate
(env) $ python -c "import bcrypt; print(bcrypt.hashpw('password'.encode('utf-8'), bcrypt.gensalt()))"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named 'bcrypt'
As you can see, the behavior of the python -c "import bcrypt..."
command changes after the source env/bin/activate
call.
In one instance, we have bcrypt
available to us, and in the next we don’t. This is the kind of separation we’re looking to achieve with virtual environments, which is now easily achieved.
To explain how this works, let’s first check out the locations of the different python
executables. With the environment “deactivated,” run the following:
$ which python
/usr/bin/python
Now, activate it and run the command again:
$ source env/bin/activate
(env) $ which python
/Users/michaelherman/python-virtual-environments/env/bin/python
After activating the environment, we’re now getting a different path for the python
executable because, in an active environment, the $PATH
environment variable is slightly modified.
Notice the difference between the first path in $PATH
before and after the activation:
$ echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:
$ source env/bin/activate
(env) $ echo $PATH
/Users/michaelherman/python-virtual-environments/env/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:
In the latter example, our virtual environment’s bin
directory is now at the beginning of the path. That means it’s the first directory searched when running an executable on the command line. Thus, the shell uses our virtual environment’s instance of Python instead of the system-wide version.