When installing software, and Python packages in particular, it’s common that you get a lot of libraries installed. You just did easy_install MyPackage and you get a dozen packages. Each of these packages has its own version.
Maybe you ran that installation and it works. Great! Will it keep working? Did you have to provide special options to get it to find everything? Did you have to install a bunch of other optional pieces? Most of all, will you be able to do it again? Requirements files give you a way to create an environment: a set of packages that work together.
If you’ve ever tried to setup an application on a new system, or with slightly updated pieces, and had it fail, pip requirements are for you. If you haven’t had this problem then you will eventually, so pip requirements are for you too – requirements make explicit, repeatable installation of packages.
So what are requirements files? They are very simple: lists of packages to install. Instead of running something like pip install MyApp and getting whatever libraries come along, you can create a requirements file something like:
MyApp
Framework==0.9.4
Library>=0.2
If you save this in requirements.txt, then you can pip install -r requirements.txt. Regardless of what MyApp lists in setup.py, you’ll get a specific version of Framework (0.9.4) and at least the 0.2 version of Library. (You might think you could list these specific versions in MyApp’s setup.py – but if you do that you’ll have to edit MyApp if you want to try a new version of Framework, or release a new version of MyApp if you determine that Library 0.3 doesn’t work with your application.) You can also add optional libraries and support tools that MyApp doesn’t strictly require, giving people a set of recommended libraries.
You can also include “editable” packages – packages that are checked out from Subversion, Git, Mercurial and Bazaar. These are just like using the -e option to pip. They look like:
-e svn+http://myrepo/svn/MyApp#egg=MyApp
You have to start the URL with svn+ (git+, hg+ or bzr+), and you have to include #egg=Package so pip knows what to expect at that URL. You can also include @rev in the URL, e.g., @275 to check out revision 275.
Requirement files are mostly flat. Maybe MyApp requires Framework, and Framework requires Library. I encourage you to still list all these in a single requirement file; it is the nature of Python programs that there are implicit bindings directly between MyApp and Library. For instance, Framework might expose one of Library’s objects, and so if Library is updated it might directly break MyApp. If that happens you can update the requirements file to force an earlier version of Library, and you can do that without having to re-release MyApp at all.
Read the requirements file format to learn about other features.
So you have a working set of packages, and you want to be able to install them elsewhere. Requirements files let you install exact versions, but it won’t tell you what all the exact versions are.
To create a new requirements file from a known working environment, use:
$ pip freeze > stable-req.txt
This will write a listing of all installed libraries to stable-req.txt with exact versions for every library. You may want to edit the file down after generating (e.g., to eliminate unnecessary libraries), but it’ll give you a stable starting point for constructing your requirements file.
You can also give it an existing requirements file, and it will use that as a sort of template for the new file. So if you do:
$ pip freeze -r devel-req.txt > stable-req.txt
it will keep the packages listed in devel-req.txt in order and preserve comments.
The requirements file is a way to get pip to install specific packages to make up an environment. This document describes that format. To read about when you should use requirement files, see Requirements Files.
Each line of the requirements file indicates something to be installed. For example:
MyPackage==3.0
tells pip to install the 3.0 version of MyPackage.
You can also request extras in the requirements file:
MyPackage==3.0 [PDF]
Packages may also be installed in an “editable” form. This puts the source code into src/distname (making the name lower case) and runs python setup.py develop on the package. To indicate editable, use -e, like:
-e svn+http://svn.myproject.org/svn/MyProject/trunk#egg=MyProject
The #egg=MyProject part is important, because while you can install simply given the svn location, the project name is useful in other places.
You can also specify the egg name for a non-editable url. This is useful to point to HEAD locations on the local filesystem:
or relative paths:
If you need to give pip (and by association easy_install) hints about where to find a package, you can use the -f (--find-links) option, like:
$ pip -f http://someserver.org/index-of-packages MyPackage==3.0
Pip will then look for a link at http://someserver.org/index-of-packages that matches version 3.0 of MyPackage – the link should be like MyPackage-3.0.tar.gz.
And if you want to install from a tarball or zip file with a direct link, you don’t need -f option, you just need to pass the absolute url, like:
$ pip install http://someserver.org/packages/MyPackage-3.0.tar.gz
Right now pip knows of the following major version control systems:
Pip supports the URL schemes svn, svn+svn, svn+http, svn+https, svn+ssh. You can also give specific revisions to an SVN URL, like:
-e svn+svn://svn.myproject.org/svn/MyProject#egg=MyProject
-e svn+http://svn.myproject.org/svn/MyProject/trunk@2019#egg=MyProject
which will check out revision 2019. @{20080101} would also check out the revision from 2008-01-01. You can only check out specific revisions using -e svn+....
Pip currently supports cloning over git, git+http and git+ssh:
-e git://git.myproject.org/MyProject.git#egg=MyProject
-e git+http://git.myproject.org/MyProject/#egg=MyProject
-e git+ssh://git@myproject.org/MyProject/#egg=MyProject
Passing branch names, a commit hash or a tag name is also possible:
-e git://git.myproject.org/MyProject.git@master#egg=MyProject
-e git://git.myproject.org/MyProject.git@v1.0#egg=MyProject
-e git://git.myproject.org/MyProject.git@da39a3ee5e6b4b0d3255bfef95601890afd80709#egg=MyProject
The supported schemes are: hg+http, hg+https, hg+static-http and hg+ssh:
-e hg+http://hg.myproject.org/MyProject/#egg=MyProject
-e hg+https://hg.myproject.org/MyProject/#egg=MyProject
-e hg+ssh://hg@myproject.org/MyProject/#egg=MyProject
You can also specify a revision number, a revision hash, a tag name or a local branch name:
-e hg+http://hg.myproject.org/MyProject/@da39a3ee5e6b#egg=MyProject
-e hg+http://hg.myproject.org/MyProject/@2019#egg=MyProject
-e hg+http://hg.myproject.org/MyProject/@v1.0#egg=MyProject
-e hg+http://hg.myproject.org/MyProject/@special_feature#egg=MyProject
Pip supports Bazaar using the bzr+http, bzr+https, bzr+ssh, bzr+sftp, bzr+ftp and bzr+lp schemes:
-e bzr+http://bzr.myproject.org/MyProject/trunk/#egg=MyProject
-e bzr+sftp://user@myproject.org/MyProject/trunk/#egg=MyProject
-e bzr+ssh://user@myproject.org/MyProject/trunk/#egg=MyProject
-e bzr+ftp://user@myproject.org/MyProject/trunk/#egg=MyProject
-e bzr+lp:MyProject#egg=MyProject
Tags or revisions can be installed like this:
-e bzr+https://bzr.myproject.org/MyProject/trunk/@2019#egg=MyProject
-e bzr+http://bzr.myproject.org/MyProject/trunk/@v1.0#egg=MyProject
If you wish, you can also refer to other requirements files, like:
-r Pylons-requirements.txt
This gives you a way of abstracting out sets of requirements. This isn’t, however, very friendly with frozen requirements, as everything in Pylons-requirements.txt will show up in your frozen file.
You can also provide values for the --index-url and --find-links options in your requirement files, like:
--index-url http://example.com/private-pypi/
Note that using --index-url removes the use of PyPI, while using --extra-index-url will add additional indexes.
--find-links is more ad-hoc; instead of a complete “index”, you only need an HTML page of links to available packages. Simply by putting all your private packages in a directory and using the Apache auto-index, you can publish your packages so pip can find them. --find-links is always additive; pip looks at everything it can find. Use it like:
--find-links http://example.com/private-packages/
Note that all these options must be on a line of their own.