python - Setuptools "development" Requirements

Question

TL;DR: Is there a way to hook setuptool's 'develop' to install a set of development requirements when running python setup.py develop?

I'm building my first python package using setuptools. I'm specifying the requirements as:

requirements = [
    'click',
    'ansible',
    'fabric',
    'gitpython',
    'pyyaml',
    'jinja2',
    'yapsy'
]

test_requirements = [
    'pytest',
    'pytest-pep8',
    'pytest-cov',
]

setup(
...
    install_requires=requirements,
    tests_require=test_requirements,
...
)

During development, I've been installing the package (in a virtual environment) with:

python setup.py develop

and uninstalling with:

python setup.py develop -u

The package uses entry_points to install some command line scripts, so this sets up the commands for me and allows me to edit the package while testing the command at the same time.

I also have some dependencies that I use for development ... sphinx + extensions and a couple other things (things that aren't needed to use the package). I'm just manually installing them in the virtual environment at the moment. I didn't see any documentation (and haven't found any examples on the googles) about how to wire them in with setuptools.

Maybe there's a way to hook 'setup.py develop' to install an additional set of requirements? Another method I haven't read about?

Answer 1

For more info on using setup.py vs requirements.txt, I found this article helpful.

Update: September 2016

I no longer use requirements.txt (see original answer below) for installing development only packages. The prevailing wisdom seems to be that requirements.txt should be used to pin deployments to specific version numbers, typically using pip freeze > requirements.txt. This ensures that the exact same versions of your project's dependencies and also your project's dependencies' dependencies are installed on all of your servers.

I instead use the extras_require option to setup.

requirements = [
    'click',
    'ansible',
    'fabric',
    'gitpython',
    'pyyaml',
    'jinja2',
    'yapsy'
]

setup({
    install_requires=requirements,
    extras_require={
        'dev': [
            'pytest',
            'pytest-pep8',
            'pytest-cov'
        ]
    }
})

Now, to install your package for development, you run pip install -e .[dev]. This installs all the regular required packages and those listed in the dev section of extras_require.

Production installs can still be done with python setup.py install or pip install . (or with a requirements.txt file).

Original Answer

Here is a way to do it that seems to be in keeping with the recommendations I've run into regarding setup.py vs requirements.txt. Specify all your production dependencies in the install_requires parameter of setup.py.

requirements = [
    'click',
    'ansible',
    'fabric',
    'gitpython',
    'pyyaml',
    'jinja2',
    'yapsy'
]

setup({
    # ...
    install_requires=requirements
    # ...
})

Then create a requirements.txt file that instructs pip to install your production dependencies from setup.py as well as your testing dependencies.

-e .

pytest
pytest-pep8
pytest-cov

Now you can install your package for development with pip install -r requirements.txt. The -e . line will install your package and its dependencies from setup.py in development mode. To install on production, you could use python setup.py install or pip install .. This will only install the dependencies listed in setup.py.