python - 'pip setup.py bdist_wheel' no longer builds forced non-pure wheels

Question

I have a project that compiles with C extensions on Linux, but without them on Windows. When I first generated the wheel files on Windows with python setup.py bdist_wheel, they became universal, and I could not upload them to PyPI as these universal wheels are preferred by pip for installation over the .tar.gz uploads (the result from python setup.py sdist).

The trick around this was to specify in the setup.py:

Distribution.is_pure = lambda *args: False

or by subclassing Distribution:

class BinaryDistribution(Distribution):
    def is_pure(self):
        return False

and calling setup() in setup.py with the extra keyword argument distclass=BinaryDistribution,.

This all worked fine on my VM running Windows XP 64 which has 32 and 64 bit versions of Python 2.6/2.7/3.3/3.4 and pypy installed just for this purpose. A simple batch file gives me:

dist/pkg-1.0-cp26-none-win32.whl
dist/pkg-1.0-cp26-none-win_amd64.whl
dist/pkg-1.0-cp27-none-win32.whl
dist/pkg-1.0-cp27-none-win_amd64.whl
dist/pkg-1.0-cp33-none-win32.whl
dist/pkg-1.0-cp33-none-win_amd64.whl
dist/pkg-1.0-cp34-none-win32.whl
dist/pkg-1.0-cp34-none-win_amd64.whl

and the appropriate package gets downloade and installed by pip when you run pip on Windows and when you run pip on Linux you get the

pkg-1.0.tar.gz

which includes the C sources which are compiled during installation.

The problem started with the fact that I don't have a spare Windows 7 licensed machine where I can install Python 3.5 (it doesn't install on the EOL XP). So I investigated Appveyor and created appveyor.yml:

environment:
  matrix:
    - PYTHON: C:Python27
    - PYTHON: C:Python33
    - PYTHON: C:Python34
    - PYTHON: C:Python35
    - PYTHON: C:Python27-x64
    - PYTHON: C:Python33-x64
      DISTUTILS_USE_SDK: '1'
    - PYTHON: 'C:Python34-x64'
      DISTUTILS_USE_SDK: '1'
    - PYTHON: 'C:Python35-x64'

install:
  - |
    %PYTHON%python.exe -m pip install --upgrade pip
    %PYTHON%python.exe -m pip install wheel

build: off

test_script:
  - echo Skipped for now

after_test:
  - |
    %PYTHON%python.exe setup.py bdist_wheel

artifacts:
  - path: dist*

With the exact same source the result from the above eight calls to python setup.py bdist_wheel are:

pkg-1.0-py2-none-any.whl
pkg-1.0-py3-none-any.whl

And if you upload these to PyPI, Linux prefers them over the .tar.gz leading to non-inclusion of the C extension code.

What causes this, and how can I use Appveyor to build my .whl files (or at least the ones for Python 3.5?

Answer 1

I've just run into this issue myself with Python v2.7 and wheel v0.29.0 on Windows 7 x64, where I build a Python package with some pre-compiled extensions (complicated VisualStudio setup with SWIG and external DLLs).

After examining the source code I have found that overriding Distribution.has_ext_modules works (automatically includes platform name and ABI tag):

from setuptools import setup
from setuptools.dist import Distribution

DISTNAME = "packagename"
DESCRIPTION = ""
MAINTAINER = ""
MAINTAINER_EMAIL = ""
URL = ""
LICENSE = ""
DOWNLOAD_URL = ""
VERSION = '1.2'
PYTHON_VERSION = (2, 7)


# Tested with wheel v0.29.0
class BinaryDistribution(Distribution):
    """Distribution which always forces a binary package with platform name"""
    def has_ext_modules(foo):
        return True


setup(name=DISTNAME,
      description=DESCRIPTION,
      maintainer=MAINTAINER,
      maintainer_email=MAINTAINER_EMAIL,
      url=URL,
      license=LICENSE,
      download_url=DOWNLOAD_URL,
      version=VERSION,
      packages=["packagename"],

      # Include pre-compiled extension
      package_data={"packagename": ["_precompiled_extension.pyd"]},
      distclass=BinaryDistribution)