shared libraries - How to include external library with python wheel package

Question

I want to create package for python that embeds and uses an external library (.so) on Linux using the cffi module.

Is there standard way to include .so file into python package?

The package will be used only internally and won't be published to pypi.

I think Wheel packages are the best option - they would create platform specific package with all files ready to be copied so there will be no need to build anything on target environments.

Answer 1

You can use auditwheel to inject the external libraries into the wheel:

auditwheel repair: copies these external shared libraries into the wheel itself, and automatically modifies the appropriate RPATH entries such that these libraries will be picked up at runtime. This accomplishes a similar result as if the libraries had been statically linked without requiring changes to the build system. Packagers are advised that bundling, like static linking, may implicate copyright concerns.

You can pre-build the external c++ library by typically executing the following:

./configure && make && make install

This will generate an my_external_library.so file and install it in the appropriate path. However, you'll need to ensure that the library path is properly set in order for the auditwheel to discover the missing dependency.

export LD_LIBRARY_PATH=/usr/local/lib

You can then build the python wheel by executing:

python setup.py bdist_wheel

Finally, you can repair the wheel, which will inject the my_external_library.so into the package.

auditwheel repair my-python-wheel-1.5.2-cp35-cp35m-linux_x86_64.whl

I successfully applied the above steps to the python library confluent-kafka-python which has a required c/c++ dependency on librdkafka.

---

Note: auditwheel is Linux-only. For MacOS, see the delocate tool.