177 lines
6.5 KiB
ReStructuredText
177 lines
6.5 KiB
ReStructuredText
=======================================
|
|
Build System Support
|
|
=======================================
|
|
|
|
What is it?
|
|
-------------
|
|
|
|
Python packaging has come `a long way <https://bernat.tech/posts/pep-517-518/>`_.
|
|
|
|
The traditional ``setuptools`` way of packaging Python modules
|
|
uses a ``setup()`` function within the ``setup.py`` script. Commands such as
|
|
``python setup.py bdist`` or ``python setup.py bdist_wheel`` generate a
|
|
distribution bundle and ``python setup.py install`` installs the distribution.
|
|
This interface makes it difficult to choose other packaging tools without an
|
|
overhaul. Because ``setup.py`` scripts allowed for arbitrary execution, it
|
|
proved difficult to provide a reliable user experience across environments
|
|
and history.
|
|
|
|
`PEP 517 <https://www.python.org/dev/peps/pep-0517/>`_ therefore came to
|
|
rescue and specified a new standard to
|
|
package and distribute Python modules. Under PEP 517:
|
|
|
|
a ``pyproject.toml`` file is used to specify what program to use
|
|
for generating distribution.
|
|
|
|
Then, two functions provided by the program, ``build_wheel(directory: str)``
|
|
and ``build_sdist(directory: str)`` create the distribution bundle at the
|
|
specified ``directory``. The program is free to use its own configuration
|
|
script or extend the ``.toml`` file.
|
|
|
|
Lastly, ``pip install *.whl`` or ``pip install *.tar.gz`` does the actual
|
|
installation. If ``*.whl`` is available, ``pip`` will go ahead and copy
|
|
the files into ``site-packages`` directory. If not, ``pip`` will look at
|
|
``pyproject.toml`` and decide what program to use to 'build from source'
|
|
(the default is ``setuptools``)
|
|
|
|
With this standard, switching between packaging tools becomes a lot easier. ``build_meta``
|
|
implements ``setuptools``' build system support.
|
|
|
|
How to use it?
|
|
--------------
|
|
|
|
Starting with a package that you want to distribute. You will need your source
|
|
scripts, a ``pyproject.toml`` file and a ``setup.cfg`` file::
|
|
|
|
~/meowpkg/
|
|
pyproject.toml
|
|
setup.cfg
|
|
meowpkg/__init__.py
|
|
|
|
The pyproject.toml file is required to specify the build system (i.e. what is
|
|
being used to package your scripts and install from source). To use it with
|
|
setuptools, the content would be::
|
|
|
|
[build-system]
|
|
requires = ["setuptools"]
|
|
build-backend = "setuptools.build_meta"
|
|
|
|
The ``setuptools`` package implements the ``build_sdist``
|
|
command and the ``wheel`` package implements the ``build_wheel``
|
|
command; the latter is a dependency of the former
|
|
exposed via :pep:`517` hooks.
|
|
|
|
Use ``setuptools``' :ref:`declarative config <declarative config>` to
|
|
specify the package information::
|
|
|
|
[metadata]
|
|
name = meowpkg
|
|
version = 0.0.1
|
|
description = a package that meows
|
|
|
|
[options]
|
|
packages = find:
|
|
|
|
.. _building:
|
|
|
|
Now generate the distribution. To build the package, use
|
|
`PyPA build <https://pypa-build.readthedocs.io/en/latest/>`_::
|
|
|
|
$ pip install -q build
|
|
$ python -m build
|
|
|
|
And now it's done! The ``.whl`` file and ``.tar.gz`` can then be distributed
|
|
and installed::
|
|
|
|
dist/
|
|
meowpkg-0.0.1.whl
|
|
meowpkg-0.0.1.tar.gz
|
|
|
|
$ pip install dist/meowpkg-0.0.1.whl
|
|
|
|
or::
|
|
|
|
$ pip install dist/meowpkg-0.0.1.tar.gz
|
|
|
|
Dynamic build dependencies and other ``build_meta`` tweaks
|
|
----------------------------------------------------------
|
|
|
|
With the changes introduced by :pep:`517` and :pep:`518`, the
|
|
``setup_requires`` configuration field was made deprecated in ``setup.cfg`` and
|
|
``setup.py``, in favour of directly listing build dependencies in the
|
|
``requires`` field of the ``build-system`` table of ``pyproject.toml``.
|
|
This approach has a series of advantages and gives package managers and
|
|
installers the ability to inspect in advance the build requirements and
|
|
perform a series of optimisations.
|
|
|
|
However some package authors might still need to dynamically inspect the final
|
|
users machine before deciding these requirements. One way of doing that, as
|
|
specified by :pep:`517`, is to "tweak" ``setuptools.build_meta`` by using a
|
|
:pep:`in-tree backend <517#in-tree-build-backends>`.
|
|
|
|
.. tip:: Before implementing a *in-tree* backend, have a look on
|
|
:pep:`PEP 508 <508#environment-markers>`. Most of the times, dependencies
|
|
with **environment markers** are enough to differentiate operating systems
|
|
and platforms.
|
|
|
|
If you add the following configuration to your ``pyprojec.toml``:
|
|
|
|
|
|
.. code-block:: toml
|
|
|
|
[build-system]
|
|
requires = ["setuptools", "wheel"]
|
|
build-backend = "backend"
|
|
backend-path = ["_custom_build"]
|
|
|
|
|
|
then you should be able to implement a thin wrapper around ``build_meta`` in
|
|
the ``_custom_build/backend.py`` file, as shown in the following example:
|
|
|
|
.. code-block:: python
|
|
|
|
from setuptools import build_meta as _orig
|
|
|
|
prepare_metadata_for_build_wheel = _orig.prepare_metadata_for_build_wheel
|
|
build_wheel = _orig.build_wheel
|
|
build_sdist = _orig.build_sdist
|
|
|
|
|
|
def get_requires_for_build_wheel(self, config_settings=None):
|
|
return _orig.get_requires_for_build_wheel(config_settings) + [...]
|
|
|
|
|
|
def get_requires_for_build_sdist(self, config_settings=None):
|
|
return _orig.get_requires_for_build_sdist(config_settings) + [...]
|
|
|
|
|
|
Note that you can override any of the functions specified in :pep:`PEP 517
|
|
<517#build-backend-interface>`, not only the ones responsible for gathering
|
|
requirements.
|
|
|
|
.. important:: Make sure your backend script is included in the :doc:`source
|
|
distribution </userguide/distribution>`, otherwise the build will fail.
|
|
This can be done by using a SCM_/VCS_ plugin (like :pypi:`setuptools-scm`
|
|
and :pypi:`setuptools-svn`), or by correctly setting up :ref:`MANIFEST.in
|
|
<manifest>`.
|
|
|
|
If this is the first time you are using a customised backend, please have a
|
|
look on the generated ``.tar.gz`` and ``.whl``.
|
|
On POSIX systems that can be done with ``tar -tf dist/*.tar.gz``
|
|
and ``unzip -l dist/*.whl``.
|
|
On Windows systems you can rename the ``.whl`` to ``.zip`` to be able to
|
|
inspect it on the file explorer, and use the same ``tar`` command in a
|
|
command prompt (alternativelly there are GUI programs like `7-zip`_ that
|
|
handle ``.tar.gz``).
|
|
|
|
In general the backend script should be present in the ``.tar.gz`` (so the
|
|
project can be build from the source) but not in the ``.whl`` (otherwise the
|
|
backend script would end up being distributed alongside your package).
|
|
See ":doc:`/userguide/package_discovery`" for more details about package
|
|
files.
|
|
|
|
|
|
.. _SCM: https://en.wikipedia.org/wiki/Software_configuration_management
|
|
.. _VCS: https://en.wikipedia.org/wiki/Version_control
|
|
.. _7-zip: https://www.7-zip.org
|