====== PEP-517 ======

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=255722 https://reviews.freebsd.org/D36290

===== Introduction =====

[[https://peps.python.org/pep-0517/|PEP-517]] introduces a new build and install paradigm for Python packages that the ports framework, as of Q2 2022, does not support. This page explains why such support is needed and discusses how to get there.

In a nutshell, PEP-517 is a minimal format/interface/standard for Python package build systems to generate wheels from source trees or distributions, which can then be installed into one's Python distribution (''%%site-packages%%'').

==== Why support PEP-517 ====

Packages from ''%%CHEESESHOP%%'' are increasingly not shipping with ''%%setup.py%%'' that ''%%USE_PYTHON=distutils%%'' executes to build and install the package in staging. Instead, they follow PEP-517 with a ''%%pyproject.toml%%'', similar to Rust/Cargo packages' ''%%Cargo.toml%%''; any additional files related to building and installing are specific to the chosen build systems in ''%%pyproject.toml%%''. setuptools no longer plays the central distutils role of building and installing; it is now one of many build backends packages can specify.

Additionally, [[https://peps.python.org/pep-0632/|distutils is deprecated with removal planned for Python 3.12]]. We need to start future-proofing the framework now, not least to avoid surprises later, but to gradually allow existing ports to transition and add new ports following PEP-517.

===== Usage =====

''%%USE_PYTHON=pep517%%'' exists to streamline ''%%do-build%%'' and ''%%do-install%%'''s common mechanics. Operation is otherwise similar to ''%%USE_PYTHON=distutils%%''. It is the porter's responsibility to parse ''%%pyproject.toml%%'' for the correct build backend and add it into ''%%BUILD_DEPENDS%%''. Example:

<code>
PORTNAME=	sample
DISTVERSION=	1.2.3
CATEGORIES=	devel

MAINTAINER=	freebsd@example.org
COMMENT=	Python sample module

BUILD_DEPENDS=	${PYTHON_PKGNAMEPREFIX}flit>0:devel/py-flit@${PY_FLAVOR}
RUN_DEPENDS=	${PYTHON_PKGNAMEPREFIX}six>0:devel/py-six@${PY_FLAVOR}

USES=		python
USE_PYTHON=	autoplist pep517

.include <bsd.port.mk>
</code>

In this configuration, during ''%%do-build%%'', ''%%devel/py-build%%'' parses ''%%pyproject.toml%%'', checks that requirements are met and executes the specified build backend, in this case ''%%devel/py-flit%%'', which generates a wheel. During ''%%do-install%%'', ''%%devel/py-installer%%'' installs the wheel into ''%%STAGEDIR%%''.

''%%USE_PYTHON=autoplist%%'' will read the installed/staged wheel's ''%%RECORD%%'' (CSV) file, prepend ''%%${PYTHONPREFIX_SITELIBDIR}%%'' to each line and strip the other columns to generate the temporary plist. If ''%%USE_PYTHON=concurrent%%'' is specified, files installed into ''%%bin/%%'' are moved to their proper names. ''%%${PREFIX}%%'' is stripped to generate the final plist. Compiled bytecode is not included in ''%%RECORD%%'', so it is not compiled unlike ''%%USE_PYTHON=distutils%%''. This also accommodates future ''%%pkg%%'' triggers that compile bytecode at install time rather than stage time.

==== Build backend ====

The build backend is usually specified in the ''%%[build-system]%%'' table. If this table is not defined, per [[https://peps.python.org/pep-0518/#build-system-table|PEP-518]], the default is:

<code>
[build-system]
# Minimum requirements for the build system to execute.
requires = ["setuptools", "wheel"]  # PEP 508 specifications.
</code>

thus both of these need to be specified in ''%%BUILD_DEPENDS%%''.

For those using flit, ''%%BUILD_DEPENDS%%'' shall include ''%%devel/py-flit%%'' and not ''%%devel/py-flit-core%%''. The latter is for bootstrapping the Python packaging ecosystem only (see below).

==== Feature notes ====

=== Supported feature comparison ===

^  Feature  ^  ''%%distutils%%''  ^  ''%%pep517%%''  ^
| ''%%concurrent%%'' | ✔ | ✔ |
| ''%%cython%%'' | ✔ | ✔ |
| ''%%cython_run%%'' | ✔ | ✔ |
| ''%%flavors%%'' | ✔ (implied) | ✔ (implied) |
| ''%%noflavors%%'' | ✘ | ✘ |
| ''%%allflavors%%'' | ✔ | ✔ |
| ''%%optsuffix%%'' | ✔ | ✔ |
| ''%%distutils%%'' | ✔ | ✘ |
| ''%%autoplist%%'' | ✔ | ✔ |
| ''%%py3kplist%%'' | ✘ | ✘ |
| ''%%pythonprefix%%'' | ✔ | ✔ |
| ''%%noegginfo%%'' | ✔ | ✘ |
| ''%%nose%%'' |  |  |
| ''%%nose2%%'' |  |  |
| ''%%pytest%%'' |  |  |
| ''%%pytest4%%'' |  |  |
| ''%%unittest%%'' |  |  |
| ''%%unittest2%%'' |  |  |

=== autoplist ===
This method of parsing ''%%RECORD%%'' only supports ''%%entry_points%%'' installed to ''%%bin/%%'' and not other ''%%data_files%%'' installed elsewhere in ''%%hier(7)%%''. [[https://github.com/pypa/setuptools/discussions/2648|At least setuptools, through PyPA guidance, have deprecated this practice]], and general consensus amongst language package managers regard this as out of their scope, left as exercises for downstream packagers. Install any such ''%%data_files%%'' ourselves, using the appropriate ''%%INSTALL%%'' variables, through ''%%post-install%%'' and append them to ''%%PLIST_FILES%%'', rather than manually constructing ''%%pkg-plist%%''. Please work with your package upstreams to comply with Python packaging standards.

===== Implementation =====

The framework will almost entirely use ''%%devel/py-build%%'' as the build frontend for ''%%do-build%%'' and ''%%devel/py-installer%%'' as the install frontend for ''%%do-install%%'' so it is important that these and their ''%%RUN_DEPENDS%%'' bootstrap properly, **without distutils or ''%%devel/py-setuptools%%'' in the environment**. The ports framework serves as the integration frontend.

==== Design goals ====

To start, provide the same features available in both ''%%USE_PYTHON=distutils%%'' and PEP-517 using the same syntax as practicable. Leave room for accommodate future Python standards or accepted practices in the wild.

==== Bootstrapping ====

Especially since none of the tooling needed for PEP-517 support exists in the base Python distribution, unlike distutils, we have to bootstrap some of the tooling. Leveraging the framework comes after successful bootstrapping.

We will follow the process for [[https://flit.pypa.io/en/latest/bootstrap.html|bootstrapping devel/py-flit-core]] since it does not depend on anything or introduce circular dependencies. Other packages during this step will use it to build themselves. The Python Team (python@) shall maintain these ports:

  - ''%%devel/py-flit-core%%'' https://reviews.freebsd.org/D34786
  - ''%%devel/py-installer%%'' https://reviews.freebsd.org/D34789
  - ''%%devel/py-build%%'' https://reviews.freebsd.org/D37921
    - ''%%devel/py-packaging%%'' https://reviews.freebsd.org/D37920
    - ''%%devel/py-pyproject_hooks%%''
      - formerly ''%%devel/py-pep517%%'' https://reviews.freebsd.org/D34875
    - ''%%textproc/py-tomli%%'' https://reviews.freebsd.org/D34876

==== "Test suite" ====
The above mentioned bootstrap ports/packages will serve as the "test suite". It is a small quantity yet diverse enough collection to exercise the framework, including overridable variables.

----

{{tag>Ports}}