Python Ports Policy

Status: Defacto. If in doubt, check with the FreeBSD Python Team

This policy document is used:

• As a reference to a well-defined target-state
• To inform discussions and technical implementations
• To provide FreeBSD Python Users and Organisations with insight into the Python@ team goals
• To orient and focus resources towards common initiatives or goals.

FreeBSD currently supports Python (CPython) branches until they are end-of-life (EoL) as defined upstream.

• Python Release Support

Note: Upstream currently only provides bug fixes for the latest minor release (and Python 2.7 at the time of this writing). In order to reduce maintenance overheads and inconsistencies between ports, it is likely that the FreeBSD Python team will modify its branch support policy to only support branches that are also supported by upstream in the future.

• Subversion directory name MUST be lowercase(py-PORTNAME)

• Python ports MUST be prefixed with py-, named category/py-<PORTNAME>, unless there is a compelling case not to.
  • Compelling cases do NOT include:
    • PyPi name or PORTNAME already contains py, python, python-* or similar prefix or suffix
    • Package is not listed in PyPI
• Python packages that are complete end-user products NOT present in PyPI AND not likely to ever be, MAY omit a py- prefix, A. BUT …
  • Check with the FreeBSD Python Team

• PORTNAME MUST exactly match the complete upstream name.
  • Ports MUST NOT modify the name if it already contains py, python, python-* or similar prefix or suffix
  • Ports MUST include all non-alpha numeric characters in the name (such as dashes (-))

• PORTNAME MUST exactly match the registered PyPI name, if the package is registered on PyPI
  • This also applies if PyPi (CHEESESHOP) is NOT used for MASTER_SITES or DISTFILES. Example: USE_GITHUB=yes

• PORTNAME MUST be lowercase(PORTNAME)

• Ports MUST use PKGNAMEPREFIX, unless there is a compelling case not to.
• PKGNAMEPREFIX MUST be set to ${PYTHON_PKGNAMEPREFIX}

• Ports MUST add python as a secondary category, unless there is a compelling reason not to.
• Ports SHOULD add multiple CATEGORIES as needed to aid discovery by users.

Ports SHOULD use MASTER_SITES=CHEESESHOP if a Python package is registered on PyPI, unless there is a compelling (temporary) case not to.

Examples where temporarily using an alternate MASTER_SITES is OK:

• Certain files are not correctly packaged/included in the PyPI sdist, such as test suite files or data. Submit an upstream issue or PR in these cases.
• A source distribution (sdist, aka tarball) has not been uploaded yet.

In these cases, issues or pull request SHOULD be created upstream to rectify the problem, and MASTER_SITES switched to CHEESESHOP when the change is released in a later version.

This ensures upstreams packaging and deployment pipeline is well tested and standardised (Python standards), as it is heavily relied upon (setuptools, autoplist, documentation, discoverability, etc).

• Python ports MUST declare USES=python :<version-spec> as completely and accurately as possible, without being incorrect, given existing syntax constraints.
  • :<version-spec> MUST match the Python versions declared supported by upstream exactly. If upstream declarations are incorrect or incomplete, fix them in the port and send the changes upstream.
• Python ports MUST NOT limit or restrict Python version support in :version-spec.
• Python ports SHOULD NOT leave :<version-spec> empty (Note: transitionary guideline. This will eventually be a MUST)

• Packages that use distutils (have a setup.py, import distutils/setuptools), SHOULD use autoplist, unless there is a compelling reason not to.

Note: autoplist can be used in combination with PLIST_FILES, PLIST_DIRS and explicit entries in pkg-plist

• concurrent MUST be set when any files (pathnames/filenames) installed by a package are NOT Python-version specific, like /usr/local/bin/script.
  • This includes ports that support a single-version of Python, as supported versions can change at any time.
  • concurrent SHOULD be set, even in cases where no such filenames are installed, as this may change in future versions.

See Also: https://reviews.freebsd.org/D20927#454268

• Packages that use distutils (have a setup.py, import distutils/setuptools), SHOULD use distutils, unless there is a compelling reason not to.

NOTE: Be careful when referencing dependencies only from requirements.txt files. Unless the setup.py parses those requirements.txt entries, only setup.py defined dependencies are canonical/actual.

• Ports MUST exactly match the dependencies declared by the upstream package.
  • Refer to setup.py:setup_requires for BUILD_DEPENDS
  • Refer to setup.py:install_requires for RUN_DEPENDS
  • Refer to setup.py:tests_requires for TEST_DEPENDS

• BE CAREFUL, dependencies can be declared inconsistently, incompletely or in multiple places (Eg: requirements.txt, test_requirements, tox.ini)
• Port maintainers SHOULD fix missing or incorrectly specified dependencies, in the port, AND upstream the changes.
• Use common sense for upstream version requirements that set maximum versions or pin very specific versions, eg: ==, <=, <

• setup.py:extras_require or otherwise OPTIONAL dependencies, SHOULD be reflected in equivalent port OPTIONS.
• OPTION name SHOULD be named after the extras_require 'key' name.
• Port OPTIONS for optional functionality, SHOULD be enabled by default (defined in OPTIONS_DEFAULT).

• If a package depends on another packages optional dependencies, the package MUST depend on those dependencies directly/explicitly.
  • Eg: Package A depends on PackageB[optional] (defined in extras_require). Package A should depend explicitly on all dependencies listed in Package B's extras_require{ optional: ..., ...} list.

• Python ports SHOULD include a do-test target if the package includes tests.
• Python ports SHOULD add TEST_DEPENDS referring to the package tests_require dependencies.

• Python ports SHOULD document their patches
  • If the patch is standalone, place the comment in the header
  • If the patch is combined with others changes, place the comment inline, in each relevant hunk

• Python port maintainers SHOULD (strongly encouraged) aim to:
  • Craft port patches that are likely to be accepted upstream, even if they are committed locally first.
  • Create upstream issues for bug reports and enhancement requests. Eg:
    • Better documented requirements in install_requires, tests_require, setup_requires
    • Optional dependencies made a part of extras_require
    • Support for python setup.py test command (for regression testing)
    • Adding LICENSE to repository and sdist
  • Regularly test and report test suite failures of Python packages on FreeBSD

• Ports that install Python modules or packages (anything going into ${PYTHON_SITELIBDIR}), MUST use ${PYTHON_CMD} at build and installation time, not python, python2 or python3.
• Executable scripts being installed MUST use ${PYTHON_CMD} as shebang line. See: Mk/Uses/shebangfix.mk

• Ports depending on Python to build MUST use${PYTHON_CMD} for the build system/command invocation.
• Ports MUST NOT use version-unspecific Python invocations such as python, python2 or python3.

This allows ports to be built with different default versions of Python.

• Ports that install Python scripts, which use a package or module being provided by another Python port, MUST use a specific Python port version. The shebang line MUST use ${PYTHON_CMD}. If the dependent Python module can be installed for multiple Python versions in parallel, the port providing the script MUST use ${PYTHON_PKGNAMESUFFIX} and the script SHOULD be suffixed, too. If the port is installed for the default Python version, the script SHOULD be symlinked to the original name.

• port devel/foobar provides script foofoo.py, which requires textproc/py-barbar
• devel/foobar MUST use USES=python,run
• devel/foobar MUST add textproc/py-barbar as RUN_DEPENDS
• foofoo.py MUST use ${PYTHON_CMD} as shebang
• devel/foobar MUST use PKGNAMESUFFIX=${PYTHON_PKGNAMESUFFIX}
• if devel/foobar and textproc/py-barbar support different Python versions:
  • foofoo.py MUST be installed as foofoo-${PYTHON_PKGNAMESUFFIX}.py
  • foofoo-${PYTHON_PKGNAMESUFFIX}.py SHOULD symlink to foofoo.py, if the Python version of the port matches the default Python version being specified by the system (DEFAULT_VERSION)

• Patches & Back-ports
  • An issue is created in the Python Bug Tracker
  • Links to issues are included with commits to ports
  • Patches are correctly commented

Description: Upstream issue summary or commit short log
Issue ID: URL to upstream Issue
Changeset: URL to upstream Commit
TODO: Comments on what needs to be done to remove local patch

• For Django packages
  • Should be added in the www/ category, with www as the primary CATEGORIES, and python as a secondary category
  • More specific categories can be added as additional/virtual categories if appropriate
• For Flask packages: TBD. Where specific policies are not defined, Python Ports Policy applies
• For zope packages: TBD. Where specific policies are not defined, Python Ports Policy applies
• For <framework> packages: TBD. Where specific policies are not defined, Python Ports Policy applies