| Both sides previous revision Previous revision Next revision | Previous revision |
| python:compiledpackages [2023/02/15 16:02] – some of 10 years worth of updates CharlieLi | python:compiledpackages [2023/02/15 21:56] (current) – typo, mention -B CharlieLi |
|---|
| interpreter will not stop (re)compiling the files unless | interpreter will not stop (re)compiling the files unless |
| |
| - ''%%PYTHONDONTWRITEBYTECODE%%'' is set in the environment | - ''%%PYTHONDONTWRITEBYTECODE%%'' is set in the environment or ''%%-B%%'' passed to ''%%${PYTHON_CMD}%%'' |
| - PEP-517 or distutils install commands pass the --no-compile parameter (to avoid the initial creation of //.pyc/////.pyo// files) | - PEP-517 or distutils install commands pass the ''%%--no-compile%%'' parameter (to avoid the initial creation of //.pyc/////.pyo// files) |
| - the CPython package directories (''%%${PYTHON_SITELIBDIR}%%'' and friends) are read-only and hence inaccessible for write operations | - the CPython package directories (''%%${PYTHON_SITELIBDIR}%%'' and friends) are read-only and hence inaccessible for write operations |
| |
| ===== Current approach ===== | ===== Current approach ===== |
| |
| This is a hybrid of not compiling bytecode during the port build process and avoiding package checksum mismatches (described in historical approaches). | This is a hybrid of not compiling bytecode during the port build process and avoiding package checksum mismatches (described in historical approaches). Using //pkg// triggers, bytecode files are compiled or removed after the pkg transactions complete, mimicking what Python's own packaging tools like pip do by default. Only operates on ''%%${PYTHON_SITELIBDIR}%%'' and follows [[https://peps.python.org/pep-3147/|PEP-3147]]. Implemented as [[https://reviews.freebsd.org/D34739|D34739]] |
| | |
| | This approach was chosen primarily due to the ''%%RECORD%%'' file from Python Packaging Authority's [[https://packaging.python.org/en/latest/specifications/recording-installed-packages/|recording installed packages]] standard not requiring bytecode to be included, and thus //devel/py-installer//, used during port build's install/stage, does not include them. (By contrast, the distutils install command's ''%%--record%%'' parameter does include bytecode.) While PEP-3147 specifies a uniform file and directory format for bytecode, it is not prudent to infer from that when such files are not included in ''%%RECORD%%''. Additionally, this provides the most flexibility for users to decide whether they want bytecode or not, especially when such an option in the //lang/python// ports is available. Implementations other than CPython do not compile or use bytecode, a CPython implementation detail, so packaging the same Python ports with those additional flavours would break with bytecode included in plists. |
| | |
| | ==== Deterministic bytecode ==== |
| | |
| | By default, the **st_mtime** checking between bytecode and original sources is not deterministic and thus results in non-reproducible builds. [[https://peps.python.org/pep-0552/|PEP-552]] addresses this by providing the option of verifying a deterministic hash instead. Passing the appropriate flags for this option to any compile commands during the port build process would have a higher chance of using packaged bytecode rather than immediate invalidation. |
| | |
| | This approach was not chosen because the aforementioned pitfalls were not addressed. Further, no installation tool that %%PyPA%% supports supplies a parameter to pass ''%%invalidation_mode%%'' to ''%%compileall%%''. |
| |
| ===== Historical approaches ===== | ===== Historical approaches ===== |