This is an old revision of the document!



Packages

Pkgng packages for all supported FreeBSD versions will be provided for i386 and amd64 in due course.

Building

Building PyPy can be done using the Ports Collection or directly from source. The former is recommended.

The PyPy build process has two phases:

  • The translation phase converts the language interpreter, written in RPython, into C code. This is done using a python interpreter (currently either cPython or PyPy). This is a single thread task.
  • The compilation phase compiles the C code into a single executable. This is a highly parallel task.

See below for detailed discussions on the various aspects of building PyPy.

To install PyPy from ports:

# make -C /usr/ports/lang/pypy install

PyPy supports a diverse set of configurations, the port supports installing multiple instances of PyPy that is built using different sets of configurations. Each set of configurations is called an instance. Four predefined instances are provided by the port (two are broken) and additional instances can be specified (see the following section).

The default instance (called DEFAULT) is always built. The remaining three instances can be included using the port's options.

Name Binary Options Description
DEFAULT pypy --Ojit Default PyPy with JIT
SANDBOX pypy-sandbox --sandbox -Ojit Sandboxed PyPy, requires special supervisor to run
CLI pypy-cli --backend=cli -O2 (BROKEN) PyPy that runs on a .NET virtual machine
JVM pypy-jvm --backend=jvm -O2 (BROKEN) PyPy that runs on a Java virtual machine

The jit optimisation level is converted to 2 for platforms that are not supported by the JIT. The predefined instances are defined at ${PORTSDIR}/lang/pypy/files/bsd.pypy.inst.mk.

Custom instances can be specified for the port to build and install. It is recommended that the custom instance specification is placed in /etc/make.conf as the full specification is required for each invocation of make (i.e. the port does not remember the specification between stages).

The list of instances to build is passed by the PYPY_INST environment variable. When this environment variable is defined all predefined instances (including DEFAULT) are not automatically added. The predefined instances may be added manually.

The list of variables that defines an instance specification are:

Variable Description
PYPY_${inst}_NAME The binary name
PYPY_${inst}_TRANSLATION_ARGS The translation arguments
PYPY_${inst}_OPT The optimisation level
PYPY_${inst}_OBJSPACE_ARGS The object space arguments

${inst} is the instance name, as passed in PYPY_INST. See PyPy documentation for a list of supported parameters.

The following example defines a custom instance, with name CUSTOM and binary name pypy-custom, and also includes the DEFAULT and SANDBOX predefined instances. DEFAULT is defined first as the first instance is used to compile the python files (.py → .pyc) and needs to be a stand-alone instance (SANDBOX requires a supervisor to work).

# cat >> /etc/make.conf << _EOF
PYPY_INST=                  DEFAULT SANDBOX CUSTOM
PYPY_CUSTOM_NAME=           pypy-custom
PYPY_CUSTOM_TRANSLATE_ARGS= --gcrootfinder=shadowstack --gc=generation
PYPY_CUSTOM_OPT=            0
PYPY_CUSTOM_OBJSPACE_ARGS=  --no-objspace-usepycfiles --objspace=thunk
_EOF
# make -C /usr/ports/lang/pypy all install

Building PyPy requires a lot of memory. Most systems are not capable of building PyPy, if this applies to your system please consider using a package as listed above. PyPy is also capable of building itself which, thanks to the JIT, is faster than using Python but also uses more memory. This behaviour is a port option, dependent on ${LOCALBASE}/bin/pypy being present.

By default PyPy builds all instances in parallel which stacks the required memory (i.e. each build will use the maximum memory listed below). If your system has sufficient memory to build one PyPy instance but not multiple instances use the build flag -DDISABLE_MAKE_JOBS to build the PyPy instances in series.

The PyPy port checks if your system has sufficient memory and refuses to build if it detects insufficient memory. This behaviour can be overwritten with the build flag -DPYPY_IGNORE_MEMORY. Each extra level of optimisation requires more memory when building PyPy, with the JIT taking the most memory, so consider using a lower level of optimisation if memory is a limitation and disabling the memory checks however the primary reason to use PyPy is for the JIT and the associated performance improvements.

The following memory requirements were determined using /usr/bin/time -l on an x86_64 system for building the DEFAULT instance:

Translator 32-bit 64-bit
python2.7 2.2GiB 4.4GiB
pypy-2.1 (min mem) 2.1GiB 3.6GiB
pypy-2.1 2.4GiB 5.8GiB
System 0.2GiB 0.4GiB

See ${PORTSDIR}/lang/pypy/Makefile for more accurate figures.

PyPy takes a long time to build. The “translation” phase of building PyPy can take most of the time. If concurrency is disabled, via -DDISABLE_MAKE_JOBS, then the compilation phase can also take a signification amount of time.

Each instance built can be translated concurrently if concurrency is not disabled.

The following times are based on an Intel Core i7-2600 (4 physical cores + hyper-threading = 8 logical cores) for building the DEFAULT instance:

Translator 32-bit 64-bit
python2.7 84 mins 81 mins
pypy-2.1 (min mem) 50 mins 72 mins
pypy-2.1 51 mins 49 mins
Compilation 6 mins 6 mins

See PyPy instructions on building PyPy directly from source.

Status

PyPy builds and runs under FreeBSD under most configurations. Some configurations are not supported upstream and others do not work under FreeBSD. Below is a list of what does not work under FreeBSD and/or upstream:

  • Boehm GC: The Boehm GC works upstream but does not work under FreeBSD
  • Unit Tests: All unit tests pass upstream, many tests do not pass under FreeBSD (and at least one test segfaults pypy). Many FreeBSD failures are fixed upstream so the stable release is not a fair reflection.
  • ASMGCC GC: The ASMGCC GC is not supported on non-Linux systems and only works with particular GCC compilers with particular compiler options (i.e. very temperamental).
  • CLI backend: The CLI (.NET) based PyPy is not supported upstream
  • JVM backend: The JVM (Java) based PyPy is not supported upstream

TODO

  • Upstream appropriate patches
  • Properly check sandbox support and create sandbox VM-thing (does upstream support sandboxing?)
  • Add pypy-based numpy port
  • Update lang/pypy-devel on a weekly basis
  • Make a jitviewer port (depends on flask pygments simplejson Jinja2 Werkzeug)
  • Support valgrind
  • Integrate PyPy into bsd.python.mk
  • python/pypy.1384614947.txt.gz
  • Last modified: 2013/11/16 15:15
  • by DavidNaylor