Thursday May 15, 09:00-12:00

The FreeBSD project has provided pre-built ready-to-install binary packages for many years on a best-effort basis. While these packages do work in a large number of cases, there are too many inconsistencies and failure combinations, from the unpredictable update frequency to dependency handling across upgrades, for them to be used on a wider scale. This session centers on a round-table brainstorm that begins with a summary of the tremendous progress made in the last 12 months, and closes with a discussion of the roadmap on how to improve binary package creation, distribution, installation and upgrading.

If you would like to participate, contact the working group chairs below and CC devsummit@. You will be then added to this page. Please include a list of things you want to talk about or the areas you are interested in. This helps us in planning the session and to bring people together with common interests.

It is possible to bring in people who cannot attend in person via video conference or chat tools. Notes during the session will be published later on for the whole community to see what we discussed.

In particular, we would like to cover the following topics. This is not an exhaustive list and if you feel there is something missing that you want to talk about, contact one of the session chairs and we will include your topic here. Note that the numbering of the topics does not represent an ordering or importance indication of any kind, but rather a reference to the second table with the “topic of interest” column.

Part one 09:00 - 10:30

Break 10:30 - 11:00

Part two 11:00-12:00

Note: General presentations about work you have done that does not require further discussions should be submitted for the FreeBSD Developer Summit track at BSDCan (see the general developer summit page).

In order to attend you need register for the developer summit as well as by email for the session and be confirmed by the working group organizer. Follow the guidelines described on the main page or what you received by email. For questions or if in doubt ask the session chairs.

bapt:

Building all packages with poudriere on large machines. 16h to build full tree. each Wed at 01:00UTC snapshot tree and do incremental build (full build if there is a security problem). All supported releases + 2x stable branches (9, 10) Quarterly branches for 9.x-RELEASE and 10.x-RELEASE

Catalogue is signed on signing machine. 4 pkgrepo mirrors (2 US, 1 UK, 1 Russia). Most people seem happy – can even run head with binary packages.

coordination with src – abi breakage.

Supported arch – i386, amd64. To add arm and one MIPs variant.

Don't mirror distfiles any more

Building a pkg of the ports tree? No progress, but it's a good idea. pkg-1.3 will have repo metadata including the revision of the prots tree.

We should drop the NG bit (bapt: “it's not me. I'm not writing pkgng for years now”)

New solver in pkg-1.3 – 'pkg upgrade' understands it needs to delete an old package and replace it with a new one, so no more need for 'pkg set' or portmaster chicanery. All due to new solver. cebka's talk on Saturday.

Added bits to cross-build ports. 'make XBUILD=armXX' some bits are in the ports tree already, rest is waiting on pkg-1.3

Security: every dangerous part of pkg upgrade/install/etc. is sandboxed

pkg can upgrade itself in jails, by forking itself

Errata notice has added default pkg.conf and keys to all supported branches, delays added when building from ports

Aim is to deprecate pkg_tools in 6 months. Usage of old style pkg_tools generates warning messsage

1st Sept for death of pkg_tools. (Same date as staging deadline)

Evaluation based on 3 months support. do we have enough manpower etc.

Snapshot of ports tree: only allow seurity, build time fixes and upgrades to pkg

MFH tag in commit messages, auth requests approval from portmgr. MFH tool in scripts to make it all easier.

So far has worked pretty well. Only a couple of problems due to insufficient testing.

commits to Q-branch should just merge the security fix: other changes

auto merge of updates to vuxml ? Or just remove vuxml from Q-branch; just have one per repository.

JA: Release notes for the quarterly branch? What is/isn't available etc.

We do extra testing etc. before branches. Use UPDATING file in pkg metadata? Not really needed – pkg -1.3 should obviate the need for UPDATING.

per-package News planned for pkg-1.4.

Machine readable info.

pkg messages: differentiate between what you need for installing from ports and what you need with binary packages

Context sensitive messsages

Various changes: staging, LIB_DEPENDS, users=foo

Huge migration affecting all ports. Expect committers to have port buildable without root, to pass QA tests etc. Still ongoing.

Needed to speed up. Maintainer lockout was slowing things down too much. New policy – easy fixes and stuff related to infrastructure changes, any committer can just do without reference to the maintainer. “Just do it” Took time for people to get used to this. Hasn't been much complaining from maintainers.

About 3800 still to be staged. We did stage twice as many ports in one year than pkgsrc managed. Success!

Committters should have DEVELOPER=yes. Enables warnings for every infrastructure change in tree, and how to convert. Only special case is staging, which refers you to the wiki.

erwin:

Change default to http: — send announcement, and do it (maybe not in that order)

distfiles on the new pkg distribution network? yes, no, ?

ben kaduk:

Athena is not yet FreeBSD based but would like it to be

started in 1983.

Long history of different hardware platforms and OS. Currently debian/ubuntu.

100s of machines, including ~400 publicly accessible machines (can get root password for the asking)

Software used to be monolithing; now split up into about 150 different packages. 4 popular metapackages What does a debian package look like? debian adds a subdir to the upstream source tarball, inside out from the ports way.

Diversions – heavily used feature. replace another package's file with my own version.

https://github.com/mit-athena/

Is there a VM? Don't think there's one currently.

Diversions and pkg(8)? Possible, but not implemented. All of what was described is doable using the ports tree now.

Athena on FreeBSD depends on AFS, which needs work.

des:

Now we have staging, split run dependencies into two different lists. What is needed at install time and what is needed at run time. Problem is we can't change meaning of RUN_DEPENDS overnight.

eg. anything that installs a DTD needs to have libxml installed so it can register DTD in catalogue.

Some dependencies have to be present for 'make stage' others only needed for 'make install'

STAGE_DEPENDS vs INSTALL_DEPENDS

bapt: STAGE_DEPENDS should just be part of BUILD_DEPENDS

More intelligent dependency tracking coming with pkg-1.3 and death of pkg_tools Will break portupgrade / portmaster

'pkg add' is a problem: without a repo, we don't have all the necessary meta-data. Merge of pkg add and pkg install was announced and implemented for 1.3, but reverted.

Idea proposed to FreeBSD Foundation. Idea only, invitation to discussion, no code yet.

Initial idea: Google's Certificate Transparency. Detect doctored certificates, certificates illegitimately issued to people who don't own domain.

Works by logging certificate issuance to log server. Can be checked by public. Domain owners can search for certs issued for their own domains.

Extend this idea to binary package management.

Model is applicable to individually signed packages. Can verify packages issued that claim to be signed by FreeBSD are authentic.

Can we do this with existing infrastructure?

sson:

Working with Qemu 2.0.0

static + dynamic binaries supported

Handful of system calls not supported (will never be)

Supported arches: MIPS{32,64} ARMv6 PPC easy to add.

32bit on 64bit host now working, some lingering issues

Future:

Fix lingering issues for 32 on 64

Add PPC and/or Aarch64

Pushing code upstream

Improved debugger support for cross development

Kernel module allows eg. suing native compiler for cross building. Good performance improvement.

Kernel module already committed to head. will be in 11 and MFC'd to head

Can I use this to build Java for power pc without a pre-existing javavm?

yes.

also could use jam java interpreter. Close to being able to build java.

~~~~ break ~~~~