This is an old revision of the document!
FreeBSD Developer Summit: Ports
Thursday May 15, 09:00-12:00
Overview
The FreeBSD project has provided pre-built ready-to-install binary packages for many years on a best-effort basis. While these packages do work in a large number of cases, there are too many inconsistencies and failure combinations, from the unpredictable update frequency to dependency handling across upgrades, for them to be used on a wider scale. This session centers on a round-table brainstorm that begins with a summary of the tremendous progress made in the last 12 months, and closes with a discussion of the roadmap on how to improve binary package creation, distribution, installation and upgrading.
If you would like to participate, contact the working group chairs below and CC devsummit@. You will be then added to this page. Please include a list of things you want to talk about or the areas you are interested in. This helps us in planning the session and to bring people together with common interests.
It is possible to bring in people who cannot attend in person via video conference or chat tools. Notes during the session will be published later on for the whole community to see what we discussed.
Goals
In particular, we would like to cover the following topics. This is not an exhaustive list and if you feel there is something missing that you want to talk about, contact one of the session chairs and we will include your topic here. Note that the numbering of the topics does not represent an ordering or importance indication of any kind, but rather a reference to the second table with the “topic of interest” column.
Topics
Part one 09:00 - 10:30
| # | Who | Topic Description |
|---|---|---|
| 1 | ? | Status update on package building |
| 2 | ? | Status update on PKGNG |
| 3 | ? | Status and evaluation of Quarterly branch |
| 4 | ? | Status update on pkg_* decommissioning |
| 5 | ? | Status and evaluation of blanket approval for infrastructure updates |
| 6 | erwin | Default to http for MASTER_SITE_BACKUP |
| 7 | bjk | MIT Project Athena |
| 8 | des | Dependencies |
| 9 | emaste | Binary Package Transparency |
| 10 | sson | cross building |
| 11 | BryanDrewery | PortCI: Build automation for Exp-runs, Package building, QAT |
| 12 | * | freebsd-version(1) |
Break 10:30 - 11:00
Part two 11:00-12:00
| # | Who | Topic Description |
|---|---|---|
| 9 | ? | Ports on FreeBSD 10 |
| 11 | ? | Changes to the ports infrastructure |
| 12 | ? | The LICENSE framework |
| 13 | ? | Plans for the ports infrastructure |
| 14 | * | Brainstorm: common complaints and wantlist |
Note: General presentations about work you have done that does not require further discussions should be submitted for the FreeBSD Developer Summit track at BSDCan (see the general developer summit page).
Attending
In order to attend you need register for the developer summit as well as by email for the session and be confirmed by the working group organizer. Follow the guidelines described on the main page or what you received by email. For questions or if in doubt ask the session chairs.
| Name | Username / Affiliation | Topics of Interest | Notes |
|---|---|---|---|
| MathieuArnold | mat | * | Session knitter |
| ErwinLansing | erwin | * | Session chair |
| BryanDrewery | bdrewery | * | Person who doesn't give slides to coordinator in time |
| MatthewSeaman | matthew | * | |
| GlenBarber | gjb | * | |
| BaptisteDaroussin | bapt | * | Ports Breaker |
| BrooksDavis | brooks | * | non-x86, default options, flavours |
| BenjaminKaduk | bjk | * | config-package-dev |
| DavidChisnall | theraven | * | toolchain, base system packaging |
| DavidCroal | Sentex | * | |
| PeterLosher | ISC | * | |
| MarkLinimon | linimon | * | probably attend incognito, with groucho-glasses |
| AllanJude | BSDNow / ScaleEngine | * | |
| DagErlingSmoergrav | des | * | |
| MichaelLucas | himself | * | |
| AndyNagle | Xinuos | * | |
| WaltCroom | Xinuos | * | |
| KrisMoore | kmoore | * | |
| AlfredPerlstein | alfred | * | |
| ClaudiaY | Norse | * | |
| MarcusHoffman | Norse | * | |
| EdMaste | emaste | * | |
| PawelJakubDawidek | pjd | * | |
| RyanSteinmetz | zi | * | |
| GavinAtkinson | gavin | * | |
| StacySon | sson | * | |
| BradDavis | brd | * | |
| SteveWills | swills | * | |
| JonathanAnderson | jonathan | * |
Notes
Results
bapt:
Status update on package building
Building all packages with poudriere on large machines. 16h to build full tree. each Wed at 01:00UTC snapshot tree and do incremental build (full build if there is a security problem). All supported releases + 2x stable branches (9, 10) Quarterly branches for 9.x-RELEASE and 10.x-RELEASE
Catalogue is signed on signing machine. 4 pkgrepo mirrors (2 US, 1 UK, 1 Russia). Most people seem happy – can even run head with binary packages.
coordination with src – abi breakage.
Supported arch – i386, amd64. To add arm and one MIPs variant.
Don't mirror distfiles any more
Building a pkg of the ports tree? No progress, but it's a good idea. pkg-1.3 will have repo metadata including the revision of the prots tree.
PKGNG Status
We should drop the NG bit (bapt: “it's not me. I'm not writing pkgng for years now”)
New solver in pkg-1.3 – 'pkg upgrade' understands it needs to delete an old package and replace it with a new one, so no more need for 'pkg set' or portmaster chicanery. All due to new solver. cebka's talk on Saturday.
Added bits to cross-build ports. 'make XBUILD=armXX' some bits are in the ports tree already, rest is waiting on pkg-1.3
Security: every dangerous part of pkg upgrade/install/etc. is sandboxed
pkg can upgrade itself in jails, by forking itself
pkg_tools decomissioning
Errata notice has added default pkg.conf and keys to all supported branches, delays added when building from ports
Aim is to deprecate pkg_tools in 6 months. Usage of old style pkg_tools generates warning messsage
1st Sept for death of pkg_tools. (Same date as staging deadline)
Quarterly branches
Evaluation based on 3 months support. do we have enough manpower etc.
Snapshot of ports tree: only allow seurity, build time fixes and upgrades to pkg
MFH tag in commit messages, auth requests approval from portmgr. MFH tool in scripts to make it all easier.
So far has worked pretty well. Only a couple of problems due to insufficient testing.
commits to Q-branch should just merge the security fix: other changes
auto merge of updates to vuxml ? Or just remove vuxml from Q-branch; just have one per repository.
JA: Release notes for the quarterly branch? What is/isn't available etc.
We do extra testing etc. before branches. Use UPDATING file in pkg metadata? Not really needed – pkg -1.3 should obviate the need for UPDATING.
per-package News planned for pkg-1.4.
Machine readable info.
pkg messages: differentiate between what you need for installing from ports and what you need with binary packages
Context sensitive messsages
Ports infrastructure
Various changes: staging, LIB_DEPENDS, users=foo
Huge migration affecting all ports. Expect committers to have port buildable without root, to pass QA tests etc. Still ongoing.
Needed to speed up. Maintainer lockout was slowing things down too much. New policy – easy fixes and stuff related to infrastructure changes, any committer can just do without reference to the maintainer. “Just do it” Took time for people to get used to this. Hasn't been much complaining from maintainers.
About 3800 still to be staged. We did stage twice as many ports in one year than pkgsrc managed. Success!
Committters should have DEVELOPER=yes. Enables warnings for every infrastructure change in tree, and how to convert. Only special case is staging, which refers you to the wiki.
erwin:
MASTER_SITE_BACKUP
Change default to http: — send announcement, and do it (maybe not in that order)
distfiles on the new pkg distribution network? yes, no, ?
ben kaduk: