This is an old revision of the document!


We would like to cover the following topics. This is not an exhaustive list and if you feel there is something missing that you want to talk about, contact the session chair and your topic will be included here.

Please send talks, brain storm topics, or other proposals to the session leader, ErwinLansing.

Who Description
AllanJude PCBSD package CDN
Recent developments
* SSP Jeremy Le Hen
* Staged installation Baptiste Daroussin
Open tasks Baptiste Daroussin
* FreeBSD 10
  • Package set policy
  • Security branch
  • pkg_* removal

(Add a list or attach slides detailing the achieved results here.)

Announcement: We now have INDEX-10 from portsnap

Packages sets would take mirrors weeks to upload.

pkgng: the master needs to be atomic. If the pkg checksum doesn't match will fall back to alternative

New high bandwidth master and using myCDN

rsync –delay-update in two passes (files, then sym-links)

zfs replication. using zxfer.

zfs-snapshot-mgmt – avoid having to resync the entire dataset even if offline for a few days. zxfer – hasn't been touched since FreeBSD 8.2. Fails on later ZFS. Forked on github with many fixes, error checking, extra options To integrate: fdpv for progress bar animation

Buildfarm only has 5Mbps – is uploading constantly. Servers sync last 15mins of traffic within seconds. Staggered updates across servers.

Automatic mirror selection. GSLB from ScaleEngine. LB checks mirrors for freshness on images and drops them from the real hosts.

Conclusion: ZFS make replication easier and allows atomicity.

PCBSD servers pull about 36TiB per day – iso.cdn about 1 TiB/day, pkg.cdn ~85 GiB/day of that.

bapt: want to look at this for FreeBSD pkg repo and steal any applicable ideas.

Need to get pkg signing working – need signing server.

Compare portsnap is about 5TiB per month

Reprise of what was in the security session.

SSP mitigation fro stack based overflows. First appeared in FreeBSD 8, only in base system. Finally is here for ports.

The first 99% is easy. Easiest path simply to patch CFLAGS.

Needs a lot of polishing and testing before it can be default.

Showstoppers:

on i386 gcc special symbols addressed in specific way and needs to link binary in special way. Some ports do not honour LDFLAGS. Solution: turned libc.so into an ldscript: Propose libssp_nonshared for linking in 10.x. Now enabled by default for i386/amd64 in 10.x amd64 usable on 9.x by adding WITH_SSP_PORT knob (doesn't do anything on i386). Idea is to activate by default for official packages but not yet.

New archs need work.

New knob for port maintainers: SSP_UNSAFE – for ports that do weird things or for kernel modules.

Every pkg building system uses staging *except* FreeBSD. staging means install somewhere else rather than braking your system. Also create package without installing. Can create packages without being root, except for some packages doing weird things, for which a new NEED_ROOT flag has been added.

Plan to create a new fakeroot thing using ptrace, but that is v. architecture dependent. Overwrites BINOWN etc. as current user to allow package-as-a-user

Can improve package building system so we don't need to wast time trying to figure out of pkg building system is doing stuff outside staging area.

Won't have packages with broken plist – pkg register reads the plist to assemble the pkg. We also test installing as a pkg every time, which didn't happen before. (Many broken package scripts were broken but never tested)

Most of post-install targets can be ripped out of ports as they'll be done by the pkg.

Standard post-installtion tasks lke mkinfo no longer need to be run. Save time on pkg build systems, allows more EXP runs.

Added NO_STAGE to all ports so default for new ports now is to use staging. Don't want to spend years switching like pkgsrc. Plan is something like 6 months. Many packages work already, or require fairly simple changes.

Proposal: in 6 months time, mark as deprecated all packages still marked as NO_STAGE. Allow 3 months deprecation time. Will highlight what is still maintained in the ports tree.

Is this a bit aggressive? Yes, but having to maintain everything in dual is painful. 6 months is probably optimistic, may need to extend. Long deprecation period should allow maintainers to produce updates. Ultimate target is for stage project to complete in 1 year.

Have ripped out the MAN1 macros. Were needed for compress man – new implementation doesn't need this. Separate Makefile build code from PLIST style stuff. Also sub-packages: will have several plists for one port (MAN1… how do you know which sub package this belongs to).

redports should already have staging spport, but may need to wait for updates on Monday.

Monthly mail to maintainers with ports still with NO_STAGE, wiki page

AUTO_PLIST as an option? Can do now (as soon bapt gets connectivity…)

EXP runs with FORCE_STAGE to find what is already stage ready. Usually the port works, but post-install needs work.

stage is the last major modification to ports we can do without breaking support for pkg_install.

STAGE give opportunity to add QA testing heuristics which analyze the stage dir and suggest eg. adding dependencies

Create debug sub-packages from stage dir (ie. compile everything with debug symbols) Conversely warn about installing binaries with debug flags when not intended.

RUNPATH in binaries – know where to find shlibs

developer sub-package: should this be automatic or manual setup of sub-packages? cf. PORTDOCS – start by doing things manually, then provide more automation once it has matured.

  • devsummit/201309/ports.1380202192.txt.gz
  • Last modified: 2013/09/26 13:29
  • by MatthewSeaman