jenkins-admin@ TODO list

Item	Responsible	Status	Note
Recruit more people
Setup ci.FreeBSD.org: config document	LiWenHsu	In progress	Setup
Setup ci.FreeBSD.org: environment provision	clusteradm	DONE	Allocate IPs, config web frontend server and firewall
Setup ci.FreeBSD.org: basic system	LiWenHsu	In progress	jenkins, nginx, ftpd, config backup/notification
Integration with reviews.FreeBSD.org	LiWenHsu	In progress	Integration_with_reviews.FreeBSD.org
Design more fine-grained pipeline			Design_more_fine-grained_pipeline
Design a way to purpose new job easier			Design_a_way_to_purpose_new_job_easier
Revive clang-scan-build job	LiWenHsu	In progress	Revive_clang-scan-build_job
Revive gcc build
Automatically ports exp-run			for all tree / one port
Check reproducible build: src			Check_reproducible_build
Check reproducible build: ports			Check_reproducible_build
Integration with redports
Non-default-on tests: DTrace test suite	LiWenHsu	In progress	Non-default-on_tests:_DTrace_test_suite
Setup stage environment	LiWenHsu	In progress	https://ci-dev.FreeBSD.org
Better FreeBSD.org authentication mechanism			Work with clusteradm to setup oauth.FreeBSD.org, and create an OAuth plugin for FreeBSD.org or general use
Design dashboard			ex: https://ci.trafficserver.apache.org/
YSV → NYI migration	LiWenHsu	DONE	We'll just setup new ci.FreeBSD.org in NYI
Performance test
Run tcp-testsuite
arm build	BradDavis
mips build		In progress	waiting for mipsn32 kernel config
powerpc build		DONE
sparc64 build		DONE
provide more images on artifact server			Provide_more_images_on_artifact_server
artifact expiring script
Document “how to submit new job”

Item	Responsible	Status	Note
Create “FreeBSD jail wrapper” plugin			Can reference chroot or docker plugin. Also put this item on IdeasPage
Create “FreeBSD.org OAuth” plugin			Can reference other OAuth plugin.

Pipeline execution time: building world & kernel is faster than running full regression tests. We might do:
- Build world & kernel as often as we can, then trigger full regression tests if there is no one running.
- Wait until a full regression test complete, then start next round building world & kernel
Create job naming convention
How to do real hardware test?
How to work with re@?
- Provide snapshot build
- Perform ABI compatibility check

Master branch of https://github.com/uber/phabricator-jenkins-plugin supports svn repository, but has no official release currently
- https://issues.jenkins-ci.org/browse/JENKINS-31335
- Pull request: https://github.com/uber/phabricator-jenkins-plugin/pull/214

Current jobs:
- https://ci.freebsd.org/view/phabric/

Phabricator build rules:
- https://reviews.freebsd.org/H75
- https://reviews.freebsd.org/H76

Phabricator build plans:

Currently new build only triggered by a patch author is in certain list (committers), for some security consideration.
- Is it possible for it to be triggered by a tag set by a committer attached to a patch?

Known issue:
- error on fetching patch, ex: https://ci.freebsd.org/job/phabric-FreeBSD-head-amd64-build/249/console
  - need to know it's setting error on phabricator site or error of jenkins's phabricator plugin
- applying a malformated patch cause infinite loop: FreeBSD-head-amd64-build-phabric build #251 generated 20G log
- if head is broken, all patch builds will fail. so need to check out the latest known buildable revision.

https://lists.freebsd.org/pipermail/freebsd-testing/2015-June/001014.html
ex: quick amd64/i386 build to detect compiling failure first → make universe / build for other architectures → test in VM

A possible way: make a “template job”, some preliminary work:
- https://github.com/freebsd/freebsd-ci/tree/master/scripts/jail
- https://github.com/freebsd/freebsd-ci/tree/master/jobs
Create “FreeBSD jail wrapper” plugin, let user can run almost arbitrary commands inside jail.
Submit ${JENKINS_HOME}/jobs/<jobname>/config.xml with some additional documents
Use jenkins-job-builder
- https://github.com/freebsd/freebsd-ci/tree/master/jjb
Let user can easily test locally

Use Clang Scan-Build Plugin
Use make analyze:

make CLANG_ANALYZE_OUTPUT_DIR=${WORKSPACE}/clangScanBuildReports CLANG_ANALYZE_OUTPUT=html analyze

Perhaps make “make analyze” recursively check all sub-directories when type in /usr/src ?
Current amd64 job: https://ci.freebsd.org/job/FreeBSD-head-scan_build/
- Latest result: https://ci.freebsd.org/job/FreeBSD-head-scan_build/lastCompletedBuild/clangScanBuildBugs/

Related tools:
- sysutils/py-diffoscope
- HTML Publisher Plugin
Issues:
- Cannot use settimeofday(2) in jail, the second build should use VM

src.conf: WITH_DTRACE_TESTS
amd64: https://ci.freebsd.org/job/FreeBSD-head-amd64-dtrace_test/
TODO: distfile dedup

need makefs(8) support to replace mount_msdosfs(8) usage in jail
- need to import new makefs(8) from NetBSD.
- In progress, see https://github.com/emaste/freebsd/blob/makefs-wip/TODO-makefs.md

Todo