Testing [FreeBSD WIKI]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
===== FreeBSD Developer Summit: Testing Working Group =====

Thursday June 8th 13:30 to 16:30 in DMS 1160

==== Overview ====

We will discuss the roadmap of the CI system, examine current [[Jenkins/TODO|TODO list]] and plan for setting up the performance testing cluster.

If you would like to participate, contact the working group chairs below and CC ''%%devsummit@%%''. You will be then added to this page. Please include a list of things you want to talk about or the areas you are interested in. This helps us in planning the session and to bring people together with common interests.

It is possible to bring in people who cannot attend in person via video conference or chat tools. Notes during the session will be published later on for the whole community to see what we discussed.

==== Goals ====

  * Let more people understand the internal of the [[https://ci.freebsd.org|FreeBSD's CI system]]
  * Check the current TODO list, have more people work on the items
  * Add new items to the TODO list
  * Plan for setting up the CI performance test cluster

==== Topics ====

^ Presenter ^ Topic Description ^
| [[/LiWenHsu|LiWenHsu]] | Internal of the %%FreeBSD%%'s CI system and Roadmap discussion [[https://docs.google.com/presentation/d/1zeycKng4krd01eoewAdqSY2J5hJmv-1Jdvbn1tWVVoI/edit#slide=id.p|slides]] |
| [[/SteveWills|SteveWills]] | Ports CI |
| [[/BradDavis|BradDavis]] | plan for [[Jenkins/PerfTesting]] |

Collaborative editor for notes: [[https://hackmd.io/KwMwLAjCAmDGBMBaARgDlQU0WZBmADIgIYgYSKoBsCA7BpQJyy4g1A==?both]]

==== Attending ====

In order to attend, you need to put your name on the general [[DevSummit/201706|developer summit]] and register yourself in this page or send email to the working group organizer (lwhsu@, brd@).

^ Name ^ Username / Affiliation ^ Topics of Interest ^ Notes ^
| [[/LiWenHsu|LiWenHsu]] | lwhsu | Organizer |  |
| [[/BradDavis|BradDavis]] | brd | Organizer |  |
| Diane Bruce | db |  |  |
| [[/MichaelDexter|MichaelDexter]] | dexter | %%FreeBSD%% regression testing |  |
| [[/PeterGrehan|PeterGrehan]] | grehan | %%FreeBSD%% regression testing |  |
| [[/SteveWills|SteveWills]] | swills | TCP and ports |  |
| [[/EdMaste|EdMaste]] | emaste |  | Likely only 2nd session |
| Siva Mahadevan | %%FreeBSD%% Foundation |  |  |
| Jason Tubnor | Invited by brd/dexter |  |  |
| Charlie Yang | %%FreeBSD%% Foundation |  |  |
| Nanako Momiyama | brd |  |  |

==== Results ====

Copied from https://hackmd.io/KwMwLAjCAmDGBMBaARgDlQU0WZBmADIgIYgYSKoBsCA7BpQJyy4g1A==

=== Continuous Integration infra (13:30-14:30) ===

=== Ports CI (14:30-15:00) ===

=== Network Performance Testing (15:30-16:30) ===

  * talk to vendors for h/w support?
  * where to put these machines? NYI
  * let venders can setup their test bed

=== CI/CD Tools ===

* https://github.com/ovh/cds << Github project
* https://ovh.github.io/cds/ << Documentation (pay attention to the nested pages)

=== Validate manpages in the base system ===

run make manlint on each first level directories:
usr.bin, bin, sbin, etc.

=== How to set up a FreeBSD ci ===

  * Basic instruction on how to setup a jenkins slave and maintain it
  * Some recommendations on how to setup complexe build (via jails? installing the dependencies on the fly, etc)

=== Building Custom Built Jenkins Builders/Images ===

seanc@ is merging these packer templates: https://github.com/sean-/joyent-freebsd

into: https://github.com/brd/packer-freebsd

=== CI for 3rd party software ===

"CFT: CI for [[/GitHub|GitHub]] on %%FreeBSD%%" on -hackers@
https://lists.freebsd.org/pipermail/freebsd-hackers/2017-May/051096.html

=== Network testing ===

Contact Olivier Cochard Labbé (olivier@%%FreeBSD%%.org)

== Performance for Network Testing ==

https://freebsd.circonus.com/

Free monitoring account donated to the project by Circonus.  Ask seanc@ for details.

  * Monitoring agent: https://github.com/circonus-labs/nad
  * https://github.com/circonus-labs/latsub : Passive TCP Latency Analysis via Circonus hisotgrams
  * https://github.com/circonus-labs/wirelatency : Go tools for analyzing wire protocols to report fine-grained telemetry

== tools ==
  * netperf
  * packetdrill
  * vegeta: https://github.com/tsenart/vegeta
  * hey / wrk https://github.com/giltene/wrk2 (nginx is using it)
  * iperf (uses select(2) and might not be a good choice)
  * samba
  * NFS

== Other tools ==
  * https://github.com/denji/awesome-http-benchmark
  * https://gist.github.com/denji/8333630