Transport [FreeBSD WIKI]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Transport ======

Notes courtesy of Tom Jones.

===== Topics =====

  * Jonathan could do his slides on transport group activity
  * timer precision and locking stragies in TCP
  * TCP models and testing and evaluation
  * Tom upstreaming crazy transport changes
  * Ilias Manus LLC Changes
  * BZ's methodology (GEM5)
  * Update on TCP-AO
  * In kernel TLS

====== Jonathan's slides on transport group activity ======

[[https://docs.google.com/presentation/d/1UK0wqcpVCxrCBAOBzAVMJZ9pcdbhHQm30rn5o7QLL9Y/edit?usp=sharing|slides]]

Q: It is great that work has happened, but where is the code?

A:
  * Fastpath and Alt stack infrastructure are in head
  * bbr and rack are not
  * blackbox in [[https://reviews.freebsd.org/D11085|review]]
  * callout is off in the future
  * pacer and timer is tied to rack and bbr
  * you can ask for the code

Q: Has anyone done anything interesting to others?

A: Limelight is sponsoring fast open client, it will work for default and bbr and rack. Expected by end of 2017.

Q: Who is working on the modular stacks?

A: Jonathan is working on creating the default stack.

GNN: Are there public packetdrill tests for the default stack?

A: Yes, in a public repro.

GNN is working on better timestamps in the mbuf, GNN would like to use a
''%%time_t%%'' but it won't quite fit. Hardware timestamps would be nice, but device
support is not consistent.

====== Tom upstreaming crazy transport changes ======

Tom explains what UDP Options is.

Is there interest in the project? Get it into a committable state and we can
talk about it.

====== Update on TCP-AO ======

Should be fairly non interfering.

====== BZ's methodology (GEM5) ======

2 use case to use the GEM5 simulator

  * locking The world has changed from high throughput bidirectional streams, maybe it is time to revisit locking problems.  Many %%FreeBSD%% TCP workloads are now unidirectional streams.

We could do some experiments with both types of workload and see where the
hot spots are.

This performance experiement could be done on gem5.

Two potential combinations: combine horizontally (socket read and write buffer locks), or combine vertically (socket and PCB locks). Prioritize testing the vertical combination.

  * callout problems could be looked at on gem5

    * Callout precision requirements: high-precision requirements vs. low-precision requirements

    * Callout run length: quick vs. long

    * Likelihood to run: likely to run vs. likely to cancel

    * How to satisfy these requirements?

    * Where is delay introduced unexpectedly?

Q: Has anyone ever measured the ioctl overhead on their running machines?

A: NF looked at setsockopts per second, remove extraneous ones and it didn't
give much improvement.

====== Timer precision and locking strategies in TCP ======
====== TCP models and testing and evaluation ======

10 years ago a formal model was created, but TCP is specifed in 30-40 RFC
documents. A newer model has been written using dtrace to capture TCP Stack
events. packetdrill is used to run tests on a tun interface.

More modern tcp features are not supported in the model.

Doesn't touch the congestion control, but will touch things like timers and IW.

This model cannot discover bugs in the stack, %%FreeBSD%% must be approximately
correct. Results are still pending.

A single test suite for evaluating tcp would be interesting.

Collecting traces from production to evaluate against the model could show
examples of different workloads than could be produced in a lab. blackbox could
be applied to generate events for the model.

====== Ilias Manus LLC Changes ======

Sigcom paper on specialised network stacks, clean slate tcp stack. Shows that
specialisation and micro architecture awareness can result in large
improvements.

As disc latencies have gone down, the end to end latency between the disk and
the nic has reduced. If it fits in the cache it is fast, if it isn't dram
bandwidth becomes a problem.

Pretty close to optimal.

Accessing memory is the problem.

lessons
  * very asynchronous network stack, end to end latency is very high
  * %%FreeBSD%% could do things differently, AES in dram, eliminating the buffer cache will help.
  * crypto is free, dcache is everything.

Use processor behaviour as input to the tcp congestion control, optimizing to
keep things in the llc.

====== In kernel TLS ======

How does NF do in kernel TLS?

SSL pushed into kernel as much as possible, modifications to openssl.
Handshake and keying as normal, then offloads into the kernel with a socket
option. %%OpenSSL%% still does framing of packets; kernel stores header, trailer and
a pointer to the data (ala sendfile). AESNI calls used to encrypt data in the buffer.

Big performance impact.

Q: how do you handle tls alerts

A: all input packets are delivered to userspace openssl, works for the
asymmetric traffic workload. When necessary, openssl can temporarily deactivate kernel encryption.