Project/routing status
Stage1: conversion
There are the following goals:
- Remove llentry read lock from fast path
- Use read rmlock for looking up lle data in fast path
- Remove rtentry lock from all data paths
- Use read rmlock for looking up routing next-hop data
Outstanding reviews
LLTABLE llentries
These changes are mostly done. Most interesting commits:
LLTABLE rmlock(9)
No major depends on performing conversion. Waiting on D4751 to timeout be approved and the new LLE lookup functions patch D4962.
The only “tricky” thing is that lltable needs to be protected by its own (per-lltable) set of locks, instead of (ab)using AFDATA interface lock.
Data path changes
The goal was to exclude struct rtentry and struct llentry from if_output() routines and provide an abstract way of passing arbitrary prepend data. Mostly done.
The last remaining step is to deal with per-ifa output traffic accounting (numbers from netstat -i).
Given that it won't be possible to do it fast (no stable ifa pointer can be returned) (at least for IPv4), there is a proposal to remove these counters: D4794
Most interesting commits:
New routing KPI conversion status
All rtalloc*() callers (along with code dereferencing struct rtentry) has to be converted to the new KPI. This is the pre-requisite for stage 2 (multipath, other advanced routing features). This is also the requirement for switching data path read lock to rmlock(9).
Most interesting commits:
Non-output consumers
Here we have lightweight routing KPI consumers: simple uRPF (reverse path versification), mtu checks, gateway or source address checks, etc.. Most of them are already converted.
The only huge problem is SCTP which caches struct rtentry and (which is much worse) actively uses its fields.
| Done | Desc | Rev | Code |
|---|---|---|---|
| ✓ | Need to return mask (e.g. RIB) | r293914 | sys/netgraph/netflow/netflow.c: rt = rtalloc1_fib((struct sockaddr *)&sin, 0, 0, r→fib); sys/netgraph/netflow/netflow.c: rt = rtalloc1_fib((struct sockaddr *)&sin, 0, 0, r→fib); sys/netgraph/netflow/netflow.c: rin6.ro_rt = rtalloc1_fib((struct sockaddr *)dst, 0, 0, r→fib); sys/netgraph/netflow/netflow.c: rin6.ro_rt = rtalloc1_fib((struct sockaddr *)src, 0, 0, r→fib); |
| ✓ | PF: basic is OK | D4763, r293311 | sys/netpfil/pf/pf.c: in_rtalloc_ign(&ro, 0, rtableid); sys/netpfil/pf/pf.c: in6_rtalloc_ign(&ro6, 0, rtableid); sys/netpfil/pf/pf.c: in6_rtalloc_ign(&ro, 0, rtableid); sys/netpfil/pf/pf.c: in_rtalloc_ign((struct route *)&ro, 0, rtableid); sys/netpfil/pf/pf.c: rt = rtalloc1_fib(sintosa(&dst), 0, 0, M_GETFIB(m0)); |
| ✓ | IPFW: basic is OK | r293626 | sys/netpfil/ipfw/ip_fw_table_algo.c: return (rtalloc1_fib(s, 0, 0, fib)); |
| ✓ | TOE: uRPF checks | r293309 | sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c: * rtalloc1, RT_UNLOCK on rt. sys/dev/cxgb/ulp/tom/cxgb_listen.c: rt = rtalloc1((struct sockaddr *)&nam, 0, 0); sys/dev/cxgb/ulp/iw_cxgb/iw_cxgb_cm.c: rtalloc(&iproute); sys/dev/cxgbe/tom/t4_connect.c: * rtalloc1, RT_UNLOCK on rt. sys/dev/cxgbe/tom/t4_listen.c: rt = rtalloc1(dst, 0, 0); sys/dev/cxgbe/iw_cxgbe/cm.c: rtalloc(&iproute); |
| TOE: tod_connect() | https://people.freebsd.org/~melifaro/toe_rt_to_nh.diff (sent to np@) | sys/netinet/tcp_offload.c: rt = rtalloc1(nam, 0, 0); |
|
| ✓ | ND6: proxyarp… | r293159 | sys/netinet6/nd6_nbr.c: rtalloc_mpath_fib((struct route *)&ro, ntohl(taddr6.s6_addr32[3]), sys/netinet6/nd6_nbr.c: in6_rtalloc(&ro, RT_DEFAULT_FIB); |
| ✓ | ND6: need rib (return prefix) | r293159 | sys/netinet6/nd6.c: rt = in6_rtalloc1((struct sockaddr *)&pr→ndpr_prefix, |
| ✓ | MTU checks, basic is OK, nearly done | r293169 | sys/netinet6/ip6_output.c: in6_rtalloc(ro_pmtu, fibnum); |
| ✓ | rtsock RIB, should be easy | r293159 | sys/net/rtsock.c: rtalloc_ign_fib(&gw_ro, 0, fibnum); |
| ✓ | TCP MTU, _ext is OK | r294712 | sys/netinet/tcp_subr.c: in_rtalloc_ign(&sro, 0, inc→inc_fibnum); sys/netinet/tcp_subr.c: in6_rtalloc_ign(&sro6, 0, inc→inc_fibnum); |
| pcb: ext is OK | sys/netinet/in_pcb.c: in_rtalloc_ign(&sro, 0, inp→inp_inc.inc_fibnum); | ||
| SCTP: NHOPS | Discussing w/ rrs@/tuexen@ | sys/netinet/sctp_pcb.c: SCTP_RTALLOC((sctp_route_t *) & net→ro, sys/netinet/sctp_output.c: SCTP_RTALLOC(ro, vrf_id, inp→fibnum); sys/netinet/sctp_output.c: SCTP_RTALLOC(ro, vrf_id, inp→fibnum); sys/netinet/sctp_output.c: SCTP_RTALLOC(ro, vrf_id, inp→fibnum); |
|
| IB: _ext should be OK | sys/ofed/drivers/infiniband/core/addr.c: rte = rtalloc1(dst_in, 1, 0); |
||
| ✓ | rib lookup | r293159 | sys/netinet/in.c: /* XXX rtalloc1_fib should take a const param */ sys/netinet/in.c: rt = rtalloc1_fib(DECONST(struct sockaddr *, l3addr), 0, 0, |
| ✓ | NFS: IPv6 SAS | r294084 | sys/fs/nfsclient/nfs_clport.c: rt = rtalloc1_fib((struct sockaddr *)&sad, 0, 0UL, sys/fs/nfsclient/nfs_clport.c: rt = rtalloc1_fib((struct sockaddr *)&sad6, 0, 0UL, |
| ✓ | IPF: ext is OK | D4764, r293628 | sys/contrib/ipfilter/netinet/ip_fil_freebsd.c: in_rtalloc(ro, M_GETFIB(m0)); sys/contrib/ipfilter/netinet/ip_fil_freebsd.c: in_rtalloc(&iproute, 0); |
| ✓ | gif: uRPF | r291993 | sys/netinet/in_gif.c: rt = in_rtalloc1((struct sockaddr *)&sin, 0, sys/netinet6/in6_gif.c: rt = in6_rtalloc1((struct sockaddr *)&sin6, 0, 0UL, |
| ✓ | mcast: ifp lookup | r292015 | sys/netinet/in_mcast.c: in_rtalloc_ign(&ro, 0, inp ? inp→inp_inc.inc_fibnum : 0); |
| ✓ | lltable: rib rtcheck | r292015 | sys/netinet/if_ether.c: rt = in_rtalloc1((struct sockaddr *)&sin, 0, 0UL, 0); sys/netinet/if_ether.c: rt = in_rtalloc1((struct sockaddr *)&sin, 0, 0UL, 0); sys/netinet6/in6.c: rt = in6_rtalloc1(DECONST(struct sockaddr *, l3addr), 0, 0, |
| ✓ | mcast: ifp lookup | r292015 | sys/netinet6/in6_mcast.c: rtalloc_ign_fib((struct route *)&ro6, 0, |
| ✓ | in6_ hlim | r292015 | sys/netinet6/in6_src.c: in6_rtalloc(&ro6, in6p→inp_inc.inc_fibnum); |
| ✓ | icmp6 redirect | r292015 | sys/netinet6/icmp6.c: rt = in6_rtalloc1((struct sockaddr *)&sin6, 0, 0UL, RT_DEFAULT_FIB); |
| ✓ | ipfw: uRPF | r291993 | sys/netpfil/ipfw/ip_fw2.c: in_rtalloc_ign(&ro, 0, fib); sys/netpfil/ipfw/ip_fw2.c: in6_rtalloc_ign(&ro, 0, fib); |
| ✓ | if_stf: uRPF | r292331 | sys/net/if_stf.c: rt = rtalloc1_fib((struct sockaddr *)&sin, 0, |
Output/Forwarding
Here is the code which is actually doing packets output. IPv4 fastforwarding/IPv6 forwarding could be converted (more or less) easily, for other code there are 2 showstoppers:
- rtentry caching in SCTP/flowtable. The plan is to implement simple nexthops (initally planned for stage 2) and cache nexthop references instead of caching struct rtentry.
- Per-ifa output packet accounting. There is no good/easy way to make this accounting work. D4794 is the review with the discussion.
| Done | Desc | Rev | Code |
|---|---|---|---|
| fib6_lookup_nhop | D5010 | sys/netinet6/ip6_forward.c: rin6.ro_rt = in6_rtalloc1((struct sockaddr *)dst, 0, 0, M_GETFIB(m)); |
|
| selectroute(). NHOPS + lookup_prepend() | sys/netinet6/in6_src.c: in6_rtalloc(ron, fibnum); /* multi path case? */ sys/netinet6/in6_src.c: rtalloc_mpath_fib((struct route *)ro, sys/netinet6/in6_src.c: ro→ro_rt = in6_rtalloc1((struct sockaddr *) |
||
| NHOPS + lookup_prepend() | sys/net/flowtable.c: rtalloc_mpath_fib(ro, hash, fibnum); sys/net/flowtable.c: rtalloc_ign_fib(ro, 0, fibnum); |
||
| fib4_lookup_prepend() | D5009 | sys/netinet/ip_fastfwd.c: in_rtalloc_ign(ro, 0, M_GETFIB(m)); |
|
| ip_forward() – no so easy: need to pass cached route to ip_output(). NHOPS + lookup_prepend() | sys/netinet/ip_input.c: rtalloc_mpath_fib(&ro, sys/netinet/ip_input.c: in_rtalloc_ign(&ro, 0, M_GETFIB(m)); |
||
| NHOPS + lookup_prepend() | sys/netinet/ip_output.c: rtalloc_mpath_fib(ro, sys/netinet/ip_output.c: in_rtalloc_ign(ro, 0, fibnum); |
Routing rmlock(9)
Depends on on rtalloc* functions conversion