Project/routing status

There are the following goals:

  • Remove llentry read lock from fast path
  • Use read rmlock for looking up lle data in fast path
  • Remove rtentry lock from all data paths
  • Use read rmlock for looking up routing next-hop data
  • D5009 (IPv4 fast forwarding conversion)
  • D5010 (IPv6 forwarding conversion)
  • D4794 (Deal with per-ifa output counters)
  • D4962 (new LLE lookup functions, no sockaddrs in lltable data path)
  • D4751 (move all lltable code to separate files)

These changes are mostly done. Most interesting commits:

  • Removal of llentry read lock from fast path: IPv4: D3688 r291853, IPv6: D3780 r292155
  • Struct llentry reshaping (addresses instead of sockaddrs, all necessary data in first 64 bytes): r286624
  • Simplify attachin link-level header in IPv6 & make nd6_resolve() behave like arpresolve(): D1469 r287861
  • Make lltable code use unified per-af callbacks: r286577
  • Split IPv4/IPv6 lltable init/hash/lookup permitting having different hash functions/hash size for each address family: r286616

No major depends on performing conversion. Waiting on D4751 to timeout be approved and the new LLE lookup functions patch D4962.

The only “tricky” thing is that lltable needs to be protected by its own (per-lltable) set of locks, instead of (ab)using AFDATA interface lock.

The goal was to exclude struct rtentry and struct llentry from if_output() routines and provide an abstract way of passing arbitrary prepend data. Mostly done.

The last remaining step is to deal with per-ifa output traffic accounting (numbers from netstat -i).

Given that it won't be possible to do it fast (no stable ifa pointer can be returned) (at least for IPv4), there is a proposal to remove these counters: D4794

Most interesting commits:

  • Link precomputation API (remove lle cache from packet output routines): D4102 r292978
  • Struct route changes (remove rtentry dereference from packet output routines): r275196 r293544

All rtalloc*() callers (along with code dereferencing struct rtentry) has to be converted to the new KPI. This is the pre-requisite for stage 2 (multipath, other advanced routing features). This is also the requirement for switching data path read lock to rmlock(9).

Most interesting commits:

  • Introduce fibX KPI: 291993
  • Transparently support current RADIX_MPATH implementation in fibX: 293657
  • Last user changing rte (rt_mtu) w/o proper locking eliminated: D1125 274611

Non-output consumers

Here we have lightweight routing KPI consumers: simple uRPF (reverse path versification), mtu checks, gateway or source address checks, etc.. Most of them are already converted.

The only huge problem is SCTP which caches struct rtentry and (which is much worse) actively uses its fields.

Done Desc Rev Code
✓ Need to return mask (e.g. RIB) r293914 sys/netgraph/netflow/netflow.c: rt = rtalloc1_fib((struct sockaddr *)&sin, 0, 0, r→fib);
sys/netgraph/netflow/netflow.c: rt = rtalloc1_fib((struct sockaddr *)&sin, 0, 0, r→fib);
sys/netgraph/netflow/netflow.c: rin6.ro_rt = rtalloc1_fib((struct sockaddr *)dst, 0, 0, r→fib);
sys/netgraph/netflow/netflow.c: rin6.ro_rt = rtalloc1_fib((struct sockaddr *)src, 0, 0, r→fib);
✓ PF: basic is OK D4763, r293311 sys/netpfil/pf/pf.c: in_rtalloc_ign(&ro, 0, rtableid);
sys/netpfil/pf/pf.c: in6_rtalloc_ign(&ro6, 0, rtableid);
sys/netpfil/pf/pf.c: in6_rtalloc_ign(&ro, 0, rtableid);
sys/netpfil/pf/pf.c: in_rtalloc_ign((struct route *)&ro, 0, rtableid);
sys/netpfil/pf/pf.c: rt = rtalloc1_fib(sintosa(&dst), 0, 0, M_GETFIB(m0));
✓ IPFW: basic is OK r293626 sys/netpfil/ipfw/ip_fw_table_algo.c: return (rtalloc1_fib(s, 0, 0, fib));
✓ TOE: uRPF checks r293309 sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c: * rtalloc1, RT_UNLOCK on rt.
sys/dev/cxgb/ulp/tom/cxgb_listen.c: rt = rtalloc1((struct sockaddr *)&nam, 0, 0);
sys/dev/cxgb/ulp/iw_cxgb/iw_cxgb_cm.c: rtalloc(&iproute);
sys/dev/cxgbe/tom/t4_connect.c: * rtalloc1, RT_UNLOCK on rt.
sys/dev/cxgbe/tom/t4_listen.c: rt = rtalloc1(dst, 0, 0);
sys/dev/cxgbe/iw_cxgbe/cm.c: rtalloc(&iproute);
TOE: tod_connect() https://people.freebsd.org/~melifaro/toe_rt_to_nh.diff (sent to np@) sys/netinet/tcp_offload.c: rt = rtalloc1(nam, 0, 0);
✓ ND6: proxyarp… r293159 sys/netinet6/nd6_nbr.c: rtalloc_mpath_fib((struct route *)&ro, ntohl(taddr6.s6_addr32[3]),
sys/netinet6/nd6_nbr.c: in6_rtalloc(&ro, RT_DEFAULT_FIB);
✓ ND6: need rib (return prefix) r293159 sys/netinet6/nd6.c: rt = in6_rtalloc1((struct sockaddr *)&pr→ndpr_prefix,
✓ MTU checks, basic is OK, nearly done r293169 sys/netinet6/ip6_output.c: in6_rtalloc(ro_pmtu, fibnum);
✓ rtsock RIB, should be easy r293159 sys/net/rtsock.c: rtalloc_ign_fib(&gw_ro, 0, fibnum);
✓ TCP MTU, _ext is OK r294712 sys/netinet/tcp_subr.c: in_rtalloc_ign(&sro, 0, inc→inc_fibnum);
sys/netinet/tcp_subr.c: in6_rtalloc_ign(&sro6, 0, inc→inc_fibnum);
pcb: ext is OK sys/netinet/in_pcb.c: in_rtalloc_ign(&sro, 0, inp→inp_inc.inc_fibnum);
SCTP: NHOPS Discussing w/ rrs@/tuexen@ sys/netinet/sctp_pcb.c: SCTP_RTALLOC((sctp_route_t *) & net→ro,
sys/netinet/sctp_output.c: SCTP_RTALLOC(ro, vrf_id, inp→fibnum);
sys/netinet/sctp_output.c: SCTP_RTALLOC(ro, vrf_id, inp→fibnum);
sys/netinet/sctp_output.c: SCTP_RTALLOC(ro, vrf_id, inp→fibnum);
IB: _ext should be OK sys/ofed/drivers/infiniband/core/addr.c: rte = rtalloc1(dst_in, 1, 0);
✓ rib lookup r293159 sys/netinet/in.c: /* XXX rtalloc1_fib should take a const param */
sys/netinet/in.c: rt = rtalloc1_fib(DECONST(struct sockaddr *, l3addr), 0, 0,
✓ NFS: IPv6 SAS r294084 sys/fs/nfsclient/nfs_clport.c: rt = rtalloc1_fib((struct sockaddr *)&sad, 0, 0UL,
sys/fs/nfsclient/nfs_clport.c: rt = rtalloc1_fib((struct sockaddr *)&sad6, 0, 0UL,
✓ IPF: ext is OK D4764, r293628 sys/contrib/ipfilter/netinet/ip_fil_freebsd.c: in_rtalloc(ro, M_GETFIB(m0));
sys/contrib/ipfilter/netinet/ip_fil_freebsd.c: in_rtalloc(&iproute, 0);
✓ gif: uRPF r291993 sys/netinet/in_gif.c: rt = in_rtalloc1((struct sockaddr *)&sin, 0,
sys/netinet6/in6_gif.c: rt = in6_rtalloc1((struct sockaddr *)&sin6, 0, 0UL,
✓ mcast: ifp lookup r292015 sys/netinet/in_mcast.c: in_rtalloc_ign(&ro, 0, inp ? inp→inp_inc.inc_fibnum : 0);
✓ lltable: rib rtcheck r292015 sys/netinet/if_ether.c: rt = in_rtalloc1((struct sockaddr *)&sin, 0, 0UL, 0);
sys/netinet/if_ether.c: rt = in_rtalloc1((struct sockaddr *)&sin, 0, 0UL, 0);
sys/netinet6/in6.c: rt = in6_rtalloc1(DECONST(struct sockaddr *, l3addr), 0, 0,
✓ mcast: ifp lookup r292015 sys/netinet6/in6_mcast.c: rtalloc_ign_fib((struct route *)&ro6, 0,
✓ in6_ hlim r292015 sys/netinet6/in6_src.c: in6_rtalloc(&ro6, in6p→inp_inc.inc_fibnum);
✓ icmp6 redirect r292015 sys/netinet6/icmp6.c: rt = in6_rtalloc1((struct sockaddr *)&sin6, 0, 0UL, RT_DEFAULT_FIB);
✓ ipfw: uRPF r291993 sys/netpfil/ipfw/ip_fw2.c: in_rtalloc_ign(&ro, 0, fib);
sys/netpfil/ipfw/ip_fw2.c: in6_rtalloc_ign(&ro, 0, fib);
✓ if_stf: uRPF r292331 sys/net/if_stf.c: rt = rtalloc1_fib((struct sockaddr *)&sin, 0,

Output/Forwarding

Here is the code which is actually doing packets output. IPv4 fastforwarding/IPv6 forwarding could be converted (more or less) easily, for other code there are 2 showstoppers:

  • rtentry caching in SCTP/flowtable. The plan is to implement simple nexthops (initally planned for stage 2) and cache nexthop references instead of caching struct rtentry.
  • Per-ifa output packet accounting. There is no good/easy way to make this accounting work. D4794 is the review with the discussion.
Done Desc Rev Code
fib6_lookup_nhop D5010 sys/netinet6/ip6_forward.c: rin6.ro_rt = in6_rtalloc1((struct sockaddr *)dst, 0, 0, M_GETFIB(m));
selectroute(). NHOPS + lookup_prepend() sys/netinet6/in6_src.c: in6_rtalloc(ron, fibnum); /* multi path case? */
sys/netinet6/in6_src.c: rtalloc_mpath_fib((struct route *)ro,
sys/netinet6/in6_src.c: ro→ro_rt = in6_rtalloc1((struct sockaddr *)
NHOPS + lookup_prepend() sys/net/flowtable.c: rtalloc_mpath_fib(ro, hash, fibnum);
sys/net/flowtable.c: rtalloc_ign_fib(ro, 0, fibnum);
fib4_lookup_prepend() D5009 sys/netinet/ip_fastfwd.c: in_rtalloc_ign(ro, 0, M_GETFIB(m));
ip_forward() – no so easy: need to pass cached route to ip_output(). NHOPS + lookup_prepend() sys/netinet/ip_input.c: rtalloc_mpath_fib(&ro,
sys/netinet/ip_input.c: in_rtalloc_ign(&ro, 0, M_GETFIB(m));
NHOPS + lookup_prepend() sys/netinet/ip_output.c: rtalloc_mpath_fib(ro,
sys/netinet/ip_output.c: in_rtalloc_ign(ro, 0, fibnum);

Depends on on rtalloc* functions conversion

  • projectsroutingproposal/conversionstatus.txt
  • Last modified: 2016/01/26 10:30
  • by AlexanderChernikov