NEW_PCIB is a kernel option to enable a “new” PCI-PCI bridge driver. The “new” driver actively manages the resource windows in the PCI-PCI bridge from which resources for child devices are allocated. The “old” driver used whatever ranges the firmware provided. This includes the resources managed include SYS_RES_MEMORY and SYS_RES_IOPORT for memory and I/O BARs as well as the recently added PCI_RES_BUS to manage the PCI bus numbers assigned to secondary buses behind bridges.
Currently, the NEW_PCIB driver is not enabled on all platforms. The goal of this page is to coordinate making that happen.
NEW_PCIB Platforms
| Platform | NEW_PCIB Enabled | PCI_RES_BUS Enabled | Notes |
|---|---|---|---|
| amd64 | Yes | Yes | |
| arm | No | No | |
| arm64 | Yes | Yes | |
| i386 | Yes | Yes | |
| mips | No | No | |
| pc98 | Yes | Yes | |
| powerpc | Yes | No | |
| riscv | Yes | Yes | |
| sparc64 | Yes | No |
Steps to support NEW_PCIB
To support, the base NEW_PCIB driver (which supports managing memory and I/O port windows), the platform specific code needs the following changes:
- The Host-PCI bridge drivers need to properly handle
bus_adjust_resource()calls forSYS_RES_MEMORYandSYS_RES_IOPORTresources. Typically, this means forwarding the requests up the tree to the device that allocates the resource from a resource manager (struct rman). That driver'sbus_adjust_resourcemethod should userman_adjust_resource(). For an example, seenexus_adjust_resource()insys/x86/x86/nexus.c.
- Any bus drivers that provide resources to child PCI devices (e.g. Host-PCI bridge drivers and any parent devices that the bridge drivers pass
bus_alloc_resourceup to) need to only configure bus space handles and tags in thebus_activate_resourceandbus_deactivate_resourcemethods. Currently, theNEW_PCIBdriver relies on the ability to initialize its own resource managers to manage the resource ranges assigned to its I/O windows. When a child device allocates a resource from a PCI-PCI bridge, a resource is allocated from that rman, but that resource is then passed up the tree viabus_activate_resourceto obtain a bus space tag and handle. This only works if the bus space tag and handle logic is self-contained inbus_activate_resourceandbus_deactivate_resource.
- Test and then enable in the platform's
DEFAULTSconfig file.
Steps to support PCI_RES_BUS
Once a platform supports the base NEW_PCIB driver, the next step is to add support for PCI bus number management. This requires the following changes:
- Define a value for
PCI_RES_BUSin<machine/resource.h. The PCI-PCI bridge driver and PCI bus driver enable support for PCI bus number management automatically if this is defined.
- The Host-PCI bridge drivers need to handle requests to allocate (
bus_alloc_resource), adjust (bus_adjust_resource), and release (bus_release_resource)PCI_RES_BUSbus numbers. The PCI layer provides a library of routines to manage these requests with a separate resource pool of bus numbers for each PCI domain (or segment). All the Host-PCI bridge driver must do is provide the appropriate domain number when calling thepci_domain_*_busmethods to implement these requests. An example of this can be found in the ACPI Host-PCI bridge driver insys/dev/acpica/acpi_pcib_acpi.c
Future Work
Support for hotplug PCI-e should build on this work (and will likely require both NEW_PCIB and PCI_RES_BUS).
One issue with the current approach is that passing a request allocated from the PCI-PCI bridge's rman up to a parent device for bus_activate_resource is a gross hack and a layering violation. Also, bus_activate_resource is currently overloaded to do two tasks: 1) mark a resource as actively in use (e.g. a PCI BAR that is active is decoding the assigned address range) and 2) allocate a bus space tag and handle so that the device driver can use the bus_space API to access a resource. Sometimes a resource needs the first property, but having the second property thrown in has unfortunate side effects (e.g. KVA wasteage on 32-bit platforms).
One idea to fix both of these problems would be to split bus_activate_resource up. However, care must be taken to avoid a huge API incompatibility. One approach that may be able to do both would be:
- Add a new pair of bus methods that allow arbitrary sub-ranges of a given resource to be “mapped” (meaning allocate a bus space tag and handle):
bus_map_resource(bus, child, resource, start, length, *tag, *handle)andbus_unmap_resource(bus, child, resource, start, length, tag, handle).
- Change
struct resourceto support having a linked list of all mappings so that when a resource is free'd, all mappings for that resource can be released. It would probably be best if this was an optional library call in the rman API that bus drivers can call to add a known mapping to a given resource rather than having them automatically added somehow.
- When the PCI-PCI bridge driver sub-allocates a resource from a window, it locates the corresponding range of the resource it has allocated from its parent backing the window and uses
bus_map_resourceto obtain a mapping for the sub-range of that resource that corresponds to the child resource to obtain a mapping for the child resource.
- To maintain API compatibility, the
RF_ACTIVEflag passed tobus_alloc_resourcewill callbus_map_resourceafterbus_activate_resourcesucceeds to allocate the “full” mapping of the resource that sets the public tag and handle members ofstruct resource. However, a newRF_UNMAPPEDflag will inhibit this step allowing a driver to mark a resource as active but chose to only map a subset of a resource for driver access. This last requirement is the trickiest as it means changing just about every bus driver in the tree that has a non-trivialbus_alloc_resourcemethod to support RF_UNMAPPED.