Show pageOld revisionsBacklinksBack to top This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong. ====== Unified busdma ====== This page describes the work done to make our current busdma implementation almost entirely machine independent (MI), with machine dependent operations done by callback functions. This project is also being referred to as busdma/mi. ===== Problems with the current implementation ===== * Mostly machine-dependent (MD), but with almost identical implementations across architectures. * There's no correlation/association between a device_t and a bus_dma_tag_t. * Bus constraints (inheritance) implemented as an after-thought (= bus_get_dma_tag). Not followed strictly. * ISA DMA is not integrated -- no generic support for third-party DMA. * Locality information between devices and memory is not present -- no NUMA awareness for DMA. * bus_space_t is assumed to map 1:1 to vm_paddr_t -- no support for I/O MMUs. * Bounce buffering cannot be used to solve scatter/gather problems -- bounce buffers are not guaranteed to be contiguous in physical memory. * bus_dmamap_sync() is lacking features and its use is sometimes awkward. * bus_dmamem_alloc() effective returns mapped/loaded memory, but drivers still call bus_dmamap_load(), but no loading is needed. * bus_dmamem_alloc() does not work with multiple segments (i.e. maxsegsz < maxsize && nsegs > 1). * With segments tied to the bus_dma_tag_t, multiple tags are needed even though they describe identical constraints. ===== Objectives of busdma/mi ===== * MI implementation of the infrastructure, with minimal or no MD code. * NUMA awareness for (physical) memory allocations. * Integrated support for both bus mastering and "third-party" DMA. * Integrated support for I/O MMUs. * Bounce buffering is implemented/treated as a software-only I/O MMU. * mmap(2) extensions to allow for mapping DMA-capable memory into process space. ===== Current API (FreeBSD vs. NetBSD) ===== For reference only: | %%FreeBSD%% | %%NetBSD%% | | //bus_dma_tag_t// **bus_get_dma_tag**(//device_t// dev) | | | //int// **bus_dma_tag_create**(//bus_dma_tag_t// parent, //bus_size_t// alignment, //bus_addr_t// boundary, //bus_addr_t// lowaddr, //bus_addr_t// highaddr, //bus_dma_filter_t *// filtfunc, //void *// filtfuncarg, //bus_size_t// maxsize, //int// nsegments, //bus_size_t// maxsegsz, //int// flags, //bus_dma_lock_t *// lockfunc, //void *// lockfuncarg, //bus_dma_tag_t *// dmat) | | | //int// **bus_dma_tag_destroy**(//bus_dma_tag_t//) | //void// **bus_dmatag_destroy**(//bus_dma_tag_t//) | | | //int// **bus_dmatag_subregion**(//bus_dma_tag_t//, //bus_addr_t//, //bus_addr_t//, //bus_dma_tag_t *//, //int//) | | //int// **bus_dmamap_create**(//bus_dma_tag_t//, //int//, //bus_dmamap_t *//) | //int// **bus_dmamap_create**(//bus_dma_tag_t//, //bus_size_t//, //int//, //bus_size_t//, //bus_size_t//, //int//, //bus_dmamap_t *//) | | //int// **bus_dmamap_destroy**(//bus_dma_tag_t//, //bus_dmamap_t//) | //void// **bus_dmamap_destroy**(//bus_dma_tag_t//, //bus_dmamap_t//) | | //int// **bus_dmamap_load**(//bus_dma_tag_t//, //bus_dmamap_t//, //void *//, //bus_size_t//, //bus_dmamap_callback_t *//, //void *//, //int//) | //int// **bus_dmamap_load**(//bus_dma_tag_t//, //bus_dmamap_t//, //void *//, //bus_size_t//, //struct proc *//, //int//) | | //int// **bus_dmamap_load_mbuf**(//bus_dma_tag_t//, //bus_dmamap_t//, //struct mbuf *//, //bus_dmamap_callback2_t *//, //void *//, //int//) | //int// **bus_dmamap_load_mbuf**(//bus_dma_tag_t//, //bus_dmamap_t//, //struct mbuf *//, //int//) | | //int// **bus_dmamap_load_mbuf_sg**(//bus_dma_tag_t//, //bus_dmamap_t//, //struct mbuf *//, //bus_dma_segment_t *//, //int *//, //int//) | | | | //int// **bus_dmamap_load_raw**(//bus_dma_tag_t//, //bus_dmamap_t//, //bus_dma_segment_t *//, //int//, //bus_size_t//, //int//) | | //int// **bus_dmamap_load_uio**(//bus_dma_tag_t//, //bus_dmamap_t//, //struct uio *//, //bus_dmamap_callback2_t *//, //void *//, //int//) | //int// **bus_dmamap_load_uio**(//bus_dma_tag_t//, //bus_dmamap_t//, //struct uio *//, //int//) | | //void// **bus_dmamap_unload**(//bus_dma_tag_t//, //bus_dmamap_t//) || | //void// **bus_dmamap_sync**(//bus_dma_tag_t//, //bus_dmamap_t//, //bus_dmasync_op_t//) | //void// **bus_dmamap_sync**(//bus_dma_tag_t//, //bus_dmamap_t//, //bus_addr_t//, //bus_size_t//, //int//) | | //int// **bus_dmamem_alloc**(//bus_dma_tag_t//, //void %%**%%//, //int//, //bus_dmamap_t *//) | //int// **bus_dmamem_alloc**(//bus_dma_tag_t//, //bus_size_t//, //bus_size_t//, //bus_size_t//, //bus_dma_segment_t *//, //int//, //int *//, //int//) | | //void// **bus_dmamem_free**(//bus_dma_tag_t//, //void *//, //bus_dmamap_t//) | //void// **bus_dmamem_free**(//bus_dma_tag_t//, //bus_dma_segment_t *//, //int//) | | | //int// **bus_dmamem_map**(//bus_dma_tag_t//, //bus_dma_segment_t *//, //int//, //size_t//, //void %%**%%//, //int//) | | | //void// **bus_dmamem_unmap**(//bus_dma_tag_t//, //void *//, //size_t//) | | | //paddr_t// **bus_dmamem_mmap**(//bus_dma_tag_t//, //bus_dma_segment_t *//, //int//, //off_t//, //int//, //int//) | ===== New API ===== Open design items: * How to get from //(client device_t, SYS_RES_DMA, rid)//, via //struct resource *//, to //(DMA device_t, channel, callback)// so that we can start and stop the third-party DMA engine properly? Maybe we can store a pointer to a control structure with rman_set_virtual() and obtain that pointer later again with rman_get_virtual()? The assumption being that there's never any virtual memory address associated with a DMA resource. Tag functions: DMA tags describe DMA constraints. Tags are created for a particular device and are therefore associated with a device. Since any device is part of the newbus hierarchy, constraints get inherited from the attached bus in a natural manner. | //int// | **busdma_tag_create** | (//device_t// dev, //bus_addr_t// align, //bus_addr_t// bndry, //bus_addr_t// maxaddr, //bus_size_t// maxsz, //u_int// nsegs, //bus_size_t// maxsegsz, //u_int// datarate, //u_int// flags, //busdma_tag_t *// tag_p) | | //int// | **busdma_tag_derive** | (//busdma_tag_t// tag, //bus_addr_t// align, //bus_addr_t// bndry, //bus_addr_t// maxaddr, //bus_size_t// maxsz, //u_int// nsegs, //bus_size_t// maxsegsz, //u_int// datarate, //u_int// flags, //busdma_tag_t *// tagp) | | //int// | **busdma_tag_destroy** | (//busdma_tag_t// tag) | | //busdma_addr_t// | **busdma_tag_get_align** | (//busdma_tag_t// tag) | | //busdma_addr_t// | **busdma_tag_get_bndry** | (//busdma_tag_t// tag) | | //busdma_addr_t// | **busdma_tag_get_maxaddr** | (//busdma_tag_t// tag) | | //busdma_tag_t// | **device_get_busdma_tag** | (//device_t// dev) | | //busdma_tag_t// | **device_set_busdma_tag** | (//device_t// dev, //busdma_tag_t// tag) | Memory segment functions: DMA memory segments are used to describe a single contiguous address range in either virtual address space, physical address space or bus address space. Each segment can have a pointer assigned to it for administrative purposes. For example, a virtual address range could be associated with a VM space or PMAP, a bus address range can be associated with a set of I/O MMU resources and a physical address range can be associated with a locality domain. | //uintmax_t// | **busdma_seg_get_addr** | (//busdma_seg_t// seg) | | //uintmax_t// | **busdma_seg_get_size** | (//busdma_seg_t// seg) | | //void *// | **busdma_seg_get_opaque** | (//busdma_seg_t// seg) | | //void *// | **busdma_seg_set_opaque** | (//busdma_seg_t// seg, //void *// opaque) | Memory descriptor functions: DMA memory descriptors provide information about the memory used for DMA. This information includes a list of (scatter/gather) segments in virtual address space, a list of segments in (CPU) physical address space and a list of segments in bus address space. Virtual address segments exist when the memory descriptor is mapped. Bus address segments exist when the memory descriptor is loaded. Physical address segments exist when the memory descriptor is not empty or is in use. | //int// | **busdma_md_create** | (//busdma_tag_t// tag, //u_int// flags, //busdma_md_t *// md_p) | | //int// | **busdma_md_destroy** | (//busdma_md_t// md) | | //int// | **busdma_md_load_bio** | (//busdma_md_t// md, //struct bio *// bio, //busdma_callback_f// cb, //void *// arg, //u_int// flags) | | //int// | **busdma_md_load_buf** | (//busdma_md_t// md, //void *// buf, //size_t// size, //busdma_callback_f// cb, //void *// arg, //u_int// flags) | | //int// | **busdma_md_load_ccb** | (//busdma_md_t// md, //union ccb *// ccb, //busdma_callback_f// cb, //void *// arg, //u_int// flags) | | //int// | **busdma_md_load_mbuf** | (//busdma_md_t// md, //struct mbuf *// mbuf, //busdma_callback_f// cb, //void *// arg, //u_int// flags) | | //int// | **busdma_md_load_mem** | (//busdma_md_t// md, //struct memdesc *// mem, //busdma_callback_f// cb, //void *// arg, //u_int// flags) | | //int// | **busdma_md_load_phys** | (//busdma_md_t// md, //vm_paddr_t// pa, //size_t// size, //busdma_callback_f// cb, //void *// arg, //u_int// flags) | | //int// | **busdma_md_load_uio** | (//busdma_md_t// md, //struct uio *// uio, //busdma_callback_f// cb, //void *// arg, //u_int// flags) | | //int// | **busdma_md_unload** | (//busdma_md_t// md) | | //u_int// | **busdma_md_get_flags** | (//busdma_md_t// md) | | //u_int// | **busdma_md_get_nsegs** | (//busdma_md_t// md) | | //busdma_tag_t// | **busdma_md_get_tag** | (//busdma_md_t// md) | | //busdma_seg_t// | **busdma_md_first_seg** | (//busdma_md_t// md, //u_int// space) | | //busdma_seg_t// | **busdma_md_next_seg** | (//busdma_md_t// md, //busdma_seg_t// seg) | Memory allocation functions: To allocate and free memory that satisfy the constraints set forth by DMA tags. | //int// | **busdma_mem_alloc** | (//busdma_tag_t// tag, //u_int// flags, //busdma_md_t *// md_p) | | //int// | **busdma_mem_free** | (//busdma_md_t// md) | DMA functions: | //int// | **busdma_start** | (//busdma_md_t// md, //busdma_access_t// acc, //busdma_callback_t// f) | | //int// | **busdma_stop** | (//busdma_md_t// md) | | //int// | **busdma_sync** | (//busdma_md_t// md, //u_int// op) | | //int// | **busdma_sync_range** | (//busdma_md_t// md, //u_int// op, //bus_addr_t// addr, //bus_size_t// size) | ==== busdma_tag_create ==== The busdma_tag_create() function creates a so-called root tag: a tag for which the constraints are not directly derived from another tag, but are derived from the bus hierarchy starting at the root and ending up to device //dev//. The //align// argument specifies the alignment requirements of any DMA segment. The //bndry// argument specifies a boundary for DMA and indicates that DMA is not to cross that boundary. It is commonly the result of limitations in the DMA counter or address computation engine as implemented by H/W. A boundary of 0 indicates that there are no restrictions. The //maxaddr// argument describes the highest address that can be handled by the device. This is commonly the result of the number of address bits that the device has. The lowest address is always 0. The //maxsz// argument specifies the maximum size of any DMA operation, irrespective of of the number of segments and the size constraints of each segment. The //nsegs// and //maxsegsz// arguments specify the amount of data that can be transferred from a single segment and the maximum number of (scatter/gather) segments that can be used for the DMA. If a device does not support scatter/gather, then DMA can only be done using a single segment. The //maxsegsz// argument is ignored if //nsegs// is 1. The //datarate// argument provides a hint to the busdma/mi infrastructure about the (theoretical) data rate associated with this device. This hint is used by the infrastructure to make decisions about how best to load a memory descriptor given resource availability and cost of use, especially with respect to having other devices in the system. The //datarate// argument in in megabits per second. A value of 0 means that no hint is provided. A value of 1 is used for devices with a throughput less than or equal to 1Mbps. The //flags// argument contains operational flags to control the behavior of the operation. The flags are still to be defined. The //tag_p// argument holds the address at which the created tag is to be returned to the caller. ==== busdma_tag_derive ==== The busdma_tag_derive() function is effectively the same as dmabus_tag_create(), except that it takes a busdma_tag_t as the base for deriving a new tag. As such, busdma_tag_derive() can only be used by drivers after they first created a so-called root tag using busdma_tag_create(). The busdma_tag_derive() function allows drivers to express different DMA constraints for the various DMA operations that a device can do. It's therefore more useful for bus masters than for third-party DMA. A device can have multiple DMA root tags and each root tag can have multiple derivatives. Derived tags can themselves have zero or more derivatives, and so on. Drivers can create tag derivation hierarchies as simple or as complex as is needed to properly control DMA. See busdma_tag_create() for the description of the common arguments. ==== busdma_tag_destroy ==== DMA tags created by busdma_tag_create() or by busdma_tag_derive() must be destroyed by the driver. This function destroys the specified tag. ==== device_get_busdma_tag ==== Returns the first root tag in the chain of tags that are associated with the specified device. ==== device_set_busdma_tag ==== Sets or replaces the first root tag in the chain of tags associated with the device. The function returns the tag that was previously the first root tag in the chain, or NULL if there were no tags. ==== busdma_md_create ==== Create a memory descriptor that is to be used to hold information about DMA memory. When created this way, no information is held and the number of segments is 0. The memory descriptor is unused this way. The //tag// argument is used the DMA constraints, such as the maximum number of segments. The //flags// argument contains operational flags to control the behavior of the operation. The flags are still to be defined. See also busdma_md_load... and busdma_mem_alloc. ==== busdma_md_destroy ==== Destroy a memory descriptor. The descriptor must not be used (i.e. does not hold information). ==== busdma_md_load... ==== Construct (load) DMA information based on the provided data structure. The data structure is either a (virtual) memory buffer, a mbuf or an uio. The load operation will create as many DMA segments as needed and stores the virtual, physical and bus addresses for each segment. When a segment does not match the constraints of the device, bus address virtualization is used to make DMA possible. Bus address virtualization is done by using I/O MMUs or bounce buffering. ==== busdma_md_unload ==== Unload a loaded memory descriptor. The memory descriptor becomes unused again. ==== busdma_md_get_nsegs ==== Returns the number of memory segments needed to perform the DMA operation. ==== busdma_md_get_busaddr ==== Returns the bus address of the respective segment. This is the address that the device needs to use when accessing this memory segment. ==== busdma_md_get_paddr ==== Returns the physical memory address of the respective segment. This is the physical address as used by the CPU to refer to the memory segment. ==== busdma_md_get_vaddr ==== Returns the virtual address of the physical memory segment. This may not be defined if the memory isn't mapped. ==== busdma_md_get_size ==== Returns the size of the respective memory segment. ==== busdma_mem_alloc ==== The busdma_mem_alloc() function allocates physical memory in accordance with the restrictions set forth by the tag. Multiple segments can be allocated if such is allowed by the tag and required to satisfy the request. The memory description that provides the information about the physical memory is returned in the mem_p argument. ==== busdma_mem_free ==== The busdma_mem_free() function reclaims the (physical) memory previously allocated and described by the mem argument. If the memory is still mapped, it will be unmapped first. ==== busdma_start ==== This function initiates DMA. Before the actual DMA transfer is started, I/O MMUs are programmed with the appropriate translations and/or bounce buffers are copied to. CPU caches are made coherent as per the platform as well. Lastly, the bus master or the third-part DMA engine is programmed to perform the DMA. The access argument specifies how the device access the memory. It can contain BUSDMA_ACCESS_READ or BUSDMA_ACCESS_WRITE or both. The meaning of BUSDMA_ACCESS_READ is that the physical memory is read by the device. The meaning of BUSDMA_ACCESS_WRITE is that the device is writing to physical memory. When the access contains BUSDMA_ACCESS_WRITE, then the CPU caches are invalidated. Dirty cache lines are not written to physical memory because the device is going to overwrite it anyway. When the access contains BUSDMA_ACCESS_READ, then the CPU caches are flushed (written back to physical memory) and bounce buffers are being copied to (i.e. are being populated). When both BUSDMA_ACCESS_READ and BUSDMA_ACCESS_WRITE is set then the actions are taken as if only BUSDMA_ACCESS_READ was set. ==== busdma_stop ==== This function stops the DMA. It is typically called when an interrupt was received to indicate that the DMA has completed. The bus master or third-party DMA engine is stopped/reset, I/O MMU translations are removed and/or bounce buffers are copied from. CPU caches are made coherent again if necessary. When the access specified for busdma_start() contains BUSDMA_ACCESS_WRITE, then CPU caches are invalidated if necessary and the contents of the bounce buffers is being copied into the final destination buffers. ==== busdma_sync ==== ===== Implementation notes ===== Tags: * The device structure keeps a list of all the driver's root tags. * All tags point to the driver's device structure. * Derived tags point to their parent tag and all derived tags directly or indirectly point to a driver's root tag. * A driver's root tag does not have a parent. * All tags maintain a list of derived tags for which they are the direct base. Memory: * The busdma_mem_t structure keeps a reference to its corresponding tag. * In it is also a list or array (to be kept abstract) of segments. * A segment has a size, bus address, physical address and virtual address, as well as an index. * For some platforms, the bus address is different from the physical address as used by the CPU. A typical example for when this happens is when an I/O MMU is involved in mapping the physical memory into a visible/valid range for the bus and device. * It may not be possible to create a single contiguous mapping in virtual memory for the physical memory segments. This can happen when multiple segments are allocated that are each smaller than a page size. DMA: * Tags contain the DMA engine's callback for starting and stopping DMA when the tag was created with a valid DMA resource. For bus masters the callback is to be provided separately. * The callback gets a reference to the memory and an argument whether to start or stop. * The busdma_mem_t holds the information about any active DMA. ---- {{tag>InactiveProject}} unifiedbusdma.txt Last modified: 2018/07/21 04:07by MarkLinimon