Section 10.3. DART

10.3. DART

With the advent of the 64-bit G5-based computers that support more than 4GB of physical memory, Mac OS X had to incorporate support for 64-bit memory addressing.^[4] However, the PCI and PCI-X busses on the G5 still employ 32-bit addressing. This causes physical addresses (64-bit) to be different from I/O addresses (32-bit) on the G5.

^[4] It is possible for a computer system based on a 32-bit processor to support more than 4GB of physical memory.

As we saw in Chapter 8, even with support for 64-bit user address spaces, the kernel's address space, including the I/O Kit's runtime environment, remains 32-bit in Mac OS X.

As we noted in Section 3.3.3.2, besides the standard memory management unit (MMU), G5-based Apple computers use an additional MMU for I/O addresses. This Device Address Resolution Table (DART) creates mappings between linear addresses and physical addresses. It is implemented as an application-specific integrated circuit (ASIC) that physically resides in the North Bridge. It translates memory accesses from HyperTransport/PCI devices. In particular, it provides dynamic DMA mapping support, and all DMA accesses are channeled through it.

The DART translates only memory accesses that fall in the range 0GB through 2GB, that is, 31 bits of memory; thus, HyperTransport/PCI devices cannot access more than 2GB of memory at a time. In other words, the DART supports up to 2GB of I/O data in some stage of transfer at any given time. The translated physical addresses are 36 bits wide. The DART driver manages the 2GB I/O space using power-of-2-sized zones, with the smallest zone size being 16KB, which, consequently, is the lower bound on the size of an allocation. The driver limits the size of a single allocation to be at most half the total size of the space. Therefore, a single mapping can be at most 1GB in size.

The AppleMacRiscPCI kernel extension implements the DART driver. The AppleDART class inherits from the IOMapper class, which in turn inherits from IOService.

Given its allocation algorithm, the DART driver is likely to return contiguous I/O memory for most allocations, even though the underlying physical memory may be fragmented. In general, the driver of the device in question will see the memory as contiguous and is likely to have improved performance for DMA transfers. This is why the DART is enabled even on G5-based systems with less than 2GB of physical memory.

A device driver is not required to directly interface with the DART or to even know of its existence. If a driver uses IOMemoryDescriptor objects for accessing and manipulating memory, the I/O Kit automatically sets up the DART. If a driver performs DMA on a G5-based system, it must use IOMemoryDescriptor and thus will implicitly use the DART. Moreover, before DMA can be initiated for an IOMemoryDescriptor object, its prepare() method must be called to prepare the associated memory for an I/O transfer. On the G5, this preparation converts the system's 64-bit addresses to 32-bit addresses for DMA, including creation of entries in the DART. Additionally, the preparation may page in memory and wire it down for the duration of the transfer. The complete() method of IOMemoryDescriptor must be called to complete the processing of memory after the I/O transfer finishes. As a general rule, a driver must call prepare() on all IOMemoryDescriptor objects before using them,^[5] whether it is for DMA or programmed I/O (PIO). The getPhysicalSegment() method of IOMemoryDescriptor is relevant for DMA, since it breaks a memory descriptor into its physically contiguous segments. Progammed I/O can be performed by calling the readBytes() and writeBytes() methods of IOMemoryDescriptor.

^[5] In some cases, if a memory descriptor describes wired memory, preparation may be automatic.

As we saw in Section 8.16.6, the I/O Kit's IOMallocContiguous() function implicitly prepares the physically contiguous memory it returns. Although it is possible to obtain the physical address of the allocated memory from IOMallocContiguous(), that address is not the actual physical address but the DART'ed (translated) physical address.^[6] The I/O Kit does not expose the real physical address to the programmer. If a real address were nevertheless presented to the I/O Kit by a driver, the operation would fail because the DART would be unable to handle the translation.

^[6] A corollary is that DMA addresses are always less than 2GB on Mac OS X.