FC - Emulex - SLIM

The Emulex LP6000 Driver

   http://web.cs.wpi.edu/~claypool/mqp/raid/html/node53.html

   The Emulex Light Pulse 6000 (LP6000) card is a high performance PCI Fibre Channel host adapter. It supports all Fibre Channel topologies (point to point, arbitrated loop, switch fabric), multiple Upper Layer Protocols (ULP), including both network (such as IP) and storage (such as FCP) loaded simultaneously. It has a maximum transfer rate of 1.0625 Gb/s at full duplex over either fiber optics or copper serial wire. The card is driven by the FireFly chipset.

   Programming the card is done by means of the Service Level Interface Memory (SLIM). This is a region of memory on the LP6000 card that is mapped into the host main memory. The SLIM is further divided into 3 main parts:
the mailbox,
the I/O Control Block (IOCB) rings, and
the registers.

The registers are used for checking time critical status portions of the card, such as the interrupt status, and some commands to the card, such as resetting the card. The mailbox is used for sending and receiving more complex commands, such as partitioning the SLIM and reading the NVRAM parameters. The IOCB rings are used for sending and receiving ULP packets.

Figure 5.1: SLIM Partitioning Scheme

Registers

The FireFly chipset has four 32 bit registers in the SLIM. The registers are used for configuring, controlling, and checking the status of the card.
The Host Attention Register (HAR) is used by the FireFly chipset to notify the host of a condition that requires attention. The sources can be any one of the IOCB rings, the mailbox, link attention, or an error.
The Chip Attention Register (CAR) is used by the host to notify the FireFly chipset of a condition that requires it's attention, typically of a command that requires processing. The source can be any one of the IOCB rings, or the mailbox.
The Control Register is used to initialize the chipset, and indicate which events should generate interrupts. The potential interrupts sources can be the mailbox, any of the IOCB rings, link attention, or errors. Each of these sources can be configured individually to generate interrupts or not. If interrupts are not enabled for any sources, then the host is expected to poll the status of that source periodically.
The Status Register is used by the FireFly chipset to indicate the status of the chipset. The conditions reported include the readiness of the chipset, readiness of the mailbox, and the various error conditions.

The Mailbox

The mailbox is used for sending commands to the FireFly chipset. The mailbox consists of eight 32 bit words mapped into host memory as part of the SLIM. It is primarily used during the initialization sequence, including partitioning the IOCB rings within the SLIM, and performing the fabric login.
The mailbox contains several fields. The first 32 bit word contains the return code, the command code, and the host contention (HC) and OWN bits. The HC bit is used as a mutex to exclude other processes on the host machine from accessing the host adapter when it is in use. The OWN bit is set to signal to the FireFly chipset that it may read the command and execute it. The chipset will clear the OWN bit when the command is complete. The remaining part of the mailbox contains any parameters necessary for the command being executed, as well as any result values.

I/O Control Block Rings

The IOCB rings are used for the actual transfer of data between the host and the card. The SLIM can contain up to 4 IOCB rings. Each ring is configured to receive a specific type of Fibre Channel message. Since our driver is oriented towards FCP only, we used 3 sets of rings: one for Fibre Channel command packets, one for Fibre Channel data packets, and one for CSH commands and any other types. Each set contains two individual rings, one for sending IOCB's, and one for receiving response IOCB's.

Figure 5.2: IOCB Ring Set

Each ring is logically presented as a circular buffer. To send a command, the driver must wait for the current entry in the buffer is clear. This can be done by enabling interrupts, which means that the host will receive an interrupt whenever a ring entry is freed, or by polling the OWN bit on the IOCB entry. The IOCB is then written into the ring, and the OWN bit is set, indicating that the transfer is complete and the chipset may start reading it.

A similar procedure is used for receiving IOCB response entries. If IOCB response interrupts have been enabled, then the host will receive an interrupt when a new response is available, otherwise the host must poll the OWN bit on the current entry. Once it is determined that the response entry can be read, the entire entry should be read from the SLIM into host memory, and then the OWN bit should be set, indicating to the chipset that it may re-use the entry.

Each IOCB entry consists of 8 32 bit words. Words 6 and 7 contain the various status bits, including a unique tag used to associate the IOCB command and it's corresponding response. The host is responsible for ensuring that each tag is unique. Words 0 through 6 are command dependent. Most typically, for data transfers, it will contain one or more Buffer Descriptor Entries (BDE). The BDE's are used to point to the specific region of memory that the card is to transfer data to or from.