Mellanox Technologies ===================== =============================================================================== MLNX_EN for XenServer README Driver Kit for Mellanox Adapter Cards with 10GigE Support Version 1.4.2, November 2009 Document No. 2959 =============================================================================== Contents: ========= 1. Overview 2. Driver Loading 2.1 Loading the Driver 2.2 Unloading the Driver 2.3 Uninstalling the Driver 3. Ethernet Driver Usage and Configuration 4. Firmware Programming 4.1 Installing Firmware Tools 4.2 Downloading the Firmware Image of the Adapter Card 4.3 Updating Adapter Card Firmware 1. Overview =========== This document provides information on the MLNX_EN XenServer driver. The driver is intended for adapter cards that identify on the PCI bus as with one of the following PCI Device IDs (decimal): 25408, 25418, 25448, 26418, 26448, 26428, 25458, 26458 and 26468. The MLNX_EN driver release exposes the following capabilities: - Single/Dual port - Up to 17 Rx queues per port - 9 Tx queues per port - Rx steering mode: Receive Core Affinity (RCA) - Tx arbitration mode: VLAN user-priority (off by default) - MSI-X or INTx - Adaptive interrupt moderation - HW Tx/Rx checksum calculation - Large Send Offload (i.e., TCP Segmentation Offload) - Large Receive Offload - Multi-core NAPI support - VLAN Tx/Rx acceleration (HW VLAN stripping/insertion) - HW VLAN filtering - HW multicast filtering - ifconfig up/down + mtu changes (up to 10K) - Ethtool support - Net device statistics - CX4 connectors (XAUI) or XFP 2. Driver Loading ====================== 2.1 Loading the Driver ---------------------- Step 1: Make sure no previous driver version is currently loaded Run: #> modprove -r mlx4_en Step 2: Load the new driver version Run: #> modprobe mlx4_en The result is a new net-device appearing in 'ifconfig -a' output. See "Ethernet Driver Usage" for details on driver usage and configuration. 2.2 Unloading the Driver ------------------------ To unload the Ethernet driver run: #> modprobe -r mlx4_en 2.3 Uninstalling the Driver -------------------------- To uninstall the mlnx_en driver run: #> /sbin/mlnx_en_uninstall.sh 3. Ethernet Driver Usage and Configuration ========================================== - To assign an IP address to the interface run: #> ifconfig eth where 'x' is the OS assigned interface number. - To check driver and device information run: #> ethtool -i eth Example: #> ethtool -i eth2 driver: mlx4_en (MT_04A0140005) version: 1.4.2 (Sep 2009) firmware-version: 2.7.0 bus-info: 0000:13:00.0 - To query stateless offload status run: #> ethtool -k eth - To set stateless offload status run: #> ethtool -K eth [rx on|off] [tx on|off] [sg on|off] [tso on|off] - To query interrupt coalescing settings run: #> ethtool -c eth - By default, the driver uses adaptive interrupt moderation for the receive path, which adjusts the moderation time to the traffic pattern. To enable/disable adaptive interrupt moderation use the following command: #>ethtool -C eth adaptive-rx on|off - Above an upper limit of packet rate, adaptive moderation will set the moderation time to its highest value. Below a lower limit of packet rate, the moderation time will be set to its lowest value. To set the values for packet rate limits and for moderation time high and low values, use the following command: #> ethtool -C eth [pkt-rate-low N] [pkt-rate-high N] [rx-usecs-low N] [rx-usecs-high N] - To set interrupt coalescing settings when adaptive moderation is disabled, use: #> ethtool -c eth [rx-usecs N] [rx-frames N] Note: usec settings correspond to the time to wait after the *last* packet is sent/received before triggering an interrupt. - To query pause frame settings run: #> ethtool -a eth - To set pause frame settings run: #> ethtool -A eth [rx on|off] [tx on|off] - To query ring size values run: #> ethtool -g eth - To modify rings size run: #> ethtool -G eth [rx ] [tx ] - To obtain additional device statistics, run: #> ethtool -S eth - To perform a self diagnostics test, run: #> ethtool -t eth The driver defaults to the following parameters: - Both ports are activated (i.e., a net device is created for each port) - The number of Rx rings for each port is the number of on-line CPUs - Per-core NAPI is enabled - LRO is enabled with 32 concurrent sessions per Rx ring - IP reassembly is enabled Some of these values can be changed using module parameters, which can be displayed by running: #> modinfo mlx4_en To set non-default values to module parameters, the following line should be added to /etc/modprobe.conf file: "options mlx4_en = = ..." Values of all parameters can be observed in /sys/module/mlx4_en/parameters/. 4. Firmware Programming ======================= The adapter card was shipped with the most current firmware available. This section is intended for future firmware upgrades, and provides instructions for (1) installing Mellanox firmware update tools (MFT), (2) downloading FW, and (3) updating adapter card firmware. 4.1 Installing Firmware Tools ----------------------------- The driver package compiles and installs the Mellanox 'mstflint' utility under /usr/local/bin/. You may also use this tool to burn a card-specific firmware binary image. See the file /tmp/mlnx_en/src/utils/mstflint/README file for details. Alternatively, you can download the current Mellanox Firmware Tools package (MFT) from http://www.mellanox.com > Downloads > Firmare Tools. The tools package to download is "MFT_SW for Linux" (tarball name is mft-X.X.X.tgz). 4.2 Downloading the Firmware Image of the Adapter Card ------------------------------------------------------ To download the correct card firmware image, please visit http://www.mellanox.com > Downloads > Firmware. For help in identifying your adapter card, please visit http://www.mellanox.com/content/pages.php?pg=firmware_HCA_FW_identification. 4.3 Updating Adapter Card Firmware --------------------------------- Using a card specific binary firmware image file, enter the following command: #> mstflint -d mtXXXXX_pci_cr0 -i b where XXXXX stands for the PCI Device ID (decimal). For burning firmware using the MFT package, please check the MFT user's manual under http://www.mellanox.com > Downloads > Firmare Tools. *** IMPORTANT NOTE *** After burning new firmware to an adapter card, reboot the machine so that the new firmware can take effect.