Mellanox Technologies =============================================================================== Mellanox FlexBoot Release Notes Rev 3.4.521, May 2015 =============================================================================== NOTE: THIS HARDWARE, SOFTWARE OR TEST SUITE PRODUCT (PRODUCT(S)) AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES AS-IS WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS. THE CUSTOMER'S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCTO(S) AND/OR THE SYSTEM USING IT. THEREFORE, MELLANOX TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST QUALITY. ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT ARE DISCLAIMED. IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES FOR ANY DIRECT, INDIRECT, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES OF ANY KIND (INCLUDING, BUT NOT LIMITED TO, PAYMENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY FROM THE USE OF THE PRODUCT(S) AND RELATED DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -------------------------------------------------------------- © Copyright 2015. Mellanox Technologies. All rights reserved. Mellanox®, Mellanox logo, BridgeX®, ConnectX®, Connect-IB®, CoolBox®, CORE-Direct®, GPUDirect®, InfiniBridge®, InfiniHost®, InfiniScale®, Kotura®, Kotura logo, MetroX®, MLNX-OS®, PhyX®, ScalableHPC®, SwitchX®, TestX®, UFM®, Virtual Protocol Interconnect®, Voltaire® and Voltaire logo are registered trademarks of Mellanox Technologies, Ltd. ExtendX™, FabricIT™, FPGADirect™, HPC-X™, Mellanox Care™, Mellanox CloudX™, Mellanox Open Ethernet™, Mellanox PeerDirect™, Mellanox Virtual Modular Switch™, MetroDX™, NVMeDirect™, Switch-IB™, Unbreakable-Link™ are trademarks of Mellanox Technologies, Ltd. All other trademarks are property of their respective owners. =============================================================================== Table of Contents =============================================================================== 1. General 2. Supported Adapters 3. Supported Firmware and Firmware Tools 4. Supported Operating Systems 5. Tested Platforms 6. Package Contents 7. Changes From Previous versions 8. Known Issues =============================================================================== 1. General =============================================================================== These are the release notes for "Mellanox FlexBoot", the software for Boot over Mellanox Technologies adapter devices supporting VPI. FlexBoot enables booting kernels or operating systems (OS) from remote servers in compliance with the PXE specification. This package is based on the open source project iPXE available at http://www.ipxe.org. =============================================================================== 2. Supported Adapters =============================================================================== The package supports the ConnectX®-2, ConnectX®-3, ConnectX®-3 Pro ConnectX®-4 InfiniBand only and Connect-IB® family Network Adapter devices and cards. 2.1 Features/Support to be Deprecated in the Next Release ------------------------------------------------------------------------------- The following support will be deprecated in the next release: - ConnectX®-2 adapter cards =============================================================================== 3. Supported Firmware and Firmware Tools =============================================================================== Firmware: - Connect-IB fw-ConnectIB version 10.0100.6440 - ConnectX-4 fw-ConnectX4 version 12.0100.6440 - ConnectX-3 Pro fw-ConnectX3pro version 2.34.5000 - ConnectX-3 fw-ConnectX3 version 2.34.5000 - ConnectX-2 fw-ConnectX2 version 2.9.1000 Tools: - MFT (Mellanox Firmware Tools) version 4.0.0 =============================================================================== 4. Supported Operating Systems =============================================================================== - Windows server 2012 - Windows server 2012R2 - ESXi5.5 - RHEL6.4 - RHEL6.6 - RHEL7.0 - RHEL7.1 - SLES11.3 - SLES12.0 - Ubuntu 14.04 =============================================================================== 5. Tested Platforms =============================================================================== Server Vendor & Model BIOS Manuf. & Version DELL M630 DELL 1.0.3 DELL R730 DELL 1.0.4 DELL Stryper 3A06 DELL C7220N 2.1.11 DELL C6220 Dell Inc. 2.5.3 DELL C6220 II Dell Inc. 2.4.2 IBM System x3850 X6 M4 (WillyCat) A8E11AUS-1.00 IBM System x3650 M4 VVE142EUS-1.80 IBM Flex system x220 KSE142AUS-1.50 IBM Flex system x240 B2E142AUS-1.50 HP DL380 Gen9 HP P89 HP DL360 G9 HP U13 HP XL230a G9 HP U13 HP BL460 G9 I36 V1.30 HP SL230 Gen8 P75 HP SE2160w Gen9 (Panther) HP U16 HP ProLiant SE2140s Gen8 (MiniPooch) P87 HP ProLiant XL230b Gen9 U12 Wiwynn WCS American Megatrends Inc. C1032.BS.1C03.GN1 Quanta T6MG American Megatrends Inc. C1031.BS.3A09 Quanta Winterfell American Megatrends Inc. F03_3B07 Supermicro X9DRW American Megatrends Inc. 3.0c ZT American Megatrends Inc. 2.59 HP Moonshot Anders M710 HP systems – H03 =============================================================================== 6. Package Contents =============================================================================== The FlexBoot package is provided as a tarball (.tgz extension) containing the files specified in section 2. Supported Adapters. A PXE ROM image file for each of the supported Mellanox network adapter devices. =============================================================================== 7. Changes From Previous versions =============================================================================== Changes From Version 3.4.460 ------------------------------------------------------------------------------ - Added iSCSI CHAP and mutual CHAP configuration - Added the GRH size when allocating receive buffer for IPoIB - Updated VLAN netdevice's settings with all the trunk's iSCSI required settings - Updated the port event handling process - Enabled console output in Debug mode - Disabled the serial output - Disabled the banner in BEV execution - Disabled function 0x04 (in int21) when serial console is disabled - Preserved COM port settings - Fixed HTTP download over IPoIB - Fixed completion with error handling process Changes From Version 3.4.306 ------------------------------------------------------------------------------ - Boot Menu support: Added new FlexBoot GUI. The device can now be configured in the POST stage. - Non volatile memory read/write support - Configurable URI boot retry and delay between retries - Configurable iSCSI settings using DHCP/NVM - Added new interface in order to update the registered devices on the PXE stage - Enabled ConnectX Ethernet adapter cards family to work with interrupts - Enabled PXE to work in promiscuous VLAN mode (configurable through the INI) - Synced version with ipxe.org: Now the latest code in iPXE is used - Added boot priority capability: iSCSI vs PXE and fallback incase one fails - Updated the Proxy DHCP request method for non-existing option 54. ProxyDHCP request is sent to port 67 with broadcast IP address if the server identifier in option 54 is zero. Packets with source port different than BOOTPS_PORT and PXE_PORT are filtered by the PROXY - SHELL CLI is currently supported on ConnectX-3 and ConnectX-3 Pro adapter cards only - Both the GUID and the MAC are printed on the screen when the port link layer is set as InfiniBand - PROXYDHCP and PXEBS settings are saved under netdevice settings - rootpath/filename/nextserver are now fetched from the netdevice settings - The cached DHCP packet are received only if working with the same net device. When pxelinux.0 receives the cached DHCP packet from the UNDI API, it constructs a new (fake) packet for the current net device. If the process is stopped and then restarted and booted from the next boot device which serves as the second port in the HCA, a new (fake) DHCP packet is not constructed. The previous packet which includes all the information of the first port (IP, MAC, Netmask, etc...) is used. If an old (fake) DHCP packet is discovered, its chaddr is compared to the chaddr in the pxe_netdev, if not similar, a new (fake) DHCP packet is created. - PXE shutdown is called if int22 with function 0x000C is called. - The server's IP address in DHCP server replies is now checked before checking the reply type. This will ignore NACK replies from servers which already were ignored by the client. In case of 2 DHCP servers in the same subnet, the client will eventually choose one of them, by sending the DHCP REQUEST with 'DHCP Server Identifier' (option 54) filled with the requested server's IP address. - Changed DHCP discover timeouts to comply with PXE spec Changes From Version 3.4.225 ------------------------------------------------------------------------------ - Added validation script for the released ROMs - Added the option to always keep SAN hook to enable WIN install on iSCSI target - Added compilation flag around the flash readout. - Added URI Boot retry. Default retries = 0. - Added Unmap MPT command in teardown. - Added support for HII ISCSI configuration. - Added 64-bit PCI BAR support (Large bar). - Added the option added for running PXE with promiscuous VLAN. - Re-added COMBOOT image support by default. - Enabled pages-function handling in Connect-IB initialization stage to work according to the PRM. - Applied additional patches from ipxe.org - Updated the window even if ACK does not acknowledge new data. - Modified the error print to debug print. - Modified the printed string when initializing devices. - Modified the error print. Added additional information to make the output more user-friendly. - Changed the size of the domain name array to 0xfd. - Disabled the waiting period for link up on trunk-net-device when VLAN is enabled on port. - Removed unsupported EQ event in Connect-IB - Fixed an issue for TLV with length 0. - Fixed an issue related to sync VLAN IRQ operation with trunk IRQ operation. - Fixed an issue which enabled a netdevice (VLAN) to open/close twice. - Fixed an issue which prevented the iSCSI initiator's name from being received from HII. - Fixed an issue related to dual port adapters; occasionally, booting from the second port resulted in TFTP download failure when the first port was already linked up with DHCP, and has received a TFTP address. - Fixed an issue which caused PXE boot failure when using a filename if iSCSI rootpath is set. - Fixed an issue which prevented the device to PXE boot from the 2nd port if first port was already downloaded. - Fixed compilation issue. - Fixed a broken VLAN issue. - Fixed a retry issue when the value is infinite. Changes From Version 3.4.142 ------------------------------------------------------------------------------ - Added additional information to the error print output. - Added compilation flag around the flash readout. - Added URI Boot retry. Default retries = 0. - Added Unmap MPT command in teardown. - Added 64-bit PCI BAR support. - Added an option for running PXE with promiscuous VLAN. - Added support for HII iSCSI configuration. - Enlarged the mailbox size to 4kb. - Enlarged the number of WQE to 64 (from 4). - Enabled multiple DHCP offers to be received before proceeding to request state. - Changed the size of the domain name array to 0xfd. - Changed error print to debug print. - Changed printed string when initializing devices. - Kept the SAN connection permanently open to enable Windows install on iSCSI target even when the iSCSI target is empty. - Readded COMBOOT image support by default. - Prevented a netdevice (VLAN) from opening/closing twice. - Removed unsupported EQ event in Connect-IB. - Disabled the waiting time for link up on trunk net device when VLAN is enabled on a port. - Fixed sync VLAN IRQ operation with trunk IRQ operaaion. - Fixed iSCSI initiator's name retrieval from HII issue. - Fixed an issue caused in dual port adapters, when the first port was already linked up with DHCP, and had received a TFTP address. Booting from the second port resulted in TFTP download failure. - Fixed retry issue when the value is infinite. - Fixed a TLV with length 0 issue. - Fixed a PXE boot failure issue occurred when using a filename when iSCSI rootpath is set. - Fixed "Impossible to PXE boot from 2nd port if first port already downloaded." issue. - Fixed compilation issue. - Fixed broken VLAN support issues. Changes From Version 3.4.000 ------------------------------------------------------------------------------- - Enabled firmware to handle the link state with the Subnet Manager - Updated the DHCP class code to NONE - Added flash access capability for reading software-to-software configurations - Enabled DHCP validation of MAC address and XID for a unique tuple - Improved randomness algorithm for DHCP XID Changes From Version 3.3.420 ------------------------------------------------------------------------------- - Increased the number of the supported systems - Added the ability to merge the PXE image with the UFI image - Fixed "no more network devices" issues Changes From Version 3.3.400 ------------------------------------------------------------------------------- - Increased DHCP timeout Changes From Version 3.0.000 ------------------------------------------------------------------------------- - Added ConnectX-3 support - Increased the number of the supported systems - All iPXE features are now part of the ROM image - Support multiple functions (Flex10) - Added support for Link Aggregation Control Protocol (LACP) - Added ifenable/ifdisable network interface command - Improved reliability =============================================================================== 8. Known Issues =============================================================================== - Firmware 2.30.3000 and below does not support booting over Ethernet with VPI adapter cards when using QSFP cables (40GE, hybrid cables) - Booting from WDS running on Windows 2008 R2 in InfiniBand mode, is currently not supported due to compatibility issues between FlexBoot 3.4.142 and Mellanox WinOF v4.40 and below - Several BIOS vendors have limited boot-vector space and may not display FlexBoot in their boot menu Workaround: Disable the embedded NIC boot agent in BIOS - In several BIOS, the server might hang during FlexBoot booting due to wrong configuration of the PMM - Large Receive Offload (LRO) and iSCSI may not interoperate due to a bug in current Linux kernel distributions. To avoid this problem when using iSCSI, disable LRO in the IPoIB module. See the Mellanox FlexBoot user's manual for details under the Diskless Machines chapter (InfiniBand Ports). - iSCSI over IB is not tested - iSCSI over DCB is not supported - Flexboot supports only 2K MTU - .mrom images larger than 128kB are currently not supported - Only EBX,ESI,DS,ES registers can be saved in Boot Entry - SLAM,FTP,HTTPS and SRP are currently not supported - FCoE, BCV are not supported - Boot over VLAN with IB port is currently not supported - Boot menu is not shown if the HCA card does not have flash - PXE boot after iSCSI boot with static configuration is currently not supported - HTTP boot over IPoIB may not function due to a Known Issue in iPXE community in HTTP stack - Boot retries is currently not functional when booting from iSCSI - Blink LEDs are currently not functional - Chainloading undionly.kpxe over Connect-IB is currently not functional - Boot over IB is supported only using the default PKey - FlexBoot supports only a single active iSCSI connection. Thus, when iSCSI boot via Port 1 succeeds to connect but fails to boot, it will fail to connect via Port 2.