Big Data Overview

Big Data analytics used to be the exclusive domain of big enterprises. Now, leveraging parallel processing technologies, the challenge of quick analysis of massive data that streamed-in from different sources at a very high speed is going mainstream. Parallel processing architecture breaks a big data analytics job into smaller jobs and runs it over tens, hundreds, or thousands of commodity servers, which keeps data center costs low and provides easy scalability. These trends have existed in the HPC industry for a long time already, but the recent emergence of massive network access devices has led to a new services-oriented infrastructure that requires a reliable high-performance networking infrastructure. A lower latency and higher throughput network infrastructure enables data centers to increase their hardware utilization and to scale up to massive capacities in an instant without having to worry about bottlenecks and bad user experiences.

Mellanox server and storage connectivity solutions have been designed to deliver the required networking and system efficiency capabilities related to bandwidth, latency, offloads, and CPU utilization for big data analytics. With networking solutions that deliver 56Gb/s throughput at less than 1 usec server-to-server latency, efficient networking hardware offload capabilities, and innovative acceleration software, the Mellanox high-performance networking solution accelerates Hadoop jobs' execution time by 2X at a lower cost and delivers the best ROI to its customers.

The Mellanox Solution

Mellanox adapters, switches, gateways, and specific acceleration technologies implement the world's fastest and most robust end-to-end networking solutions for a complete, high-performance Big Data analytics infrastructure. These capabilities ensure optimum data center performance with:

  • Up to 56GbE/s fabric speed to support servers (or blade servers) and storage connectivity in any configuration
  • As low as one microsecond application latency, leveraging the most efficient remote direct memory access (RDMA) between servers and storage
  • RDMA-based solutions that speed transaction processing by offloading significant portions of a communication job from the server CPU and operating system to the IO controller

Mellanox's Unstructured Data Accelerator (UDA) accelerates Hadoop networks and improves the scaling of Hadoop clusters executing data analytics intensive applications. A novel data moving protocol, which uses RDMA in combination with an efficient merge-sort algorithm, enables Hadoop clusters based on Mellanox InfiniBand and 10/40GbE with RoCE (RDMA over Converged Ethernet) adapter cards to efficiently move data between servers, accelerating the Hadoop framework.

Mellanox's Unified Fabric Manager (UFM) is a platform to manage scale-out Ethernet and InfiniBand computing environments. It enables data center operators to efficiently provision, monitor, and operate the modern data center fabric with high uptime and failover capabilities.

  • Lossless transmission, ensuring that no packets are dropped and avoiding retransmissions of data
  • Multi-protocol bridging - Mellanox BridgeX® gateways allow for data center fabric consolidation, unifying InfiniBand, Fibre Channel, iSCSI, and Ethernet over a reliable high-speed interconnect.

Mellanox's Key Technology Advantages

  • The world's fastest interconnect, supporting up to 56GbE/s
  • The world's lowest latency, as low as 1 microsecond
  • CPU offloads with the flexibility of RDMA capabilities
  • UDA for efficient Hadoop clusters
  • Lossless packet transmission
  • Multi-protocol bridging for unified data center fabrics
  • High Speed storage fabric manager
  • Workload optimized network fabric manager