Yearly Archives: 2013

Accelerating Red Hat’s new OpenStack cloud platform with Mellanox Interconnect

Red Hat Enterprise Linux OpenStack Platform is a new leading Infrastructure-as-a-Service (IaaS) open-source solution for building and deploying cloud-enabled workloads. This new cloud platform gives customers the agility to scale and quickly meet customer demands without compromising on availability, security, or performance.

Red Hat built an industry leading certification program for their OpenStack platform. By achieving this technology certification, partners can assure customers that their solutions have been validated with Red Hat OpenStack technology.  Anyone who earns this new certification will be able to show that they can accomplish the following tasks:

•             Install and configure Red Hat Enterprise Linux OpenStack Platform.

•             Manage users, projects, flavors, and rules.

•             Configure and manage images.

•             Add compute nodes.

•             Manage storage using Swift and Cinder.

 

Mellanox is listed in the Red Hat marketplace as a certified Hardware partner for Networking (Neutron) and Block Storage (Cinder) services. This ensures that Mellanox ConnectX-3 hardware was tested, certified, and now supported with Red Hat OpenStack technology.

Mellanox Technologies offers seamless integration between its products and Red Hat OpenStack services and provides unique functionality that includes application and storage acceleration, network provisioning, automation, hardware-based security, and isolation. Furthermore, using Mellanox interconnect products allows cloud providers to save significant capital and operational expenses through network and I/O consolidation and by increasing the number of virtual machines (VMs) per server.

With Mellanox ConnectX-3 card and OpenStack plugins, customers will benefit from superior performance and native integration with Neutron:

 

Mellanox OpenStack solution extends the Cinder project by adding iSCSI running over RDMA (iSER). Leveraging RDMA, Mellanox OpenStack delivers 5x better data throughput (for example, increasing from 1GB/s to 5GB/s) and requires up to 80% less CPU utilization.

Eli Blog 092013 Img1

 

Mellanox ConnectX-3 adapters equipped with onboard embedded switch (eSwitch) are capable of performing layer-2 switching for the different VMs running on the server. Using the eSwitch will gain higher performance levels in addition to security and QoS. The eSwitch configuration is transparent to the Red Hat Enterprise Linux OpenStack Platform administrator by using the Mellanox neutron plugin. By implementing a technology called SR-IOV (Single Root IO Virtualization) and running RDMA over eSwitch, we were able to show a dramatic difference (x20) compared to when using para-virtualized vNIC running a TCP stream connectivity.

Eli Blog 092013 Img2

Learn more:

Mellanox and Red Hat OpenStack joint solution - click here

View the Mellanox certificationclick here

eli karpilovski
Author: Eli Karpilovski manages the Cloud Market Development at Mellanox Technologies. In addition, Mr. Karpilovski serves as the Cloud Advisory Council Chairman. Mr. Karpilovski served as product manager for the HCA Software division at Mellanox Technologies. Mr. Karpilovski holds a Bachelor of Science in Engineering from the Holon Institute of Technology and a Master of Business Administration from The Open University of Israel.

Virtual Modular Switch (VMS): A Network Evolution Story – Part 1

Traditionally, while Ethernet networks were serving low end and non-performance driven applications, the network topology was based on an access layer with a very high port count and a very low rate of traffic generation. This drove a very high and acceptable blocking ratio and a situation where a single (or two in case of need for high availability) uplink would serve for all purposes and connect to an all mighty aggregation chassis that catered for the whole network.

While applications were continuously evolving into becoming more bandwidth hungry, latency sensitive and capacity driven, the need for a wider pipe between the access and aggregation elements in the network became the enabler for the entire evolution of the network. This in turn, drove users towards usage of more interfaces on the aggregation chassis and the network into a gridlock of price to performance ratio.

The need for a high port count of high capacity interfaces on the aggregation switch translates to a very large and complicated chassis. Now although these are available, they are traditionally a step behind the physical evolution or Ethernet technologies;  late to arrive with the proper amount of higher speeds interfaces and limiting in terms of their capability to carry the extra volume in terms of power, cooling, control tables and switching matrix. This situation can be resolved by eventually replacing the existing chassis with a newer model with the promise to be more future tolerant than its predecessor and of course accepting the additional cost spent on a huge device (or two in case of need for high availability).

VMS Part 1

An alternative to hanging your entire network from a single element is to use a fabric of smaller, simpler and more cost effective elements, in order to create a network entity with the required port count, capacity and other performance attributes. This essentially means replacing your modular switch with a Virtual Modular Switch– or how we like to call it–the VMS.

A VMS is a fat tree topology of Ethernet switches with OSPF routing used for topology discovery and ECMP used for load balancing traffic between leaf (access) elements of the VMS via spine (core) elements of it.

Stay tuned to further exploration of the pros and cons in deploying a VMS vs. deploying a modular chassis.

 ran-almog Author: Since 2011, Ran has served as Sr. Product Manager for Ethernet Products. Prior to joining Mellanox, Ran worked at Nokia Siemens Networks as a solution sales and marketing specialist for the packet networks business unit. Ran holds a BSc. In Electrical Engineering and Computer Sciences from the University of Tel Aviv, Israel.

Deploying HPC Clusters with Mellanox InfiniBand Interconnect Solutions

High-performance simulations require the most efficient compute platforms. The execution time of a given simulation depends upon many factors, such as the number of CPU/GPU cores and their utilization factor and the interconnect performance, efficiency, and scalability. Efficient high-performance computing systems require high-bandwidth, low-latency connections between thousands of multi-processor nodes, as well as high-speed storage systems.

Mellanox has released “Deploying HPC Clusters with Mellanox InfiniBand Interconnect Solutions”.  This guide describes how to design, build, and test a high performance compute (HPC) cluster using Mellanox® InfiniBand interconnect covering the installation and setup of the infrastructure including:

  • HPC cluster design
  • Installation and configuration of the Mellanox Interconnect components
  • Cluster configuration and performance testing

 

 Scot Schlultz Author: Scot Schultz is a HPC technology specialist with broad knowledge in operating systems, high speed interconnects and processor technologies. Joining the Mellanox team in March 2013 as Director of HPC and Technical Computing, Schultz is 25-year veteran of the computing industry. Prior to joining Mellanox, he spent the past 17 years at AMD in various engineering and leadership roles, most recently in strategic HPC technology ecosystem enablement. Scot was also instrumental with the growth and development of the Open Fabrics Alliance as co-chair of the board of directors. Scot currently maintains his role as Director of Educational Outreach, founding member of the HPC Advisory Council and of various other industry organizations.

ConnectX-3 Pro Hardware Offload Engines

ConnectX-3 Pro,  a new addition to the ConnectX-3 family, is showing significant CPU overhead reduction and performance improvement while running NVGRE, dramatically improving ROI for cloud providers by reducing the application running cost.

We conducted initial tests to measure the performance improvements and the CPU overhead reduction while utilizing the ConnectX-3 Pro NVGRE hardware offload engines.

Blog 091613 Pic 1

Results show 2x performance improvement and 40% CPU overhead reduction!

Blog 091613 Pic 2

ConnectX-3 Pro supports VXLAN hardware offload engines on top of the NVGRE one and is the first adapter in the market that supports hardware offload engines for overlay networks, i.e., NVGRE and VXLAN.

 Gadi Singer Author: Gadi Singer – Product Manager, Adapter Drivers. Gadi manages the Adapters Product Line at Mellanox Technologies. He served as Marketing Product Manager for the HCA Software division at Mellanox Technologies from 2012 to 2013. Prior to joining Mellanox, Gadi worked at Anobit and PMC-Sierra as a Product Line Manager. Mr. Singer holds a BSc degree in Electrical Engineering from Ben-Gurion University in Israel.

How to Increase Virtual Desktop Infrastructure (VDI) Efficiency

Every IT professional’s goal is to improve TCO. In a Virtual Desktop Infrastructure (VDI) application, the objective is to increase the efficiency by maximizing the number of virtual desktops per server while maintaining response times to users that would be comparable to a physical desktop. In addition, the solution must be resilient since downtime of the VDI application causes the idling of hundreds to thousands of users and consequently reduces overall organizational productivity and increases user frustration.

Low latency data requests from storage or other servers are the key to enabling more VDI sessions without increasing user response times. Legacy Fibre Channel-connected storage subsystems provide shared storage which enables moving virtual machines between physical servers. Leveraging an existing Ethernet infrastructure saves costs by combining networking and storage I/O over the same cable. iSCSI Extensions for RDMA (iSER) is a computer network protocol that extends the Internet Small Computer System Interface (iSCSI) protocol to use Remote Direct Memory Access (RDMA). It basically uses the upper layers of iSCSI for session management, discovery, recovery, etc., and thus compatible with all the features and functions supported by iSCSI. However, using iSER eliminates the bottleneck through the following mechanisms:

  • Uses zero copy via RDMA technology
  • CRC is calculated by hardware
  • Works with message boundaries instead of streams
  • The transport protocol is implemented in hardware (minimal CPU cycles per IO)

Motti - diagram2 for blog 091313

Recently, at VMworld’13, LSI Corporation and Mellanox Technologies presented a joint solution that accelerates the access storage. The solution includes LSI’s Nytro MegaRAID NMR 8110-4i card which has 200GB of on-card flash and eight SAS HDDs and Mellanox’s ConnectX®-3 Pro adapter supports 10Gb/s RoCE storage connectivity between the servers. VDI performance (over TCP/IP and RoCE) was measured using Login VSI’s VDI load generator which creates the actual workload of a typical Windows user using Microsoft Office.

Running Login VSI showed that when running over 10GE TCP/IP only 65 virtual desktop responded within 5 seconds or less, versus 140 when running over 10GE RoCE. This translates into more than 2X cost saving of the VDI hardware infrastructure and proven to be an excellent economical alternative to legacy Fibre Channel based storage subsystems.

 mottibeck Author:  Motti Beck is the Director of Marketing, Enterprise Data Center market segment at Mellanox Technologies, Inc. Before joining Mellanox, Motti was a founder of several setup companies including BindKey Technologies that was acquired by DuPont Photomask (today Toppan Printing Company LTD) and Butterfly Communications that was acquired by Texas Instruments. Prior to that, he was a Business Unit Director at National Semiconductors. Motti holds a B.Sc in computer engineering from the Technion – Israel Institute of Technology.

Advancing Applications Performance With InfiniBand

High-performance scientific applications typically require the lowest possible latency in order to have the parallel processes be in sync as much as possible.  In the past, this requirement drove the adoption of SMP machines, where the floating point elements (CPU, GPUs) were placed as much as possible on the same board. With the increased demands for higher compute capability, and lowering the cost of adoption for making large scale HPC more available, we have witnessed the increase of clustering as the preferred architecture for high-performance computing.

 

 

We introduce and explore some of the latest advancements in the areas of high speed networking and suggest new usage models that leverage the latest technologies that meet the desired requirements of today’s demanding applications.   The recently launched Mellanox Connect-IB™ InfiniBand adapter introduced a novel high-performance and scalable architecture for high-performance clusters.  The architecture was designed from the ground up to provide high performance and scalability for the largest supercomputers in the world, today and in the future.

The device includes a new network transport mechanism called Dynamically Connected Transport™ Service (DCT), which was invented to provide a Reliable Connection Transport mechanism — the service that provides many of InfiniBand’s advanced capabilities such as RDMA, large message sends, and low latency kernel bypass — at an unlimited cluster size.  We will also discuss optimizations for MPI collectives communications, that are frequently used for processes synchronization and show how their performance is critical for scalable, high-performance applications.

 

Presented by:  Pak Lui, Application Performance Manager, Mellanox – August 12, 2013 – International Computing for the Atmospheric Sciences Symposium, Annecy, France

 

 

Mellanox Delivers High Speed Interconnect Solutions for New IBM NeXtScale System

IBM recently introduced their new NeXtScale System, a flexible computing platform that provides 3X as many cores as current one-unit rack servers, making it ideal for the fastest growing workloads such as social media, analytics, technical computing and cloud delivery.

NeXtScale n1200 Enclosure
IBM NeXtScale System Chassis front fully loaded

IBM and Mellanox have worked closely to develop a platform that addresses multiple large-scale markets and solves a variety of complex research and business issues.

Through the use of ConnectX-3 FDR 56Gb/s InfiniBand and 10/40GbE adapters and SwitchX-2 FDR 56Gb/s InfiniBand and 10/40GbE switches, we can provide IBM NeXtScale customers with unrivaled interconnect performance to address the needs for:

  • Large data centers requiring efficiency, density, scale, and scalability;
  • Public, private and hybrid cloud infrastructures;
  • Data analytics applications like customer relationship management, operational optimization, risk/financial management, and new business models;
  • Internet media applications such as online gaming and video streaming;
  • High-resolution imaging for applications ranging from medicine to oil and gas exploration;
  • “Departmental” uses where a small solution can increase the speed of outcome prediction, engineering analysis, and design and modeling

Mellanox’s technology, combined with the IBM NeXtScale compute density, provides customers with sustainable competitive advantage in building scale out compute infrastructures. Customers deploying the joint Mellanox-IBM solution will receive maximum bandwidth, lower power consumption and superior application performance.

cecilia-blog-IBM-v2

Resources:

 

 

Driving Innovation with OpenEthernet

Authored by: Amir Sheffer, Sr. Product Manager

For years, data center Ethernet switching equipment has been based on closed, proprietary vendor implementation, providing very limited flexibility for the user. The progress made in open source applications and software can be leveraged in Ethernet switches to create a new generation of open, flexible and customizable solutions.  

Open Source Enables New Solutions / Trends / Technologies

Open Source Enables New Solutions / Trends / Technologies

Switches based on the OpenEthernet approach will replace traditional closed-code switches and will allow data center customization for optimized and efficient operation. The OpenEthernet switch is based on functionality developed by the equipment vendor and integration with public, open cores and tools that can be freely downloaded from the internet.

As a leader of this approach, Mellanox is investing in the integration and development of such tools, which when combined, can provide complete functionality. Examples for such tools can be OpenFlow–for flow configuration; Puppet and Chef–for switch configuration, Quagga for routing protocols, etc.

Open Ethernet

Mellanox switch software runs over Linux.  Even if the Linux kernel provides good infrastructure for the switch, it lacks functionality to connect it to the switching and routing functions. For example, a routing reflector unit is required to synchronize between the Linux kernel, the routing stack and the silicon data path. For this purpose, we are developing and opening such “reflector” units to the open community.

Another example can be the hardware driver or the software development kit (SDK) application interface (API) for the switch. By opening the API to the community, we will be the first ones to enable full flexibility and ease-of implementation to our customers and we believe other will follow.

In parallel, Mellanox is participating in industry-wide groups that are taking a similar approach.  One example can be the OpenStack community, in which Mellanox is an active member. Another example for such group can be the Open Compute Project (OCP), which is defining open and standard equipment for data centers. Mellanox already builds OCP-compatible NICs and has recently contributed the hardware design documents of the SX1024 switch system to OCP.

So far, we have briefly touched several aspects of OpenEthernet. An important feature that will be explained in the coming weeks is the hardware – software separation.

 To be continued…..

The Storage Fabric of the Future Virtualized Data Center

Guest post by Nelson Nahum, Zadara Storage

It is evident that the future data center will be based on cutting-edge software and virtualization technologies to make the most effective use of hardware, compute power, and storage needs to perform essential analytics and to increase the performance of media-related and advanced web applications. And it turns out that the wires that will connect all this technology together are no less crucial to next-generation data centers and clouds than the software and virtualization layers that run within them.


There are multiple storage fabrics and interconnects available today, including Fibre Channel, Ethernet and SAS. Each has various pros and cons, and fabrics were chosen according to need of performance, compatibility and cost efficiencies.

 

As an enterprise storage as-a-service provider, delivering a software-based cloud storage solution for public, private and hybrid cloud models based on commodity hardware, Zadara Storage provides storage as-a-service in multiple public cloud and colocation facilities around the globe. Consistency, high availability and predictability are key in supplying the scalable, elastic service our customers expect, regardless of their location, facility or the public cloud they employ. The hardware we use needs to be dependable, pervasive and cost-efficient in order to sustain the performance and cost-level of our service, anywhere and at any scale.

 

When choosing our fabric, Ethernet came as a clear choice. Ethernet is likely to become the new standard, and boasts several advantages vital to our product:

  • Ethernet’s speed roadmap is aggressive: from 10GbE to 40GbE, and upcoming 100GbE
  • Ethernet is ubiquitous: we can employ it with no complication at any data center or colocation facility around the globe
  • The latency we have found to be more than manageable, specifically as we use advanced techniques such as IO virtualization and data passthrough
  • Ethernet is the most cost effective: an as-a-service company needs to have a competitive pricing edge.

The future of enterprise storage
The future of Enterprise Storage lies in software and a choice of hardware (premium or commodity). Software-defined storage can scale performance more easily and cost effectively than monolithic hardware, and by combining the best of hardware of software, the customer wins. Ethernet is a critical element of our infrastructure, and Mellanox switches offer significant higher performance and consistent dependability that enables our storage fabric and meets our customer’s needs.

 

Zadara Storage at the Mellanox Booth at VM World 2013
Wednesday, August 28, at 2:15pm
At the Mellanox Booth at VM World 2013, Zadara Storage CEO, Nelson Nahum, will present the Zadara™ Storage Cloud, based on the patent-pending CloudFabric™ architecture, and providing a breakthrough cost structure for data centers. Zadara’s software-defined solution employs standard, off-the-shelf x86 servers, and utilizes Ethernet as its only interconnect to provide performant, reliable, SSD- and spindle-based SAN and NAS as a service.

 

About Zadara Storage
An Amazon Web Services and Dimension Data Technology Partner and winner of the VentureBeat, Tie50, Under the Radar, and Plug and Play cloud competitions, Zadara Storage offers enterprise-class storage for the cloud in the form of Storage as a Service (STaaS). With Zadara Storage, cloud storage leapfrogs ahead to provide cloud servers with high-performance, fully configurable, highly available, fully private, tiered SAN and NAS as a service. By combining the best of enterprise storage with the best of cloud and cloud block storage, Zadara Storage accelerates the cloud by enabling enterprises to migrate existing mission-critical applications to the Cloud.

ConnectX-3 Leverages Network Services in SDN Era

Guest blog by: Alon Harel

 

If your job is related to networking, be it a network admin, an R&D engineer, an architect, or any other job involving networks, it is very likely you have heard people around you (or GASP! maybe even heard yourself) express doubts about the proliferation of Software Defined Networking (SDN) and OpenFlow. How many times have you encountered skepticism about this new revolutionary concept of decoupling control and data planes and “re-inventing the wheel”? Many people used to think “this is hype; it will go away like other new technologies did, and it will never replace the traditional network protocols…” Well, if you perceive SDN/OpenFlow only as a replacement for the current network distributed protocol, these doubts may be turn out to be valid. The concept of saying “OpenFlow is here to replace the old strict protocols” is pretty much the message one gets from reading the old white papers regarding OpenFlow. These papers used to describe the primary motivation for moving to OpenFlow as the determination to introduce innovation in the control plane (that is, the ability to test and apply new forwarding schemes in the network).

 

This long preface is the background for the use case we present below. This use case is not about a new forwarding scheme, nor is it about re-implementing protocols; rather, it is a complementary solution for existing traditional networks. It is about adding network services in an agile way, allowing cost-efficient scalability. It is innovative and fresh and, most importantly, it could have not been done prior to the SDN era. Its simplicity and the fact that it relies on some very basic notions of OpenFlow can only spark the imagination about what can be done further using the SDN toolbox.

 

RADWARE’s security appliance, powered by Mellanox’s OpenFlow-enabled ConnectX®-3 adapter, brings a new value proposition to the network appliance market, demonstrating the power of SDN by enabling the addition of network services in a most efficient and scalable way.

 

Security and attack mitigation service is applied for pre-defined protected objects (servers) identified by their IP address. Prior to SDN, the security appliance had to be a ‘bump in the wire’ because all traffic destined for the protected objects must traverse through it. This, of course, dictates network physical topology, limited by the appliance’s port bandwidth and imposing high complexity when scale comes into play.

 

RADWARE’s DefenseFlow software is capable of identifying abnormal network behavior by monitoring the amount of bytes and packets of specific flows destined for the protected objects. The monitoring is performed by installing specific flows in the forwarding hardware only for the sake of counting the amount of data traversing it. Flow configuration and counter information is retrieved via standard OpenFlow primitives. The naïve approach would be to use the OpenFlow switches to accommodate the flows (counters); however, the limited resource capacity of commodity switches (mainly TCAM, which is the prime resource for OpenFlow) rules out this option. (Note that a switch may be the data path for hundreds or thousands of VMs, each with several monitored flows). Thus, the viability of the solution must come from somewhere else. Enter Mellanox’s OpenFlow-enabled ConnectX-3 SR-IOV adapter.

 

ConnectX-3 incorporates an embedded switch (or eSwitch) enabling VM communication to enjoy bare metal performance. The HCA driver includes OpenFlow agent software, based on the Indigo-2 open source project, which enables the eSwitch to be controlled using standard OpenFlow protocol.

 

Installing the flows (counters) on the edge switch (eSwitch) makes a lot of sense. First, each eSwitch is responsible only for a relatively small amount of protected objects (only those servers running on a specific host), therefore the scale obstacle becomes a non-issue. Moreover, more clever or sophisticated monitoring (for example, event generation when a threshold is crossed) can easily be added, offloading the monitoring application (DefenseFlow in this case).

 

You might think, “What’s new about that? We already have Open vSwitch (OVS) on the server which is OpenFlow capable.” Well, when performance is the name of the game, OVS is out and SR-IOV technology is in. While in SR-IOV mode, VM communication is performed by interfacing the hardware, directly bypassing any virtual switch processing software; therefore, in this mode OVS’s OpenFlow capabilities cannot be used (as it is not part of the data path).

 

Let’s take a look at this practically by describing the setup and operation of the joint solution. The setup is based on standard servers equipped with Mellanox’s ConnectX-3 adapter and OpenFlow-enabled switch and with RADWARE’s DefensePro appliance and DefenseFlow software, which interacts with the Floodlight OpenFlow controller.

SDN bog iamge1.png

Figure 1 – Setup

 

Here’s a description of the joint solution operation, as depicted in Figure 2:

  • DefenseFlow installs the relevant flows on each ConnectX-3 adapter.
  • The security appliance does not participate in the normal data path.
  • ConnectX-3 counts traffic matching the installed flows.
  • Flow counters are retrieved from ConnectX-3.
  • Once an attack is identified, only relevant traffic is diverted to the security appliance (where it is cleared of malicious flows and inserted back toward its destination).

 

 

SDN bog iamge2.png

Figure 2 -Joint Solution

 

I would argue that every skeptic seeing this example use case and the added value it brings to existing network environments using these very basic OpenFlow knobs, would have to reconsider his SDN doubts…