Building a large scale data center is not an easy task and one that includes considerable cost. The larger the cluster is, the larger the core switching element needs to be to carry traffic between servers and storage elements of the data center.
Multiple redundancy and distribution mechanisms are needed to avoid network outages, make implementations resilient and reduce the business impact of failed network elements.
The Virtual Modular Switch (VMS) solution provides a distributed core element to the data center. The VMS is logically placed where you would traditionally place a chassis. Its benefit is targeted for increased resiliency by offering built-in redundancy and distribution of the networking load between multiple elements.
VMS, in practice, is a non-blocking fat tree topology of 1RU sized switches, same topology as used in designing of chassis based switches. The principle difference is that the VMS also exposes the links connecting between its leaf and spine switches. A chassis keeps these links or midplane hidden from the user and reveals a big black box that leaves predictability as something to be desired.
In order to reduce infrastructure costs, these internal links are best implemented using passive copper cables that also provide lowest power consumption, higher MTBF and a certain level of flexibility allowed by the length of cables supported.
Built using two (2) tiers and a basic building block of 36x40GbE ports, the VMS can scale to 648 ports and using the 40% bandwidth boost on the internal links, can provide over 700x40GbE ports of a non-blocking switching solution. In order to support a larger topology than this, you would need to add a third (3) tier to the network. This is same as you would when building your network with a chassis. However, this solution provides better granularity as it is using an atomic building block and not a hidden multichip design.
Still, managing 54 switches instead of just two can be expected to be more demanding. The fact is that there is a different aspect you should be looking at: Let’s say you have two aggregation elements to your network and these provide 750x40GbE ports. These ports are connected to the top of rack (ToR) layer of the network (typically with 4 links per ToR switch) so you have almost 200 ToR switches to manage. Now, managing 250 switches or 200 switches are in fact tasks of similar magnitude.
Last point before I close, VMS is a standard topology based on standard implementation of commonly used network protocols. The benefit of Mellanox VMS lies in the superior performance attributes (latency, power and throughput) of the building block used which is a Mellanox designed ASIC of 36 40Gb/s or 56Gb/s ports.
Author: Since 2011, Ran Almog has served as Sr. Product Manager for Ethernet Products. Prior to joining Mellanox, Ran worked at Nokia Siemens Networks as a solution sales and marketing specialist for the packet networks business unit. Ran holds a BSc. In Electrical Engineering and Computer Sciences from the University of Tel Aviv, Israel.