All posts by Brad Smith

About Brad Smith

Brad is the Director of Marketing at Mellanox, based in Silicon Valley for the LinkX cables and transceivers business focusing on hyperscale, Web 2.0, enterprise, storage and telco markets. Recently, Brad was Product Line Manager for Intel’s Silicon Photonics group for CWDM4/CLR4 and QSFP28 product lines and ran the 100G CLR4 Alliance. Director of Marketing & BusDev at OpSIS MPW Silicon Photonics foundry. President/COO of LuxSonar Semiconductors ( Cirrus Logic) and co-founder & Director of Product Marketing of NexGen, a X86-compatible CPU company sold to AMD - now the X86 product line. Brad also has ~15 years in technology market research as Vice president of the Computer Systems group at Dataquest/Gartner; VP/Chief Analyst at RHK and Light Counting networking research firms. Brad started his career at Digital Equipment near Boston with the VAX 11/780 and has served as CEO, president/COO and on the board-of-directors of three start-up companies. Brad has a BSEE degree from the University of Massachusetts; MBA from University of Phoenix and holds 2 optical patents

Introducing the DynamiX QSA™ Family of Port Adapters

Connects Any SFP DAC, AOC or Transceiver to a QSFP Port

The DynamiX QSATM family of port adapters is the answer to many Data Center professional’s prayers because it resolves the systems linking issues between different line rates and different form-factors used in switches and network adapters such as SFP and QSFP. In addition, it is inexpensive and a life saver in many system upgrade applications.

System Problems:

The trend for faster line rates and new MSA form-factors continues to expand. The market is flooded with a jumble of buzz-words and products that are loaded with specific technical booby traps. In addition, mistakes are becoming very expensive. Systems designers are discovering that the variety and flavors of interconnect products is increasing exponentially. This is all causing IT professionals an increasing amount of irritation and no small amount of confusion as they try to align all these elements, especially when attempting to seamlessly upgrade to new equipment while still supporting legacy systems.

Switch and network adapters are currently offered in QSFP+, QSFP28 and SFP+ and SFP28 form-factors and based on 1G, 10G, or 25G-per-line rates. So, IT managers can easily find themselves with a 25G QSFP-based system and 10G-per-line SFP-based network adapter yet have no clue how to link them up. Different line rates and different form-factors present a challenge but FYI, the answer is: DynamiX QSATM port adapters.

LinkX DynamiX QSATM Solutions:

The DynamiX QSATM family of port adapters is designed, patented and manufactured by Mellanox. The adapter fits into a QSFP 4-channel port in a switch or network adapter and accepts a SFP-based device inserted in the adapter end. This enables passing through a single-channel to the SFP-based device inserted into a larger QSFP port. The faster speed devices are backwards compatible with the slower line speed devices such as:

  • QSA+ 10Gb/s supports 1Gb/s and SFP+ devices
  • QSA28 25Gb/s supports 1Gb/s, 10Gb/s and 25Gb/s and both SFP+ and SFP28 devices

An example of linking a 10G SFP+ transceiver to a 100Gb/s QSFP28 switch using SFP+, SFP28 and QSFP28 devices with the DynamiX QSA™ adapter

 

Product Specifics:

  • Patented design manufactured by Mellanox US patent 7-934-959
  • Available in two speed versions, 10G and 25G
  • Contain an EPROM to tell the host what the device configuration is.
  • Operates independently from the SFP device inserted
  • Passes through SFP device configuration EPROM information
  • Consumes no power – passive except for configuration setup
  • Induces no signal latency
  • Can be used in QSFP+ and QSFP28-based switches and network adapters
  • Supports a wide range of SFP-based DAC and AOC cables plus both multi-mode and single-mode optical transceivers:
    • Cables
      • CR: DAC copper SFP (3-5m)
      • AOCs: SFP multi-mode (100m)
    • Transceivers
      • SR: SFP multi-mode transceiver (100m)
      • LR: SFP single-mode transceiver (10km)
      • SX: 1G SFP+ multi-mode transceiver (500m)
      • Base-T: 1G SFP converter using CAT-5 copper UTP cables

DynamiX QSA Supported SFP Devices

 

The new DynamiX QSATM QSA28 model sports a black, low mass, small-profile, easy-pull tab that is becoming more and more popular in very crowded data center racks. The low-mass, loop design enables more free airflow through the crowded systems and does not flap-in-the-wind as with larger tabs.

In high-density system racks, one can visualize over 100 cable ends and transceivers tabs flapping in the wind.  Besides the restriction in air flow, the larger tabs also create a lot of noise and eventually reliability issues. The tiny, looped design eliminates these factors

DynamiX QSA™ QSA28 Model Low-mass, Looped Tab Design Versus Traditional Flat and Long Tab

Summary:

  • The new DynamiX QSATM family of port adapters resolves many system upgrade issues with respect to linking different form-factors and line rates to switch and network adapter ports
  • The solutions from Mellanox enable a smooth transition when upgrading to newer and faster systems while continuing to support slower speed equipment.
  • Inexpensive, easy-to-use, and plug & play.

 

Supporting Resources:

Contact your Mellanox sales representative for availability and pricing options and stay tuned to my blog more interconnect news and tips.

Sign up for free ebook download here: LinkX Cables and Transceivers ebook

 

About Mellanox & LinkX

Mellanox Technologies, Ltd. (NASDAQ: MLNX), a leading supplier of optical transceivers and high-performance, end-to-end smart interconnect solutions for data center servers and storage systems. Mellanox offers a full line of 10G to 200Gb/s cables and transceivers for hyperscale, enterprise, telecom and storage datacenter applications for Ethernet and InfiniBand protocols.

  • LinkX 25G/50G/100G/200Gb/s DAC & AOC cables and transceivers
  • New Quantum switches with 40 ports of 200Gb/s QSFP28 in 1RU chassis
  • New ConnectX®-6 adapters with two ports of 200Gb/s QSFP28
  • Silicon Photonics 100Gb/s Optical Engines and IC components

Download Free LinkX Ebook on DAC, AOCs and Transceivers for High-Speed Data Centers

What’s New?

Mellanox has introduced a new ebook that describes the latest developments in Direct Attach Copper (DAC) cables, Active Optical Cables (AOCs) and both multi-mode and single-mode optical transceivers for use in modern high-speed data centers. With 33 pages of descriptive text and diagrams, this ebook is a must-read for anyone involved in high-speed networking.

The new Mellanox ebook tries to simplify all the various buzz-words and acronyms and boil it all down to simple-to-understand concepts. It is designed as a quick-read, is blessedly equation free and includes a variety of artwork and photos to make complex issues more understandable and visually interesting.

With a focus on the latest 25Gb/s line rates, the ebook examines the latest copper and optical interconnect technologies for linking top-of-rack, leaf, and spine switches to servers, storage and network appliances using network adapters. These system design techniques have been recently adopted by many hyperscale data center builders and the techniques they are using are rippling throughout the data center industry, migrating to small and large enterprise data centers, and even new telecom data centers as they represent the most cost-effective high-speed interconnect solutions available.

The ebook focuses on the SFP28 and QSFP28 form-factors and interconnect reaches from 0.5 meters to 10 km in both copper and optics which covers the majority of data center links.

About Mellanox & LinkX

Mellanox Technologies, Ltd. (NASDAQ: MLNX), a leading supplier of both Ethernet and InfiniBand switches, network adapters, cables and transceivers for 10G/40G and 25G/100G systems. Mellanox is one of the few companies that designs and builds switching systems, adapters, cables and transceivers including the silicon and offers complete “end-to-end” interconnect solutions for modern data center servers and storage systems.

In addition to manufacturing complete network switches, network adapters, cables and transceivers, Mellanox also designs its own switch and network adapter CMOS silicon ICs, BiCMOS transceiver ICs and Silicon Photonics technologies.

Supporting Resources:

Contact your Mellanox sales representative for availability and pricing options and stay tuned to my blog more interconnect news and tips.

Sign up for free ebook download here: LinkX Cables and Transceivers ebook

LinkX is the Mellanox trademark and name for its cables and transceivers product line

Mellanox Showcases 100G CPRI Transceiver, Optical Engines and Silicon Photonics Wafers at ECOC and CIOE

With networks and data centers all going wireless and into the clouds, Mellanox is announcing the volume production of a new 96G/100Gb/s SR4 multi-mode 4-channel transceiver with one of the top worldwide wireless infrastructure suppliers. The solution features an extended temperature range for supporting outside plant optical front haul links in the Fifth Generation of Mobile Networks, and supports dual line rates as well as both protocols of Ethernet and CPRI or Common Public Radio Interface which are used primarily between remote radio heads (RRH) and base band units (BBU).

Solution Highlights:

  • Based on 100G SR4 Ethernet multi-mode transceiver shipping in high volume
  • Employs Mellanox-designed transceiver ICs
  • Boasting dual line rates and protocols of 24.3G CPRI and 25.8G Ethernet
  • Offers an extended temperature range
  • Available now in volume shipments

“Front haul” links are named thus because the traffic is carried from the cellular antennas to base station controllers, which is at the front of the wireless network. Since wireless antennas are generally found in an outside plant environment, the SR4 CPRI transceiver has a wider temperature rating.

With trials beginning in China and around the world in late 2018, and with full deployments estimated to begin in 2019, this new transceiver is suitable for short-reach outside plant optical transmission between remote radio head (RRH) and base band units (BBU). Multi-mode transceivers create the lowest cost optical links and can be used both inside base stations as well as for short reaches up to 100 meters running up to the top of antenna structures. More expensive single-mode transceivers are used for longer reaches.

CPRI/fiber links are rapidly replacing microware systems and are positioned to play well with Centralized Radio Access Networks (C-RAN) and software-defined-everything in the future.

What’s Driving the Move to 5G?

All of these demand factors are coming at Internet systems all at once which is driving data traffic through the roof and is at the root of multi-acre-sized hyperscale data centers which seem to be popping up everywhere:

  • IoT: Internet connections are expanding and linking nearly everything Internet of Things (IoT) and poses to link everything from power and water grids, autos, homes, appliances, tools, pets, watches to smart water bottles!
  • UHDTV: 4K and 8K resolution HDTV is a reality now. PCs, HDTVs, even cell phones support 4K video today. Moscow 2018 and Japan 2020 Olympics with 8K video will drive network demand. 4K offers not only four times the pixel resolution of standard 2K HDTV but also has many advance features as well.
  • 4K cellphone photos – already people are texting 4K cat photos and videos!
  • Video Streaming – TV and Movies; video calls; drones with 4K video cameras
  • Internet connected autos
  • Virtual and augmented reality; real-time gaming
  • Everything available from anywhere – on cellphones, pads and PCs

 

When Will 5G Be Relevant?

The bottom line is that carriers and everyone in the content supply chain knows there will be big money in 5G and new revenue streams from new 5G applications and capabilities. The pressure is on to accelerate deployments and so are the standards battles for alternative approaches poised to delay everything.

The new 5th generation wireless network scheduled to come on line beginning in 2019 will provide a much faster and efficient connection to cellular wireless equipment. Offering more than just a faster line rate, 5G will improve the connection quality and range. Preliminary tests have shown data rates as high as 10 Gb/s to the cell phone which reflects an application game changer!

Mellanox 100G SR4 CPRI Transceiver Features

The newly announced 100Gb/s SR4 transceiver for wireless networks has its genesis in the highly popular, standard temperature version of the 100G SR4 for hyperscale, Web 2.0 Cloud and enterprise data centers. Unique features include:

  • 4-channels of 24.33G or 25.78G supporting CPRI 97G or Ethernet 100G
  • Standard QSFP28 form-factor compliant to the SFF-8655
  • MPO multi-mode optical connector, 8-fibers
  • IEEE CAUI-4 electrical specification
  • Selectable Tx and Rx retiming (CDRs)
  • Supports digital diagnostic monitoring of supply voltage, temperature, transmit/receive power, and laser bias.
  • Boasts ISFU – In-Service-Firmware-Upgrade a capability that enables upgrading the transceiver firmware while running traffic in addition to module being hot pluggable
  • Tested extensively in Mellanox switching systems
  • Bit Error Ratio (BER) better than 1E-15, about 1,000 times fewer bit errors than competitor’s products

The CPRI SR4 transceiver is designed using Mellanox’s own designed transceiver control ICs for VCSEL laser drivers and TIA receiver amplifiers. The close matching of driver and receiver electronics with optics enables the SR4 to boast a 1.5W typical power consumption without retiming and 2.2W with retiming – some of the lowest power ratings in the industry. With power budgets skyrocketing, every Watt saved at the component level translates into several Watts of system cooling and related power consumption driving fans, and AC equipment. Powering remotely located equipment is very expensive.

Besides Mellanox designing and building an entire transceiver, Mellanox also builds switching systems, and the transceivers are tested in real switching systems such as the 32-port 100G SN2700 switch, and not simulated on a test bench as most competitors do. By building our own electronics and transceivers, as well as testing in real systems, this guarantees they will work out of the box and at a bit error ratio (BER) of 1E-15. This is about 1,000 fewer bit errors than the IEEE industry standard of 5E-5 and using forward error correction (FEC) to achieve 1E-12. The additional BER rating enables a wider margin for components in difficult operating conditions of outside plant.

Three Converging Factors Changing CPRI Networks:

  • Line Rates Converge: CPRI line rates have not been aligned with the Ethernet rates due to historical reasons when long reach coaxial copper wires were used. Rates evolved from 3G, 6G, 12G and now to 24.33G and is now very close to the Ethernet 25.78G rate.
  • Adoption of Four-Channels: Most of the CPRI networks use single-channel SFP transceivers but the CPRI committee is now considering using the 4-channel to increase port density to handle the huge influx of traffic, simpler fiber management and one optical connector for four instead of only one channel.
  • QSFP Form-factor: 3 or 97.2G too close to the Ethernet 100G rates to ignore. Hence, the CPRI committee has considered using the 4x25G (100G) QSFP28 form-factor standard Ethernet transceivers now available in huge volumes.

These factors lead to CPRI network builders adopting standard 4x25G (100G) QSFP28 Ethernet transceivers already being produced in huge unit volumes which should enable lower network costs and faster deployments. Dual rate capability makes it a double hit.

Bottom line:

While standards, software architectures, hardware architecture of central office-based or distributed BBU, C-RAN or not, and buzz-words are all in a high state of flux, a few things are clear:

  • A hard fiber link is needed from the antennas to base stations no matter what else happens beyond that in “software land”. Fiber has the bandwidth and the long reach capabilities. The days of microwave and copper coax wire are over.
  • Multi-mode optics is clearly the least expensive solution, making the 100G SR4 dual rate transceivers a perfect match going forward over the short reach.
  • It is also clear that both CPRI and Ethernet will play side by side roles in advanced 5G networks due to the unique capabilities of each protocol as well as the high unit volume cost advantages of Ethernet versus specialized CPRI transceivers.

Mellanox’s dual protocol, dual rate SR4 transceiver using Mellanox designed ICs, manufactured and tested by Mellanox in Mellanox systems and optimized for very low-power consumption and low bit error rates makes it a perfect low-cost choice for CPRI/Ethernet networks.

 

Supporting Resources:

LinkX is the Mellanox trademark and name for its cables and transceivers product line

 

Mellanox Announces 100G CPRI Transceiver and Accelink PSM4 Optical Engine Partnership

On display at Two Big Tradeshows CIOE & ECOC

Mellanox is showcasing its LinkX cables and transceivers, with ConnectX adapters and Ethernet and InfiniBand switches at the September CIOE and ECOC trade shows.  ECOC and CIOE are the two biggest interconnect events of the year besides the Optical Fiber Conference (OFC) in March.

  • CIOE: China International Opto-electronics Expo, Shenzhen, China Sept 6-9
  • ECOC: European Convention on Optical Communication, Gothenburg, Sweden, Sept 18-20

Mellanox announces:

  • 100G SR4 CPRI transceiver: wireless front haul protocol, multi-mode transceiver with an extended temperature rating of -10C-+75C and announcing we are entering volume production. With trials beginning in China and around the world this year, and with full deployments estimated to begin in 2019, this new transceiver is suitable for short-reach outside plant optical transmission between remote radio head (RRH) and base band units (BBU).

The CPRI transceiver is targeted at the next generation 5G wireless infrastructure build out and the potential unit volumes are staggering.  5G will enable ~10Gb/s to your cell phone, virtual reality, IOT, and 4K video – four times the resolution of your HDTV today.

  • Mellanox/Accelink partnership: Using our 100G PSM4 Silicon Photonics optical engine, Accelink will to build 1550nm PSM4 transceivers. Accelink is a leading Chinese opto-electronics components supplier with a one of the most comprehensive end-to-end product lines and one-stop solutions in the industry.

The 1550nm PSM4 relationship will create multiple industry sources for PSM4 transceivers based on Mellanox’s Silicon Photonics optical engine and transceiver ICs.

 

Mellanox CIOE booth #1A22-1, Hall 1 and ECOC booth 531 where we will show:

  • Full line of 100Gb/s transceivers for hyperscale and datacenter applications
  • LinkX 25G/50G/100Gb/s DAC & AOC cables and 100G SR4 & PSM4 transceivers
  • New Quantum switches with 40 ports of 200Gb/s QSFP28 in 1RUchassis
  • New ConnectX®-6 adapters with two ports of 200Gb/s QSFP28
  • Silicon Photonics Optical engines and components

Supporting Resources:

  • Learn more about LinkX cables and transceivers: LINK
  • Learn more about Mellanox complete 100GbE switches and adapters: LINK
  • Follow Mellanox on: TwitterFacebookGoogle+LinkedIn, and YouTube
  • Mellanox 25G/100G SR/SR4 transceivers: BLOG
  • Mellanox 100G PSM4 transceiver blog on: BLOG

 

 

Why So Many Types of High-speed Interconnects?

Rationale behind the myriad of different interconnect technologies and products

Creating high-speed interconnect links between servers and storage, switches and routers involves many different types of technologies in order to minimize the cost involved. With large data centers buying tens of thousands of devices, costs add up quickly. A 3-meter long DAC cable is priced at approximately $100 but a 10km reach single-mode transceiver $4,000-$5,000. AOCs and multi-mode transceivers priced in between.

Today, most modern data centers have zeroed in on the SFP and QSFP form-factors for use with DAC and AOC cabling and optical transceivers. By focusing on only a few types and ordering in high unit volumes, greater leverage of scale can be achieved, not only in the cables and transceivers, but also all the equipment they link to such as switches and network adapters that may reside in servers, storage HDD, SSD, NVMe arrays. Add to this, the spare parts that also need to be stocked.

Currently, the modern data center uses SFP and QSFP (“+” for 10G and “28” for 25G) in DAC, AOCs and both multi-mode and single-mode transceivers. DAC uses copper wires. Parallel 4-channel AOC and transceivers (SR4 & PSM4) use 8 optical fibers; single-channel transceivers and AOCs (SR, LR) use 2 fibers. CWDM4 and LR4 transceivers use 2 fibers and multiplex four channels in one fiber to save fibers costs over long reaches.

 

High-speed interconnects all strive to:

  • Implement the lowest cost links
  • Achieve the highest net data throughput (i.e., fastest data rate with least amount of data errors, data retransmissions and minimal latencies).
  • Transmit over various distances

 

To achieve these goals, various technologies are often used each of which has its own set of benefits and limitations. Data center professionals want to build all links with single-mode fiber, duplex LC connectors and single-mode transceivers. Build the fiber into the data center infrastructure once and forget it using single-mode fiber as it does not have reach limitations that DAC copper and multi-mode fiber do; then upgrade the transceivers with each new transceiver advancement.

While the fibers and LC connectors are already at the lowest cost points, the problem is the single-mode transceivers, which are very complex to build requiring many different material systems, are hard to manufacture and therefore expensive. Basically, the longer the reach needed to send the data, the higher the price as the technology gets more complicated and the harder to manufacture.

Most single-mode transceivers are built using a great deal of manual labor and piece part assembly in processes designed to address the low volume telecom market. The new hyperscale data centers are ordering parts in record numbers and the piece part manufacturing method is difficult to scale up. Silicon Photonics technology attempts to use CMOS silicon IC processes to integrate many of the devices and sub-micron alignments required.

As a result, data centers often use an array of different high-speed interconnects matching each interconnect type to specific reach requirements. DAC is the lowest cost however, after about 3-5 meters, the wire acts like a radio antenna and the signal becomes unrecognizable.  AOCs are used from 3 meters to about 30 meters after which installing long cables becomes difficult. More expensive multi-mode transceivers, with detachable optical connectors, can reach up to 100 meters, then the large 50-um fiber core causes the signal to scatter and become unrecognizable. Some multi-mode transceivers (eSR4) and links can be engineered to 300-400 meters (eSR4), but it gets a little tricky matching the special transceivers, fibers, and optical connector in the link.

Single-mode fiber uses a tiny 9-um light carrying core so the signal pulse stays together over very long distances and can travel literally between continents. Parallel single-mode transceivers (PSM4) with 8-fibers can reach up to 500m-2km. The PSM4.MSA standard is 500m, but Mellanox’s PSM4s can reach up to 2km; about four times the reach of the PSM4 spec.

After 500 meters, the cost of 8 fibers adds up with each meter, so multiplexing the four channels signals into only two single fibers is more economical over long fiber runs. CWDM4 is used for up to 2 km and LR4 up to 10 km.

In the chart below on the bottom axis, as the reach gets longer, different technologies are used. Starting with DAC cables on the left and ending with 10km LR4 transceivers on the far right. Also note on the vertical axis, the faster the data rates, the shorter in reach the DAC and multi-mode optics (SR, SR4) becomes while single-mode fiber remains largely reach independent.

 

Data Rate versus Interconnect Reach

 

Summary

All the different technologies and cable product types are designed to minimize the costs involved in building data center high-speed interconnects. While many enterprise data centers might use 5,000-10,000 devices, hyperscale builders with 2 million servers are ordering interconnects in the hundreds of thousands.

Mellanox sells end-to-end solutions, and designs and manufactures not only the switches and network adapters systems including the silicon ICs, but also designs and manufactures the cables and transceivers – including the silicon VCSEL and Silicon Photonics ICs driver and TIAs ICs.  For single-mode transceivers, Mellanox has its own Silicon Photonics product line and internal wafer fab for its PSM4 and AOC transceivers.

Mellanox sells state-of-the-art 25/50/100G products in copper DAC and optical AOC cables and both multi-mode and single-mode transceivers.

Mellanox recently announced it shipped its 100,000th 100G DAC cable and 200,000 th 100G transceiver/AOC module and is a leading supplier in all four interconnect product areas.

 

More Information

Mellanox LinkX™ Cables Connect the Scientific & Engineering Community

As many know, the circumference of the earth is 40,075 km. Doing the math, we start with the fact that Mellanox has shipped over 2 million copper DAC cables to-date. As the length of wire in 2 million cables is 144,000 km, that is essentially enough to circle the earth at the equator 3.5 times or get one a third of the way to the moon! Math below:

 

(3 wires/lane bundle) x (8 wire bundles in QSFP DAC) x (3 meters long average) x (2 million DACs) = 144,000 km

 

This fun fact got me thinking about how Mellanox connects the scientific and engineering community to their vital research in very real and tangible ways.

Mellanox’s approach to computer networking is to keep the CPU doing the most important tasks and leave the rest of tasks to the network. This approach preserves the most expensive resource, the CPUs, to do the important computing tasks and the network to manage the data traffic in and out of the CPU and storage sub-systems where the CPU is not needed. Mellanox calls this “Data-Centric CPU Offload” versus “CPU-centric OnLoad”. When the CPU is involved with simply moving data around, it is idle literally 40-80 percent of the time waiting for data. With OnLoad architectures, the CPU is involved with simply moving data around – not actually doing computing work. Every data transfer induced time delays or latency. This is like making an appointment to ask your CEO permission every time you have to sharpen a pencil. Lots of “busy” but not much actual “work”. Mellanox’s switches and network adapters, along with LinkX line of cables and transceivers, enables optimizing the movement of data and keeps it on the network there by enabling the CPU to do its best job. This is a fundamental theme at Mellanox and one of the factors of our success and applies to both Ethernet and InfiniBand protocols. The network switches and adapter ICs have billions of transistors to perform logical processing at the network level and not get the CPU involved.

Most engineering and scientific research requires massive amounts of data and iterations but often the formula being computed is pretty simple. So, the main problem in engineering and scientific computing is moving data fast enough to keep the CPU fully fed – which Mellanox’s switches, adapters and interconnects are ideally suited to do with our CPU-Offload architectures. The ConnectX-5 Multi-Host Socket adapters can increase the ROI at the server with 30 -60 percent better CPU utilizations, 50 -80 percent lower data transfer latency and 15 -28 percent faster data throughput. All of these benefits are derived by using intelligent network adapter and switches to keep the data moving on the network and not moving in and out of the CPU needlessly.

InfiniBand Systems ROI: Switches, Adapters and Interconnects

Mellanox Ethernet and InfiniBand-based systems deliver the lowest latency, fasted computing solutions for all kinds of engineering and scientific applications such as: Aerospace, automotive electronics, molecular dynamics, genetic engineering, chemistry, weather analysis, structural engineering, etc.

Mellanox Systems Used to Design Mellanox Systems!

Believe it or not, Mellanox even uses its Ethernet and InfiniBand switches, network adapters, cables and transceivers in CAE/CAD engineering systems to design the ICs and electronics that go inside the switches, network adapters, cables and transceivers that we sell! Think of it like a Martin Escher drawing of a hand with a pencil draws the picture of a hand with a pencil!

The Mellanox LinkX product line of cables and transceivers are all designed by Mellanox engineers – from the internal ICs to the complete assemblies. IC CAE systems are used to design, simulate and layout transceiver control ICs used in both multi-mode and single-mode Silicon Photonics transceivers. Optical engineering software is used to model the high-speed optics ray tracing and reflections inside the Silicon Photonics and fibers. Mechanical and Thermal CAE systems are used to design the mechanical aspects of the transceiver ends and thermal modeling. Electromagnetic design software is used to model the high-speed signals inside DAC copper cables and Silicon Photonics optical transceivers and the EMI/RFI emissions to meet industry standards. Lastly, the entire DAC, AOC, multi-mode and single-mode transceiver assemblies are all designed and modeled by Mellanox engineers.

Only a couple of actual formulas need to be computed but massive amounts of data in the simulations and designs – ideally suited for Mellanox OffLoad Ethernet and InfiniBand switches, adapters, cables and transceivers.

Supporting Resources:

SFP-DD – Next Generation SFP Module to Support 100G

New transceiver MSA form-factor enables doubling the SFP bandwidth and supporting fast line rates while maintaining SFP backwards compatibility.

Recently, a group of industry suppliers gathered to form a new transceiver module form-factor or Multi-Source Agreement (MSA). The agreement aims to foster the development of the next generation of SFP form-factor used in DAC and AOC cabling as well as optical transceivers. Mellanox often leads these sorts of technology developments and is a founding member of the SFP-DD MSA, as well as both QSFP-DD and OSFP MSAs.

While all the specs are not final yet, it’s called the SFP-DD or Small Form-factor Pluggable – Double Density. The “double density” refers to offering two rows of electrical pins enabling two-channels instead of the traditional one-channel in an SFP architecture – the smallest industry standard form-factor available today for data center systems.

New designs offer improved EMI and thermal management and will enable 50G and 100G PAM4 signaling in each channel for 100G and 200G support and up to 3.5 Watts of thermal dissipation; the same as the current QSFP28 which is about 2.5 times larger than the SFP-DD.

Bottom line:

First products on the market will likely be based on 50G PAM4 signaling and will feature two channels, offering 100G in SFP-DD form-factor. These new switches and network adapter configurations will enable increased bandwidth switch plate density, essentially doubling today’s current density.

This advancement will enable 100G (2x50G PAM4) in a tiny SFP port and 50G and 100G links to servers and storage in the smallest MSA available, with the highest number of 100G front-panel pluggable ports in a Top-of-Rack switch.  Eventually, two channels of 100G PAM4 will enable 200G per SFP-DD device.

 

Maintaining Popular Breakout Cabling to Servers

With the advent of new 8-channel form factors such as QSFP-DD, OSFP and COBO, a new 2-channel form factor was needed to enable 4-to-1 breakouts for servers and storage.

These time-tested data center Top-of-Rack breakout or splitter cable configurations can be maintained going forward to 400G with the SFP-DD in both copper DAC and AOC cables enabling supporting 10G, 25G, 50G, 100G and eventually 200G to the server such as:

  • 40G QSFP+ -to-Quad 10G SFP+
  • 100G QSFP28-to-Quad 25G SFP28
  • 100G QSFP28-to-Dual 50G QSFP28
  • 400G QSFP-DD-to-Quad 100G SFP-DD
  • 400G QSFP-DD-to-Dual 200G SFP-DD

Servers today typically support one or two CPUs per server but are heading towards supporting four and eight CPUs per server in the future, with additional DRAM and FLASH on board and PCIe Gen4 at 16GT/s requiring more server uplink bandwidth.  Today, 10G and 25G uplinks are popular and some hyperscale companies also require 50G uplinks. At four and eight CPUs per server, 100G and 200G uplinks will be required.

Mellanox recently introduced two new 100G AOCs breakout cables that feature 100G-to-Quad 25G SFP28 and 100G-to-Dual 50G QSFP28. They are also available in copper DAC cabling. These breakout configurations can also be made using transceivers and splitter passive fiber cables if optical connectors are needed to detach fibers from the transceivers.

Similarly, new QSFP-DD and SFP-DD breakout cables will be available in the future to support new 50G PAM4-based switches and network adapters.

Mellanox 100G DAC and AOC Product Line based on QSFP28 and SFP28

 

The SFP-DD new form-factors tie with Mellanox’s recent 200GbE Spectrum-2 switch IC announcement which is based on 50G PAM4 signaling and points to future 200G and 400G switch, network adapter, cable and transceiver developments from Mellanox.

 

Poised to Support the Next 5-10 years

By doubling the number of lanes, and at the same time doubling the number of bits per clock sent with PAM4 modulation, the SFP-DD bandwidth can transfer 100G versus the SFP28 at 25G. This translates into or four times the bandwidth of SFP28. In the future, the SFP-DD MSA goal is to support 100G PAM4 modulation enabling 200G (2x100G) per SFP-DD package which translates to eight times the current SFP28 bandwidth in the same physical space.

PAM4 Modulation versus NRZ

100G in a SFP-DD form-factors is the so-called Ethernet Alliance “holy grail” in high-speed interconnects. 100G is likely to be the next “10G” which has been the main stay in data centers for the last 10+ years. SFP-DD enables 100G in the smallest form-factors available and is likely to be around for many years to come – starting out in hyperscale and later moving into the enterprise and storage.

 

SFP-DD MSA Goals

The MSA members will develop operating parameters, signal transmission speed goals, and protocols for the SFP-DD interface, which expands on the popular SFP pluggable form factor.  Targets include:

 

  • DAC reach: 3-meter 28 AWG Direct Attach Copper (DAC) aka Twinax,

 

  • SFP Backwards Compatibility: with the SFP28 and SFP+ so that upgrades are easy and support for slower devices in new 50G PAM4 systems.

 

  • Break Out Support: Using the next generation 8-channel 400G QSFP-DD in a switch, the SFP-DD can be used in a quad breakout configuration of four 100G. Similarly, for dual breakouts of 200G to dual 100G or quad 50G.

 

  • Higher power dissipation: With advanced thermal designs, the SFP-DD goal is to support up to 3.5 Watts – equal to the current and much larger QSFP28 MSA.

Comparison of SFP-DD with QSFP28 and QSFP-DD

 

The SFP-DD MSA founding members include: Mellanox Technologies, Alibaba, Broadcom, Brocade, Cisco, Dell EMC, Finisar, HPE, Huawei, Intel, Juniper Networks, Lumentum, Molex, and TE Connectivity.

Mellanox offers complete end-to-end solutions of switches, network adapters, cables and transceivers supporting both the SFP+ for 10G line rates and the SFP28 for 25G line rates. Soon 50G PAM4 for 200G and 400G systems and interconnects for both Ethernet and InfiniBand.

 

Supporting Resources:

 

 

QSA Adapters Get Even Better at 25Gb/s

QSA solves problems linking different port sizes and speeds equipment together

Problems: You have a 4-channel QSFP port on a switch or network adapter but you’ve got a single-channel subsystem that uses SFP and you want to connect older equipment, storage or a 10G device. Or you have a new shiny 25Gb/s-based Spectrum switch or ConnectX-4 or -5 network adapters and you want to connect to slower 10Gb/s equipment. How do you connect the different port types and speeds together?

Answer: Get the Mellanox QSA Adapter. QSFP-to-SFP Adapter – now supporting 25Gb/s!

Sometimes, the simplest things can solve big problems and frustrations. The QSA is one such device and costs less than a dinner for one.

 

What is a QSA?

The QSA is a Mellanox designed and patented, mechanical adapter that fits neatly inside a QSFP port and enables plugging in a smaller, single-channel SFP device into a QSFP 4-channel port.  Only the one-channel gets passed through even though the mechanical port is 4-channels.  The QSA contains a configuration EPROM to tell the host what it is and what speed to run at.  Unless one is configuring it to run a slower line rate, it is plug-and-plug again-and-play – nothing to configure in software.

 

Features and Notes

  1. QSAs are available in 2 versions: 10G and 25G.
    • 10G version also supports 1G
    • 25G version supports 1G and 10G
  2. QSA accepts a huge range of 10G and 25G cables and transceiver types:
    • CR DAC copper SFP (3-7m)
    • SR SFP multi-mode transceiver (100m)
    • SFP multi-mode AOCs (100m)
    • LR SFP single-mode transceiver (10km)
    • SX 1G SFP+ multi-mode transceiver (500m)
    • Base-T 1G SFP converter that uses CAT-5 copper UTP cables (100m)
  3. Passive and consumes no power
  4. Does not induce and signal latency delays
  5. Contains an EPROM to tell the switch port what it is – used in the initial configuration
  6. Only one channel passes through to the QSFP port
  7. Supports Ethernet-only as InfiniBand doesn’t generally use SFP single-channel links.
  8. There is even a DAC adapter cable with SFP on one end and QSFP on the other.
  9. MC2309130-xxx to 3m and to 3 meters and MC2309124-xxx to 7 meters.

 

The copper DACs have a maximum reach of 3 – 7 meters but with a LR transceiver module with single-mode fiber can reach as far a 10km or 6.25 miles!

10G and 25Gb/s Cables and Transceivers Options For Use in QSA Adapters

 

In the past, Mellanox offered network adapters in SFP and QSFP versions of the cards. But starting with ConnectX-6, only the QSFP28 versions will be offered and if a SFP single-channel is required, the QSA will be the solution to create the connection.

Summary

Not everything in the world runs or needs to run at 25Gb/s so the QSA is a neat way to link slower 10G sub-systems to new high-speed Spectrum switches and ConnectX-5 network adapter and later upgrade the slower equipment to 25Gb/s.

More Information: 

LinkX is the Mellanox trademark and name for its cables and transceivers product line

 

 

100G PSM4: The Most Configurable & Lowest-Cost Single-Mode Transceiver Available

Single-mode transceivers now priced for high-volume data center use

Parallel Single Mode 4-channel (PSM4) is a type of single-mode transceiver that uses a parallel fiber design for reaches from up to 2 km and for reaches beyond the limits of 100-meter Short Reach 4-channel (SR4) multi-mode transceivers. PSM4s will be the transceiver that enables single-mode fiber to become popular in next-generation data centers due to its low cost and high configurability.

PSM4 is built using one laser (instead of four), split into four paths or channels and separately modulated with electrical data signals. Each channel has its own fibers and is separated throughout the link. PSM4 uses eight-fibers with four-fiber for transmission and four-fibers for receiving. A parallel, eight fiber Multiple Push On (MPO) optical connector is used.

100Gb/s PSM4 Transceiver

 

 

The PSM4 transceiver is the lowest cost, 100Gb/s transceiver on the market capable of using single-mode fiber for long reaches up to 2km. The best use case is at reaches less than 500 meters. When tallying up the cost of longer reaches, the cost of the eight fibers adds up for each meter and the CWDM4, using two fibers, becomes more economical.

 

What Data Centers are 2 km or 1.2 Miles Long?

While most data centers are not 2km (1.2 miles) long, the 2km spec is another way of stating the optical power of the laser. Measured in powers of ten called dBs (Decibels), the Mellanox PSM4 offers ~3.3 dBs of optical power which is enough to push through hundreds of meters of a lossy fiber infrastructure consisting of dirty and/or misaligned optical connectors, jumpers, optical patch panels and other interferences to the light path. This is similar to needing a very powerful flash light to shine through a dense forest of twigs, branches and leaves in the way even though the distance is relatively short.

 

Single mode fibers are cheap but transceivers expensive – reverse for multi-mode? Huh?

Interestingly, the multi-mode (large core 50-um diameter) fibers are more expensive than single-mode (9-um tiny core diameter) fibers but the transceivers are the reverse! Single-mode fiber is used by the telecom industry and ordered in hundreds of thousands of miles per year – and so it is inexpensive. Multi-mode fiber is used exclusively in data centers and the amount made is relatively small so it is about three times more expensive.

On the other hand, the multi-mode core diameter is large and easy to align with VCSEL lasers and detectors. The 9-um single-mode fiber is very hard to build and align transceiver components with and requires very expensive alignment equipment. Therefore, single-mode transceivers have always been significantly more expensive than multi-mode transceivers.  Multi-mode transceivers are less expensive than single-mode transceivers and exactly the reverse for the fibers.

Typically, single-mode transceivers use 10-20 different tiny parts that all need to be mechanically aligned to sub-micron tolerances. This requires a lot of manual labor, expensive test and alignment equipment and results in a high reject rate.

 

Silicon Photonics – Solves the Manufacturing Problems

Silicon Photonics does away with most of these problems and integrates the optical components and waveguides into a silicon wafer – the same basic technology used to build CMOS semiconductor electronic chips. This is how single-mode transceivers will eventually become more price competitive with multi-mode transceivers over time. Mellanox builds PSM4s using its internally developed Silicon Photonics technologies located in southern California which has been building and shipping Silicon Photonics products used as transceivers and Variable Optical Attenuators (VOAs) for nearly a decade

Mellanox Designed PSM4 Silicon Photonics and Control ASICs

 

Numerous PSM4 Applications and Configurations for Any Need.

The PSM4 has many different configuration application uses. It can bus 100Gb/s point-to-point over 2km or can be broken out into dual 50Gb/s or quad 25Gb/s links for linking to servers, storage and other subsystems. Additionally, the breakouts can be made using passive fiber splitter cables or a transceiver/AOC hybrid called a, “pigtail”. The following diagram illustrates the Mellanox, “end-to-end” system solutions consisting of switches and network adapters with cables and transceivers.

25G, 50G, 100G PSM4 Transceiver Applications

 

PSM4 Breakouts to Servers & Storage

Beside long reach 2km point-to-point links, PSM4 channels can also be split out individually. The diagram below shows a 100G PSM4 transceiver split using a passive breakout splitter cable with an MPO on one end and either dual MPOs (50G) or quad LC connectors (25G) on the other ends. CWDM4 cannot do this feature and can only bus 100Gb/s point-to-point.

Passive Fiber Breakout Configurations

 

PSM4 “Pigtail” Transceiver

Transceivers have their fibers attached to a detachable optical connector (MPO or LC). Active Optical Cables (AOCs) are two transceivers with the fibers permanently attached inside and not removable. A “Pigtail” (shown above) is a hybrid of both transceivers and AOCs and has the fibers attached to the transceivers with a short 1 meter length of fiber and one of three connector configurations:

  • One 100Gb/s MPO (Four 25G channels) QSFP28
  • Two 50Gb/s MPOs (Two 25G channels) QSFP28
  • Eight 25Gb/s duplex LC (One 25G channel) SFP28

While it only has 1 meter of fiber, it can reach 500 meters in the fiber infrastructure. Pigtails are used to link Top-of-Rack switches to 25Gb/s or 50Gb/s servers and storage subsystems or to plug into passive optical distribution patch panels that connect to other parts of the data center.

Bottom line, the Pigtail saves the material cost and maintenance for one optical connector as well as about 1 dB of optical loss gained back; small but when big data center builder orders tens of thousands, it all adds up.

 

Not All MPO Connector are the Same!

One thing to note: the MPO used in the SR4 multi-mode MPO (colored aqua) is not the same as the MPO/APC (colored green or yellow) for the PSM4. Optical connectors pass through “most” of the light and some gets reflected back towards the laser from the inside surface of the fiber end in the connector. The PSM4 uses tiny core single-mode fiber and it concentrates any back reflections in the connector infrastructure and aims it back at the laser which can destroy it. So, the single-mode fiber infrastructure polishes the fiber ends at an angle to divert the back reflections away from the laser. Hence, the name Angle Polished Connector (APC). Multi-mode fiber has a big 50-um core and the large area disperses the back reflections making it less of a problem. In reality, the MPO connectors use 12 fibers with four unused.

MPO Optical Connectors

Flat and Angle Polished Fibers

 

Mellanox PSM4-1550nm Interoperates with Most Industry PSM4s

Most PSM4 transceivers use a PIN detector that has a wide bandwidth spanning both 1310nm and 1550nm. The Mellanox 1550nm PSM4 can talk to almost any 1310nm PSM4 transceiver and the reverse even though the wavelengths are different. Many of our customers have interoperate tested more than ten different suppliers without any issues.

At the Optical Fiber Conferences (OFC), Mellanox demonstrated our 100Gb/s 1550nm PSM4 interoperating with PSM4 transceivers from Innolight and AOI and in breakout configurations at 25Gb/s with the LR transceivers with Oclaro, Hisense and Ligent.

Interoperability Demo 1310nm & 1550nm PSM4s

 

200Gb/s HDR

The PSM4 will make another debut at 200Gb/s using the QSFP28 form-factors in late 2017 supporting 200Gb/s HDR for InfiniBand and also in a 1:2 splitter configuration split into two 100Gb/s HDR100 QSFP28.

 

Summary

The PSM4 transceiver is the lowest-cost single-mode transceiver available today for use in next-generation data centers as it employs the low-cost and long reach features of single-mode fiber. It is a very flexible transceiver that can link 100Gb/s point-to-point or be split out into individual channels combinations of 25Gb/s or 50Gb/s to servers, storage and other subsystems.

While 100G PSM4s are fairly new to the market, as the popularity climbs and volume manufacturing efficiencies kick in, the PSM4 has a chance at challenging the 100G SR4 multi-mode transceiver in similar market prices when the transceiver and fiber link costs are added up. PSM4s will be the transceivers that enables single-mode fiber to become popular in next-generation data centers.

 

More Information:

 

LinkX is the Mellanox trademark and name for its cables and transceivers product line

DesignCon Trade Show – Connecting All the Signaling Dots

Stop by Booth #120 to see Mellanox 25G and 100Gb/s DAC, AOCs, SR4, PSM4 Cables and Transceivers

DesignCon at the Santa Clara Convention Center, to be held Jan. 31-Feb. 2, is the “chippie geek” trade show of the year and you better know how to speak PAM4 fluently this year if you attend. Electronic signaling and optical buzz-words are flying around like never before! So, you better get your CAUI-4 PAM4 and NRZ vernacular straight for SFP28, QSFP28, in SR, SR4 LR4, CWDM4, and PSM4 because 200G/400G 400AUI-8, QSFP-DD and OSFP, DR4, FR4, LR8, DR8 and FR8 are on their way!

 

Mellanox will be exhibiting its full line of 25Gb/s and 100Gb/s cables, transceivers, switches and network adapters. Not many companies have a complete line of end-to-end systems and interconnect products. Most tradeshows exhibit a lot of “tomorrow-land” products, promises and demos. But Mellanox’s 25Gb/s and 100Gb/s LinkX cables and transceivers as well as Spectrum switches and ConnectX® network adapter are available and shipping in volume today and in a wide deployment of hyperscale, HPC and enterprise markets. In fact, with respect to 100Gb/s network adapters, Mellanox was first to market and now has more than 90 percent market share. We are also the market leader in 100Gb/s SR4 transceivers.

 

We will be displaying our end-to-end solutions of SN2700 32-port QSFP28 and SN2410 25Gb/s SFP28 Top-of-Rack switches in a full system rack packed with DAC cables and splitters, AOCs, multi-mode SR4 and PSM4 optical transceivers in various configurations and showing the broad configuration and wide flexibility available from Mellanox.

Mellanox Switches, Network Adapters, Cables and Transceivers

 Picture2

 

Brad Smith and Arlon Martin from the LinkX Interconnect marketing will be at the show to answer any questions. Stop by our booth, number 120, or, if you want to set up a meeting, send me an email at BradS@Mellanox.com.

 

More Information:  

Learn more about LinkX cables and transceivers at: Mellanox.com/LinkX

3