#25gbe

Nowadays, the requirement for bandwidth in cloud data centers is increasing strikingly. To meet the demand for higher bandwidth, networking and the Ethernet industry are moving toward a new direction. Discussions previously focusing on 10GbE and 40GbE are now shifting onto 25GbE. It seems that 25GbE is more preferred and accepted by end users, which poses a threat to 10GbE and 40GbE. Why does it happen? This post will lead you to interpret 25GbE in an all-sided perspective.

25GbE—An Emerging Standard

25GbE is a standard developed by IEEE 802.3 Task Force P802.3by in July 2014, used for Ethernet servers and switches connectivity in a data center environment. The single lane design of 25GbE gives it a low cost per bit, which enables cloud providers and large-scale data center operators to deploy fewer switches and cables to meet the needs while still scaling their network infrastructure. The following table provides a summary of key upcoming IEEE standard interfaces that specify 25GbE.

Figure 1: IEEE 802.3 standard Interfaces that specify 25GbE

Cloud Will Drive to QSFP28 and SFP28

The 25GbE physical interface specification supports two main form factors—SFP28 and QSFP28. SFP28 is used for a single 25GbE port. The SFP28-25G-SR-S is an 850nm VCSEL 25GbE transceiver which is designed to transmit and receive optical data over 50/125µm multimode fiber (MMF) and support up to 70m on OM3 MMF and 100m on OM4 MMF.

The QSFP28 transceiver and interconnect cable is a high-density, high-speed product solution designed for applications in the telecommunication. The interconnect offers four channels of high-speed signals with data rates ranging from 25 Gbps up to potentially 40 Gbps, and will meet 100 Gbps Ethernet (4x25 Gbps) and 100 Gbps 4X InfiniBand EDR requirements.

Figure 2: FS.COM optical transceivers for 25GbE solution

Why Choose 25GbE

While 10GbE is fine for many existing deployments, it cannot efficiently deliver the bandwidth required by next-generation cloud and web-scale environments. And 40GbE isn't cost-effective or power-efficient in ToR switching for cloud providers. Thus, 25GbE was designed to break through the dilemma.

Number of SerDes Lanes

SerDes is an integrated circuit or transceiver used in high-speed communications for converting serial data to parallel interfaces and vice versa. The transmitter section is a serial-to-parallel converter, and the receiver section is opposite. Currently, the rate of SerDes is 25 Gbps. That is to say, we can only use one SerDes lane at the speed of 25 Gbps to connect from 25GbE card to the other end of 25GbE card. In contrast, 40GbE needs four 10GbE SerDes lanes to achieve connection. As a result, the communication between two 40GbE cards requires as many as four pairs of fiber. Furthermore, 25 Gbps Ethernet provides an easy upgrade path to 50GbE and 100GbE networks, which utilize multiple 25GbE lanes.

Figure 3: Numbers of lanes needed in different Gigabit Ethernet

Utilization of PCIe Lanes

At present, the mainstream Intel Xeon CPU only provides 40 lanes of PCIe 3.0. The lane bandwidth of a single PCIe 3.0 is about 8 Gbps. These PCle lanes are used for many connections. Therefore, it is necessary to consider the utilization of limited PCIe lanes by the network cards. Single 40GbE NIC needs at least one PCIe 3.0 x8 slot, so two 40GbE cards need to occupy two PCIe 3.0 x8 lanes. Even if the two 40GbE ports can run full of data at the same time, the actual lane bandwidth utilization is only: (40G+40G) / 8G*16= 62.5%. On the contrary, 25GbE card only needs one PCIe 3.0 x8 lane, and then the utilization efficiency is 25G*2 / (8G*8) = 78%. Apparently, 25GbE is significantly more efficient and more flexible than 40GbE in terms of the use of PCIe lanes.

10GbE vs 25GbE vs 40GbE

25GbE enables resellers and their customers to provide 2.5X the performance of 10GbE, making it a cost-effective upgrade to the 10GbE infrastructure. Since 25GbE is delivered across a single lane, it provides greater switch port density and network scalability compared to 40GbE, which is actually four 10GbE lanes. Thus, it costs less, requires lower power consumption and provides higher bandwidth. What’s more, 25GbE can run over existing fiber optic cable plant designed for 10, 50 or 100GbE and also 40GbE by changing the transceivers.

Figure 4: Bandwidth comparison for 25GbE and other Ethernet speeds

Summary