Shall We “Converge”?
Jun 23rd, 2008 by Sadry Tavana
10 Gigabit Ethernet has brought tremendous excitement and potential for big change in the datacenter I/O infrastructure and connectivity. Everyday, we are being bombarded with new terminologies and buzzwords like DCE (Data Center Ethernet), FCoE (Fibre Channel over Ethernet), CEE (Congestion Enhanced Ethernet), CFE (Congestion Free Ethernet), Unified Wire, Converged Fabrics, etc.
So, what is so special about DCE?
DCE’s promise is use a single Ethernet interface on the server for all the types of traffic (i.e. SAN, LAN, IPC, etc.). The cost of managing and administrating multiple separate networks for LAN and SAN has already become prohibitive in large enterprise datacenters. I’ve seen data from IDC that predicts the cost of management and admin for a server will be 65% of total cost of ownership in 2010. 10G Ethernet provides enough bandwidth to converge and consolidate all of a server’s network and storage traffic out of a single spigot. Users love Ethernet’s high degree of plug-and-play and ease of use. What classical Ethernet MAC layer protocol lacks is delivery guarantees, control over jitter, and decent Layer 2 QoS. This is what DCE brings to the party.
Most, if not all, DCE changes are being standardized in 802.1 Ethernet Bridging standards body of IEEE. There is much coverage on the web on the details of Ethernet facelifts to make it DCE ready, so I will not elaborate much on those here. DCE will benefit not only data centers, but any application that relies on a reliable L2 transport. Most of the enhancements are intended to make Ethernet switches and NICs have more predictable behavior under load and congestion. It also provides a finer grain link level control over different traffic classes traversing the LAN. Ethernet is really a best effort connection-less datagram protocol. Retransmissions above L2 can be expensive and impact performance, especially when you talk about SAN protocols like Fibre Channel that rely on link level congestion management. Even high performance iSCSI which uses TCP congestion management can experience performance issues if the network and the IP stack are not properly tuned and architected.
Looks like vendors are mainly implementing DCE features in their second generation 10 Gb/s NIC and switch products and are not even bothering with the older 1 Gb/s technology. The IEEE802.1au/az work is starting to gel, and semiconductor and system vendors alike are feverishly trying to be first to market with their version of “almost standard” products.
For me, it has been interesting to watch all this excitement about convergence. While the rest of the world is just now learning about the economics and benefits of convergence on unified wire in data centers, we have been delivering and demonstrating the TCO benefits of converged fabric server products for over 8 years now!
Thanks for the post
Yes, Egenera has set a very fine example, but so did IBM when they invented the 360 architecture, mainly by looking ahead and implementing an expandable and scalable architecture. It’s still alive today in a way, but it doesn’t fit a very broad range of businesses any more. Similarly Egenera doesn’t start small enough for many businesses and doesn’t scale far enough for many others. As you outgrow an enclosure, you loose almost all typical Egenera benefits. And an upgrade of the backplane looks urgend to me, if you consider the processing and I/O capabilities of what can be put in to a blade these days. Considering the initial investment and the vendor lock-in risk, it simply doesn’t seem an acceptable risk, unless you need to replace a couple of thousand servers in a hurry. 3 leaf systems Infiniband based V-8000 server and now DCE simply offer a far easier migration path towards unified wire and far less dependency on a single vendor.
That’s why I welcome your tendency to develop your PAN manager into a generic data center operating system, that is hardware and hypervisor agnostic and your willingness to license that. Going further down the proprietary hardware road seems like a sure way into oblivition to me.
Regards, Thomas