Server interconnect fabrics

Our group's focus is on the design of interconnection networks used for server and clustering networks, both for high-performance-computing and for commercial environments. The increasing degree of parallelism at all hardware levels, from processor to server and data center, is leading to architectures that are increasingly network-centric. Building on our group's extensive experience in designing and implementing packet switch chips, most notably the former line of IBM PowerPRS Packet Routing Switches [1], our mission is to develop high-performance interconnect fabrics with cost-effective, scalable architectures and technology for IBM's high-end server products. Results on optimizing this architecture for HPC (high-performance computer) interconnection networks have been published in [2]. Our expertise covers flow control, queuing and scheduling, performance analysis (by simulation as well as analytically), congestion control, logic design and fabric system design.

In a joint project with Corning Inc., we have advanced the state of the art in optical switch technology for future high-end HPC systems. As these switches will be bufferless for the foreseeable future, they require a centralized arbiter to resolve contention. Our main contribution is a novel, highly distributed and highly scalable scheduler architecture, which achieves low latency, high throughput, and supports multicast communication. We have designed and implemented this architecture for a 64-port optical switch technology demonstrator with a line rate of 40 Gb/s per port [3].

Data centers are a significant area of growth for IBM's business. A key challenge faced by data-center interconnection networks is scalability. In such networks, link-level flow control and congestion management are instrumental in meeting reliability, performance, and cost targets. Congestion management is required to prevent severe congestion in the form of saturation trees from causing catastrophic performance collapse. We have performed an extensive evaluation of the Congestion Control Annex proposed for the InfiniBand networking technology, demonstrating how the congestion management parameters should be set depending on network size and traffic scenario [4]. Currently, we are actively contributing to the IEEE 802.1au working group, which is in the process of defining a congestion management method for Gigabit and 10-Gigabit Ethernet networks. Moreover, we have prototyped a novel Ethernet congestion management scheme that also functions on legacy networks, as it is implemented entirely in software at the network edges.

From a system perspective, evaluating the performance of a computer interconnection network is of secondary importance. The primary metric is how much faster the system can run its applications. To this end, such a performance evaluation typically involves simulating a set of benchmark applications. As this is often not a computationally feasible approach, we are also performing end-to-end evaluations of computer interconnection networks based on traces collected from actual application runs. We are also measuring traffic directly within the network switches by using monitoring capabilities built into IBM Federation switches.