net127

Dr. Joachim Stadel, Institute of Theoretical Physics, University of Zurich
writes , via Slashdot:

Our in-house designed (Joachim Stadel & Ben Moore 2003), massively parallel supercomputer for running our cosmological N-body simulations. This machine consists of 288 AMD Athlon-MP 2200+ (1.8 GHz) CPUs within a few cubic meters. Under load it produces about 45 kW of heat, about equivalent to 45 electric hair dryers operating continuously! This amount of heat, combined with the extremely high density necessitated a new design for efficient cooling. The 144 nodes (2 CPUs per node) are connected using an SCI fast interconnect supplied by Dolphin in a 12×12 2-dimensional torus. The layout of the machine is ring-like, thereby allowing very short “ribbon” cables to be used between the nodes. This fast interconnect network attains a peak bisection bandwidth of 96 Gbits/sec, with a node-node write/read latency as low as 1.5/3.5 microseconds. Additionally the zBox has 11.5 TBytes of disk (80 GBytes/node) and 3 Gbits/s I/O bandwidth to a frontend server with 7 TB of RAID-5 storage. This is among the fastest parallel computers in the world! At “first light” it ranked in the top 100, but the technology advances quickly. (see top500, June 2003: Rank 144) (see top500, November 2003: Rank 276)

We greatly acknowledge the aid of the Physics Mechanical Workshop at the University of Zurich for: 1) turning the “napkin-sketch” into a proper CAD/CAM design of the machine; 2) providing numerous suggestions which improved the detailed design; 3) providing a gigantic room for the construction of the boards; 4) and, well, building the thing! We thank the companies of Dolphin (dolphinics.com) for supplying the high speed network and COBOLT Netservices for supplying the majority of parts. We would like to especially thank the individuals: Doug Potter and Simen Timian Thoresen for their great help in setting up the linux kernel and root file system, getting netbooting to work correctly, and resolving several operating system related problems. Finally we thank all who helped in the construction of the zBox (assembly of boards, etc), Tracy Ewen, Juerg Diemand, Chiara Mastropietro, Tobias Kaufman.