This presentation discusses the following topics: (1) Red Sky Background; (2) 3D Torus Interconnect Concepts; (3) Difficulties of Torus in IB; (4) New Routing Code for IB a 3D Torus; (5) Red Sky 3D Torus Implementation; and (6) Managing a Large IB Machine. Computing at Sandia: (1) Capability Computing - Designed for scaling of single large runs, Usually proprietary for maximum performance, and Red Storm is Sandia's current capability machine; (2) Capacity Computing - Computing for the masses, 100s of jobs and 100s of users, Extreme reliability required, Flexibility for changing workload, Thunderbird will be decommissioned this quarter, Red Sky is our future capacity computing platform, and Red Mesa machine for National Renewable Energy Lab. Red Sky main themes are: (1) Cheaper - 5X capacity of Tbird at 2/3 the cost, Substantially cheaper per flop than our last large capacity machine purchase; (2) Leaner - Lower operational costs, Three security environments via modular fabric, Expandable, upgradeable, extensible, and Designed for 6yr. life cycle; and (3) Greener - 15% less power-1/6th power per flop, 40% less water-5M gallons saved annually, 10X better cooling efficiency, and 4x denser footprint.
Gigabit Passive Optical Networks (GPON) is a networking technology which offers the potential to provide significant cost savings to Sandia National Laboratories in the area of network operations. However, a large scale GPON deployment requires a significant investment in equipment and infrastructure. Before a large scale GPON system was acquired and built, a small GPON system manufactured by Motorola was acquired and tested. The testing performed was to determine the suitability of GPON for use at SNL. This report documents that testing. This report presents test results of GPON system consisting of Motorola and Juniper equipment. The GPON system was tested in areas of data throughput, video conferencing, VOIP, security, and operations and management. The GPON system performed well in almost all areas. GPON will not meet the needs of the low percentage of users requiring a true 1-10 Gbps network connection. GPON will also most likely not meet the need of some servers requiring dedicated throughput of 1-10 Gbps. Because of that, there will be some legacy network connections that must remain. If these legacy network connections can not be reduced to a bare minimum and possibly consolidated to a few locations, any cost savings gained by switching to GPON will be negated by maintaining two networks. A contract has been recently awarded for new GPON equipment with larger buffers. This equipment should improve performance and further reduce the need for legacy network connections. Because GPON has fewer components than a typical hierarchical network, it should be easier to manage. For the system tested, the management was performed by using the AXSVison client. Access to the client must be tightly controlled, because if client/server communications are compromised, security will be an issue. As with any network, the reliability of individual components will determine overall system reliability. There were no failures with the routers, OLT, or Sun Workstation Management platform. There were however four ONTs that failed. Because of the small sample size of 64, and the fact that some of the ONTs were used units, no conclusions can be made. However, ONT reliability is an area of concern. Access to the fiber plant that GPON requires must be tightly controlled and all changes documented. The undocumented changes that were performed in the GPON test lab demonstrated the need for tight control and documentation. In summary, GPON should be able to meet the needs of most network users at Sandia National Laboratories. Because it supports voice, video, and data, it positions Sandia National Laboratories to deploy these services to the desktop. For the majority of corporate network users at Sandia National Laboratories GPON should be a suitable replacement for the legacy network.
This report presents the result of an effort to re-implement the Parallel Virtual File System (PVFS) using Portals as the transport. This report provides short overviews of PVFS and Portals, and describes the design and implementation of PVFS over Portals. Finally, the results of performance testing of both stock PVFS and PVFS over Portals are presented.
This report describes testing of prototype InfiniBand{trademark} host channel adapters from Intel Corporation, using the Linux(reg sign) operating system. Three generations of prototype hardware were obtained, and Linux device drivers were written which exercised the data movement capabilities of the cards. Latency and throughput results obtained were similar to other SAN technologies, but not significantly better.