Publications

In the past few years much effort has been devoted to finding faster and more convenient ways to exchange data between nodes of massively parallel distributed memory machines. One such approach, taken by Thorsten von Eicken et al. is called Active Messages. The idea is to hide message passing latency and continue to compute while data is being sent and delivered. The authors have implemented Active Messages under SUNMOS for the Intel Paragon and performed various experiments to determine their efficiency and utility. In this paper they concentrate on the subset of the Active Message layer that is used by the implementation of the Split-C library. They compare performance to explicit message passing under SUNMOS and explore new ways to support Split-C without Active Messages. They also compare the implementation to the original one on the Thinking Machines CM-5 and try to determine what the effects of low latency and low band-width versus high latency and high bandwidth are on user codes.

SUNMOS is an acronym for Sandia/UNM Operating System. It was originally developed for the nCUBE-2 MIMD supercomputer between January and December of 1991. Between April and August of 1993, SUNMOS was ported to the Intel Paragon. This document provides a quick overview of how to compile and run jobs using the SUNMOS environment on the Paragon. The primary goal of SUNMOS is to provide high performance message passing and process support an example of its capabilities, SUNMOS Release 1.4 occupies approximately 240K of memory on a Paragon node, and is able to send messages at bandwidths of 165 megabytes per second with latencies as low as 42 microseconds using Intel NX calls. By contrast, Release 1.2 of OSF/1 for the Paragon occupies approximately 7 megabytes of memory on a node, has a peak bandwidth of 65 megabytes per second, and latencies as low as 42 microseconds (the communication numbers are reported elsewhere in these proceedings).

The compute power of the individual nodes of massively parallel systems increases steadily, while network latencies and bandwidth have not improved as quickly. Many researches believe that it is necessary to use explicit message passing in order to get the best possible performance out of these systems. High level parallel languages are shunned out of fear they might compromise performance. In this paper we have a look at one such language called Split-C. It fits into a middle ground between efforts such as High Performance Fortran (HPF) and explicit message passing. HPF tries to hide the underlying architecture from the programmer and let the compiler and the run time system make decision about parallelization, location of data, and the mechanisms used to transfer the data from one node to another. On the other hand, explicit message passing leaves all the decision to the programmer. Split-C allows access to a global address space, but leaves the programmer in control of the location of data, and offers a clear cost model for data access. Split-C is based on Active Messages. We have implemented both under the SUNMOS operating system on the Intel Paragon. We will discuss performance issues of Split-C and make direct comparisons to the Thinking Machines CM-5 implementation. We will also scrutinize Active Messages, discuss their properties and drawbacks, and show that other mechanisms can be used to support Split-C.