Kitten OS Demonstrates Near-Native Network Performance on Cray Aries Interconnect

 One of the challenges in designing, developing, and deploying a research operating system (OS) for high- performance computing platforms has been the integration of device drivers for peripheral hardware, such as disk drives and network adapters. Developing and maintaining a device driver, which essentially allows the operating system to make use of the attached hardware, can be a tedious and time- consuming activity. Since nearly all vendors provide Linux device drivers for their hardware, it would be ideal if research OSs could simply use these existing drivers. Recently Sandia researchers along with collaborators from the University of Pittsburgh have successfully employed a multi-kernel approach that eliminates the need for a custom OS to have a custom device driver. The Kitten Lightweight Kernel OS was ported to the Cray XC architecture by extending the Cray Linux Environment to partition a node’s resources to be able to run Kitten alongside Linux. To enable applications running on Kitten to use Cray’s Aries interconnect, an approach was developed that passes control plane operations from Kitten to the Aries device driver running on Linux. Data plane transmissions flow directly from Kitten application memory to the Aries, enabling near-native MPI networking performance. Performance tuning is underway to try to optimize remaining performance differences.

These plots show the MPI ping-pong latency (a) and bandwidth (b) achieved by Cray Linux, the Kitten LWK running natively, and Kitten running in a virtual machine for the Cray Aries interconnect. Kitten is able to achieve near-native MPI performance without a custom device driver by employing a multi-kernel strategy that uses the Linux device driver for Aries.
These plots show the MPI ping-pong latency (a) and bandwidth (b) achieved by Cray Linux, the Kitten LWK running natively, and Kitten running in a virtual machine for the Cray Aries interconnect. Kitten is able to achieve near-native MPI performance without a custom device driver by employing a multi-kernel strategy that uses the Linux device driver for Aries.
Contact
Kevin Pedretti, ktpedre@sandia.gov

June 1, 2017