CS 442 / EECE 432 Fall '97
Introduction to Parallel Computing
(really Parallel Scientific Computing)

Instructors

Bruce Hendrickson (bah@cs.sandia.gov, 845-7599)
Steve Plimpton (sjplimp@cs.sandia.gov, 845-7873)

Schedule

Class: Tues/Thurs 4:00-5:15 in Mitchell 115
Office Hours: Tues 3:00-4:00 in Ferris 317

Texts

Required: Parallel Programming with MPI by Peter Pacheco
Recommended: Introduction to Parallel Computing: Design and Analysis of Algorithms by Kumar, Grama, Gupta & Karypis
Recommended: Designing and Building Parallel Programs by Ian Foster

Summary

This course will introduce students to the practical algorithmic issues involved in using current parallel computers for scientific computations. The emphasis will be more on parallelism than on scientific computing. The course will be structured around programming assignments using MPI on parallel computers at the Albuquerque Resource Center (or other machines you have access to.) We will cover parallel algorithms for a number of important scientific computing kernels including solving linear systems of equations, performing finite element analysis and simulating many-body interactions.

Course Outline

  1. Introductory Material
    1. What's and why's of parallel computing
    2. Introduction to MPI
    3. Performance metrics and modeling
    4. Simple collective communication operations
  2. Simple Scientific Computing Kernels
    1. Matrix-vector multiplication
    2. Structured grid calculations
    3. Master/slave parallelism
    4. More sophisticated collective communication operations
  3. Tougher Scientific Computing Kernels
    1. Particle simulations
    2. Solving dense linear systems
    3. Iterative solvers for sparse systems / Graph partitioning
    4. Sorting
  4. Advanced Topics (details depend on time & student interests)
    1. Dynamic load balancing
    2. Random number generators
    3. Shared memory programming
    4. Other scientific computing applications
  5. Student Project Presentations

Assignments

  • Programming Assignment 1.
  • Programming Assignment 2.
  • Programming Assignment 3.
  • Programming Assignment 4.
  • Programming Assignment 5.

  • Problem Set 1.
  • Problem Set 2.

    • Useful Web Pages

  • Source code for programs in Pacheco's textbook Parallel Programming with MPI.

  • Information about accounts at Sandia.

  • Ian Foster's book Designing and Building Parallel Programs.

  • MPI: The Complete Reference by Marc Snir, Steve Otto, Steven Huss-Lederman, David Walker, and Jack Dongarra.

  • Bill Gropp's MPI Page.

  • Case studies in parallel applications from Cal Tech in the book Parallel Computing Works.

  • Jim Demmel's lecture notes from Berkeley's CS 267.

  • Prith Banerjee's lecture notes from CSE 302 at the University of Illinois.

  • Alan Edelman's lecture notes from MIT.