The most exciting development in parallel computer architecture is the convergence of traditionally disparate approaches on a common machine structure. This book explains the forces behind this convergence of shared-memory, message-passing, data parallel, and data-driven computing architectures. It then examines the design issues that are critical to all parallel architecture across the full range of modern design, covering data access, communication performance, coordination of cooperative work, and correct implementation of useful semantics. It not only describes the hardware and software techniques for addressing each of these issues but also explores how these techniques interact in the same system. Examining architecture from an application-driven perspective, it provides comprehensive discussions of parallel programming for high performance and of workload-driven evaluation, based on understanding hardware-software interactions.

Key Features

* synthesizes a decade of research and development for practicing engineers, graduate students, and researchers in parallel computer architecture, system software, and applications development * presents in-depth application case studies from computer graphics, computational science and engineering, and data mining to demonstrate sound quantitative evaluation of design trade-offs * describes the process of programming for performance, including both the architecture-independent and architecture-dependent aspects, with examples and case-studies * illustrates bus-based and network-based parallel systems with case studies of more than a dozen important commercial designs

Table of Contents

  • 1 Introduction
    • 1.1 Introduction
    • 1.2 Why Parallel Architecture
      • 1.2.1 Application Trends
      • 1.2.2 Technology Trends
      • 1.2.3 Architectural Trends
      • 1.2.4 Supercomputers
      • 1.2.5 Summary
    • 1.3 Convergence of Parallel Architectures
      • 1.3.1 Communication Architecture
      • 1.3.2 Shared Memory
      • 1.3.3 Message-Passing
      • 1.3.4 Convergence
      • 1.3.5 Data Parallel Processing
      • 1.3.6 Other Parallel Architectures
      • 1.3.7 A Generic Parallel Architecture
    • 1.4 Fundamental Design Issues
      • 1.4.1 Communication Abstraction
      • 1.4.2 Programming Model Requirements
      • 1.4.3 Naming
      • 1.4.4 Ordering
      • 1.4.5 Communication and Replication
      • 1.4.6 Performance
    • 1.5 Concluding Remarks
    • 1.6 References
    • 1.7 Exercises
  • 2 Parallel Programs
    • 2.1 Introduction
    • 2.2 Parallel Application Case Studies
      • 2.2.1 Simulating Ocean Currents
      • 2.2.2 Simulating the Evolution of Galaxies
      • 2.2.3 Visualizing Complex Scenes using Ray Tracing
      • 2.2.4 Mining Data for Associations
    • 2.3 The Parallelization Process
      • 2.3.1 Steps in the Process
      • 2.3.2 Parallelizing Computation versus Data
      • 2.3.3 Goals of the Parallelization Process
    • 2.4 Parallelization of an Example Program


      No. of pages:
      © 1998
      Morgan Kaufmann
      eBook ISBN:
      Print ISBN:
      Print ISBN:

About the authors

David Culler

David Culler led the Berkeley Network of Workstations (NOW) project, which sparked the current commercial revolution in high-performance clusters. Anoop Gupta co-led the Stanford DASH multiprocessor project, which developed the shared-memory technology increasingly used in commercial machines.

Affiliations and Expertise

University of California, Berkeley

Jaswinder Pal Singh

Jaswinder Pal Singh led the development of the SPLASH and SPLASH-2 suites of parallel programs, which have defined the workloads and methodology used to drive decisions and evaluate trade-offs in shared- memory parallel architecture.

Affiliations and Expertise

Princeton University

Anoop Gupta

Dr. Anoop Gupta is a Distinguished Scientist at Microsoft Research. He works on cross-disciplinary projects that have potential for large business or societal impact. His recent projects focus on areas of education, communication, collaboration, and natural user interfaces.

Affiliations and Expertise

Microsoft Research Redmond, WA, USA