Description

If you need to learn CUDA but don't have experience with parallel computing, CUDA Programming: A Developer's Introduction offers a detailed guide to CUDA with a grounding in parallel fundamentals. It starts by introducing CUDA and bringing you up to speed on GPU parallelism and hardware, then delving into CUDA installation. Chapters on core concepts including threads, blocks, grids, and memory focus on both parallel and CUDA-specific issues. Later, the book demonstrates CUDA in practice for optimizing applications, adjusting to new hardware, and solving common problems.

Key Features

  • Comprehensive introduction to parallel programming with CUDA, for readers new to both
  • Detailed instructions help readers optimize the CUDA software development kit
  • Practical techniques illustrate working with memory, threads, algorithms, resources, and more
  • Covers CUDA on multiple hardware platforms: Mac, Linux and Windows with several NVIDIA chipsets
  • Each chapter includes exercises to test reader knowledge

Readership

Software engineers, programmers, hardware engineers, students / advanced students

Table of Contents

Preface

Chapter 1. A Short History of Supercomputing

Introduction

Von Neumann Architecture

Cray

Connection Machine

Cell Processor

Multinode Computing

The Early Days of Gpgpu Coding

The Death of the Single-Core Solution

Nvidia and Cuda

Gpu Hardware

Alternatives to Cuda

Conclusion

Chapter 2. Understanding Parallelism with GPUs

Introduction

Traditional Serial Code

Serial/Parallel Problems

Concurrency

Types of Parallelism

Flynn’s Taxonomy

Some Common Parallel Patterns

Conclusion

Chapter 3. CUDA Hardware Overview

PC Architecture

GPU Hardware

CPUs and GPUs

Compute Levels

Chapter 4. Setting Up CUDA

Introduction

Installing the Sdk Under Windows

Visual Studio

Linux

Mac

Installing a Debugger

Compilation Model

Error Handling

Conclusion

Chapter 5. Grids, Blocks, and Threads

What it all Means

Threads

Blocks

Grids

Warps

Block Scheduling

A Practical Example—Histograms

Conclusion

Chapter 6. Memory Handling with CUDA

Introduction

Caches

Register Usage

Shared Memory

Constant Memory

Global Memory

Texture Memory

Conclusion

Chapter 7. Using CUDA in Practice

Introduction

Serial and Parallel Code

Processing Datasets

Profiling

An Example Using AES

Conclusion

References

Chapter 8. Multi-CPU and Multi-GPU Solutions

Introduction

Locality

Multi-CPU Systems

Multi-GPU Systems

Algorithms on Multiple GPUS

Which GPU?

Single-Node Systems

Streams

Multiple-Node Systems

Conclusion

Chapter 9. Optim

Details

No. of pages:
600
Language:
English
Copyright:
© 2013
Published:
Imprint:
Morgan Kaufmann
eBook ISBN:
9780124159884
Print ISBN:
9780124159334

About the author

Shane Cook

Shane Cook is Technical Director at CUDA Developer, a consultancy company that helps companies exploit the power of GPUs by re-engineering code to make the optimal use of the hardware available. He formed CUDA Developer upon realizing the potential of heterogeneous systems and CUDA to disrupt existing serial and parallel programming technologies. He has a degree in Applied Software Engineering, specializing in the embedded software field. He has worked in senior roles with blue chip companies over the past twenty years, always seeking to help to develop the engineers in his team. He has worked on C programming standards including the MISRA Safer C used by widely in the automotive software community, and previously developed code for companies in the Germany automotive and defense contracting industries as well as Nortel and Ford Motor Company.

Affiliations and Expertise

Technical Director, CUDA Developer, Munich, Germany

Reviews

"I must mention chapters 7, which deals with the practicalities of using the SDK, and 9, which offers advice and a detailed breakdown of areas that can limit the performance of a CUDA application. Together, these chapters transform this good book into the kind of excellent text that all CUDA developers can find useful, regardless of their relative experience."--ComputingReviews.com, July 12, 2013
"This book is one of the most comprehensive on the subject published to date…it will guide those acquainted with GPU/CUDA from other books or from NVIDIA product documentation through the optimization maze to efficient CUDA/GPU coding."--ComputingReviews.com, April 25, 2013