CUDA by Example An Introduction to General-Purpose GPU Programming (Jason Sanders,Edward Kandrot)（Z-Library）

【One-Line Pitch】 A hands-on introduction to GPU programming with CUDA C, this book teaches you to write parallel code that runs on NVIDIA graphics processors through practical examples, making it ideal for programmers with C experience who want to harness GPU power for general-purpose computing. 【Book Arc】 - **Opening (~0%–10%)**: Explains why parallel computing matters and how CUDA emerged as a solution to the limitations of early GPU programming, which required graphics APIs like OpenGL and shading languages. - **Early (~10%–23%)**: Introduces the CUDA C basics—writing your first kernel with `__global__` functions, launching code on the device, and querying GPU properties like compute capability and multiprocessor count. - **Early (~23%–32%)**: Covers parallel programming fundamentals, starting with vector addition using thread and block indices, then moving to more complex examples like the Julia Set to illustrate 2D indexing and device-side computation. - **Middle (~32%–42%)**: Delves into thread cooperation, explaining how to split work across blocks and threads, and introduces shared memory (`__shared__`) for fast, on-chip communication between threads within a block. - **Middle (~42%–48%)**: Demonstrates synchronization techniques to avoid race conditions, using examples like summation reduction and image processing to show how threads coordinate via shared memory buffers. - **Late (~48%–100%)**: Advances to constant memory for performance optimization, CUDA events for timing, streams for concurrent kernel execution, multiple GPU support, and a final overview of CUDA tools like CUFFT and CUBLAS. 【Key Takeaways】 - **CUDA C is C with extensions** (Early): Adding qualifiers like `__global__` to functions and using angle-bracket syntax (`<<<blocks, threads>>>`) lets you run code on the GPU, making the transition from CPU to GPU programming straightforward for C programmers. - **Thread and block indexing map to parallelism** (Early): Using `blockIdx.x`, `threadIdx.x`, and `blockDim.x` lets you assign each thread a unique data element, enabling vector operations like addition to run hundreds of computations in parallel. - **Grid and block dimensions are flexible** (Middle): You can launch kernels with 1D or 2D grids and blocks, as shown in image-processing examples where each thread computes a pixel value based on its (x, y) coordinate. - **Shared memory enables fast thread communication** (Middle): Declaring variables with `__shared__` creates per-block, on-chip memory that threads can read and write, offering lower latency than global memory and serving as a software-managed cache. - **Synchronization prevents race conditions** (Middle): When threads communicate via shared memory, you need a barrier mechanism to ensure writes complete before reads, as demonstrated in reduction algorithms where threads sum values in stages. - **Constant memory optimizes read-only data access** (Late): For data that all threads read uniformly, constant memory provides performance benefits, though the excerpts only hint at this topic without full code examples. - **CUDA events measure kernel performance** (Late): Using events to time GPU operations helps you profile and optimize your code, a critical skill for writing efficient parallel programs. 【Reading Tips】 - **Skim the history in Chapter 1** (~0%–10%): The early GPU programming struggles are interesting context, but you can move quickly to the practical CUDA C introduction starting around 10%. - **Deep-read the vector addition example** (~23%): This is the foundation for understanding thread indexing—master how `blockIdx.x` and `threadIdx.x` map to array elements before moving on. - **Focus on the Julia Set example** (~29%): It shows 2D indexing and device-side computation with complex numbers, which is a great template for image-based or grid-based problems. - **Pay close attention to shared memory and synchronization** (~39%–48%): These concepts are trickier and more advanced; work through the reduction example carefully to understand how threads cooperate without race conditions. - **Treat the later chapters as reference material** (~48%–100%): Constant memory, streams, and multi-GPU topics are advanced; skim them to know what exists, then return when you need those features. 【Coverage Limits】 The excerpts cover roughly the first half of the book (through shared memory and synchronization) plus chapter objectives for later topics; detailed code for constant memory, streams, and multi-GPU programming is not fully included in this guide.