Metal by Tutorials Beginning Game Engine Development with Metal (Caroline Begbie Marius Horga) (z-library.sk, 1lib.sk, z-lib.sk)

Third Edition — Xcode 13 ■ Metal 2.4 Metal by Tutorials Beginning Game Engine Development With Metal By the raywenderlich Tutorial Team Caroline Begbie & Marius Horga

Metal by Tutorials Metal by Tutorials Metal by Tutorials By Caroline Begbie & Marius Horga Copyright ©2022 Razeware LLC. Notice of Rights All rights reserved. No part of this book or corresponding materials (such as text, images, or source code) may be reproduced or distributed by any means without prior written permission of the copyright owner. Notice of Liability This book and all corresponding materials (such as source code) are provided on an “as is” basis, without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose, and noninfringement. In no event shall the authors or copyright holders be liable for any claim, damages or other liability, whether in action of contract, tort or otherwise, arising from, out of or in connection with the software or the use of other dealing in the software. Trademarks All trademarks and registered trademarks appearing in this book are the property of their own respective owners. raywenderlich.com 2

Metal by Tutorials Table of Contents: Overview Book License............................................................................. 17 Before You Begin.............................................................18 What You Need......................................................................... 19 Book Source Code & Forums...................................................20 Acknowledgments.................................................................... 23 Introduction.............................................................................. 24 Section I: Beginning M etal.............................................28 Chapter 1: Hello, M eta l!................................................ 29 Chapter 2: 3D Models.................................................... 44 Chapter 3: The Rendering Pipeline............................... 69 Chapter 4: The Vertex Function.....................................97 Chapter 5: 3D Transformations...................................117 Chapter 6: Coordinate Spaces..................................... 134 Chapter 7: The Fragment Function..............................156 Chapter 8: Textures...................................................... 179 Chapter 9: Navigating a 3D Scene.............................. 212 Chapter 10: Lighting Fundamentals............................239 Section II: Intermediate M e ta l.................................. 267 Chapter 11: Maps & Materials.................................... 268 Chapter 12: Render Passes..........................................300 Chapter 13: Shadows...................................................321 raywenderlich.com 3

Metal by Tutorials Chapter 14: Deferred Rendering................................343 Chapter 15: Tile-Based Deferred Rendering..............371 Chapter 16: GPU Compute Programming..................395 Chapter 17: Particle Systems...................................... 411 Chapter 18: Particle Behavior.....................................437 Section III: Advanced M eta l......................................458 Chapter 19: Tessellation & Terrains............................459 Chapter 20: Fragment Post-Processing......................491 Chapter 21: Imaged-Based Lighting............................510 Chapter 22: Reflection & Refraction........................... 544 Chapter 23: Animation................................................ 568 Chapter 24: Character Animation............................... 593 Chapter 25: Managing Resources............................... 619 Chapter 26: GPU-Driven Rendering........................... 637 Section IV: Ray Tracing................................................ 665 Chapter 27: Rendering W ith Rays.............................. 666 Chapter 28: Advanced Shadows..................................694 Chapter 29: Advanced Lighting................................... 722 Chapter 30: Metal Performance Shaders...................738 Chapter 31: Performance Optimization..................... 755 Chapter 32: Best Practices..........................................777 Conclusion............................................................................788 raywenderlich.com 4

Metal by Tutorials Table of Contents: Extended Book License............................................................................. 17 Before You Begin.............................................................18 What You Need.........................................................................19 Book Source Code & Forums.................................................. 20 About the Authors.............................................................................22 About the Editors.............................................................................. 22 Acknowledgments....................................................................23 In troduction ............................................................................. 24 About This Book................................................................................24 How Did Metal Come to Life?............................................................ 25 Why Would You Use Metal?..............................................................25 When Should You Use Metal?........................................................... 26 Who This Book Is For?.......................................................................27 How to Read This Book?....................................................................27 Section I: Beginning M e ta l........................................... 28 Chapter 1: Hello, M e ta l!.......................................................... 29 What is Rendering?........................................................................... 30 What is a Frame?................................................................................32 Your First Metal App..........................................................................32 The Metal View................................................................................. 33 Rendering..........................................................................................38 Challenge..........................................................................................42 Key Points..........................................................................................43 Chapter 2: 3D M odels............................................................. 44 What Are 3D Models?.......................................................................45 Creating Models With Blender......................................................... 49 raywenderlich.com 5

Metal by Tutorials 3D File Formats................................................................................. 50 Exporting to Blender..........................................................................51 The .obj File Format........................................................................... 53 The .mtl File Format........................................................................... 54 Material Groups................................................................................57 Vertex Descriptors.............................................................................59 Metal Coordinate System.................................................................. 62 Submeshes........................................................................................ 65 Challenge ........................................................................................ 67 Key Points..........................................................................................68 Chapter 3: The Rendering Pipeline.........................................69 The GPU and CPU.............................................................................70 The Metal Project............................................................................. 71 The Render Pipeline ........................................................................ 81 Challenge ........................................................................................ 95 Key Points........................................................................................ 96 Chapter 4: The Vertex Function..............................................97 Shader Functions ............................................................................. 98 The Starter Project .......................................................................... 98 Rendering a Quad.............................................................................. 99 Calculating Positions.......................................................................102 More Efficient Rendering............................................................... 104 Vertex Descriptors ........................................................................ 107 Adding Another Vertex A ttribute....................................................110 Rendering Points.............................................................................113 Challenge.........................................................................................115 Key Points....................................................................................... 116 Chapter 5: 3D Transformations............................................ 117 Transformations.............................................................................. 118 The Starter Project & Setup.............................................................119 raywenderlich.com 6

Metal by Tutorials Translation......................................................................................122 Vectors & Matrices..........................................................................122 Creating a Matrix.............................................................................125 Scaling.............................................................................................127 Rotation.......................................................................................... 129 Key Points....................................................................................... 133 Chapter 6: Coordinate Spaces............................................... 134 The Starter Project..........................................................................139 Uniforms.........................................................................................141 Projection....................................................................................... 146 Refactoring the Model Matrix..........................................................152 Key Points....................................................................................... 155 Where to Go From Here?................................................................. 155 Chapter 7: The Fragment Function....................................... 156 The Starter Project..........................................................................157 Screen Space...................................................................................158 Metal Standard Library Functions....................................................161 Normals.......................................................................................... 167 Loading the Train Model With Normals...........................................169 Depth.............................................................................................. 172 Hemispheric Lighting.......................................................................175 Challenge.........................................................................................177 Key Points....................................................................................... 178 Where to Go From Here?................................................................. 178 Chapter 8: Textures............................................................... 179 Textures and UV Maps....................................................................180 The Starter A pp .............................................................................. 183 sRGB Color Space........................................................................... 190 Capture GPU Workload.................................................................. 192 Samplers.......................................................................................... 195 raywenderlich.com 7

Metal by Tutorials Mipmaps..........................................................................................200 The Asset Catalog........................................................................... 204 Texture Compression.......................................................................209 Key Points.......................................................................................211 Chapter 9: Navigating a 3D Scene.........................................212 The Starter Project..........................................................................213 Scenes.............................................................................................213 Cameras..........................................................................................215 Input................................................................................................219 Delta Time.......................................................................................221 Mouse and Trackpad Input..............................................................227 Arcball Camera................................................................................228 Orthographic Projection.................................................................232 Challenge........................................................................................ 237 Key Points.......................................................................................238 Chapter 10: Lighting Fundamentals......................................239 The Starter Project..........................................................................240 Representing Color..........................................................................241 Normals..........................................................................................243 Light Types..................................................................................... 244 Directional Light.............................................................................. 244 The Phong Reflection Model........................................................... 247 The Dot Product.............................................................................. 248 Creating Shared Functions in C++.................................................. 252 Point Lights..................................................................................... 260 Spotlights........................................................................................ 263 Key Points.......................................................................................266 Where to Go From Here?.................................................................266 Section II: Intermediate M eta l...................................267 Chapter 11: Maps & M ateria ls..............................................268 Li raywenderlich.com 8

Metal by Tutorials Normal Maps...................................................................................269 The Starter A pp .............................................................................. 275 Using Normal Maps........................................................................ 275 Materials........................................................................................ 285 Physically Based Rendering (PBR).................................................. 291 Channel Packing.............................................................................. 296 Challenge........................................................................................ 298 Where to Go From Here?.................................................................299 Chapter 12: Render Passes....................................................300 Render Passes................................................................................. 301 Object Picking................................................................................. 302 The Starter A pp .............................................................................. 303 Setting up Render Passes.................................................................304 Creating a UInt32 Texture............................................................... 305 Adding the Render Pass to Renderer............................................... 308 Adding the Shader Function............................................................ 309 Adding the Depth Attachment........................................................ 312 Load & Store Actions.......................................................................315 Reading the Object ID Texture........................................................ 316 Key Points.......................................................................................320 Chapter 13: Shadows............................................................. 321 Shadow Maps...................................................................................322 The Starter Project..........................................................................323 Identifying Problems.......................................................................334 Visualizing the Problems.................................................................336 Solving the Problems.......................................................................339 Cascaded Shadow Mapping............................................................ 341 Key Points.......................................................................................342 Chapter 14: Deferred Rendering.......................................... 343 The Starter Project..........................................................................346 raywenderlich.com 9

Metal by Tutorials The G-buffer Pass........................................................................... 347 Updating Renderer..........................................................................357 The Lighting Shader Functions........................................................ 358 Adding Point Lights..........................................................................360 Blending..........................................................................................367 Key Points.......................................................................................370 Chapter 15: Tile-Based Deferred Rendering........................371 Programmable Blending.................................................................. 373 Tiled Deferred Rendering............................................................... 373 The Starter Project..........................................................................374 Stencil Tests.................................................................................... 384 Create the Stencil Texture............................................................... 387 Configure the Stencil Operation..................................................... 388 Masking the Sky.............................................................................. 392 Challenge........................................................................................ 394 Key Points.......................................................................................394 Where to Go From Here?.................................................................394 Chapter 16: GPU Compute Programming............................ 395 The Starter Project..........................................................................396 Winding Order and Culling..............................................................397 Reversing the Model on the CPU..................................................... 398 Compute Processing.......................................................................400 Reversing the Warrior Using GPU Compute Processing..................404 The Kernel Function........................................................................ 406 Atomic Functions.............................................................................408 Key Points.......................................................................................410 Chapter 17: Particle Systems.................................................411 Particle........................................................................................... 412 Emitter........................................................................................... 413 The Starter Project..........................................................................413 raywenderlich.com 10

Metal by Tutorials Creating a Particle and Em itter...................................................... 414 The Fireworks Pass..........................................................................417 Particle Dynamics........................................................................... 421 Implementing Particle Physics........................................................422 Particle Systems.............................................................................. 423 Rendering a Particle System...........................................................426 Configuring Particle Effects............................................................ 432 Fire................................................................................................. 434 Key Points.......................................................................................436 Where to Go From Here?.................................................................436 Chapter 18: Particle Behavior...............................................437 Behavioral Animation..................................................................... 438 Swarming Behavior..........................................................................439 The Starter Project..........................................................................440 Velocity........................................................................................... 443 Behavioral Rules..............................................................................445 Key Points.......................................................................................457 Where to Go From Here?.................................................................457 Section III: Advanced M e ta l......................................458 Chapter 19: Tessellation & Terrains......................................459 Tessellation..................................................................................... 460 The Starter Project..........................................................................460 The Tessellation Kernel....................................................................465 Multiple Patches.............................................................................472 Tessellation By Distance.................................................................474 Displacement.................................................................................. 479 Shading By Slope.............................................................................485 Challenge........................................................................................ 488 Key Points.......................................................................................489 Where to Go From Here?.................................................................490 raywenderlich.com 11

Metal by Tutorials Chapter 20: Fragment Post-Processing............................... 491 The Starter A pp ..............................................................................492 Alpha Testing.................................................................................. 493 Depth Testing.................................................................................. 495 Stencil Testing.................................................................................495 Scissor Testing.................................................................................496 Alpha Blending............................................................................... 497 Opacity........................................................................................... 498 Blending..........................................................................................499 Antialiasing..................................................................................... 505 Fog...................................................................................................507 Key Points.......................................................................................509 Where to Go From Here?.................................................................509 Chapter 21: Imaged-Based Lighting......................................510 The Starter Project..........................................................................511 The Skybox..................................................................................... 511 Procedural Skies.............................................................................. 518 Reflection.......................................................................................525 Image-Based Lighting..................................................................... 528 Challenge........................................................................................ 542 Key Points.......................................................................................543 Where to Go From Here?.................................................................543 Chapter 22: Reflection & Refraction.................................... 544 The Starter Project..........................................................................545 Rendering Rippling Water............................................................... 546 1. Creating the Water Surface......................................................... 547 2. Rendering the Reflection ........................................................... 549 3. Creating Clipping Planes..............................................................555 4. Rippling Normal Maps.................................................................557 5. Adding Refraction........................................................................ 560 6. The Fresnel Effect....................................................................... 563 raywenderlich.com 12

Metal by Tutorials 7. Adding Smoothness Using a Depth Texture.............................. 565 Key Points.......................................................................................567 Where to Go From Here?.................................................................567 Chapter 23: Anim ation.......................................................... 568 The Starter Project..........................................................................569 Animation.......................................................................................569 Procedural Animation..................................................................... 570 Animation Using Physics.................................................................571 Keyframes.......................................................................................574 Quaternions ................................................................................... 581 USD and USDZ Files........................................................................ 585 Animating Meshes........................................................................... 586 Challenge........................................................................................ 592 Key Points ...................................................................................... 592 Chapter 24: Character Animation.........................................593 Skeletal Animation..........................................................................594 The Starter App ............................................................................. 600 Implementing Skeletal Animation....................................................601 Loading the Animation....................................................................604 The Joint Matrix Palette.................................................................. 605 The Inverse Bind Matrix.................................................................. 607 Updating the Vertex Shader............................................................ 611 Function Specialization....................................................................613 Key Points ...................................................................................... 617 Where to Go From Here?............................................................... 618 Chapter 25: Managing Resources.........................................619 The Starter Project..........................................................................620 Argument Buffers........................................................................... 622 Resource Heaps.............................................................................. 629 Key Points.......................................................................................636 raywenderlich.com 13

Metal by Tutorials Chapter 26: GPU-Driven Rendering.................................... 637 The Starter Project..........................................................................638 Indirect Command Buffers..............................................................639 GPU-Driven Rendering....................................................................651 Challenge........................................................................................ 663 Key Points.......................................................................................663 Where to Go From Here?.................................................................664 Section IV: Ray Tracing................................................ 665 Chapter 27: Rendering W ith Rays.........................................666 Getting Started................................................................................667 Ray Casting..................................................................................... 668 Ray Tracing..................................................................................... 670 Path Tracing.................................................................................... 671 Raymarching...................................................................................673 Signed Distance Functions..............................................................675 The Starter Playground.................................................................. 676 Using a Signed Distance Function....................................................676 The Raymarching Algorithm........................................................... 677 Creating Random Noise.................................................................. 684 Marching Clouds.............................................................................691 Key Points.......................................................................................693 Chapter 28: Advanced Shadows.......................................... 694 The Starter Playground.................................................................. 695 Hard Shadows................................................................................. 695 Soft Shadows...................................................................................703 Ambient Occlusion..........................................................................712 Key Points.......................................................................................721 Where to Go From Here?.................................................................721 Chapter 29: Advanced L ighting............................................722 The Rendering Equation.................................................................. 723 raywenderlich.com 14

Metal by Tutorials Reflection.......................................................................................724 Getting Started................................................................................725 Refraction.......................................................................................731 Raytraced W ater.............................................................................735 Key Points.......................................................................................737 Where to Go From Here?.................................................................737 Chapter 30: Metal Performance Shaders............................ 738 Overview........................................................................................ 739 The Sobel Filter................................................................................739 Image Processing.............................................................................740 The Starter Project..........................................................................744 The Blit Command Encoder............................................................ 746 Gaussian Blur...................................................................................747 Matrix / Vector Mathematics......................................................... 750 Challenge........................................................................................ 754 Key Points.......................................................................................754 Chapter 31: Performance Optimization............................... 755 The Starter A pp .............................................................................. 756 Profiling..........................................................................................756 GPU Workload Capture.................................................................. 758 GPU Timeline...................................................................................764 Instancing.......................................................................................768 Removing Duplicate Textures......................................................... 770 CPU-GPU Synchronization..............................................................771 Key Points.......................................................................................776 Chapter 32: Best Practices....................................................777 General Performance Best Practices............................................... 778 Memory Bandwidth Best Practices.................................................780 Memory Footprint Best Practices....................................................784 Where to Go From Here?.................................................................787 raywenderlich.com 15

Metal by Tutorials Conclusion 788 raywenderlich.com 16

Book License By purchasing Metal by Tutorials, you have the following license: • You are allowed to use and/or modify the source code in Metal by Tutorials in as many apps as you want, with no attribution required. • You are allowed to use and/or modify all art, images and designs that are included in Metal by Tutorials in as many apps as you want, but must include this attribution line somewhere inside your app: “Artwork/images/designs: from Metal by Tutorials, available at www.raywenderlich.com”. • The source code included in Metal by Tutorials is for your personal use only. You are NOT allowed to distribute or sell the source code in Metal by Tutorials without prior authorization. • This book is for your personal use only. You are NOT allowed to sell this book without prior authorization, or distribute it to friends, coworkers or students; they would need to purchase their own copies. All materials provided with this book are provided on an “as is” basis, without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose and noninfringement. In no event shall the authors or copyright holders be liable for any claim, damages or other liability, whether in an action of contract, tort or otherwise, arising from, out of or in connection with the software or the use or other dealings in the software. All trademarks and registered trademarks appearing in this guide are the properties of their respective owners. raywenderlich.com 17

Before You Begin This section tells you a few things you need to know before you get started, such as what you’ll need for hardware and software, where to find the project files for this book and more. raywenderlich.com 18

W hat You Need To follow along with the tutorials in this book, you need the following: • A Metal-capable Mac running macOS Monterey 12.0 or later. All Macs built since 2012 should run Metal, although not all of them will be able to run the most recent features in Metal 2. Nvidia GPUs will have issues, in some cases serious, as drivers have not been updated since macOS High Sierra. One chapter includes Apple GPU-specific code, for which you’ll need either an Ml GPU or a recent iPhone or iPad. • Xcode 13.3 or later. • [optional] A Metal-capable iPhone or iPad running iOS 15 or later. Any iOS device running the A7 chip or later will run Metal. The latest features, such as tile shading and imageblocks, will only run on the Al 1 (or later) chipset. The projects will build and run on macOS, and most of them will run on the iOS Simulator, so using an iOS device is optional. If you wish to make an iOS game, the game engine you build while reading this book will have an iOS target as well. raywenderlich. com 19

Book Source Code & Forums Where to download the materials for this book The materials for this book can be cloned or downloaded from the GitHub book materials repository: • https://github.eom/ravwenderlich/met-materials/tree/editions/5.0 Forums We’ve also set up an official forum for the book at https:// forums.ravwenderlich.eom/c/books/metal-bv-tutorials. This is a great place to ask questions about the book or to submit any errors you may find. raywenderlich. com 20