Building Recommendation Systems in Python and JAX Hands-On Production Systems at Scale (Bryan Bischof, Hector Yee) (Z-Library)

📄 File Format: PDF

💾 File Size: 10.3 MB

📖 Read Online ⬇️ Download

Registered users can read the full content for free

Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.

📄 Page 1

(This page has no text content)

📄 Page 2

MACHINE LE ARNING “A must read for practitioners aspiring to build real-world systems, not just train models.” —Jacopo Tagliabue Co-founder of Bauplan, co-creator of RecList and evalRS “This book will sit on my bookshelf as the standard reference on this topic from now on.” —Will Kurt AI engineer and author of Bayesian Statistics the Fun Way and Get Programming with Haskell Building Recommendation Systems in Python and JAX Twitter: @oreillymedia linkedin.com/company/oreilly-media youtube.com/oreillymedia Implementing and designing systems that make suggestions to users are among the most popular and essential machine learning applications available. Whether you want customers to find the most appealing items at your online store, videos to enrich and entertain them, or news they need to know, recommendation systems (RecSys) provide the way. In this practical book, authors Bryan Bischof and Hector Yee illustrate core concepts and provide examples to help you create a RecSys for any industry or scale. You’ll learn the math, ideas, and implementation details you need to succeed. This book includes the RecSys platform components, relevant MLOps tools in your stack, plus code examples and helpful suggestions in PySpark, SparkSQL, FastAPI, and Weights & Biases. You’ll learn: • The data essential for building a RecSys • How to frame your data and business as a RecSys problem • Ways to evaluate models appropriate for your system • Methods to implement, train, test, and deploy the model you choose • Metrics you need to track to ensure your system is working as planned • How to improve your system as you learn more about your users, products, and business case Bryan Bischof leads AI at Hex, and is an adjunct professor in the Rutgers Masters of Business and Analytics program where he teaches Data Science. Previously, he was the Head of Data Science at Weights and Biases, where he built the DS, ML, and Data Engineering teams. Hector Yee is a staff software engineer at Google who’s worked on projects that include the first content-based ranker on image search and self-driving car perception. 9 7 8 1 4 9 2 0 9 7 9 9 0 5 7 9 9 9 US $79.99 CAN $99.99 ISBN: 978-1-492-09799-0

📄 Page 3

Praise for Building Recommendation Systems in Python and JAX Bryan and Hector have created something special here, introducing concepts that take most people years to learn within the RecSys domain and then providing clear code examples that put them into practice. I wish I had this book when I started out on my RecSys journey. —Even Oldridge, Director of Engineering, Recommender Systems, NVIDIA This is a book I’ve been waiting for, making recommendation systems accessible using JAX. The only other thing you need is your laptop. —Shaked Zychlinski, former Head of Recommendations at Lightricks Bryan and Hector have distilled decades of recommendation system advancements into a concise, yet practical guide. Bridging the gap between theory and application, this book is packed with easy-to-understand Python and JAX examples. This is an indispensable guide for RecSys practitioners at all levels, from novices to experts. —Eugene Yan, Applied Scientist, Amazon This book takes a holistic approach to building recommender systems, synthesizing math, code, systems design, and business application. It covers all the nuances that practitioners need to consider to implement real-world solutions. The intuitive examples using publicly available datasets enables the reader to turn abstract concepts into concrete learnings. —Eric Colson, AI Advisor, Former Chief Algorithms Officer at Stitch Fix, Former VP of Data Science and Engineering at Netflix

📄 Page 4

Recommender systems are among the most impactful ML systems ever deployed. This book brilliantly navigates the balance between principled modeling, clear code examples, and architectural best practices. A must read for practitioners aspiring to build real-world systems, not just train models. —Jacopo Tagliabue, Co-founder of Bauplan, Adjunct Professor of ML Systems at NYU, Co-creator of RecList and evalRS For years I’ve found there is a tremendous gap between recommendation systems as described in texts and as practiced in the field. Yee and Bischof ’s excellent Building Recommendation Systems in Python and JAX closes this gap and will make readers finally feel initiated into this vital area of data science. —Will Kurt, AI Engineer and author of Bayesian Statistics the Fun Way and Get Programming with Haskell This book is an essential resource for anyone interested in the information retrieval (IR) space. The authors take special care to do the incredibly important and nuanced work of preparing the reader to solve problems in this space. With this book as a reference, you will be able to think through how to set up the IR problem, think through the practical steps to take, and then get building. —Eric Schles, Research Scientist, Johns Hopkins University

📄 Page 5

Bryan Bischof and Hector Yee Building Recommendation Systems in Python and JAX Hands-on Production Systems at Scale Boston Farnham Sebastopol TokyoBeijing

📄 Page 6

978-1-492-09799-0 [LSI] Building Recommendation Systems in Python and JAX by Bryan Bischof and Hector Yee Copyright © 2024 Bryan Bischof and Resonant Intelligence LLC. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com. Acquisitions Editor: Nicole Butterfield Development Editor: Jill Leonard Production Editor: Aleeya Rahman Copyeditor: Sharon Wilkey Proofreader: Piper Editorial Consulting, LLC Indexer: Judith McConville Interior Designer: David Futato Cover Designer: Karen Montgomery Illustrator: Kate Dullea December 2023: First Edition Revision History for the First Edition 2023-12-04: First Release See http://oreilly.com/catalog/errata.csp?isbn=9781492097990 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Building Recommendation Systems in Python and JAX, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the authors and do not represent the publisher’s views. While the publisher and the authors have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

📄 Page 7

Table of Contents Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii Part I. Warming Up 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Key Components of a Recommendation System 4 Collector 4 Ranker 4 Server 4 Simplest Possible Recommenders 5 The Trivial Recommender 5 Most-Popular-Item Recommender 6 A Gentle Introduction to JAX 7 Basic Types, Initialization, and Immutability 7 Indexing and Slicing 9 Broadcasting 9 Random Numbers 10 Just-in-Time Compilation 11 Summary 12 2. User-Item Ratings and Framing the Problem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 The User-Item Matrix 13 User-User Versus Item-Item Collaborative Filtering 16 The Netflix Challenge 17 Soft Ratings 19 Data Collection and User Logging 19 What to Log 20 v

📄 Page 8

Collection and Instrumentation 23 Funnels 24 Business Insight and What People Like 26 Summary 27 3. Mathematical Considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 Zipf ’s Laws in RecSys and the Matthew Effect 29 Sparsity 32 User Similarity for Collaborative Filtering 34 Pearson Correlation 35 Ratings via Similarity 36 Explore-Exploit as a Recommendation System 37 ϵ-greedy 38 What Should ϵ Be? 39 The NLP-RecSys Relationship 40 Vector Search 40 Nearest-Neighbors Search 42 Summary 42 4. System Design for Recommending. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 Online Versus Offline 43 Collector 44 Offline Collector 44 Online Collector 45 Ranker 45 Offline Ranker 46 Online Ranker 46 Server 46 Offline Server 47 Online Server 47 Summary 47 5. Putting It All Together: Content-Based Recommender. . . . . . . . . . . . . . . . . . . . . . . . . . 49 Revision Control Software 50 Python Build Systems 51 Random-Item Recommender 52 Obtaining the STL Dataset Images 54 Convolutional Neural Network Definition 55 Model Training in JAX, Flax, and Optax 56 Input Pipeline 58 Summary 70 vi | Table of Contents

📄 Page 9

Part II. Retrieval 6. Data Processing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 Hydrating Your System 73 PySpark 73 Example: User Similarity in PySpark 77 DataLoaders 82 Database Snapshots 84 Data Structures for Learning and Inference 85 Vector Search 86 Approximate Nearest Neighbors 87 Bloom Filters 88 Fun Aside: Bloom Filters as the Recommendation System 89 Feature Stores 90 Summary 94 7. Serving Models and Architectures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 Architectures by Recommendation Structure 95 Item-to-User Recommendations 96 Query-Based Recommendations 96 Context-Based Recommendations 98 Sequence-Based Recommendations 99 Why Bother with Extra Features? 99 Encoder Architectures and Cold Starting 100 Deployment 103 Models as APIs 103 Spinning Up a Model Service 104 Workflow Orchestration 105 Alerting and Monitoring 108 Schemas and Priors 108 Integration Tests 109 Observability 110 Evaluation in Production 111 Slow Feedback 111 Model Metrics 112 Continuous Training and Deployment 113 Model Drift 113 Deployment Topologies 114 The Evaluation Flywheel 117 Daily Warm Starts 117 Lambda Architecture and Orchestration 118 Logging 119 Table of Contents | vii

📄 Page 10

Active Learning 122 Summary 125 8. Putting It All Together: Data Processing and Counting Recommender. . . . . . . . . . . . 127 Tech Stack 128 Data Representation 129 Big Data Frameworks 130 Cluster Frameworks 132 PySpark Example 132 GloVE Model Definition 142 GloVE Model Specification in JAX and Flax 143 GloVE Model Training with Optax 145 Summary 146 Part III. Ranking 9. Feature-Based and Counting-Based Recommendations. . . . . . . . . . . . . . . . . . . . . . . . 149 Bilinear Factor Models (Metric Learning) 149 Feature-Based Warm Starting 153 Segmentation Models and Hybrids 155 Tag-Based Recommenders 155 Hybridization 157 Limitations of Bilinear Models 157 Counting Recommenders 159 Return to the Most-Popular-Item Recommender 159 Correlation Mining 160 Pointwise Mutual Information via Co-occurrences 162 Similarity from Co-occurrence 163 Similarity-Based Recommendations 164 Summary 165 10. Low-Rank Methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 Latent Spaces 167 Dot Product Similarity 169 Co-occurrence Models 171 Reducing the Rank of a Recommender Problem 172 Optimizing for MF with ALS 174 Regularization for MF 176 Regularized MF Implementation 177 WSABIE 199 Dimension Reduction 199 viii | Table of Contents

📄 Page 11

Isometric Embeddings 203 Nonlinear Locally Metrizable Embeddings 204 Centered Kernel Alignment 206 Affinity and p-sale 206 Propensity Weighting for Recommendation System Evaluation 208 Propensity 209 Simpson’s and Mitigating Confounding 210 Summary 212 11. Personalized Recommendation Metrics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213 Environments 214 Online and Offline 214 User Versus Item Metrics 215 A/B Testing 215 Recall and Precision 216 @ k 218 Precision at k 219 Recall at k 219 R-precision 220 mAP, MMR, NDCG 220 mAP 220 MRR 221 NDCG 222 mAP Versus NDCG? 223 Correlation Coefficients 223 RMSE from Affinity 224 Integral Forms: AUC and cAUC 224 Recommendation Probabilities to AUC-ROC 225 Comparison to Other Metrics 225 BPR 226 Summary 227 12. Training for Ranking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 Where Does Ranking Fit in Recommender Systems? 229 Learning to Rank 230 Training an LTR Model 231 Classification for Ranking 231 Regression for Ranking 231 Classification and Regression for Ranking 232 WARP 233 k-order Statistic 234 BM25 236 Table of Contents | ix

📄 Page 12

Multimodal Retrieval 238 Summary 238 13. Putting It All Together: Experimenting and Ranking. . . . . . . . . . . . . . . . . . . . . . . . . . . 241 Experimentation Tips 241 Keep It Simple 242 Debug Print Statements 242 Defer Optimization 243 Keep Track of Changes 244 Use Feature Engineering 244 Understand Metrics Versus Business Metrics 245 Perform Rapid Iteration 245 Spotify Million Playlist Dataset 246 Building URI Dictionaries 248 Building the Training Data 249 Reading the Input 252 Modeling the Problem 254 Framing the Loss Function 257 Exercises 261 Summary 262 Part IV. Serving 14. Business Logic. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265 Hard Ranking 266 Learned Avoids 266 Hand-Tuned Weights 267 Inventory Health 268 Implementing Avoids 269 Model-Based Avoids 271 Summary 272 15. Bias in Recommendation Systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 Diversification of Recommendations 274 Improving Diversity 274 Applying Portfolio Optimization 276 Multiobjective Functions 277 Predicate Pushdown 278 Fairness 279 Summary 280 x | Table of Contents

📄 Page 13

16. Acceleration Structures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281 Sharding 282 Locality Sensitive Hashing 282 k-d Trees 284 Hierarchical k-means 288 Cheaper Retrieval Methods 290 Summary 290 Part V. The Future of Recs 17. Sequential Recommenders. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293 Markov Chains 294 Order-Two Markov Chain 295 Other Markov Models 295 RNN and CNN Architectures 297 Attention Architectures 299 Self-Attentive Sequential Recommendation 300 BERT4Rec 300 Recency Sampling 301 Merging Static and Sequential 301 Summary 303 18. What’s Next for Recs?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305 Multimodal Recommendations 306 Graph-Based Recommenders 307 Neural Message Passing 308 Applications 310 Random Walks 311 Metapath and Heterogeneity 313 LLM Applications 313 LLM Recommenders 314 LLM Training 314 Instruct Tuning for Recommendations 317 LLM Rankers 317 Recommendations for AI 318 Summary 319 Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321 Table of Contents | xi

📄 Page 14

(This page has no text content)

📄 Page 15

Preface How did you come to find this book? Did you see an ad for it on a website? Maybe a friend or mentor suggested it; or perhaps you saw a post on social media that referenced it. Could it be that you found it sitting on a shelf in a bookstore—a bookstore that your trusty maps app led you to? However you came to find it, you’ve almost certainly come to this book via a recommendation system. Implementing and designing systems that provide suggestions to users is among the most popular and most essential applications of machine learning (ML) to any business. Whether you want to help your users find the best clothing to match their tastes, the most appealing items to buy from an online store, videos to enrich and entertain them, maximally engaging content to surface from their networks, or the news highlights they need to know on that day, recommendation systems provide the way. Modern recommendation system designs are as diverse as the domains they serve. These systems consist of the computer software architectures to implement and execute product goals, in addition to the algorithmic components of ranking. Meth‐ ods for ranking recommendations can come from traditional statistical learning algorithms, linear-algebraic inspirations, geometric considerations, and, of course, gradient-based methods. Just as the algorithmic methods are diverse, so too are the modeling and evaluation considerations for recommending: personalized ranking, search recommendations, sequence modeling, and the scoring for all of these are now need-to-know for the ML engineer working with recommendation systems. The abbreviation RecSys is often used by practitioners to describe the field of recommendation systems. Therefore, in this book, we use RecSys when referring to the field, and recommendation sys‐ tem when referring to what we build. xiii

📄 Page 16

1 Some may quibble that Apple also has core recommendation systems at the heart of its company. While it’s certainly true that the App Store forms a crucial strategic product for the company, we remain conservative in our four-out-of-five assessment and say that recommendation systems are not Apple’s primary revenue- generating capability. If you’re an ML practitioner, you are probably aware of recommendation systems, and you may know one or two of the simplest modeling approaches and be able to speak intelligently about the relevant data structures and model architectures; however, RecSys frequently falls outside the core curriculum of data science and ML. Many senior data scientists with years of experience in the industry know little about actually building a recommendation system and may feel intimidated when the topic comes up. Despite drawing on similar foundations and skills as other ML problems, RecSys has a vibrant community with a fast-moving focus that can make it easy to relegate building recommendation systems to other data scientists who have already invested the time, or are willing to stay on top of the latest information. The reason this book exists, is to break through those perceived barriers. Understand‐ ing recommendation systems at a practical level is not only useful for business cases requiring content to be served to users, but the underlying ideas of RecSys often bridge gaps between an incredibly diverse set of other types of ML. Take, for exam‐ ple, an article recommendation system that may utilize natural language processing (NLP) to find representations of the articles, sequential modeling to promote longer engagement, and contextual components to allow user queries to guide results. If you’re approaching the field from a purely academic interest, no matter what aspects of mathematics you’re interested in, sooner or later, there appears a link or applica‐ tion in RecSys! Finally, if connections to other fields, applications of nearly all of mathematics, or the obvious business utility aren’t enough to get you interested in RecSys, the stunning cutting-edge technology might: RecSys is at and beyond the forefront of ML at all times. One benefit of having obvious revenue impact is that companies and practitioners need to always be pushing the boundaries of what is possible and how they go about it. The most advanced deep learning architectures and best code infrastructures are brought to bear on this field. That’s hardly a surprise when you consider that at the heart of four of the five letters in FAANG—which stands for Meta (formerly Facebook), Apple, Amazon, Netflix, and Google—lies one or many recommendation systems.1 As a practitioner, you’ll need to understand how to do the following: • Take your data and business problem and frame it as a RecSys problem • Identify the essential data to get started building a RecSys xiv | Preface

📄 Page 17

• Determine the appropriate models for your RecSys problem and how should you evaluate them. • Implement, train, test, and deploy the aforementioned models • Track metrics to ensure that your system is working as planned • Incrementally improve your system as you learn more about your users, prod‐ ucts, and business case This book illustrates the core concepts and examples necessary to complete these steps, whatever the industry or scale. We’ll guide you through the math, ideas, and implementation details for building recommendation systems—whether it’s your first or your fiftieth. We’ll show you how to build these systems with Python and JAX. If you’re not yet familiar, JAX is a Python framework from Google that seeks to make autodifferentiation and functional programming paradigms first-class objects. Additionally, it uses a NumPy API style especially convenient for ML practitioners from a variety of backgrounds. We will show code examples and architecture models that capture the essential concepts necessary and provide the way to scale these systems to production applica‐ tions. Conventions Used in This Book The following typographical conventions are used in this book: Italic Indicates new terms, URLs, email addresses, filenames, and file extensions. Constant width Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords. Constant width bold Shows commands or other text that should be typed literally by the user. Constant width italic Shows text that should be replaced with user-supplied values or by values deter‐ mined by context. This element signifies a tip or suggestion. Preface | xv

📄 Page 18

This element signifies a general note. This element indicates a warning or caution. Using Code Examples The included code snippets reference notebooks that will run on moderate-size and, in most cases, free resources. To facilitate easy experimentation and exploration we provide the code via Google Colab notebooks. Supplemental material (code examples, exercises, etc.) is available for download at ESRecsys on GitHub. If you have a technical question or a problem using the code examples, please send email to bookquestions@oreilly.com. This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission. We appreciate, but generally do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Building Recommenda‐ tion Systems in Python and JAX by Bryan Bischof and Hector Yee. Copyright 2024 Bryan Bischof and Resonant Intelligence LLC, 978-1-492-09799-0.” If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at permissions@oreilly.com. xvi | Preface

📄 Page 19

O’Reilly Online Learning For more than 40 years, O’Reilly Media has provided technol‐ ogy and business training, knowledge, and insight to help companies succeed. Our unique network of experts and innovators share their knowledge and expertise through books, articles, and our online learning platform. O’Reilly’s online learning platform gives you on-demand access to live training courses, in-depth learning paths, interactive coding environments, and a vast collection of text and video from O’Reilly and 200+ other publishers. For more information, visit http://oreilly.com. How to Contact Us Please address comments and questions concerning this book to the publisher: O’Reilly Media, Inc. 1005 Gravenstein Highway North Sebastopol, CA 95472 800-889-8969 (in the United States or Canada) 707-829-7019 (international or local) 707-829-0104 (fax) support@oreilly.com https://www.oreilly.com/about/contact.html We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at https://oreil.ly/build_rec_sys_python_jax. For news and information about our books and courses, visit https://oreilly.com. Find us on LinkedIn: https://linkedin.com/company/oreilly-media Follow us on Twitter: https://twitter.com/oreillymedia Watch us on YouTube: https://youtube.com/oreillymedia Acknowledgments Hector would like to thank his husband, Donald, for his loving support during the writing of this book and for the snacks his sister Serena sends all the time. He would also like to dedicate this book to his relatives who have passed. A big thank you goes to the Google reviewers Ed Chi, Courtney Hohne, Sally Goldman, Richa Nigam, Mingliang Jiang, and Anselm Levskaya. Thanks to Bryan Hughes for reviewing the Wikipedia code. Preface | xvii

📄 Page 20

Bryan would like to thank his colleagues from Stitch Fix, where he learned many of the key ideas in this book—in particular, Ian Horn’s patient guidance on transfer learning, Dr. Molly Davies’s mentorship on experimentation and effect estimates, Mark Weiss’s deep partnership on understanding the relationship between availability and recommendations, Dr. Reza Sohrabi’s introduction to transformers, Dr. Xi Chen’s encouragement on GNNs for recs, and Dr. Leland McInnes for his careful advice on dimension reduction and approximate nearest neighbors. Bryan benefitted a lot from conversations with Dr. Natalia Gardiol, Dr. Daniel Fleischman, Dr. Andrew Ho, Jason Liu, Dr. Dan Marthaler, Dr. Chris Moody, Oz Raza, Dr. Anna Schneider, Ujjwal Sarin, Agnieszka Szefer, Dr. Daniel Tasse, Diyang Tang, Zach Winston, and others he has almost certainly forgotten. Outside of his incredible Stitch Fix colleagues, he especially wants to thank Dr. Eric Bunch, Dr. Lee Goerl, Dr. Will Chernoff, Leo Rosenberg, and Janu Verma for collaboration over the years. Dr. Brian Amadio as an excellent colleague and originally suggested that he write this book. Dr. Even Oldridge for encouraging him to actually try it. Eugene Yan and Karl Higley—neither of whom he’s met but has been significantly inspired by. He’d like to thank Dr. Zhongzhu Lin and Dr. Alexander Rosenberg, who both had formative impacts on his career. Cianna Salvatora, who assisted in early literature review, and Valentina Besprozvannykh, who greatly assisted in reading early draft notes and providing guidance. Both authors thank Tobias Zwingmann, Ted Dunning, Vicki Boykis, Eric Schles, Shaked Zychlinski, and Will Kurt, who spend much time giving careful technical feedback on book manuscripts—without which this book would have been incom‐ prehensible. Rebecca Novack, who harangued us into signing up for this project. And Jill Leonard, who removed nearly 100 erroneous instances of the word utilize from the manuscript, and who offered an incredible amount of patient partnership on the book text. xviii | Preface