AI Engineering Building Applications with Foundation Models (Chip Huyen) (Z-Library)

# AI Engineering: Building Applications with Foundation Models ## 【One-Line Pitch】 A practical, end-to-end guide for engineers and technical product managers who want to build real AI applications on foundation models—covering everything from choosing and adapting models to optimizing inference and shipping production systems. ## 【Book Arc】 - **Opening (~0%–10%)**: Introduces the AI engineering landscape, contrasting it with traditional ML engineering, and lays out the book's structure—from understanding foundation models through building complete applications. - **Early (~10%–23%)**: Explores why foundation models matter for creative and enterprise tasks, discusses human-in-the-loop design patterns (Crawl-Walk-Run framework), and addresses product defensibility when building on top of AI. - **Early (~23%–32%)**: Compares prompt engineering versus finetuning approaches, then dives into model development layers—training, dataset engineering, and inference optimization—as distinct responsibilities. - **Middle (~32%–42%)**: Explains what makes foundation models work: training data distribution, transformer architecture dominance, model sizing decisions, and how parameter counts translate to compute and memory requirements. - **Middle (~42%–48%)**: Covers advanced model design topics including mixture-of-experts (MoE) architectures, the impact of AI-generated training data, and the value of proprietary data for differentiation. - **Late (~48%–end)**: Moves into practical generation techniques—sampling strategies like top-p and min-p, stopping conditions—then transitions into finetuning, dataset engineering, and inference optimization before culminating in end-to-end application building. ## 【Key Takeaways】 - **Tokens are the fundamental unit of language models** (Early): They balance vocabulary size and meaning retention, allowing models to process unknown words by breaking them into meaningful components like "chatgpt" into "chatgpt" + "ing." - **Two model families serve different purposes** (Early): Masked language models (like BERT) excel at understanding tasks—sentiment analysis, classification, code debugging—while autoregressive models generate text by predicting the next token sequentially. - **Human-in-the-loop is a graduated spectrum** (Early): The Crawl-Walk-Run framework lets you start with mandatory human review, progress to AI interacting with internal employees, and finally allow direct AI-user interaction as quality improves—measured by metrics like acceptance rates. - **Prompt engineering and finetuning are complementary, not competing** (Early): Prompt engineering is easier, requires less data, and enables rapid experimentation, but finetuning unlocks capabilities impossible without weight changes—like adapting to entirely new tasks. - **Model size is only meaningful relative to training data** (Middle): A 13B-parameter model trained on a single sentence will underperform a much smaller model trained on more data; newer models like Llama 3-8B can outperform older, larger models like Llama 2-70B. - **Mixture-of-experts decouples parameter count from compute cost** (Middle): Mixtral 8x7B has 46.7B total parameters but only activates 12.9B per token, meaning its cost and speed match a much smaller model while retaining greater capacity. - **Sampling strategies shape output quality** (Late): Top-p sampling focuses on the most contextually relevant tokens and has proven effective in practice despite theoretical ambiguity; stopping conditions help control latency and cost but risk malformed outputs like truncated JSON. ## 【Reading Tips】 - **Skim Chapter 1's application examples** (Early): The creative and enterprise use cases are illustrative but not technical—focus instead on the human-in-the-loop framework and defensibility discussion, which have lasting strategic value. - **Deep-read the model architecture sections** (Middle): The transformer discussion, parameter-to-memory calculations, and MoE explanations are foundational for making informed model choices—don't rush through these. - **Pay special attention to the prompt engineering vs. finetuning comparison** (Early): This decision framework will guide your entire project approach; understand the trade-offs before diving into implementation details. - **Treat the finetuning and dataset engineering chapters as a unit** (Late): They're deeply interconnected—data quality directly determines finetuning success, so read them together rather than in isolation. - **Keep the end-to-end application chapter in mind throughout** (Late): The book explicitly builds toward this final synthesis, so note how earlier concepts connect as you read rather than treating chapters as standalone topics. ## 【Coverage Limits】 This guide covers the book's opening through the sampling strategies section (~48% of the book). The later chapters on finetuning techniques, dataset engineering, inference optimization, and end-to-end application building are referenced but not detailed here. ##