Hands-On LLM Serving and Optimization (Chi Wang, Peiheng Hu)（Z-Library）

【One-Line Pitch】 A practical, code-backed guide to deploying and optimizing large language models in production, aimed at engineers who must turn an LLM prototype into a reliable, cost-efficient service. If you own an LLM endpoint and need to reason about latency, throughput, and GPU spend, this book is your systems companion. 【Book Arc】 - **Opening (~0%–10%)**: Frames why LLM serving is its own discipline—autoregressive, stateful, prefill/decode phases, dominant inference cost—and why classic ML deployment thinking breaks down. - **Early (~10%–30%)**: Defines the audience, prerequisites, and scope; walks through model anatomy (data, architecture, execution code) and the ML lifecycle from training to serving. - **Middle (~30%–55%)**: Explains LLM execution mechanics (attention, prefill, decoding), core serving metrics, and how to design single-model and multi-model serving systems from scratch. - **Late (~55%–80%)**: Moves into best practices and optimization—agent and RAG pipelines, enterprise architectures, buy vs. self-host decisions, then continuous batching, quantization, kernel fusion, and prefix caching. - **Ending (~80%–100%)**: Covers advanced techniques (speculative decoding, multi-GPU parallelization, prefill-decode disaggregation, KV cache management) and surveys frameworks like vLLM, TensorRT-LLM, SGLang, and llama. 【Key Takeaways】 - **LLMs are not just bigger ML models** (Opening): Their stateful, autoregressive nature and split prefill/decode phases create new physics, economics, and stakes that demand dedicated serving strategies. - **Serving is the product experience, not backstage infrastructure** (Opening): Because LLMs sit inside interactive UX, latency is visible, streaming is mandatory, and reliability directly drives trust and revenue. - **Inference cost dominates** (Opening): GPU memory and scheduling efficiency translate directly into dollars, making optimization a business concern, not just an engineering one. - **Models are executable programs, not static data** (Middle): Treating a model as architecture + execution code + data clarifies packaging, versioning, and deployment decisions. - **Serving sits late in the ML lifecycle but determines realized value** (Middle): Training only pays off once a model is deployed, served reliably, and operated at scale. - **Optimization has a layered toolkit** (Late): Continuous batching, quantization, kernel fusion, and prefix caching are broadly applicable; speculative decoding, multi-GPU parallelism, and prefill-decode disaggregation are advanced levers. - **Framework choice matters** (Ending): vLLM, TensorRT-LLM, SGLang, and llama each embody different trade-offs; understanding why specialized frameworks exist guides selection. - **Buy vs. self-host is a first-class decision** (Late): Public APIs, cloud endpoints, serving libraries, and self-managed services each carry distinct cost, safety, and control implications. 【Reading Tips】 - **Deep-read Chapters 2–3 and 6–7**: Execution mechanics, system design, and optimization techniques are the book's core; skim the intro chapters if you already know the LLM landscape. - **Run the code**: The hands-on value comes from experimenting with batch sizes, model sizes, sequence lengths, and schedules—then measuring the differences yourself. - **Pair with a general LLM text**: If transformers or attention are unfamiliar, read a foundational book alongside this one and treat this as your serving companion. - **Focus on trade-offs, not recipes**: The durable skill is reasoning about latency, throughput, and cost under changing workloads and evolving frameworks. - **Use the framework survey as a map, not a verdict**: Chapter 8 helps you evaluate options; expect the landscape to shift and revisit your choices. 【Coverage Limits】 This guide is based on stratified excerpts covering the front matter, introduction, and early-to-middle chapters; later chapters on advanced optimization and framework surveys are summarized from the book's own chapter outline rather than detailed excerpt content.