动手学大语言模型 (Jay Alammar, Maarten Grootendorst)（Z-Library）

【One-Line Pitch】 A hands-on, highly visual guide to understanding and building with large language models, covering everything from tokenization and embeddings to fine-tuning and multimodal systems—ideal for practitioners who want to move from theory to working code without needing a massive GPU budget. 【Book Arc】 - **Opening (~0%–13%)**: Introduces the book's mission, audience, and practical setup. It frames LLMs broadly (including non-generative models) and sets expectations for running examples on modest hardware like Google Colab's free T4 GPU. - **Early (~13%–32%)**: Builds the conceptual foundation of language AI. It traces the evolution from bag-of-words and word2vec to RNNs and the Transformer architecture, explaining key model families (encoder-only like BERT, decoder-only like GPT) and the two-step training paradigm of pretraining and fine-tuning. - **Early–Middle (~32%–42%)**: Covers the practicalities of interacting with LLMs. It discusses proprietary vs. open-source models, API access, and walks through loading a model (Phi-3) and tokenizer with Hugging Face Transformers to generate a first piece of text. - **Middle (~42%–48%)**: Dives into the mechanics of tokenization and embeddings. It shows how a prompt is converted into token IDs, how subword tokenization works (with examples of partial words and special tokens), and why this step is critical for model input and output. - **Late (beyond ~48%)**: The excerpts indicate the book progresses into advanced topics—prompt engineering, semantic search and RAG, multimodal LLMs, and fine-tuning (including SFT, QLoRA, and preference tuning)—though the provided material does not detail these chapters' content. 【Key Takeaways】 - **LLMs are more than just text generators** (Early): The book deliberately includes encoder-only models, embedding models, and even bag-of-words approaches under the "LLM" umbrella, arguing that "large" is arbitrary and capability matters more than size. - **The Transformer is the backbone of modern language AI** (Early): Understanding self-attention, encoder-decoder stacks, and masked attention in decoders explains why models like BERT and GPT work—and why they can be trained in parallel unlike earlier RNNs. - **Tokenization is the hidden first step** (Middle): A prompt is not fed to a model as text but as a sequence of token IDs, where tokens can be whole words, word fragments, or punctuation. Decoding these IDs back to text is essential for interpreting model output. - **Training is a multi-stage process** (Early): Pretraining on massive corpora creates base models, while fine-tuning adapts them to specific tasks or behaviors. This separation saves resources, as pretraining (e.g., Llama 2's 2-trillion-token dataset) is prohibitively expensive for most. - **You can work with LLMs on limited hardware** (Early): The book is written for "GPU-poor" readers, using Google Colab's free T4 (16 GB VRAM) as a baseline and offering local alternatives to paid APIs where possible. - **Responsible use is part of the craft** (Early): Bias, transparency, harmful content, IP questions, and regulation (like the EU AI Act) are not afterthoughts—they shape how and whether you deploy LLM-based systems. - **Hands-on code is the core learning method** (Middle): Using Hugging Face's `transformers` library, you can load a model and tokenizer, create a pipeline, and generate text with just a few lines of Python—making the concepts immediately testable. 【Reading Tips】 - **Skim the historical overview in Chapter 1** if you already know what word2vec and RNNs are; focus instead on the "training paradigms" and "responsible use" sections, which frame the rest of the book. - **Deep-read the tokenization chapter (Chapter 2)**: It's easy to skip, but understanding subword tokens and special tokens (like `<s>` and `<|assistant|>`) is essential for debugging prompts and outputs later. - **Run the code as you go**: The book is designed for hands-on learning. Use the provided Google Colab notebooks to follow along, especially for the first text-generation example with Phi-3. - **Treat the Transformer chapter (Chapter 3) as a reference**: The architecture details (multi-head attention, positional embeddings) are dense; revisit them when you hit later chapters on fine-tuning or multimodal models. - **Don't worry about the latest model names**: The book explicitly aims for model-agnostic examples, so focus on the underlying techniques (e.g., RAG, QLoRA) rather than chasing specific releases. 【Coverage Limits】 This guide is based on excerpts covering roughly the first half of the book (through tokenization). Topics like prompt engineering, RAG, multimodal LLMs, and fine-tuning are mentioned in the table of contents but not detailed in the provided material.