Designing Data-Intensive Applications, 2nd Edition (Martin Kleppmann, Chris Riccomini)（Z-Library）

【One-Line Pitch】 A definitive guide to the principles and trade-offs behind modern data systems, this book helps engineers and architects choose, design, and operate reliable, scalable, and maintainable databases and data pipelines. If you want to move beyond buzzwords and understand the "why" behind tools like Postgres, Kafka, and Spark, this is your map. 【Book Arc】 - **Opening (~0%–9%)**: The book sets the stage by framing data systems as a set of trade-offs between reliability, scalability, and maintainability. It introduces the core vocabulary and the central distinction between systems of record (primary databases) and derived data systems (indexes, caches), establishing a mental model for the rest of the book. - **Early (~9%–25%)**: This section dives into the fundamentals of distributed systems, contrasting cloud computing with supercomputing and detailing the realities of hardware faults. It covers the importance of redundancy and fault tolerance, using concrete examples like Twitter's timeline to illustrate the nonfunctional requirements that drive system design. - **Early (~25%–34%)**: The focus shifts to data models and query languages. It explores the relational model versus the document model, discussing schema flexibility, locality, and the challenges of joins. This part also introduces object-relational mapping (ORM) and its pitfalls, such as the N+1 query problem, helping you choose the right model for your data. - **Middle (~34%–47%)**: This stage deepens the exploration of data modeling, covering advanced topics like graph data models (Property Graphs, Cypher, SPARQL) and their query languages. It also introduces event sourcing and CQRS, explaining how to use immutable event logs as a source of truth and derive read-optimized views from them. - **Middle (~47%–53%)**: The book transitions to storage and retrieval, examining the internals of storage engines. It covers log-structured storage, B-Trees, and LSM-Trees, comparing their trade-offs for OLTP workloads. It also touches on embedded storage engines like RocksDB and SQLite, and begins to discuss column-oriented storage for analytics. 【Key Takeaways】 - **Systems of record vs. derived data** (Early): The distinction between where data is first written and where it is transformed for reads is crucial. Understanding this clarifies system architecture and helps you design data pipelines for integration. - **Hardware faults are normal, not exceptional** (Early): At scale, CPU errors, RAM corruption, and even datacenter outages are expected. The book argues for designing systems that tolerate these faults through redundancy and distributed architectures, rather than relying on the reliability of individual machines. - **Cloud vs. supercomputing are different worlds** (Early): Cloud systems prioritize high availability and multi-tenant security, while supercomputers focus on raw performance for batch jobs. This distinction shapes network topologies, fault-handling strategies, and communication methods. - **Document vs. relational models have real trade-offs** (Middle): Document models offer schema flexibility and locality, but relational models excel at joins and many-to-many relationships. The choice depends on your data's structure and access patterns, not on hype. - **ORMs are a double-edged sword** (Middle): While they reduce boilerplate, ORMs can hide complexity, generate inefficient queries (like the N+1 problem), and struggle with non-relational systems. Understanding the underlying data model is still essential. - **Event sourcing and CQRS are powerful patterns** (Middle): Using an immutable event log as the source of truth and deriving materialized views (CQRS) allows for flexible, auditable systems. This pattern is rooted in state machine replication and is a key approach for building scalable, event-driven architectures. - **Storage engines are built on fundamental structures** (Middle): Log-structured storage and B-Trees are the two main families for OLTP. Their trade-offs in write amplification, read performance, and compaction strategies are central to database performance. 【Reading Tips】 - **Skim the opening chapters** (~0%–9%) for the core vocabulary and the systems-of-record concept; they are foundational but can be read quickly. - **Deep-read the data model chapters** (~25%–47%) as they are the heart of the book. Pay close attention to the graph model and event sourcing sections, as they are conceptually dense but highly applicable. - **Focus on the "why" in the storage chapter** (~47%+). You don't need to memorize B-Tree internals, but understanding the trade-offs between LSM-Trees and B-Trees will help you choose databases in practice. - **Treat the book as a reference, not a novel**. Jump to chapters that address your current problems (e.g., consistency, replication, or pipelines) rather than reading linearly. - **Take away the decision frameworks**, not the specific technologies. The book's value is in teaching you how to evaluate trade-offs, not in recommending a specific tool. 【Coverage Limits】 This guide is based on excerpts covering roughly the first half of the book (up to ~53%). It does not cover later chapters on replication, partitioning, transactions, consensus, or batch/stream processing, which are essential for a complete understanding of distributed data systems.