AI guide
【One-Line Pitch】
A practical field guide to keeping distributed systems alive when things go wrong, blending hard-won engineering patterns with lessons from safety-critical industries and human factors. Best for developers, architects, SREs, and technical leaders who already run server-based software and want to move beyond circuit breakers and retries toward genuinely resilient, AI-aware systems.
【Book Arc】
- **Opening (~0%–10%)**: Frames the problem — safeguards exist yet systems still fail — and sets out the book's ambition to merge technical resilience patterns with safety science and human factors, including the new unpredictability of AI components.
- **Early (~10%–30%)**: Lays the foundations in Part I: defining resilience, stability fundamentals, timeouts, retries and idempotency, thundering herds, rate limiting, and a dedicated chapter on AI's opportunities and risks.
- **Middle (~30%–55%)**: Moves into Part II's harder failure modes — machine failure and load balancing, message brokers and delivery guarantees, disaster recovery and cross-site replication, and progressive collapse (small faults cascading into system-wide outages).
- **Late (~55%–85%)**: Extends into sociotechnical territory in Part III — organizational culture, team dynamics, human error, observability, incident learning, and game days — treating people as both a source of instability and the key to resilience.
- **Ending (~85%–100%)**: A concluding chapter that ties the technical and human threads together, reinforced by a glossary, appendix, and bibliography for ongoing reference.
【Key Takeaways】
- **Resilience is a whole-system property, not a bolt-on** (Early): It spans software, hardware, people, and process — you cannot assess it by inspecting code or infrastructure in isolation.
- **Give up deliberately** (Early): Timeouts, retries, and idempotency are the first line of defense; setting them well means balancing user expectations against what the system can actually absorb.
- **Not all load spikes are alike** (Early): Thundering herds can be internal or external, malicious or accidental — even a sign of success — and each shape needs different mitigations, from rate limiting to backpressure and load shedding.
- **Stateful processing makes machine failure painful** (Middle): Load balancing and stateless design keep services alive when individual machines die; message brokers add delivery guarantees but must themselves survive failure.
- **Plan for the data center, not just the server** (Middle): Disaster recovery needs a simple model for defining requirements and a strategy for cross-site replication before an outage forces the question.
- **Progressive collapse is the real catastrophe** (Middle): One small fault can cascade into wider system failure; the goal is graceful degradation, not heroic recovery.
- **AI cuts both ways** (Early): LLMs can help build and operate resilient systems, but embedded AI components introduce unpredictability — and AI's effect on human operators must be weighed too.
- **Culture and learning are resilience infrastructure** (Late): Human error should not automatically mean outage; observability, blameless learning, and game days turn incidents into organizational capability.
【Reading Tips】
- **Read Part I end-to-end** — the timeout/retry/idempotency/thundering-herd material is the conceptual spine and pays off everywhere later.
- **Skim Part II selectively** if you already run brokers or multi-site infrastructure; deep-read the progressive collapse and disaster recovery chapters, which are the book's distinctive contributions.
- **Don't skip Part III** even if you're purely technical — the sociotechnical framing is what separates this book from pattern catalogs.
- **Treat the AI chapter as a decision aid**, not a tutorial: use it to ask the right questions before wiring agents into production.
- **Keep the glossary and bibliography nearby** for follow-up on safety-critical systems literature.
【Coverage Limits】
The excerpts cover the book's structure, chapter map, and core definitions well, but do not include the detailed pattern mechanics, case studies, or worked examples from the middle chapters; this guide reflects the book's architecture rather than its full technical depth.
Passage locations
Excerpt 1
this book belongs within reach of every software architect. Scott Hanselman, VP and member of technical staff, Microsoft/GitHub Building resilient distribute...
View in text
Excerpt 2
book for people who are brand-new to working with software. The technical sections of this book (Parts I and II ) assume that you at least have experience in...
View in text
Excerpt 3
system. This is a phenomenon known as progressive collapse . Drawing on case studies in the built environment, this chapter will look at the lessons we can l...
View in text
Excerpt 4
is ever more important for us to be able to rely on things. Attempting to bring stability to the global economy, relations between nation-states, or the clim...
View in text