The software architecture landscape has evolved dramatically over the past decade. Microservices have displaced monoliths. Data and applications are increasingly becoming distributed and decentralised. But composing disparate systems is a hard problem. More recently, software practitioners have been rapidly converging on event-driven architecture as a sustainable way of dealing with complexity — integrating systems without increasing their coupling. In Effective Kafka, Emil Koutanov explores the fundamentals of Event-Driven Architecture — using Apache Kafka — the world's most popular and supported open-source event streaming platform. You'll learn: • The fundamentals of event-driven architecture and event streaming platforms • The background and rationale behind Apache Kafka, its numerous potential uses and applications • The architecture and core concepts — the underlying software components, partitioning and parallelism, load-balancing, record ordering and consistency modes • Installation of Kafka and related tooling — using standalone deployments, clusters, and containerised deployments with Docker • Using CLI tools to interact with and administer Kafka classes, as well as publishing data and browsing topics • Using third-party web-based tools for monitoring a cluster and gaining insights into the event streams • Building stream processing applications in Java 11 using off-the-shelf client libraries • Patterns and best-practice for organising the application architecture, with emphasis on maintainability and testability of the resulting code • The numerous gotchas that lurk in Kafka's client and broker configuration, and how to counter them • Theoretical background on distributed and concurrent computing, exploring factors affecting their liveness and safety • Best-practices for running multi-tenanted clusters across diverse engineering teams, how teams collaborate to build complex systems at scale and equitably share the cluster with the aid of quotas • Operational aspects of running Kafka clusters at scale, performance tuning and methods for optimising network and storage utilisation • All aspects of Kafka security —including network segregation, encryption, certificates, authentication and authorization. The coverage is progressively delivered and carefully aimed at giving you a journey-like experience into becoming proficient with Apache Kafka and Event-Driven Architecture. The goal is to get you designing and building applications. And by the conclusion of this book, you will be a confident practitioner and a Kafka evangelist within your organisation — wielding the knowledge necessary to teach others.