Event_driven
Chapter 13: Event-Driven Architecture
Section titled “Chapter 13: Event-Driven Architecture”Building Reactive Systems with Events
Section titled “Building Reactive Systems with Events”13.1 What is Event-Driven Architecture?
Section titled “13.1 What is Event-Driven Architecture?”Event-Driven Architecture (EDA) is a software architecture paradigm where services communicate by producing and consuming events rather than making direct synchronous calls.
Request-Driven vs Event-Driven =============================
REQUEST-DRIVEN: EVENT-DRIVEN:
Client Client | | v v Service A Service A (Producer) | | | Call | Publish Event v v Service B Message Broker | | | Call | Consume Event v v Service C Service B (Consumer) | v Service C
Tight coupling Loose coupling Synchronous Asynchronous13.2 Event Concepts
Section titled “13.2 Event Concepts”Types of Events
Section titled “Types of Events”| Event Type | Description | Example |
|---|---|---|
| Notification | Something happened | ”User signed up” |
| Command | Request an action | ”Process order” |
| Document | Data snapshot | ”Order created” |
Event Structure
Section titled “Event Structure” Event Message Format ===================
{ "eventId": "uuid-12345", "eventType": "OrderCreated", "timestamp": "2024-01-15T10:30:00Z", "producer": "order-service", "data": { "orderId": "order-789", "userId": "user-456", "amount": 99.99, "items": [...] } }13.3 Event Patterns
Section titled “13.3 Event Patterns”13.3.1 Pub/Sub (Publish-Subscribe)
Section titled “13.3.1 Pub/Sub (Publish-Subscribe)” Pub/Sub Pattern ===============
Producer Topic Consumers +--------+ +--------+ +--------+ | Order | ----------->| orders | --------->|Email | |Service | +--------+ |Service | +--------+ +--------+ +--------+ +--------+ +--------+ | | --------->|Analytics| +--------+ +--------+ +--------+ +--------+ | | --------->|Inventory| +--------+ +--------+
Characteristics: - One event, multiple consumers - Decoupled producers/consumers - Event broadcast to all subscribers13.3.2 Event Streaming
Section titled “13.3.2 Event Streaming” Event Streaming ==============
Kafka/Event Hub: Topics as Logs +------------------------------------------------+
Partition 0: [E1] -> [E2] -> [E5] -> [E7] -> ... Partition 1: [E3] -> [E4] -> [E6] -> ... Partition 2: [E8] -> [E9] -> [E10] -> ...
Consumers read from offset: +--------------------------------------------+ | Consumer A: offset=5 (processing E7) | | Consumer B: offset=2 (processing E5) | +--------------------------------------------+
Benefits: - Replay events - Exactly-once processing - Event retention - Partition ordering13.3.3 Event Sourcing
Section titled “13.3.3 Event Sourcing” Event Sourcing ==============
Instead of storing current state, store all events:
Traditional: Event Sourcing: +--------------------+ +--------------------+ | Order Table: | | Events: | | order_id: 123 | | OrderCreated | | status: shipped | | OrderPaid | | total: 99.99 | | OrderShipped | +--------------------+ +--------------------+
Rebuild state by replaying events: +----------------------------------+ | Created(99.99) | | + Paid() | | + Shipped() | | = Current: Shipped, 99.99 | +----------------------------------+
Benefits: - Complete audit trail - Temporal queries - Event replay/debugging13.4 Message Brokers
Section titled “13.4 Message Brokers”Popular Options
Section titled “Popular Options”| Broker | Characteristics |
|---|---|
| Apache Kafka | High throughput, distributed, persistent |
| RabbitMQ | Flexible routing, older, mature |
| AWS SQS | Fully managed, simple |
| AWS SNS | Pub/sub, push notifications |
| Google Pub/Sub | Managed, global |
Kafka Architecture
Section titled “Kafka Architecture” Kafka Architecture =================
+--------------------------------------------------+ | Kafka Cluster | +--------------------------------------------------+
+----------+ +----------+ +----------+ | Broker 1 | | Broker 2 | | Broker 3| | (Leader) | | (Follower| |(Follower)| +----------+ +----------+ +----------+
Topics are partitioned and replicated: +------------------------------------------+ | Topic: Orders | | Partition 0: Leader on Broker 1 | | Partition 1: Leader on Broker 2 | | Partition 2: Leader on Broker 3 | +------------------------------------------+
Producers write to topics: +------------------------------------------+ | Producer: Order Service | | - Keys partition by order_id | | - Balances across partitions | +------------------------------------------+
Consumers read from topics: +------------------------------------------+ | Consumer Group: email-service | | - Each partition assigned to one worker| | - Offsets managed by Kafka | +------------------------------------------+13.5 Benefits of Event-Driven Architecture
Section titled “13.5 Benefits of Event-Driven Architecture”Advantages
Section titled “Advantages”| Benefit | Description |
|---|---|
| Loose Coupling | Producers don’t know consumers |
| Scalability | Easy to add consumers |
| Resilience | Failed events can be replayed |
| Auditability | Complete event history |
| Real-time | Low latency event processing |
| Flexibility | Add services without changing producers |
13.6 Challenges
Section titled “13.6 Challenges”Complexities
Section titled “Complexities”| Challenge | Description |
|---|---|
| Eventual Consistency | Data may be temporarily inconsistent |
| Debugging | Hard to trace distributed flows |
| Ordering | Events may arrive out of order |
| Duplicates | Idempotency needed |
| Schema Evolution | Event schema changes over time |
Handling Challenges
Section titled “Handling Challenges” Challenge: Eventual Consistency =================================
Synchronous: Eventual: +---------+ +---------+ +--------+ +--------+ | Service | -> | Service | |Service | -> | Queue | +---------+ +---------+ +--------+ +--------+ | | | v v v Immediate Slow Consumer consistency consistency processes
Solution: Compensating Transactions ===================================
Order Service: Create order Inventory Service: Reserve items (or compensate)
If inventory fails: 1. Send "ReserveFailed" event 2. Order Service rolls back order 3. Send "OrderCancelled" event13.7 CQRS Pattern
Section titled “13.7 CQRS Pattern”Command Query Responsibility Segregation
Section titled “Command Query Responsibility Segregation” CQRS ====
Traditional: CQRS: +----------+ +----------+ +----------+ | Service | | Command | | Query | | Read/ | => | Service | | Service | | Write | | (Write) | | (Read) | +----------+ +----------+ +----------+ | | v v +-------+ +-------+ | Write | | Read | | DB | | DB | +-------+ +-------+
Separates read and write models!Implementation with Events
Section titled “Implementation with Events” CQRS with Event Sourcing =======================
Command Side: +--------+ +--------+ +--------+ |Client | -> | Command| -> | Event | -> Write DB |Request | | Handler| | Store | (append-only) +--------+ +--------+ +--------+
Query Side: +--------+ +--------+ +--------+ |Client | -> |Query | -> | Read | <- Materialized |Request | |Handler | | Model | Views +--------+ +--------+ +--------+
Event Flow: 1. Command processed -> Event stored 2. Event published -> Projectors update views 3. Query reads from materialized views13.8 Saga Pattern
Section titled “13.8 Saga Pattern”Managing Distributed Transactions
Section titled “Managing Distributed Transactions” Saga Pattern ============
Instead of ACID across services, use Choreography or Orchestration:
CHOREOGRAPHY (Event-based): ==========================
Order Service -> [OrderCreated] -> Inventory Service | v [InventoryReserved] | v Payment Service <----------------- [ReserveFailed] | v [PaymentProcessed]
ORCHESTRATION (Central coordinator): ====================================
Order Service -> Orchestrator | +---------+---------+ | | | v v v Inventory Payment Shipping | | | +---------+---------+ | v Complete13.9 Best Practices
Section titled “13.9 Best Practices”Event Design
Section titled “Event Design”| Practice | Description |
|---|---|
| Immutable Events | Don’t modify after creation |
| Versioning | Include version in event |
| Schema Registry | Track event schemas |
| Idempotency | Handle duplicate events |
| Correlation IDs | Track related events |
Event Schema
Section titled “Event Schema”{ "eventId": "uuid", "eventType": "OrderCreated", "eventVersion": "1.0", "timestamp": "2024-01-15T10:30:00Z", "correlationId": "order-123", "causationId": "command-456", "producer": "order-service", "data": { ... }}Summary
Section titled “Summary”Key event-driven concepts:
- Events over requests - Loose coupling, async communication
- Pub/Sub - One event, multiple consumers
- Event sourcing - Store events, rebuild state
- CQRS - Separate read and write models
- Saga pattern - Handle distributed transactions
- Challenges - Eventual consistency, ordering, debugging
- Schema evolution - Plan for change
Next: Chapter 14: CQRS Pattern