Diagram
Side-by-side comparison showing Pub/Sub leasing individual messages for parallel independent processing versus partition-based systems requiring ordered processing limited by partition count.
Why Decoupling Makes Ingestion Scalable and Reliable
Pub/Sub also uses per-message parallelism instead of partition-based messaging to maximize subscriber throughput. Pub/Sub leases individual messages to subscriber clients and tracks whether each message is successfully processed. By contrast, partition-based systems force subscribers to process messages in each partition in order and limit the number of concurrent clients to the number of partitions.
| Approach | Parallelism unit | Processing constraint |
|---|---|---|
| Pub/Sub | Individual message leased to a subscriber client | Messages can be processed in parallel across many subscribers, with success tracked per message |
| Partition-based messaging | Partition assigned to a subscriber | Messages in each partition must be processed in order, and concurrent clients are limited by partition count |
Domain 5 of 5
27 practice questions
Practice this DomainLast updated