Queues, Topics, and Event Routing

Goal

Differentiate work distribution, fan-out publication, and event matching for asynchronous integration.

The Short Version

Queues distribute work, topics distribute publications, and event routers match facts to interested destinations.

Queues buffer work and support competing consumers; topics publish to multiple subscriptions; event routers match and forward events by rules. The important architecture move is to make the boundary explicit before selecting a cloud feature. A feature can automate a decision, but it cannot decide the organization's risk tolerance, ownership model, or acceptable failure behavior.

Use this section as a decision tool. Name the actor or producer, the protected or affected resource, the contract between them, the conditions under which the action is allowed, and the evidence that will show whether the design worked.

Mental Model

Compare a work queue, pub-sub topic, and event router using the same producer and multiple consumers.

flowchart LR
    N1["Producer intent"] --> N2["Contract"] --> N3["Delivery mechanism"] --> N4["Consumer"] --> N5["Outcome"] --> N6["Failure evidence"]

Read the diagram from left to right, then review it from right to left. The forward pass explains how work succeeds. The reverse pass asks whether an operator can trace an outcome back to its initiating identity, input, policy, route, or contract.

Architecture Decisions

The answers should be recorded with the workload design. If the answer is merely "the platform default," the team has discovered an assumption, not completed an architecture decision.

Core Ideas

Queues buffer work and support competing consumers; topics publish to multiple subscriptions; event routers match and forward events by rules

Queues buffer work and support competing consumers; topics publish to multiple subscriptions; event routers match and forward events by rules. Make the owner, boundary, failure behavior, and evidence explicit. That turns the concept from a configuration choice into a design decision that another team can review and operate.

Delivery is commonly at least once, so consumers need duplicate-safe behavior and explicit acknowledgement semantics

Delivery is commonly at least once, so consumers need duplicate-safe behavior and explicit acknowledgement semantics. Make the owner, boundary, failure behavior, and evidence explicit. That turns the concept from a configuration choice into a design decision that another team can review and operate.

Retention, visibility or lock time, dead-letter handling, backpressure, schema ownership, and consumer isolation shape reliability

Retention, visibility or lock time, dead-letter handling, backpressure, schema ownership, and consumer isolation shape reliability. Make the owner, boundary, failure behavior, and evidence explicit. That turns the concept from a configuration choice into a design decision that another team can review and operate.

Provider Lenses

AWS provides API Gateway, SQS, SNS, EventBridge, Kinesis, and Step Functions among its implementation choices. Azure provides API Management, Service Bus, Event Grid, Event Hubs, Logic Apps, and Durable Functions. Choose first among request-response, work queue, publish-subscribe, event routing, durable stream, and workflow semantics; then map those semantics to a provider service.

Provider lenses are useful for implementation planning, but they should come after the provider-neutral decision. This keeps the reasoning portable and makes differences in defaults, scope, and operational responsibility easier to detect.

Example: Applying the Decision

Order acceptance should trigger inventory, fulfillment, analytics, and notification work with different ownership and retry needs.

Keep checkout validation synchronous, submit accepted orders durably, orchestrate payment and inventory when the business needs visible process state, publish outcome events for fulfillment and notifications, and make every state-changing consumer idempotent.

A review should finish with named owners, a testable success path, expected failure modes, and observable evidence. That is more useful than a diagram that only proves the components can connect.

Common Mistakes

• Using a topic when only one worker should own each task. It hides a boundary or enlarges the impact of failure, so record the safer alternative in the design.

• Treating delivery as exactly once without designing idempotent effects. It hides a boundary or enlarges the impact of failure, so record the safer alternative in the design.

• Letting one failing subscriber block or poison unrelated consumers. It hides a boundary or enlarges the impact of failure, so record the safer alternative in the design.

If Someone Asks You

How do you choose between a queue, a topic, and event routing?

A clear answer is:

Queues distribute work, topics distribute publications, and event routers match facts to interested destinations.

I would then show the decision path, the responsibility boundary, and the evidence used to review it. That explanation connects the technical mechanism to its architecture purpose.

What You Should Remember

Queues distribute work, topics distribute publications, and event routers match facts to interested destinations.

Questions to Check Your Understanding

1. Which topology fits competing consumers?

2. Why must consumers tolerate duplicate delivery?

3. How can subscriptions isolate consumer failures?