Streams, Workflows, Orchestration, and Choreography
Goal
Separate continuous record processing from multi-step business coordination.
The Short Version
Streams preserve records for ongoing processing; workflows coordinate progress; orchestration names a coordinator; choreography distributes reactions.
Streams support retained, partitioned, ordered records and multiple processors with independent positions. The important architecture move is to make the boundary explicit before selecting a cloud feature. A feature can automate a decision, but it cannot decide the organization's risk tolerance, ownership model, or acceptable failure behavior.
Use this section as a decision tool. Name the actor or producer, the protected or affected resource, the contract between them, the conditions under which the action is allowed, and the evidence that will show whether the design worked.
Mental Model
Use side-by-side diagrams of a stream processor, an orchestrated workflow, and a choreographed event chain.
flowchart LR
N1["Producer intent"] --> N2["Contract"] --> N3["Delivery mechanism"] --> N4["Consumer"] --> N5["Outcome"] --> N6["Failure evidence"]Read the diagram from left to right, then review it from right to left. The forward pass explains how work succeeds. The reverse pass asks whether an operator can trace an outcome back to its initiating identity, input, policy, route, or contract.
Architecture Decisions
| Decision dimension | Question to make explicit |
|---|---|
| Intent | Is this a query, command, notification, fact, stream, or process step? |
| Coupling | Must the producer wait for a named consumer? |
| Delivery | What can be duplicated, delayed, reordered, or lost? |
| State | Where is process progress, correlation, and replay position stored? |
| Change | How do contracts evolve without forcing coordinated releases? |
The answers should be recorded with the workload design. If the answer is merely "the platform default," the team has discovered an assumption, not completed an architecture decision.
Core Ideas
Streams support retained, partitioned, ordered records and multiple processors with independent positions
Streams support retained, partitioned, ordered records and multiple processors with independent positions. Make the owner, boundary, failure behavior, and evidence explicit. That turns the concept from a configuration choice into a design decision that another team can review and operate.
Workflow engines persist step state, waits, retries, compensation, and progress for long-running processes
Workflow engines persist step state, waits, retries, compensation, and progress for long-running processes. Make the owner, boundary, failure behavior, and evidence explicit. That turns the concept from a configuration choice into a design decision that another team can review and operate.
Orchestration improves centralized visibility; choreography reduces central coordination but can obscure end-to-end process state
Orchestration improves centralized visibility; choreography reduces central coordination but can obscure end-to-end process state. Make the owner, boundary, failure behavior, and evidence explicit. That turns the concept from a configuration choice into a design decision that another team can review and operate.
Provider Lenses
AWS provides API Gateway, SQS, SNS, EventBridge, Kinesis, and Step Functions among its implementation choices. Azure provides API Management, Service Bus, Event Grid, Event Hubs, Logic Apps, and Durable Functions. Choose first among request-response, work queue, publish-subscribe, event routing, durable stream, and workflow semantics; then map those semantics to a provider service.
Provider lenses are useful for implementation planning, but they should come after the provider-neutral decision. This keeps the reasoning portable and makes differences in defaults, scope, and operational responsibility easier to detect.
Example: Applying the Decision
Compare real-time order analytics with a fulfillment process that may wait for payment, inventory, human review, and shipment.
Keep checkout validation synchronous, submit accepted orders durably, orchestrate payment and inventory when the business needs visible process state, publish outcome events for fulfillment and notifications, and make every state-changing consumer idempotent.
A review should finish with named owners, a testable success path, expected failure modes, and observable evidence. That is more useful than a diagram that only proves the components can connect.
Common Mistakes
Using stream terminology for any sequence of events. It hides a boundary or enlarges the impact of failure, so record the safer alternative in the design.
Implementing long-running process state in ad hoc application retries. It hides a boundary or enlarges the impact of failure, so record the safer alternative in the design.
Choosing choreography for a process that needs a clear owner and customer-visible status. It hides a boundary or enlarges the impact of failure, so record the safer alternative in the design.
If Someone Asks You
What is the difference between a stream, a workflow, orchestration, and choreography?
A clear answer is:
Streams preserve records for ongoing processing; workflows coordinate progress; orchestration names a coordinator; choreography distributes reactions.
I would then show the decision path, the responsibility boundary, and the evidence used to review it. That explanation connects the technical mechanism to its architecture purpose.
What You Should Remember
Streams preserve records for ongoing processing; workflows coordinate progress; orchestration names a coordinator; choreography distributes reactions.
Questions to Check Your Understanding
What makes a stream replayable?
Which responsibilities does a workflow engine hold?
What visibility trade-off separates orchestration and choreography?