Sagas: transactions across services
Replace distributed transactions with sagas - a sequence of local transactions with compensations - orchestrated or choreographed.
- Explain why one ACID transaction can’t span services, and why two-phase commit is rarely used
- Design a saga with compensating actions, and order steps around the point of no return
- Compare orchestration and choreography
Placing an order at Galactic Noodle Express touches four services: Orders records it, Payments charges the card, Kitchen reserves ingredients and cooks, Delivery launches a rocket. In the monolith, one database transaction made that all-or-nothing. With database-per-service, there’s no transaction that spans them.
Two-phase commit (2PC) can coordinate a distributed transaction, but every participant must hold locks while waiting for the coordinator, one slow service stalls everyone, and most brokers and modern databases don’t support it across services. So microservices use a saga:
- a sequence of local transactions, one per service, each committed immediately;
- for each step that might need undoing, a compensating transaction that semantically reverses it (refund the card, release the ingredients, cancel the order);
- if a step fails, run the compensations of the completed steps in reverse order.
Try it
Saga simulator
This is the order saga. Pick a step to fail and read the log.
- Fail Payments: what gets undone?
- Fail Delivery, at the end: why is the saga stuck? You can’t un-cook noodles.
- Look at the step order. Where should the irreversible steps go?
Choose which step fails:
- 1. Orderscreate order (pending)undo: cancel order
- 2. Paymentscharge cardundo: refund card
- 3. Kitchenreserve ingredientsundo: release ingredients
- 4. Kitchencook noodlescan’t be undone
- 5. Deliverylaunch rocketcan’t be undone
Saga log
- ✓Orders: create order (pending)
- ✓Payments: charge card
- ✓Kitchen: reserve ingredients
- ✓Kitchen: cook noodles
- ✓Delivery: launch rocket
Saga completed: every service committed its step.
Compensations are semantic, not magic undo: a refund doesn’t erase the charge, it adds a reversing entry - and the customer may briefly see both. Some steps can’t be undone at all (cooking, sending an email, launching a rocket). Structure the saga around a pivot:
- Compensatable steps first - things that can be undone.
- The pivot - the point of no return.
- Retriable steps after it - steps that are guaranteed to succeed eventually if you keep retrying (idempotently), so they never need compensating.
In the simulator, cooking is the pivot - and launching the rocket must therefore be retried until it succeeds (with another rocket if necessary), not compensated.
Orchestration or choreography?
| Orchestration | Choreography | |
|---|---|---|
| Who decides the next step | a central orchestrator sends commands (ChargeCard) and waits for replies | each service reacts to the previous service’s events (OrderCreated → Payments charges) |
| Where the flow is visible | in one place - the orchestrator’s state machine | spread across services; you reconstruct it from events |
| Coupling | services depend on the orchestrator’s commands | services depend only on events |
| Good for | complex flows with many steps and branches | simple flows with few participants |
Workflow engines such as Temporal, AWS Step Functions and Camunda are durable orchestrators: they persist the saga’s state, so it survives crashes and resumes where it left off.
1steps = [("Orders", "create order", "cancel order"),
2 ("Payments", "charge card", "refund card"),
3 ("Kitchen", "reserve ingredients", "release ingredients")]
4
5def run(fail_at):
6 done = []
7 for index, (service, action, undo) in enumerate(steps):
8 if index == fail_at:
9 print(f" {service}: {action} FAILED - compensating")
10 for done_service, done_undo in reversed(done):
11 print(f" {done_service}: {done_undo}")
12 return "rolled back"
13 print(f" {service}: {action}")
14 done.append((service, undo))
15 return "completed"
16
17print("happy path:", run(None))
18print("kitchen out of noodles:", run(2))Orders: create order Payments: charge card Kitchen: reserve ingredients happy path: completed Orders: create order Payments: charge card Kitchen: reserve ingredients FAILED - compensating Payments: refund card Orders: cancel order kitchen out of noodles: rolled back
Key takeaways
A saga is a sequence of local transactions with compensating transactions instead of one distributed transaction.
On failure, compensate completed steps in reverse order; compensations are semantic reversals, not erasure.
Put compensatable steps first, then the pivot, then retriable steps that can’t fail permanently.
Orchestration centralizes the flow; choreography lets services react to events. Durable workflow engines make orchestrators crash-proof.
Lesson quiz
7 questions · pass with 5 correct · up to 50 XP
Passing this quiz completes the lesson and keeps your streak going. Questions you miss come back in review sessions later.
Practice: simulate microservice patterns in Python
Build small Python simulations of the patterns - routers, sagas, outboxes, circuit breakers, traces - and run them against sample inputs. They run locally in your browser; no servers or containers needed.
Run the saga
Each input line is a saga step Service | action | compensation (compensation - if it can’t be undone); the last line is fail: N (1-based) or fail: none. Run the saga: print ✓ Service: action for each completed step, ✗ Service: action for the failing one, then for each completed step in reverse ↩ Service: compensation or ⚠ Service: cannot undo action. Finish with outcome: completed, outcome: compensated or outcome: needs manual repair.
- Payment declined
- Rocket failure after cooking
- Happy path
Python runs in a sandboxed browser worker with a 60 second time limit. Its runtime loads from the Pyodide CDN; your code stays in this browser.
An orchestrator with retries
Each input line is a step: Service action compensation retriable|compensatable attempts..., where attempts are the results of successive tries (ok, fail). The orchestrator tries each step up to 3 times:
- If an attempt succeeds, print
Service: action (attempt N)and move on. - If a compensatable step fails 3 times, print
Service: action FAILED, compensate the completed steps in reverse (undo Service: compensation) and stop withsaga rolled back. - A retriable step must not give up: if it fails 3 times, print
Service: action still failing - parked for manual retryand stop withsaga parked.
If every step succeeds, print saga completed. A step with fewer listed attempts than tries treats the missing ones as ok.
- Flaky payment, then success
- Out of noodles
- Rocket stuck
Python runs in a sandboxed browser worker with a 60 second time limit. Its runtime loads from the Pyodide CDN; your code stays in this browser.
Questions about this lesson
Stuck? Ask. Figured something out? Share it. Explaining is one of the best ways to learn.
Loading posts…