Mixed messages on one stream#
Payments flow through a Redis stream. Release 2026.11.3 produces {"order_id", "amount_cents"} and charges in EUR. Release 2026.11.4 adds currency. During the rollout both produce, and both consume from the same consumer group. We ran this on Redis 7.0.15 from Python 3.11:
redis.xadd("payments", {"body": json.dumps({"order_id": 1, "amount_cents": 1999})}) # 2026.11.3
redis.xadd("payments", {"body": json.dumps({"order_id": 2, "amount_cents": 2500, "currency": "CHF"})}) # 2026.11.4
Three consumers read both messages:
def old_consumer(m): # 2026.11.3
return f"charge {m['amount_cents']} EUR"
def new_consumer_strict(m): # 2026.11.4, as first written
return f"charge {m['amount_cents']} {m['currency']}"
def new_consumer_tolerant(m): # 2026.11.4, accepting both shapes
return f"charge {m['amount_cents']} {m.get('currency', 'EUR')}"
| Consumer | Message 1 (no currency) | Message 2 (CHF) |
|---|---|---|
2026.11.3 | charge 1999 EUR | charge 2500 EUR |
2026.11.4 strict | KeyError: 'currency' | charge 2500 CHF |
2026.11.4 tolerant | charge 1999 EUR | charge 2500 CHF |
The old consumer's result is the costly one: a wrong charge, with no error. Ignoring unknown fields is usually the right default for a consumer, and here it is exactly what went wrong, because the new field changes the meaning of one the old consumer does read. The strict consumer's failure at least stops the message.
The order#
- Ship consumers that accept both shapes, defaulting the missing field to the old meaning, while producers still send only the old shape.
- When no consumer older than step 1 can run, including as a rollback target, ship producers that send the new field.
If producers and consumers are the same service, those are two releases. A consumer group shares messages among all its consumers, so any old consumer still attached can take any message.
When a field changes meaning#
If the old consumer would mishandle the new messages, as with currency, the producer must not send them while it might be attached. Count the consumers attached to the group and their idle times before step 2:
XINFO CONSUMERS payments billing
Name consumers after their release so the list says which code is reading. Messages already in the stream when a rollback happens are the other half of the problem, covered in new-format messages still in the queue after a rollback.
The compatibility test#
Put one message of each shape on a test stream and have each release's consumer process both. Pass when every consumer that can run produces the intended result for both, and when no producer sends the new meaning until the last old consumer is gone.