The shared key#
Release 2026.11.3 caches a user as {"name", "phone"} under user:42. Release 2026.11.4 changes the shape to a list of phones and writes the same key. We ran this against Redis 7.0.15 from Python 3.11:
# 2026.11.4 caches the user
redis.set("user:42", json.dumps({"name": "Ana", "phones": ["+49 30 1234"]}), ex=3600)
# 2026.11.3 reads it
json.loads(redis.get("user:42"))["phone"]
# KeyError: 'phone'
Every user that a new-release pod cached is now an error on every old-release pod, for as long as the entry lives. A reader written with .get("phone") fails differently: it shows no phone number, which nobody reports as an error. A rollback leaves the same entries behind for the restored release to trip over.
Version the key#
def cache_key(user_id):
return f"user:v2:{user_id}" # 2026.11.4
With separate keys, both releases read their own entries. After one read from each release, Redis held both:
keys: ['user:42', 'user:v2:42']
Both releases returned the same phone number. The cost is memory for two copies while both releases run, and a cold cache for the new key.
The cold start#
We simulated 100,000 users read 20,000 times a minute, with a few users read far more than the rest, and counted how many reads of the new key found an entry:
| Minute after the first new pod | Reads served from cache |
|---|---|
| 1 | 63% |
| 5 | 82% |
| 15 | 92% |
| 60 | 98% |
In the first minute, 7,484 of 20,000 reads missed and went to the database instead of the usual 2% or so. A rolling deploy spreads that over the rollout, because new pods arrive one at a time. A deploy that replaces every pod at once does not. Check that the database can take the miss rate of the first minute before using a new key prefix on a hot cache.
Entries the old release must still be able to drop#
Versioned keys create a second problem: a write by the old release removes only the entry it knows about. Keep old and new cache entries consistent when both releases write shows that, and the order that avoids it.
The compatibility test#
With both releases pointed at one Redis in staging, have each read a user the other cached. Pass when neither raises an error or shows an empty field, and measure the database's query rate in the first minute after the first new pod starts.