Learning center

Change a cached object's format during a rolling deploy

When the new release caches an object in a new format under the same Redis key, the old release reads that entry and fails, or reads an empty value. Put the format version in the key so each release reads only entries it wrote. The new key starts empty: in our simulation the new release served 63% of reads from cache in its first minute and 98% after an hour, and the misses went to the database.

The shared key#

Release 2026.11.3 caches a user as {"name", "phone"} under user:42. Release 2026.11.4 changes the shape to a list of phones and writes the same key. We ran this against Redis 7.0.15 from Python 3.11:

# 2026.11.4 caches the user
redis.set("user:42", json.dumps({"name": "Ana", "phones": ["+49 30 1234"]}), ex=3600)

# 2026.11.3 reads it
json.loads(redis.get("user:42"))["phone"]
# KeyError: 'phone'

Every user that a new-release pod cached is now an error on every old-release pod, for as long as the entry lives. A reader written with .get("phone") fails differently: it shows no phone number, which nobody reports as an error. A rollback leaves the same entries behind for the restored release to trip over.

Version the key#

def cache_key(user_id):
    return f"user:v2:{user_id}"          # 2026.11.4

With separate keys, both releases read their own entries. After one read from each release, Redis held both:

keys: ['user:42', 'user:v2:42']

Both releases returned the same phone number. The cost is memory for two copies while both releases run, and a cold cache for the new key.

The cold start#

We simulated 100,000 users read 20,000 times a minute, with a few users read far more than the rest, and counted how many reads of the new key found an entry:

Minute after the first new podReads served from cache
163%
582%
1592%
6098%

In the first minute, 7,484 of 20,000 reads missed and went to the database instead of the usual 2% or so. A rolling deploy spreads that over the rollout, because new pods arrive one at a time. A deploy that replaces every pod at once does not. Check that the database can take the miss rate of the first minute before using a new key prefix on a hot cache.

Entries the old release must still be able to drop#

Versioned keys create a second problem: a write by the old release removes only the entry it knows about. Keep old and new cache entries consistent when both releases write shows that, and the order that avoids it.

The compatibility test#

With both releases pointed at one Redis in staging, have each read a user the other cached. Pass when neither raises an error or shows an empty field, and measure the database's query rate in the first minute after the first new pod starts.

One month of free code review.

New organizations get one calendar month of hosted code review for private and public GitHub repositories. No card or cloud connection required. Eligibility and fair-use terms.

Start free with GitHub

Related reading