The simulation#
10,000 users each make 20 requests during a session. The new release reads from the new source for 10% of reads. We compared two ways to choose, in Python 3.11:
import hashlib, random
def per_request():
return random.random() < 0.10
def per_user(user_id, salt="orders-read-state"):
h = hashlib.sha256(f"{salt}:{user_id}".encode()).hexdigest()
return int(h[:8], 16) % 100 < 10
| Split | Users who saw both sources | Users on the new source |
|---|---|---|
| Random per request | 8,773 | most, some of the time |
| Hash of user ID | 0 | 978 |
The random result matches the arithmetic: a user avoids mixing only if all 20 requests land on the same side, and 1 − 0.9²⁰ − 0.1²⁰ of users do not, about 88%. At 1% the share is still about 18%. Any difference between the sources, from a backfill that has not finished to a value converted differently, appears to those users as data that changes between page loads.
Why the hash works across pods#
Every pod of the new release computes the same bucket for a user, because the hash depends only on the salt and the ID. Raising the percentage from 10 to 20 moved 967 more users to the new source and none back, because a bucket below 10 is also below 20. Lowering it moves users back in the reverse order.
Change the salt only when you start a new rollout. A new salt reshuffles every user, which is a fresh random split.
What the split does not fix#
The old release does not read the flag, so during a deploy a user can still alternate between an old-release pod reading the old source and a new-release pod reading the new one. The bucketed split limits the mixing to the deploy itself, and the reconciliation guides limit what the user sees during it: find every backfill mismatch with IS DISTINCT FROM should be clean before the first user moves.
Hash the entity a user sees together. If an order page and an invoice page show the same order, bucket by account or order, so both pages read the same source.
The check before raising the percentage#
Log the bucket with each read for a sample of requests. Pass when no user ID appears with both sources outside a deploy window. To compare the sources on live traffic before switching anyone, see compare old and new reads on live traffic.