Learning center

A reviewer says "use a set here": check how often the code runs

An unmeasured micro-optimization is worth taking only when the code runs per request or per item over input that grows. Find where the line runs and how large its input can get. If it runs once or on a few items, decline the suggestion, or accept it only because it reads better. If it sits on a path that grows, benchmark the function and then the whole operation before you change it.

What the suggestion assumes#

Comments such as "use a set for O(1) lookups", "avoid this copy" or "precompile this regex" rest on an unstated premise: that the line runs often enough for its cost to show. The reviewer usually cannot see that premise. It sees the diff, not the call graph or the size of the input, so it reasons from the shape of the code. Your job is to supply the two facts it lacked: how often the line runs, and on what size of input.

A suggestion that cut both ways#

In April 2026, a vLLM pull request converted tokenizer.all_special_ids, a list, to a set in several model processors. Its description said: "These are initialization-time calls (@cached_property), not per-request hot paths". Pull request

Gemini Code Assist argued against several of the conversions. On a one-off assertion it said: "Creating a set for a single membership check is less efficient than a direct list lookup for small collections". Review comment On the CLIP processor's apply() it said: "The apply method is a per-request hot path." It added: "Creating a new set from all_special_ids on every call introduces unnecessary allocation and hashing overhead." Review comment

The author agreed: "Agreed — for the CLIP apply() path and the single-membership assert, the set allocation overhead isn't worth it for a <10-element list." They kept one change, in SigLIP, where "the membership check runs inside a range(vocab_size) loop (32k+ iterations)", adding that "the set really does help there." Author's reply Ten days later they closed the pull request unmerged: "this change is too narrow to meet the substantive-work bar on its own." Closing comment

Check each premise against the code#

The CLIP check runs only when a request carries image data, and it short-circuits. On main in November 2026, it reads if all(tok in special_tokens for tok in inputs.prompt), and CLIP accepts an image only with a prompt made of special tokens. CLIP processor So the scanned prompt is a few tokens long. To see the size of the difference, I timed both forms with Python's timeit:

import timeit
special = [49406, 49407]  # CLIP's start and end tokens
def make(prompt):
    def list_scan():
        return all(t in special for t in prompt)
    def set_per_call():
        s = set(special)
        return all(t in s for t in prompt)
    return list_scan, set_per_call
n = 200_000
for prompt in ([49406, 49407], [49406] + [49407] * 76):
    for f in make(prompt):
        t = min(timeit.repeat(f, number=n, repeat=5))
        print(f"{len(prompt)} tokens, {f.__name__}: {t / n * 1e9:.0f} ns per call")

Over five runs on Python 3.11 on a shared machine, the list scan was faster or level with two tokens in every run, by at most about 240 nanoseconds. With 77 tokens, building the set per call was faster in most runs, by up to about 550 nanoseconds. The bot's direction was right for the prompts this path sees, and the difference is a fraction of a microsecond once per image request.

The SigLIP change is in a @cached_property, which runs once per processor. Its loop is next(token_id for token_id in range(tokenizer.vocab_size) if token_id not in all_special_ids), and next() stops at the first id that is not special. Diff The 32,000 iterations are an upper bound the loop reaches only if nearly every id is special. Neither side of the thread was on a hot path, which the description had already said.

When a maintainer asks for the end-to-end number#

A January 2026 vLLM pull request replaced list.index() lookups with a dictionary in the LoRA mapping code, and included a microbenchmark of that function. Gemini's review said that "the inclusion of new unit tests, especially the performance test, is excellent for verifying the improvement." Review A maintainer replied: "Thank you for your contribution, could you please provide the E2E performance comparison?" Maintainer's comment No comparison followed, and the pull request closed for inactivity in June.

A function that gets several times faster can still be invisible in the operation that calls it. The maintainer asked for the number that decides whether the review and the extra code are worth having.

Decide from where the code runs#

Where the line runsEvidence to citeDecision
Once, at startup or inside a cached propertyThe decorator or the call siteDismiss, or accept only if the new form reads better
Once per request, on input with a small fixed sizeThe bound on the inputDismiss, or accept if equally readable. No benchmark needed
Per item, on input that grows with data or trafficThe input's realistic sizeBenchmark the function, then the whole operation
On a path a profiler already shows as hotProfiler output from cProfile or your production profilerChange, with the before and after numbers in the pull request

When you decline, give the location and the size in one sentence, for example "runs once per processor, in a cached_property". When you accept for readability, say so, so that nobody later reads the change as a measured speedup. A reviewer that keeps asking for changes like these on cold paths is a candidate for a custom review rule that names your hot paths.

One month of free code review.

New organizations get one calendar month of hosted code review for private and public GitHub repositories. No card or cloud connection required. Eligibility and fair-use terms.

Start free with GitHub

Related reading