Learning center

Change, test or dismiss: triage an AI code review comment by its claim

Every AI review comment makes a claim about the code, and each kind of claim has one fact that settles it. Name the claim, find that fact, then change the code, add a test or dismiss the comment with the fact as your reason.

Read the comment as a claim#

A comment that says "this may cause an N+1 query" asserts two things: something per row reads a related object, and the query feeding the rows does not load it in advance. A comment that says "this value might be None" asserts that some caller, default or data source can produce the None. Neither assertion is settled by how confident the comment sounds, by its severity label or by how much code it quotes. Each is settled by a fact you can usually establish in minutes: a query count, a declared type, a timing, a test run at a boundary.

Three outcomes follow from that fact. When it confirms the claim, change the code. When reading cannot settle it but a run can, write the test first, because a test that fails before the fix and passes after it answers the comment and keeps the answer. When the fact disproves the claim, dismiss the comment and put the fact in your reply, so the next reader does not have to find it again.

One pull request, two answers#

A September 2026 pull request in Gyrinx, a Django application, shows both ends. It drew each user's badge beside their name on campaign screens, and Copilot reviewed it several times. Two of its comments made the same kind of claim.

One comment said a shared row template was also rendered by the signed-in dashboard, whose query did not load the owner's badges, "so a page with several campaigns regresses to N+1." Copilot review The author counted: "the dashboard costs 20 queries with one campaign listed and 20 with five (warm session, CaptureQueriesContext), so nothing grows with the number of rows." Author's reply A flat count disproves an N+1 claim, so the comment was declined with the numbers.

The other said a refused form submission redrew the page and loaded the badge data lazily. The author confirmed it and wrote a test that compares the redraw's query count with the GET's. It "fails against the reviewed guard (23 vs 21) and passes with the join." Author's reply Same reviewer, same kind of claim, same deciding fact; one comment earned a dismissal and the other a change with its test. The N+1 guide follows both threads through the merged code.

Find the fact for each kind of claim#

The guides below each take one kind of comment, follow public review threads to their resolution and say which evidence points to a change, a test or a dismissal.

Correctness#

Comment saysFact that settles itGuide
This might be nullWhat produces the value: its type, constructor and defaultsNull checks
Off by oneA test at the first value, the last value and one past itOff-by-one
Race conditionA point where a second operation can run between two stepsRace conditions
Missing awaitWhat the call returns and whether anything awaits itMissing await
Overflow or truncationThe type's width and the largest value the code really seesInteger overflow
Timezone bugWhether the value is an instant or a wall-clock timeTimezones

Errors and resources#

Comment saysFact that settles itGuide
The error is swallowedWhether anyone downstream needs to know it failedSwallowed errors
The error is not handledA realistic cause and the nearest layer that can act on itUnhandled errors
This leaksWho owns the resource and how long the owner livesResource leaks
Retrying is unsafeWhat the server does when the same request arrives twiceRetries
Validate this inputWhether the value crosses a trust boundary hereInput validation

Security#

Comment saysFact that settles itGuide
SQL injectionWhether untrusted text reaches the SQL before it is parsedSQL injection
Hardcoded secretWhat the string lets someone who copies it doHardcoded secrets
Path traversalWhere the final path lands after every decoding stepPath traversal
The log leaks a tokenWhat the logged value contains at runtimeSensitive logging
ReDoSWho controls the input, and timings in your regex engineRegex ReDoS
XSSWhere the string comes from and which context it lands inXSS

Performance and data volume#

Comment saysFact that settles itGuide
N+1 queryThe query count at one row and at severalN+1 queries
Use a faster constructHow often the line runs and how large its input getsMicro-optimizations
The cache can go staleA write path that changes an input and skips invalidationCache invalidation
No limit on this queryHow many rows it reads and how fast they growUnbounded queries
This grows without limitThe code that removes entries, and the bound it impliesMemory growth

Compatibility and dependencies#

Comment saysFact that settles itGuide
This breaks callersWho the callers are and what you promised themBreaking changes
The default changedWho relies on the old defaultChanged defaults
This API is deprecatedThe source or changelog of the version you pinDeprecations
This version is wrongThe registry, the changelog and your support policyDependency versions
Use this API insteadWhether it exists and behaves the same on your oldest supported versionSuggested APIs

Suggestions and style#

Comment saysFact that settles itGuide
A committable suggestionWhether the replaced lines are complete on their ownSuggested changes
A suggested fixWhat the code does on the named input before and afterSuggestions that add bugs
Rename thisWhether the name is false or breaks a written conventionNaming
Extract the duplicateWhether the copies change for the same reasonDuplicate code
The docstring is staleWhich side the callers and tests rely onStale docstrings
Use the idiomThe language specification for the version you build withLanguage idioms

Tests#

Comment saysFact that settles itGuide
Add a testWhether a test already fails on the old codeMissing tests
This is a bugA test with the claimed input, run before and after the fixRegression tests
The assertion is weakWhether the test fails when you put the defect inWeak assertions
This test is flakyThe source of nondeterminism and whether varying it fails the testFlaky tests
This mocks too muchWhether breaking the unit turns the test redOver-mocked tests

Check where the comment sits#

Some comments are decided less by their claim than by their place in the pull request. A comment on code the pull request did not change may describe a real problem that belongs in its own issue (comments on unchanged code). A comment GitHub marks outdated may or may not still apply to the current head (outdated comments). A question asks for an answer that becomes a change, a test or a recorded reason (question comments). Two reviewers can ask for opposite changes (conflicting comments), and a reviewer can flag the very behavior the pull request means to change (comments against the intended change).

Leave the fact in the thread#

Whichever way you decide, write the deciding fact where the comment is. A change links its commit and, where one exists, the test that fails without it. A dismissal names the type, the count, the caller or the version that disproves the claim, and says what you did not check. The dismissal reply guide shows how to write one that a later reader can verify. Teams scoring a reviewer across a trial can then count the thread as the dismissed finding guide describes, without asking the author again.

One month of free code review.

New organizations get one calendar month of hosted code review for private and public GitHub repositories. No card or cloud connection required. Eligibility and fair-use terms.

Start free with GitHub

Related reading