---
title: "How to catch a suggested fix that would add a bug | Proofline"
description: "Before applying a bot's suggested fix, write down what the code does now on the input the comment names, and what it would do after the change."
url: https://proofline.sh/learn/ai-review-suggestion-adds-bug/
---

[Learning center](/learn/)

# How to catch a suggested fix that would add a bug

Before applying a bot's suggested fix, write down what the code does now on the input the comment names, and what it would do after the change. If the two differ in a way the comment did not ask for, the suggestion changes behavior: decline it, or make the narrower fix yourself. Running the old and new code on that input, or a test that pins the current behavior, settles the question.

Code review The Proofline team November 22, 2026 4 min read

## A refactor the bot said had changed behavior

In October 2026, a pull request in biop-ai, a fork of Biopython, moved the nested closures of the `read_PIC` parser in `Bio/PDB/PICIO.py` into a private parser class, with one method per record type. The author treated it as a pure refactor. gemini-code-assist disagreed about the new `_parse_atom` method: "This changes the parser's behavior, causing it to silently ignore malformed `ATOM` lines instead of failing." It suggested a rewrite: "To preserve the original behavior, `_parse_atom` should explicitly return `False` when the line cannot be parsed." [Review comment](https://github.com/BioAGI-org/biop-ai/pull/158#discussion_r4200868657)

The claim is specific enough to check, and the author checked it: "Not a behaviour change, so leaving it." The original `elif line.startswith("ATOM ")` branch takes every line that starts with `"ATOM "`, and when neither regular expression matches, "the `if m:` body is skipped and the loop moves on." The author ran the old module and the branch on a residue line followed by a malformed `ATOM` line: "Both return a `Structure`, with identical stdout." Then the conclusion: "Applying this suggestion would make `read_PIC` return `None` for such files." [Author's reply](https://github.com/BioAGI-org/biop-ai/pull/158#discussion_r4206718273)

The pull request's diff agrees. In the removed code, the `ATOM` branch is an `if m:` block with no `else`, so a line that matches neither pattern falls through to the next line of input; it never reaches the `else:` branch that tries the edron pattern and can return `None`. The bot's comment describes the old code reaching that branch, which it could not. Its suggestion would have turned a tolerated malformed line into a failed parse, the behavior change it was warning about. The author kept the code, and the pull request merged on 8 October 2026.

## A rename that missed one reference

The opposite outcome happened in GoogleCloudSamples/client-library-samples in November 2026. gemini-code-assist recommended renaming the parameter `id` to `build_id` in two Cloud Build samples, "to avoid shadowing Python's built-in `id()` function." [Review](https://github.com/GoogleCloudSamples/client-library-samples/pull/173#pullrequestreview-5810566365) Both suggestions were committed from the review, and the pull request merged on 10 November 2026.

The suggestion changed the signature, the docstring and the request. It could not reach a line further down that still used the old name:

```
def cancel_build(project_id: str, build_id: str) -> None:
    ...
    except exceptions.NotFound:
        print(f"Build not found: {id}")
```

Before the rename, `{id}` was the build ID. After it, the name resolves to the built-in function. A reduced copy of the function prints this when the lookup fails:

```
Build not found: <built-in function id>
```

The merged `cancel_build` file contains that line, and so did the `main` branch when I fetched it on 22 November 2026. The `get_build` sample has no such reference, so its rename was complete. The line runs only when the build does not exist, so the ordinary path behaves as before and nothing failed. The renaming advice was sound, and the suggested patch was incomplete. Applied as written, it replaced a theoretical shadowing problem with a wrong error message.

## What to check for each kind of suggestion

| Suggestion | Check before applying | Evidence that it adds a bug |
| --- | --- | --- |
| Rename a variable, parameter or field | Search the function, file and callers for the old name | A use outside the suggested lines that now resolves to something else, or a keyword caller that breaks |
| "Preserve the original behavior" in a refactor | Run the base branch and your branch on the input the comment describes | The outputs already match, and the suggestion makes them differ |
| Add an early return, `raise` or `return False` | Find every caller and what it does with that result | A caller that turns the new value into a failure or a skipped record |
| Tighten a condition or a type | List the inputs that pass today and would not pass after | A valid input that the stricter version rejects |
| Replace a call with a "more idiomatic" one | Read the library documentation for both calls | Different handling of empty input, `None`, ordering or errors |

If the evidence column is empty after you look, the suggestion may still be right. Apply it and let the tests run.

## Run the old code beside the new

The biop-ai author's check is the general method. Load the base version of the module beside the branch version, feed both the input the comment names, and compare the outputs. When they match, the refactor preserved behavior on that input and the suggestion would change it; reply with the input and the result. When they differ, the comment found a real regression and the suggestion is a candidate fix, which you then run the same way.

Keep the input as a test when it is cheap to do so. A test that feeds a malformed `ATOM` line and asserts that `read_PIC` still returns a structure would turn the author's reply into something the next reviewer, human or bot, cannot misread. Where a suggestion is right in substance and short in reach, as in the rename, accept the idea and make the complete edit yourself; [the guide to committing suggestions](/learn/ai-review-suggested-change-block/) lists the boundary checks.

Record what you ran in the reply. A dismissal that cites an input and an output is one a maintainer can verify in a minute, which matters when you later [score the dismissed finding](/learn/reconstruct-dismissed-review-finding/).

## One month of free code review.

New organizations get one calendar month of hosted code review for private and public GitHub repositories. No card or cloud connection required. [Eligibility and fair-use terms](/free-code-review/).

[Start free with GitHub](https://proofline.sh/orgs/plan?offer=free_month)

## Related reading

-   ### [Before you click "Commit suggestion" on a bot's suggested change](/learn/ai-review-suggested-change-block/)
    
    Code review The Proofline team November 22, 2026 4 min read
    
-   ### [Read a dismissed code review finding before you score it](/learn/reconstruct-dismissed-review-finding/)
    
    Code review The Proofline team November 20, 2026 4 min read
