Columns › 8. Security and working with AI › Part 35
Debugging a Project with AI: From Reproduction to Fix
用 AI 在專案裡 debug:從重現到修好
A disciplined way to debug the shop system with an AI coding agent: describe the symptom precisely, reproduce it with a failing test, feed evidence instead of guesses, let git bisect run find the culprit commit, test one hypothesis at a time, fix the root cause, and prove the fix with tests, -race and repeated runs.
In this article
- What this article solves
- Step 1: Describe the symptom precisely: expected, actual, scope and frequency
- Step 2: Reproduce first: let the agent write a failing test, and confirm it fails for the right reason
- Step 3: Feed it evidence, not guesses
- Step 4: Let the agent find the culprit commit automatically with git bisect run
- Step 5: Test one hypothesis at a time, and have the agent read the whole path
- Step 6: Fix the root cause, in the shared code, after finding every caller
- Step 7: Prove it is fixed, not just fixed by luck
- Step 8: Manage the session: branches, checkpoints and context
- Step 9: Leave a regression test, and write up the symptom, root cause and evidence in the PR
- Common pitfalls
- Key takeaways
- Next
In the previous article, “Automating Deployments and Troubleshooting with AI Workflows”, we dealt with production incidents: an alert fires, an agent with read-only permissions gathers evidence, and a human decides to roll back first. The rollback brings the service back, but the problem itself is still in the code; and many more bugs never reach production at all, because they are found during development, testing, load testing or when a colleague tries something out. This article goes back to the developer's own machine and covers the other half of the work: chasing a bug in the repo all the way from "someone says something looks odd" to "there is a test proving it is fixed", with an AI agent that genuinely helps along the way.
What this article solves
Pasting an error message into an AI and asking it to "fix this for me" is the most intuitive way to use it, and the most dangerous. It will usually come back quickly with a plausible explanation and a change, and the tests may even go green. The problem is that you don't know whether it fixed the real cause, or whether it just made the symptom go away for now. What an AI agent is good at is reading lots of code, running commands and organising output; what it is not good at is admitting "I don't know", or telling "makes sense" apart from "is true".
What this article builds is a debugging process that does not depend on the AI's judgement. Every step produces something that can be checked: a failing test, a piece of evidence produced by a tool, an experiment with a predicted outcome, a commit hash, a test that goes from red to green. The AI speeds up each step; the basis for judgement comes from these results, which anyone can rerun.
The example continues the shop system from the whole series: module example.com/shop, services api-gateway, order, payment and inventory, orders as Order{ID, UserID, Items, Total, Status}, and the amount Total stored as an int64 in cents. Three demonstration bugs come up again and again in this article:
| Bug | Symptom | What makes it hard |
|---|---|---|
| Discounted total is one cent short | For a few orders, Total differs by 1 cent from what customer support calculates by hand | Most orders are right; only certain combinations of amount and discount are wrong |
| Occasional overselling under load | With 10 items in stock, a load test occasionally sells 11 | All unit tests pass; it only shows up a few times in a hundred runs |
| More goroutines after a deployment | The order service's goroutine count keeps rising over time, and memory rises with it | No error messages, just a curve slowly climbing upwards |
The AI agent in the examples is Claude Code. Its basic setup (CLAUDE.md, permissions, hooks, plan mode) was covered in “Building Go Microservices with Claude Code: Rules, Permissions and Security Review”, so it is not repeated here; this article is only about debugging itself.
Figure: Describe first, then reproduce, then gather evidence; test hypotheses one at a time and go back to the list when one is rejected; once confirmed, fix the root cause, prove the fix with tests and repeated runs, and finally leave a regression test and a write-up.
Step 1: Describe the symptom precisely: expected, actual, scope and frequency
The first output of debugging is not code but a description. It is for the AI, and also for your future self and the reviewer. If the description is vague, the AI fills the gaps with its most common guess, and that guess may not apply to your system.
A good symptom description answers at least four things:
- Expected and actual. Not "the amount is wrong", but "subtotal 1990 cents, 15% discount, expected 1692 cents, actual 1691 cents". Only with concrete numbers can you turn it into a test.
- Scope. Which service, which endpoint, starting from which version; whether every order is wrong or only some combinations; whether it only happens in production or locally as well.
- Frequency. Every time, occasionally (roughly one in how many), or only under load or after running for a long time. Frequency decides how you reproduce it: something that happens every time just needs a unit test, while something intermittent needs repeated runs and the race detector.
- What is already known. What changed recently, what has already been ruled out, and links to any evidence. Writing down what has been ruled out stops the AI from spending time retracing the path you have already walked.
The table below shows two ways of describing the same problems:
| Vague description | A description you can debug with |
|---|---|
| The discount is calculated wrong | Subtotal 1990 cents, 15% discount: Total is 1691, but per the requirements it should round to 1692; 1999 cents at 15% is correct |
| Load testing oversells | inventory's Reserve, stock 10, 100 concurrent requests: in 100 runs, a few runs successfully reserve 11 or more |
| order seems to have a memory leak | Since deploying v1.9.0, go_goroutines grows by a roughly fixed amount every hour, drops to zero on restart and climbs again; v1.8.x does not do this |
Notice that the right-hand column contains no "causes". The symptom description deliberately leaves out guesses, because once you write "it's probably a floating-point problem", the AI will tend to search straight in that direction and stop at the first place that fits. You can have guesses, but put them under "what is already known", mark them as guesses, and ask it to check other possibilities too.
Tip: Write the symptom description in an issue or ticket, not only in the conversation. Conversations end and get compacted; tickets stay, and the eventual PR can link straight to them.
Step 2: Reproduce first: let the agent write a failing test, and confirm it fails for the right reason
Without a reproduction, there is no way to prove a fix. The goal of this step is a failing automated test, or at least a minimal script that reliably triggers the problem. This is the step most worth spending time on, and also where an AI agent helps most: it can quickly read the relevant code, find the path that calculates the total, write a table-driven test, run it and paste the result.
But there is one thing you must confirm yourself: the test fails for exactly the symptom you described. A test can fail for many reasons: a compile error, a wrong import path, a typo in the test data, an inverted assertion, testing a different function. An agent writing a failing test and then reporting "reproduced" proves nothing on its own. What you need to look at is the failure message: does it say ApplyDiscount(1990, 15) = 1691, want 1692, or undefined: ApplyDiscount?
Below is a debugging prompt handed to the agent. It does not ask it to fix anything, only to reproduce:
# Reproduce: discounted total is one cent short (issue #412)
## Symptom
- Expected: subtotal 1990 cents, 15% discount, `Total` = 1692 (requirement: round to the cent, 0.5 rounds up).
- Actual: 1691. Subtotal 1999 cents with a 15% discount is correct (1699).
- Scope: both order creation in `order` and refunds in `payment` use the discount calculation; it reproduces locally too.
## What is already known
- The calculation lives in `internal/pricing`; three weeks ago there was a "unify discount calculation" refactor.
- Guess (unverified): it might be floating point. Please do not check only this direction.
## This round is reproduction only: do not modify any non-test files
1. Read `internal/pricing` and its callers, and describe the calculation path (file:line).
2. Write a table-driven test in `internal/pricing/discount_test.go`: the two cases above, exact division, 0%, 100%,
and any other edge cases you think might go wrong; note on each row how the expected value was worked out.
3. Run `go test -run TestApplyDiscount -count=1 -v ./internal/pricing/` and paste the full output.
4. Explain why each row fails; if a failure message has nothing to do with the symptom (e.g. a compile error), fix the test itself first.The last line, "do not modify any non-test files", matters. Letting it start fixing during reproduction is the same as skipping "confirm why it fails"; and with the fix and the test mixed in the same diff, it is hard to tell whether the test was written to fit the change.
The test the agent produces looks roughly like this:
// internal/pricing/discount_test.go
package pricing
import "testing"
// Rule (from the requirements): the discounted amount is rounded to the cent, with 0.5 of a cent rounding up.
func TestApplyDiscount(t *testing.T) {
tests := []struct {
name string
cents int64
percent int64
want int64
}{
{"exact division", 10000, 10, 9000},
{"no discount", 1999, 0, 1999},
{"full discount", 1999, 100, 0},
{"fraction below 0.5 rounds down", 1999, 15, 1699}, // 1699.15
{"fraction of exactly 0.5 rounds up", 1990, 15, 1692}, // 1691.5: the reported order
{"floating-point error", 2900, 30, 2030}, // exactly 2030; must not become 2029
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := ApplyDiscount(tt.cents, tt.percent); got != tt.want {
t.Errorf("ApplyDiscount(%d, %d) = %d, want %d", tt.cents, tt.percent, got, tt.want)
}
})
}
}The result:
$ go test -run TestApplyDiscount -count=1 -v ./internal/pricing/
=== RUN TestApplyDiscount
=== RUN TestApplyDiscount/exact_division
=== RUN TestApplyDiscount/no_discount
=== RUN TestApplyDiscount/full_discount
=== RUN TestApplyDiscount/fraction_below_0.5_rounds_down
=== RUN TestApplyDiscount/fraction_of_exactly_0.5_rounds_up
discount_test.go:24: ApplyDiscount(1990, 15) = 1691, want 1692
=== RUN TestApplyDiscount/floating-point_error
discount_test.go:24: ApplyDiscount(2900, 30) = 2029, want 2030
--- FAIL: TestApplyDiscount (0.00s)
--- PASS: TestApplyDiscount/exact_division (0.00s)
--- PASS: TestApplyDiscount/no_discount (0.00s)
--- PASS: TestApplyDiscount/full_discount (0.00s)
--- PASS: TestApplyDiscount/fraction_below_0.5_rounds_down (0.00s)
--- FAIL: TestApplyDiscount/fraction_of_exactly_0.5_rounds_up (0.00s)
--- FAIL: TestApplyDiscount/floating-point_error (0.00s)
FAIL
FAIL example.com/shop/internal/pricing 0.012s
FAILThis output deserves a careful read, because it already tells you more than the original report. First, the reported case (1990, 15%) does fail, and the failure message is the symptom itself, so the reproduction holds. Second, the "floating-point error" row the agent added also fails: 2900 cents at 30% off is exactly 2030, yet the result is 2029. That row and "0.5 must round up" are two different phenomena: the latter is a wrong rounding rule, the former is the arithmetic itself being imprecise. A good reproduction test often widens your understanding of the problem like this.
Third, you have to check every row's expected value yourself. The want values in the test were calculated by the agent, and it can get them wrong too; 1990 × 0.85 = 1691.5, which per the requirements rounds up to 1692, and that has to be checked against the requirements document, not against what the current code outputs. Expected values come from the requirements, not from the existing code; that is the biggest difference between a reproduction test and a "test that describes the status quo".
An intermittent problem cannot be reproduced with an ordinary unit test, but the principle is the same: write a test that can trigger it, then amplify the probability by running it repeatedly. The overselling example can be written as a test in which 100 goroutines reserve at the same time, asserting "the number of successful reservations does not exceed stock", and then run repeatedly with -count (flags are covered in the next step). The goroutine leak can be written as a test that "after N calls, runtime.NumGoroutine() returns close to its starting value", or by checking for leaks directly at the end of the test. “Concurrency Pitfalls: From Data Races to Overselling” covers the causes and fixes of these kinds of problems, and you can reuse its test patterns when reproducing.
Step 3: Feed it evidence, not guesses
With a reproduction in hand, the next job is finding the cause. The most common mistake at this step is handing the AI your own guess as if it were fact: "I think it's the database connection pool, take a look." The AI will happily go along, find some "possibly problematic" spot in the connection pool code, and give you a story that makes sense. What you get is not the cause but an echo of your own guess.
A better approach is to give it evidence produced by tools, so that it reasons from the evidence rather than from your guess. Below are the most common kinds of evidence in a Go project and how to get them:
| Evidence | How to get it | Good for |
|---|---|---|
| Full output of a single test | go test -run '^TestApplyDiscount$' -count=1 -v ./internal/pricing/ | Calculation errors, logic errors |
| Full panic stack trace | The test output itself; add GOTRACEBACK=all when you need every goroutine | nil pointers, index out of range |
| All goroutines when it hangs | go test -timeout 30s ...: on timeout it panics and prints the stacks of all goroutines | Deadlocks, channels that never deliver |
| Data race report | go test -race -count=50 -run TestReserveConcurrent ./internal/inventory/ | Occasional wrong results, overselling |
| Order dependence | go test -shuffle=on -count=1 ./...; on failure it prints a seed you can reproduce with | Tests that pass alone but fail together |
| Goroutine and memory snapshots | curl -s 'localhost:6060/debug/pprof/goroutine?debug=1'; compare two snapshots with go tool pprof -base before.pb.gz after.pb.gz | Goroutine leaks, memory growth |
| Logs for a single request | Filter by trace ID; see “Observability in Go: Logs, Metrics and Tracing” | Cross-service errors, timing problems |
| Change history of the code | git log -p -S 'ApplyDiscount' -- internal/pricing/ (finds the commits that added or removed this string) | "It used to work" |
A few flags deserve an extra word. -count=1 disables the test result cache, so you see what actually ran this time rather than a cached result from last time; adding it every time while debugging avoids the "I definitely changed it, so why is the result the same?" confusion. -run takes a regular expression, so TestApplyDiscount also matches TestApplyDiscountMatchesExactRounding; to target one test exactly, write '^TestApplyDiscount$'. -race can only find races that actually execute, so it needs tests that trigger concurrency and enough repetitions; the two stack traces it reports (one write, and one read or write) are the most direct evidence there is. The details of pprof are in “Performance Tuning in Go: pprof and Benchmarks”; for goroutine leaks, debug=1 merges goroutines with identical stacks and shows a count, so you can see at a glance "which line has started several thousand goroutines".
Figure: Test output, stack traces, logs filtered by trace ID, race reports, pprof and git history are produced by tools; the agent connects them and produces hypotheses, each backed by evidence.
Letting the agent run these commands itself is usually better than running them yourself and pasting the output: it sees the full output rather than the excerpts you picked, and it can decide the next command based on the result. But there are two boundaries.
First, don't hand it production data directly. Production logs contain users' email addresses, home addresses and order contents, and possibly tokens that were printed by accident. Debugging usually needs only a few lines: the entries for one trace ID, the format of an error message, the value of a field. Filter first, mask next, and only then hand it over; if it can be reproduced with test data, don't use real data. The previous article stressed the same thing for incident troubleshooting, and it applies equally during development.
Second, evidence must be rerunnable. When the agent says "I ran the tests and they pass now", you need to see the actual command it ran and its output; rerun it yourself when necessary. Treat "I ran it" as a claim that needs evidence, not as a fact.
Step 4: Let the agent find the culprit commit automatically with git bisect run
Many bugs have one thing in common: "it used to work". The discount calculation was right three weeks ago and is wrong now; the goroutine count was flat on v1.8.x and started climbing with v1.9.0. In that situation, rather than reading code and guessing at the cause, go straight to finding which commit introduced it. Once you find that commit, the scope of the problem shrinks from the whole repo to a single diff.
git bisect does this with a binary search: you give it a version known to be good and a version known to be bad, it checks out the commit in the middle each time and asks whether that version is good, and the range halves. A thousand commits take about ten rounds. git bisect run hands the "good or bad?" decision to a script: exit code 0 means good, 1 to 127 (except 125) means bad, and 125 means this commit cannot be judged and is skipped.
This is exactly the kind of work to hand to an agent: mechanical, repetitive, with clear-cut results. You only need to confirm two things: that the start and end points really are good and bad respectively, and that the judging script tests what you want.
# 1. First save the reproduction test outside the repo: old commits don't have this file, so it has to be copied in every round
cp internal/pricing/discount_test.go "$TMPDIR/discount_repro_test.go"
# 2. Confirm the endpoints: v1.8.0 should be good and HEAD should be bad (run each once and see the result with your own eyes)
git stash -u # the working tree must be clean for bisect to switch commits
# 3. Start the binary search
git bisect start HEAD v1.8.0
git bisect run sh -c '
cp "$TMPDIR/discount_repro_test.go" internal/pricing/discount_test.go
# A commit that does not compile cannot be judged: return 125 to skip it rather than treating it as bad
if go test -count=1 -run "^$" ./internal/pricing/ >/dev/null 2>&1; then
go test -count=1 -run "^TestApplyDiscount$" ./internal/pricing/
rc=$?
else
rc=125
fi
# Restore the original files of this commit and delete the copied-in test, so the next round can switch
git checkout -- internal/pricing/ && git clean -fq internal/pricing/
exit $rc
'
# 4. Save the log and go back to the original branch
git bisect log > "$TMPDIR/bisect-412.log"
git bisect reset
git stash pop
# 5. See what that commit changed
git show --stat <the commit found>
git show <the commit found> -- internal/pricing/This script has a few easy-to-miss traps, which are also exactly the ones an agent tends to miss:
- The test file has to be brought in from outside. The newly written reproduction test only exists in your working tree; old commits don't have it. Copy it in each round and restore afterwards, otherwise switching commits will conflict with the untracked file, or you will end up testing the old version's own test.
- A compile failure does not mean bad. An older version may have a different function signature, or some intermediate commit may simply not compile.
-run "^$"compiles the tests without running any of them, and if it doesn't compile we return 125 to skip; if you letgo testreturn 1 directly, bisect will mistake "doesn't compile" for "introduced the bug" and find a completely unrelated commit. - Amplify the probability of intermittent problems first. With something like overselling, a single test run may happen to pass, bisect will mark a bad commit as good, and the whole result is wrong. The judging script needs repetitions such as
-race -count=50to turn "occasional" into "almost certain"; the cost is that each round is slower, but that is far cheaper than finding the wrong commit. - Confirm the endpoints yourself. If v1.8.0 is actually bad too, bisect will give you an answer that looks very certain and in fact means nothing.
Figure: Each round tests the commit in the middle and goes left or right depending on the script's exit code; 125 means it cannot be judged, and the commit is skipped.
Once you have found the commit, hand its diff to the agent and ask it to explain how this change produces the result seen in the test. Now its explanation has an anchor: not "I think it might be here", but "this commit changed (cents*(100-percent)+50)/100 into int64(float64(cents)*(1-float64(percent)/100)); the int64() conversion always truncates, and 0.7 cannot be represented exactly in binary floating point". You can verify an explanation like that word by word against the diff.
Tip: bisect only works for problems where "some version was good". If a new feature was simply written wrong, or the problem comes from data and environment rather than code, bisect will get the same result on every commit; in that case go back to step 3 and work from the evidence.
Step 5: Test one hypothesis at a time, and have the agent read the whole path
With evidence in hand, you can start listing hypotheses. A good hypothesis has three characteristics: it explains all the observed evidence, not just part of it; it comes with an experiment you can run; and it states up front which outcome means it holds and which means it doesn't.
Taking overselling as an example, after reading inventory's code and the -race report, the agent might list:
| Hypothesis | Supporting evidence | Experiment | Prediction |
|---|---|---|---|
Reserve reads stock and then writes separately (check-then-act) | The -race report points at a read and a write of the same field | Put the read and write under the same lock or in a single conditional UPDATE, rerun 200 times | If it holds, overselling drops to zero |
| Retries cause the same reservation to be executed twice | The same order_id appears with reserved twice in the logs | Check for duplicates with an idempotency key; measure the proportion of duplicates | If it holds, the oversold quantity equals the number of duplicate reservations |
| The test's own counting has a race | None | Count with atomic.Int64 and rerun | If it holds, the test stops failing once fixed |
The third row looks redundant, but it matters: the reproduction test itself can have bugs. If the test accumulates the success count in a plain int across multiple goroutines, the race detector will point at the test rather than the product code, and the conclusion is completely different.
Next, test only one at a time. If you change all three places at once and the overselling disappears, you won't know which change did it, and one of them may have introduced a new problem. Run one experiment at a time, write the prediction down first, then look at the result; cross out rejected hypotheses and add new evidence to the list. This sounds slow, but it is much faster than "changed a bunch of things, seems fixed, back again two weeks later".
During this process, ask the agent to read the whole call path, not just the line the error message points at. The root cause of overselling often isn't in Reserve itself but in how it is called: a handler checks stock outside the transaction, a message consumer processes a redelivered message twice, or the gateway's retry lets the same request in twice (“Timeouts, Retries and Circuit Breakers”, “Idempotency and Distributed Locks”). The same goes for goroutine leaks: pprof tells you which line the goroutines are stuck on, but the real problem is usually "who started it, and who should make it finish", for example a time.NewTicker that is never Stopped, a goroutine waiting on a channel that nobody will ever close, or a background job not wired to the service's context and left out of the shutdown sequence designed in “Configuration and Graceful Shutdown in Go”.
When asking the agent to read the path, require it to report files and line numbers, for example "internal/order/handler.go:88 calls service.Create, and internal/order/service.go:142 calls inventory.Check before the transaction starts". That way you can open those lines and check for yourself, rather than trusting a summary. Wide-ranging searches can be handed to a subagent: it reads files and searches in its own separate context and only hands back its conclusions, so the main conversation isn't flooded with search results.
Tip: Ask the agent to write the "evidence against" next to each hypothesis. A hypothesis that only lists supporting evidence is often a conclusion looking for evidence; only when it volunteers "this hypothesis can't explain why 1999 cents comes out right" is it really comparing against the evidence.
Step 6: Fix the root cause, in the shared code, after finding every caller
Only once the cause is confirmed do you start fixing. There are two choices of where to fix: patch in a special case where the problem shows up, or correct it where the problem is caused. The former fixes the symptom; the latter fixes the root cause.
For the discount, a symptom fix might be an extra +1 correction when order creates an order, or another pass with math.Round after the calculation. The test will pass, but payment's refunds call the same ApplyDiscount and are still wrong; and the next new feature that calls it will be wrong too. AI agents are especially prone to this kind of fix, because the only thing in their context is that failing test, and the shortest path is to make that test pass.
So before touching anything, find every caller. You can have the agent search with grep -rn 'ApplyDiscount(' --include='*.go' ., or use gopls's "find references" to find every reference (including calls through interfaces), and then put them in a table: where each caller is, where the arguments it passes come from, and whether its behaviour will change after the fix. This table is also part of the PR description: the reviewer needs to know which places the change affects.
// internal/pricing/discount.go
package pricing
// ApplyDiscount returns the amount in cents after a discount of percent (0–100), rounded to the cent with 0.5 rounding up.
// It uses integer arithmetic only; cents*(100-percent) does not overflow while cents is below 9e16.
// The ranges of the amount and percentage are validated when orders and discount codes are created, so they are not checked again here.
func ApplyDiscount(cents, percent int64) int64 {
return (cents*(100-percent) + 50) / 100
}This fix deliberately keeps the function signature unchanged: none of the callers need changing, and the reproduction test from step 2 becomes the regression test untouched. It also deliberately doesn't change anything else "while it's at it", such as switching the percentage to float64, adding a rounding-mode parameter, or refactoring the whole pricing package. Those changes may be worthwhile, but they belong in another PR; mixed into a bug fix, they make it impossible for the reviewer to tell which lines are the fix and which are refactoring, and they make bisect harder to use later. Why amounts should be calculated in integer cents, and when you need math/big or a decimal type, is discussed in full in “Handling Money in Go: Minor Units, Decimals, Currencies, Rounding and Allocation”.
When fixing the root cause, watch out for two AI habits. First, it may use APIs that don't exist: a very plausible-looking math.RoundHalfUp, an option a package doesn't have, or a method that only existed in an old version of some library. The compiler catches most of these, but not "exists, but doesn't behave the way you think": for example, math.Round rounds half away from zero, while math.RoundToEven is banker's rounding, and the two handle .5 differently. Whenever it relies on the behaviour of the standard library or a package, check the documentation or confirm it with a small test. Second, it may over-modify: changing twenty files to fix one rounding problem. The bigger the diff, the harder the review and the more surprises hide in it. Require it to "only modify the function where the root cause lies and the callers that must change, and list any other changes separately as suggestions".
Step 7: Prove it is fixed, not just fixed by luck
A green test doesn't mean the problem is solved. This step rules out three ways of being "fixed by luck": the change only works for the few numbers in the test, the intermittent problem just didn't happen this time, and the test itself has been changed so that it can't fail.
Verify over a wider range, not just those few cases. The reported order is only one sample. For a pure calculation function, you can compute the exact value with an independent, obviously correct method, and then compare one by one across an entire range:
// internal/pricing/discount_exact_test.go
package pricing
import (
"math/big"
"testing"
)
// Compute the exact value with math/big rationals, round it, and compare with ApplyDiscount case by case.
// This checks a whole range rather than only the reported case, to avoid "fixing just that one number".
func TestApplyDiscountMatchesExactRounding(t *testing.T) {
half := big.NewRat(1, 2)
for cents := int64(0); cents <= 20000; cents++ {
for percent := int64(0); percent <= 100; percent += 5 {
exact := new(big.Rat).SetFrac64(cents*(100-percent), 100)
exact.Add(exact, half)
want := new(big.Int).Quo(exact.Num(), exact.Denom()).Int64() // non-negative, so Quo simply truncates
if got := ApplyDiscount(cents, percent); got != want {
t.Fatalf("ApplyDiscount(%d, %d) = %d, want %d", cents, percent, got, want)
}
}
}
}On the original buggy version, this test fails on the very first case it reaches: ApplyDiscount(1, 5) should be 1 (0.95 rounded), but the floating-point version gives 0. In other words, the original bug affected not just the few reported orders but a large number of small-amount combinations. A "compare against a reference implementation" test like this isn't hard to write, and it proves the fix is complete far better than ten hand-written cases. For more complex inputs, Go's built-in fuzzing (func FuzzXxx(f *testing.F)) can follow the same idea, letting the tool generate inputs and comparing against the reference implementation.
Confirm the regression test really catches this bug. Temporarily revert the fix (git stash only the implementation change, or ask the agent to change that line back), rerun the test and confirm it fails; then put the fix back and confirm it passes. This "red, green, red again, green again" check takes a minute, yet it proves that there is a real causal link between the test and the fix, rather than a test that passes on any version.
Run intermittent problems repeatedly. For something like overselling, one passing run after the fix means almost nothing. Run it at the same number of repetitions as the reproduction, or more, for example go test -race -count=200 -run TestReserveConcurrent ./internal/inventory/. If several of 200 runs failed before the fix and all 200 pass after it, that is convincing evidence; write down the before and after counts in the PR and the reviewer can judge for themselves. By the same logic, a fix for a goroutine leak should be judged by the curve over a period of time or the difference between two pprof snapshots, not by "looks fine now".
Finally, run the full checks. go vet ./..., go test -race ./..., and whatever else the project runs in CI. When you fix shared code, failures often show up where you didn't expect them: some other test depended on the wrong behaviour, or some golden file contains a wrong amount. When that happens, stop and decide: is that test right and the fix wrong, or did that test treat the error as expected all along? In the latter case, confirm with the requirements before changing it, and explain it in the PR.
Tip: Watch whether the agent, to get a test passing, changed the test's expected values, added
t.Skip, deleted a row, or loosened an assertion. In PR review, look at the_test.godiff first: regression tests should only ever be added, never weakened. As “Testing in Go: Table-Driven Tests, httptest and Integration Tests” discussed, the value of a test is that it represents the requirements, not the program's current behaviour.
Step 8: Manage the session: branches, checkpoints and context
Debugging is often a very long conversation: reading files, running commands, trying a direction, giving up, trying another. This way of working carries some specific risks, which are worth handling with the tool's features.
Work on a branch or worktree. Debugging produces a lot of experimental changes, some to keep and some to throw away. Keep them on a separate branch so the main branch stays clean; what you finally commit is a tidied-up fix and tests, not the history of the experiments.
Use checkpoints to go back, but know their limits. Claude Code automatically creates a checkpoint every time you send a prompt, and /rewind (or pressing Esc twice when the input box is empty) can roll the code, the conversation or both back to an earlier point. This suits "this direction is wrong, go back to where we were" well. But according to the official documentation, checkpoints track changes it made with its file-editing tools: files changed only by Bash commands are not tracked, changes made by subagents usually aren't reverted either, and it is not a replacement for version control. Debugging often involves running commands that generate files or change data, so the truly reliable way back is still git: commit a clean state before experimenting, and diff or restore directly when needed.
Control the context. A long debugging conversation fills up with early wrong hypotheses, rejected experiments and large amounts of command output, and the agent is sometimes misled by this old information, going back to explanations that have already been ruled out. A few practices: write confirmed facts and rejected hypotheses back into the issue, and when needed start a new session that continues with only that tidied-up summary; hand wide-ranging searches to a subagent; keep the hypothesis table from step 5 in a file, not only in the conversation.
Plan first, then change. Once the cause is confirmed and the fix touches several files, use plan mode to have it first list what it will change and how it will verify, and only start changing once you have approved. This is the same thing as "find every caller first" in step 6: see the scope clearly before doing anything.
Step 9: Leave a regression test, and write up the symptom, root cause and evidence in the PR
Once it's fixed, the results of this debugging session should turn into two things that last.
The first is a regression test. Both the reproduction test from step 2 and the reference-implementation comparison from step 7 should go into the repo and run in CI. The test names and comments should say clearly what they guard: "fraction of exactly 0.5 rounds up: the reported order" is far more useful than "case 5", because the next person to make it fail will start understanding why from there. Intermittent problems can run with fewer repetitions in CI and with more repetitions in a scheduled job.
The second is the PR description. A bug-fix PR should let the reviewer answer at least these questions within a few minutes:
- Symptom: expected, actual, scope, linked to the original issue.
- Root cause: a sentence or two on why it was wrong, with the commit that introduced the problem (if found with bisect) and the relevant file:line references.
- Fix: what changed and why it was changed here; the list of affected callers, and whether their behaviour changes.
- Evidence: output showing the test failing before the fix and passing after it; the number of repetitions and results for intermittent problems; the result of
go test -race ./.... - What was not done: for example, "historical order amounts were not corrected; whether to backfill needs a separate assessment". This item is easily overlooked, yet it is often the most important follow-up work: orders and refunds already created with wrong amounts need to be agreed with finance or customer support.
- AI assistance: which parts were produced by the agent and how you verified them. “Bringing AI Workflows into the Team Process: PR Review, CI and Human Gates” discussed the purpose of this field: not to assign blame, but to let the reviewer know where to focus.
You can ask the agent to draft this description too, since it has the whole process at hand. But the "root cause" and "evidence" fields must be confirmed by you: they have to match the commands that were actually run and their output, not the agent's recollection of the process.
Common pitfalls
Plausible but wrong explanations. The biggest risk with AI is not that it can't answer, but that it confidently gives an explanation that is reasonable yet not true. The countermeasure is to tie every explanation back to evidence: which line of code, which piece of output, which commit. Anything that can't be tied back is only a hypothesis.
Changing the test so that it passes. Changing expected values, adding t.Skip, deleting the failing row, turning == into "within 1". These changes can be well hidden inside a large diff. In review, look at the test files' diff first.
Fixing the symptom instead of the root cause. Adding a recover() to swallow a panic, a time.Sleep so that the race is less likely to show up, a retry so that the occasional error disappears, a one-line correction in one particular caller. The symptom disappears, the problem remains, and it becomes harder to find.
Hallucinated APIs. Functions that don't exist, flags that don't exist, behaviour that only exists in some version. Compiling and the documentation are the best checks; the same goes for tool flags: when unsure, run go help testflag first or read the official documentation.
A huge diff that nobody reads carefully. Refactoring, reformatting or upgrading dependencies "while at it". The smaller a bug fix, the better; open separate PRs for other changes.
Pasting production data or secrets into prompts. Logs, database exports, configuration files and environment variables can all contain personal data and tokens. Filter and mask first, and if it can be reproduced with test data, don't use real data; if you accidentally paste a secret, treat it as leaked and rotate it immediately.
Running an intermittent problem once and declaring it fixed. Without the same level of repetition as before the fix, "passed" means nothing. The same applies to bisect: if the judging script doesn't amplify the probability, it will find the wrong commit.
Believing "I've already run the tests". You need to see the actual command and output, and rerun it yourself when necessary; in particular, confirm that -count=1 was used rather than looking at cached results.
Skipping reproduction and going straight to the fix. Without a failing test, there is no way to prove it is fixed, and no way to stop it coming back.
Key takeaways
- Write the symptom first: expected, actual, scope, frequency and what has been ruled out; don't write guesses as facts.
- Reproduce first: have the agent write a failing test, confirm yourself that the failure message is the symptom itself, and take expected values from the requirements, not from the existing code.
- Feed it evidence:
go test -run '^X$' -count=1 -v,-race -count=N,-shuffle=on, the stacks printed by-timeout, pprof snapshot comparison, logs filtered by trace ID,git log -p -S; filter and mask production data and secrets first. - When "it used to work", use
git bisect run: bring the reproduction test in from outside the repo, return 125 when it doesn't compile, amplify the probability of intermittent problems first, and confirm the endpoints yourself. - A hypothesis must explain all the evidence and come with an experiment and a predicted outcome; test only one at a time, and have the agent report file and line numbers for the whole call path.
- Fix it in the shared code where the root cause lies, after listing every caller; keep the change small, put refactoring in a separate PR, and verify the behaviour of any API it relies on.
- Prove the fix with a reference-implementation comparison, a red-green check before and after the fix, the same level of repetition as the reproduction, and a full
go test -race ./.... - Use branches and git as the reliable way back; checkpoints are just a convenient aid; tidy long conversations into a summary before continuing.
- Put regression tests in CI, and in the PR spell out the symptom, root cause, fix, evidence, unhandled follow-ups and the parts the AI helped with.
Next
Debugging finds the places where a program "doesn't work as expected"; security problems are often a program "working exactly as written", except that what was written allows things that shouldn't be allowed. The next article, “Security Checks with AI: Threat Modelling, Tracing Input and Verifying Findings”, applies the same "evidence required, must be rerunnable" discipline to how AI can help find security problems in the shop system without being led astray by its false positives.